WikiDB/CHANGELOG

From TestWiki
Jump to: navigation, search

This page lists the changes between releases. Newer versions are on top. See WikiDB/Versioning for details about what the version numbers mean.

Code re-factoring and other non-functional changes are not listed.

v5[edit]

5.0.3[edit]

  • I've implemented many compatibility fixes for newer MediaWiki versions. The extension should now run correctly on all MediaWiki versions from 1.16 to 1.35.
    • Thanks to Ted LaPage for helping me debug various compatibility issues.
  • I've implemented a number of compatibility fixes for new PHP versions, so it now runs on all PHP versions supported by those versions of MediaWiki (from 5.1 to 7.4).
  • Fixed potential encoding issues when using old versions of PHP, prior to UTF-8 becoming the standard encoding, or if there is a non-standard encoding set in php.ini.
  • As reported by a user, when setting $wgWikiDBNamespaces to an integer value (rather than array), things would break if you loaded the extension prior to setting this variable. Now, you can define the setting at any point in LocalSettings.php.
  • I've updated the maintenace scripts to use the Maintenance class, which has existed since MW 1.16. This means that using the --help option on these scripts now gives more useful output, including a description of the script and details about any arguments.

5.0.2[edit]

Unreleased.

5.0.1[edit]

Unreleased.

5.0.0[edit]

This is the first release under the new versioning scheme.

Compatibility:

  • WikiDB now requires MediawWiki 1.16 or above.
    • This means that support has now been dropped for MediaWiki 1.7 through 1.15.
    • In practice, due to limitations in earlier versions of MediaWiki, WikiDB functionality on those versions has always been incomplete. 1.16 has been the recommended minimum version for quite a long time.
    • See WikiDB/Legacy versions if you still need to run WikiDB on an older version of MediaWiki.
  • WikiDB should now be compatible with all supported MediaWiki versions (1.16 to 1.32)
    • I have gone through the release notes for all MediaWiki versions up to 1.32 and fixed any deprecations/backwards-compatibility issues they raised.
  • WikiDB should now be compatible with all supported versions of PHP (5.2 to 7.3).
    • I have gone through the migration notes for all PHP versions up to PHP 7.3 and fixed any issues they raised.
  • Note that it is impossible to test all WikiDB functionality on all PHP/MediaWiki combinations. It is also not uncommon for release notes to be incomplete. Therefore there may still be some compatibility issues remaining, despite my best efforts. Please let me know if you encounter any problems.
  • Fixed a fatal PHP error that occurred when editing pages on MediaWiki 1.27, as reported by JSmith.

Internals:

  • I have changed the way WikiDB is versioned so that it now uses semantic versioning. For full details about the new versioning scheme, see WikiDB/Versioning.
  • Updated all code files to refer to the correct license (CC BY-SA 2.0 UK) as per LICENSE.txt. Previously some files referred to a slightly different version of the CC-BY-SA license. Note that this is not a licensing change, just a clarification.
  • I tidied the information that we display on the Special:Version page, to be consistent with modern best-practice.
  • WikiDB now uses the MediaWiki auto-loader for all class files. This should give a small performance improvement for most page views.
  • WikiDB now includes unit tests. There is only a small amount of coverage so far, focussing on the extensible type system and the built-in types, but more tests will be added in the future.

Data type handling:

  • Numbers are now fully internationalised. Previously, numbers were rendered in the local format, but they had to be entered in English. The extension now uses the MediaWiki number parsing code, which means that numbers may be entered in the local format for the wiki, as well as in English (which MediaWiki always recognised), and in English there is now more flexibility, e.g. digit grouping is allowed (e.g. "12,345.50").
  • Number formatting has had some minor improvements in certain edge-cases.
  • Detection of numbers for untyped fields has been improved. As well as being fully internationalised it is also a bit stricter about what it considers to be 'probably a number', to cut-down on false positives.
  • Fixed a fatal PHP error that occurred if a 'link' field contains characters that are not allowed in an internal link (e.g. colons), as reported by JSmith.

Other:

  • If you only have a single table namespace, you can now declare $wgWikiDBNamespaces to the appropriate namespace ID (e.g. $wgWikiDBNamespaces = 100;) rather than having to use the array syntax ($wgWikiDBNamespaces = array(100 => true);). This is in addition to the old method, to simplify the most common use-case.
  • Some of the parser tags weren't passing-on the supplied $frame argument to subsequent parsing code. This has now been resolved. This will presumably improve parsing in some instances, but I'm not sure when or how!
  • We now set 'noindex, nofollow' as the robot policy when viewing the raw table data, to stop search engines from indexing the raw data as this is not very helpful. Query results output on article pages are still indexed, however.
  • Fixed empty page content when specifying action=viewdata for a page that is not a valid table. WikiDB now shows an error page in this situation.

v4[edit]

r1376[edit]

This release contains one new feature:

  • The edit page now includes details about all data defined on the page, similarly to the way MediaWiki shows a list of all templates used in the page.

It also contains a number of bug/compatiblity fixes:

  • The redirect link that is provided when attempting to view data for a table that is a redirect now includes any query arguments that were passed to the page, meaning that these are not lost over the redirect.
  • Fixed a fatal error on special pages provided by WikiDB, if the limit argument was passed in via pretty URLs (e.g. /Special:AllTables/10 as opposed to /index.php?title=Special:AllTables&limit=10).
  • You will no longer get fatal errors on your wiki if WikiDB is enabled but the required database tables do not exist (as will be the case - for a brief period at least - when the extension is first installed). This was particularly problematic as the installation instructions direct users to check that WikiDB is installed via the Special:Version page prior to running the install script, which would have given a fatal error prior to this fix.
  • Fixed a number of issues that prevented WikiDB from running on MediaWiki 1.28 and 1.29.
  • Fixed a bug that prevented users of MW < 1.16 from being able to apply database updates when upgrading WikiDB.
  • Fixed highly rare bug that stopped the install script working for users of MW < 1.17 running on PHP < 5.2.1 who were using Postgres as their database back-end.

r1345[edit]

Various compatibility fixes for MW 1.27 and above. I went through the changelogs for 1.26, 1.27 and 1.28 and added fixes for any removed/deprecated items listed there, so hopefully I've caught everything!

r1333[edit]

Minor bug/compatibility fixes:

  • Field names that begin with a period (e.g. ".Field") are now handled correctly.
  • Attempts to query a table with an invalid name (e.g. "..") now output an appropriate 'bad table name' message rather than a fatal PHP error.
  • Fixed situations where query error messages were being assigned incorrectly, and so were never displayed to the user.
  • Fixed a number of E_STRICT errors across the code. This should hopefully mean the extension can now be used with strict-standards enabled.
  • Compatibility fix for MW 1.28 (as reported here).
  • Fixed incorrect version number being reported.

r1143[edit]

This is a major-release that adds a large amount of new functionality:

  • There has been a major rewrite of the criteria code, for <repeat> tags:
    • You can now use AND, OR and XOR for joining expressions. Previously expressions could only be joined by commas (which were equivalent to AND, and which are still supported).
    • We now support parentheses in expressions, which can be used to indicate precedence when AND and OR are used in the same expression.
    • A number of operator synonyms are now recognised, e.g. LT for <, && for AND, etc. (full list)
    • Expressions now support field names containing spaces and other special characters (so long as they aren't operators, e.g. =).
    • It is now possible to quote field names (using backticks) and strings (using single- or double-quotes). This is required for field names that contain characters that are otherwise recognised as operators or if you want to use alternative expression ordering (as per the next point).
    • Previously the expressions always needed to be of the form FIELDNAME OPERATOR VALUE, e.g. Name = John. This is still the default for unquoted values, but if you quote the operands then either or both sides of the expression may contain a field name or a literal value, therefore constructs such as "John" = `Name`, `CountryOfBirth` = `CountryOfDeath` and so on are now possible.
  • Field data is now rendered consistently in all cases. Previously alias resolution, field options and type-guessing were being applied inconsistently.
  • You can now specify criteria and ordering options on the 'data' tab, allowing you to easily filter/sort the table data.
  • Fixed an issue whereby 4-digit numbers are being displayed with a thousands-separator, which means that years (such as "1976") are being displayed incorrectly (as "1,976").
  • All HTML tables that are generated by the extension are now output consistently. Previously the default output of the <repeat> tag was configurable, but all other tables (e.g. field definitions, data tab, etc.) were hard-coded. Now they all use CSS classes for styling (as opposed to inline styles), and they all apply the WikiDBTable class and whatever custom classes are defined in $wgWikiDBDefaultClasses, as well as an additional class defining its type (e.g. WikiDBFieldDefinitions). This makes styling a lot easier and more consistent.
  • As a result of the above change, styling of WikiDB may be slightly different to that of previous versions.
  • If you put a <data> tag inside a page in a Table namespace and you leave the table attribute blank, then it now defaults to the table you are in. Previously, this would output an error and the tag would be ignored (which is still the case in non-table namespaces).
  • All user-visible text should now be translatable/editable via the standard MediaWiki message system. If you come across anything that's been missed, please let me know! Note that some messages have changed name and/or contents, so if you have localised any messages you may wish to review your customisations.
  • Sorting now treats missing and empty values as equivalent, as intended (previously missing values would be sorted before empty values).
  • Various compatibility fixes for MediaWiki < 1.16.
  • There has been a large amount of internal refactoring in this release, so if you have modified the code then this might be a tricky upgrade. I am happy to help you with this if you need me to.

r972[edit]

Compatibility fix for MediaWiki 1.24 and above.

  • wfViewPrevNext() was removed in MediaWiki 1.24, so WikiDB gave a fatal error on special pages and the 'data' tab. WikiDB has been upgraded to use Language::viewPrevNext() instead, on versions of MediaWiki that provide it (for older versions, we still use the old method).

r971[edit]

  • Fixed fatal PHP error that occurred if you output rows from a table containing Image or Link fields which are unpopulated.

r970[edit]

  • Fixed an issue whereby variable expansion in custom <repeat> tags wasn't working as expected. Thanks to User:Okino for the bug report and detailed test-case.
  • Fixed an issue whereby typed fields weren't being formatted correctly in custom <repeat> tags, and were being output as if untyped.

r944[edit]

A bugfix release for older MediaWiki versions. If you are using MediaWiki 1.18 or later then you do not need to upgrade.

  • Fixed SQL error on special pages in MW < 1.18
  • Fixed fatal PHP error on special pages on MW 1.17 only.
  • Fixed mangled HTML generated when <repeat> tags contain headings, on MW < 1.18

r937[edit]

  • WikiDB now fully supports PostgreSQL, and is fairly well-tested on PostgreSQL v9.3. Please let me know if you encounter any issues, especially when running on older versions.
  • Fixed a bug whereby page deletions would throw an error and leave old data in your WikiDB database when using SQLite.
  • I've added some compatibility fixes so that WikiDB now runs correctly on MediaWiki 1.23.
  • Fixed an issue where the 'view data' tab was not being displayed as selected on MediaWiki 1.19 and above.
  • The pagination links on the WikiDB special pages are no longer shown if there are no items to display.
  • Updated WikiDB to make use of the new feature in MediaWiki 1.23 whereby license details can be displayed on the Special:Version page (does not affect older MediaWiki versions).

r909[edit]

A minor update to fix three bugs:

  • The RebuildWikiDB script didn't work properly on SQLite back-ends.
  • Using {{{_Row}}} in a <repeat> tag would cause a fatal error.
  • There was an issue that affected the Semantic MediaWiki extension, but which may equally apply to other extensions. This was due to the class-type checks being too strict. The code expected an 'Article' object, whereas now it also recognises any child-class of 'Article', too.

r906[edit]

This update is a major re-working of the WikiDB internals, and you are therefore recommended to run a complete rebuild of your WikiDB database tables to ensure that all table data is up-to-date. If you skip this step then things will probably still work, but you may get incorrect results when querying or viewing table data.

You are recommended to drop all three WikiDB tables from your database (wikidb_tables, wikidb_rowdata and wikidb_fielddata) and run SetupTables.php followed by RebuildWikiDB.php in order to completely update the schema and data.

  • The rebuild is required as the way templates are parsed has changed, which may affect existing stored data.
  • Dropping and re-creating the tables is recommended as there have been some minor changes to the schema which can't be fixed automatically (e.g. indexes have been renamed) and also the character set used in very old versions of WikiDB was incorrect (it now uses the character set of the host wiki in all cases, rather than forcing latin1). These may not affect you, but a full rebuild will ensure your schema is completely up-to-date and correct.

Note that it is always safe to drop and rebuild the WikiDB tables - they do not contain any data that is not stored elsewhere on your wiki. However, on large wikis this can be very slow, so expect some down-time!

Main user-visible fixes in this revision:

  • The way that WikiDB interacts with templates has been completely revised in this new release, which means I have fixed nearly all of the outstanding bugs - most of which were to do with template parsing.
    • Using <repeat> tags in templates now work as expected, and any template variables in the tag body will be correctly expanded.
    • I have updated the <data> tag so that it handles transclusion properly. Now it is possible to put a <data> tag inside a template and use template variables as the field values. This means that any page which transcludes the template will correctly have the relevant data added to the database, assigned to the page which is doing the transcluding. By default the <data> tag is not expanded for the template page itself as in most cases this is not what you want. However I have added a new attribute to the tag, called expand_when, which can be used to over-ride this behaviour (both for templates and for normal pages).
    • Any WikiDB tags now allow template variables in their tag attributes when they are used within a template (so, for example, you can use a template variable to define the criteria for a <repeat> tag).
    • The body of <repeat> tags is now properly passed using the MediaWiki parser, with any field/value pairs that were returned by the query being treated in the same way as template variables, so:
      • You can specify default values for fields that are not present or are blank.
      • You can have the default value of a field being the value of another field.
      • You can now use templates within repeat tags in a way that passes the fields into the template for further expansion.
      • It should hopefully work better with other extensions (e.g. <cite>) - though this is untested.
    • And no doubt fixes to many other parsing issues that had not yet been reported.
  • Fixed an issue where page moves resulted in stale WikiDB data. This has been issue for some time, and affected MediaWiki 1.15 and above.
  • WikiDB is now compatible with SQLite database back-ends.
  • WikiDB has had a number of changes made which may now make it compatible with PostgreSQL database back-ends. However, I have so far been unable to test this. Please let me know if you have successes or problems running WikiDB with PostgreSQL.
    UPDATE: Have tested and there are still major problems. PostgreSQL remains unsupported for now -- HappyDog 23:21, 9 November 2014 (UTC)

Important Backwards-Compatibility Note: Most of the above changes will only have an effect if you are using MediaWiki 1.16 and above (although there are a couple that work on MediaWiki 1.12 and above). WikiDB will still install and run correctly on all versions back to MediaWiki 1.7, but you will be lacking the enhanced template functionality unless you upgrade to 1.16 or later. Earlier versions of MediaWiki will still be supported for now, but may not include all new features.

Note that I will continue to accept patches against older versions of MediaWiki, back to 1.7.

v3[edit]

r856[edit]

  • Updated the <repeat> code so that it is now possible to query for blank/non-blank fields. Example: criteria="Field=" or criteria="Field!="
  • Fixed some compatibility issues with older MW versions (< 1.17).

r848[edit]

This update fixes the following two issues:

  1. ParserOptions::setUseDynamicDates() was removed in MW 1.21, which meant that saving a page with a <data> tag in would cause a fatal error and the data would not end up in the DB.
  2. Numbers were not being sorted correctly and equality comparisons may have failed when the part after the decimal point contains leading or trailing zeros.
    • 1.1, 1.01, 1.001, etc. were all treated as equal to 1.1
    • 1.1 and 1.10 were *not* being considered as equal

This update also added a --force option to the RefreshStaleData script, which can be used to force all rows to be marked as stale prior to running the update. This is useful in situations where neither table data nor article data needs to be reparsed, but we want to rebuild tblFields, for example if the FormatForSorting code has changed for an existing data type, as is the case with the second issue, above.

Most users won't have been affected by either of the issues here, but if you think you may have been you are advised to run the appropriate update script:

r844[edit]

This is a compatibility upgrade. It is also the first release of WikiDB to no longer support PHP4 or MediaWiki versions below 1.7.

  • WikiDB no longer supports MW 1.6.x or below. A fatal error will be raised if you install WikiDB on an unsupported version of MW, with advice about upgrading. The previous version (v3, r816) is still available to install on older wikis, if required.
  • As a consequence of the above, PHP4 is also no longer supported. MW 1.7 and above require PHP5, so this is now also the minimum requirement for WikiDB.
  • Because we no longer support PHP4, I have been able to mark our static calls as such, using PHP5's static keyword. This means that WikiDB now works without errors when E_STRICT error reporting is enabled.
  • I have made a number of other compatibility fixes for new MW versions, so it now runs properly on MW 1.21 (previously special pages were broken, due to architectural changes to MW). I also fixed some (but not all) 'deprecated' notices, which should hopefully reduce the risk of future breakages, when these deprecated functions are removed.

r816[edit]

This version just fixes a single bug. If you already have WikiDB installed and working, there is no need to upgrade.

  • In MW 1.17 and above, the installer no longer adds code to set the include path in LocalSettings.php. This caused a fatal error in WikiDB due to it being unable to find some files. Now it explicitly looks in the right place and does not depend on the include_path setting. This bug only affected fresh installations of MW >= 1.17. Upgrades from previous versions are unlikely to have been affected.

r792[edit]

  • Various compatibility improvements - WikiDB should now work properly on MW 1.20.
  • At last, we have an enhanced syntax for the <repeat> tag, which allows you to specify optional header/footer sections. This means that custom table output is now possible.
  • There is a new config variable, $wgWikiDBDefaultClasses, which can be used to apply styling to WikiDB data tables.
  • Field names are now handled consistently across the system, and are properly sanitised in all situations. Specifically, we now enforce the following restrictions:
    • White-space at start or end is now always removed.
    • Mustn't start with an underscore, as this is reserved for internal use. Underscores are therefore stripped (so FieldName and _FieldName are equivalent).
    • Must be <= 255 characters along, for DB storage reasons. Longer names are usable, but will be truncated, and so fields must be unique within their first 255 characters (unlikely to be a problem!).
    • Fields can't contain [, - or : characters. These indicate the end of the field name in a table definition (and the start of some other part of the definition) so they can't be used in other contexts either. In this situation, the name is considered completely invalid and will be ignored.
    • Mustn't be blank - blank names (after sanitising) are considered invalid and will be ignored.
  • I've made a whole bunch of fixes to the built-in data type handlers:
    • The Min/Max parameters for number fields weren't working correctly.
    • Numeric fields now only accept completely valid numeric representations. Previously it used PHP's is_numeric() function, so entries like "23 Skidoo" would be read as the number 23, whereas now they are invalid (resulting in an empty field).
    • Data in numeric fields are now output in the appropriate format for the user's locale, rather than always in English format.
    • 'integer' fields now use float values internally, rather than integers, which means a greater range of numbers can be represented.
    • 'image' fields weren't recognising all aliases for the image namespace, only the canonical name and the local name. Since 'File:' was introduced as the canonical namespace this meant that 'Image:' was no longer recognised. We now correctly identify all valid aliases.
    • 'image' fields were not matching the namespace in a case-insensitive manner, so 'File:' would be recognised as an image, but 'file:' wouldn't be.
    • 'link' and 'image' fields are now sorted by their canonical name, meaning "Image:My Pic.jpg" and "Image:My_Pic.jpg" are now treated as equivalent.
    • Fixed output of 'image' fields so that if they refer to an image that doesn't exist we get a proper red link. Previously the link would be there, but the text for the link would always be "75px".
  • Fixed a bug whereby the field aliases table on a table definition page might display strangely under certain circumstances.
  • If you view data for a table that is a redirect, you now see a link pointing you to the data tab for the actual table where the data has ended up. Previously you just got a 'no data' message.

The improvements to field name sanitising and the new <repeat> tag syntax means that there may be certain unlikely situations where your DB may hold stale data. (Specifically, if you happened to use the new syntax in an old version of WikiDB, or if you had any field names that will be sanitised differently under the new rules). It is unlikely that this will affect you, but if you think it does - or you want to be certain - you should run the RebuildWikiDB script to ensure your DB holds the correct values.

r668[edit]

  • More compatibility fixes for MW 1.18. Data wasn't saving properly, and the 'viewdata' tab wasn't appearing in the table namespace. Both issues fixed.

r661[edit]

  • Compatibility fixes for MW 1.18 (plus a couple for 1.17). There has been a really annoying habit recently of pointlessly breaking backwards-compatibility with each new MediaWiki release. I'm going to avoid the temptation to write a long rant about this, but it is getting really, really frustrating!
  • Performance improvements: I have massively improved the performance of the <repeat> tag. For a test with a dataset of about 4000 records, and a <repeat> tag that pulled 60 records from it, the execution time has been improved from 1.017s to 0.025 - that's about 40 times faster!
  • As part of the above fix, I've rewritten the query code so that it no longer uses temporary tables. Therefore the CREATE TEMPORARY TABLES permission is no longer required for your database user.
  • Improved i18n by moving a lot of hard-coded strings into system messages, customisable via the MediaWiki namespace. There's still more work to be done, but it's a big improvement.
  • Added pagination to all special pages, and to the data tab.

You should not need to update your database if upgrading from the previous release. Note that if you have customised any UI strings via the MediaWiki namespace then you may need to revisit this, as a few strings have been renamed and a couple have been combined into one (via use of the inline {{#PLURAL:}} function).

r620[edit]

  • Some compatibility fixes for MW 1.17 and MW 1.18
  • Fixed bug whereby namespaces in the 'SourceArticle' column were in the language of the user viewing the page, rather than the wiki's language, meaning that the links were broken if these were not the same.

r495[edit]

  • Fixed bug that caused table definitions to not be saved.

You should run UpdateTableNS.php to fix any missing definitions caused by the above bug.

r489[edit]

  • Compatibility fix.
  • Added README.TXT and LICENSE.TXT to distribution.

r485[edit]

  • A few compatibility fixes.

r480[edit]

This is the first update to WikiDB for about a year and a half, partly due to a very busy life, and partly due to this being quite a major reworking of some of the core internals. This new version includes the following new features:

  • Proper alias resolution (you can now rename tables and watch as the data magically follows the redirect).
  • An extension of the <data> tag which allows you to define multiple data rows within a single tag.
  • Fixes to the sorting/filtering algorithm so that the <repeat> tag now works a lot better (and hopefully a lot faster!).
  • Updated the way parsing works when saving a page. It is now a lot less hacky and less likely to break if the MW parser object is updated. It also now ensures the article is parsed consistently in all cases (previously it was done differently when saving to when viewing the page, which in some rare cases could result in a mismatch between the data you see and what is actually in the DB).
  • A number of display bugs are now fixed, particularly in relation to the way the SourceArticle field was output.
  • Much less chance of SQL injection.
  • We now have maintenance scripts! No more copy/paste SQL! No more null edits to fix parsing issues!
  • A whole load of other little bug fixes, and a general code cleanup.
  • No doubt some other little bits I've forgotten.

v2 / v1[edit]

Changelogs were not kept for earlier versions, I'm afraid. However, there were some upgrade notes provided when v2 was released:

Changes that require re-parsing all pages:
  • Table data is now parsed AFTER the parser has stripped nowiki, pre, etc. so on pages where you had either <data> tags or field defintions (i.e. lines in a table namespace starting with >) you may have extra rows in the database that shouldn't be there. For example, if you had a data definition within <nowiki> tags, it would have been counted as table data, whereas now it is correctly skipped. Any pages where this was the case will continue to exhibit the previous incorrect behaviour until re-parsed.
  • Table data is now parsed AFTER template expansion, so on pages where you used a template to create a data tag or a field definition the data should now correctly be saved to the DB. If you had pages that used these constructs then the data was not being added correctly in previous versions. Any pages where this was the case will continue to exhibit the previous incorrect behaviour until re-parsed.
Changes that require data-massaging:
  • Aliases to fields whose name was longer than 255 characters were broken in earlier versions (as the name was truncated for the field, but not for the alias). This is now fixed, however any existing table definitions which include aliases of fields longer than 255 chars will stay broken until they are re-parsed.
  • Minor display bug in a few edge cases (with old data from previous versions). If you have a table definition with 'Options' set but no 'Type' then the options are passed to the guessed type. Now they are ignored. In the current codebase this should not be possible, but in previous versions it was possible to set options without a type (e.g. > FieldName:(0,5)). This didn't cause incorrect functioning at all, but could cause incorrect display on the table definition page, and the incorrect data could still be sitting in the DB somewhere.