Difference between revisions of "WikiDB/Roadmap"

From TestWiki
Jump to: navigation, search
(BUG: Using Repeat tag on a Table where the filter results in Zero results yields ERROR)
(BUG: Using Repeat tag on a Table where the filter results in Zero results yields ERROR)
Line 117: Line 117:
  
 
<pre>
 
<pre>
// TODO - this is VERY crude at the moment, and may well fail in certain edge cases  
+
581 // TODO - this is VERY crude at the moment, and may well fail in certain edge cases  
// (or even, certain 'non-edge' cases!)
+
582 // (or even, certain 'non-edge' cases!)
     function pWikiDB_ExpandVariables($Input, $Data) {
+
583     function pWikiDB_ExpandVariables($Input, $Data) {
         if (count($Data) > 0) {
+
584         if (count($Data) > 0) {
         // Substitute variables.  Note that we need to replace $ with \$ within $Value, to stop it being
+
585         // Substitute variables.  Note that we need to replace $ with \$ within $Value, to stop it being
         // interpreted as a PHP variable reference.
+
586         // interpreted as a PHP variable reference.
             foreach ($Data as $Key => $Value)
+
587             foreach ($Data as $Key => $Value)
                 $Input = preg_replace("/{{{" . $Key . "}}}/", str_replace('$', '\$', $Value), $Input);
+
588                 $Input = preg_replace("/{{{" . $Key . "}}}/", str_replace('$', '\$', $Value), $Input);
         }
+
589         }
         return $Input;
+
590         return $Input;
     }
+
591     }
 
</pre>
 
</pre>
  
 
I see this is commented as crude, but I'm wondering since the error above mentioned an invalid argument for the foreach() construct, why would the $Data variable contain ANYTHING if the table is completely EMPTY?  Apparently, an EMPTY table is somehow passing the '''if (count($Data) > 0)''' test.  Any ideas as to what's going on?
 
I see this is commented as crude, but I'm wondering since the error above mentioned an invalid argument for the foreach() construct, why would the $Data variable contain ANYTHING if the table is completely EMPTY?  Apparently, an EMPTY table is somehow passing the '''if (count($Data) > 0)''' test.  Any ideas as to what's going on?
 
-- [[User:Mdrayman|Mdrayman]] 06:47, 6 July 2007 (BST)
 
-- [[User:Mdrayman|Mdrayman]] 06:47, 6 July 2007 (BST)
 +
 +
After digging into the WikiDB.php PHP code and playing around with some print() statements inside of the pWikiDB_ExpandVariables() function as well as its calling function, wfWikiDB_RepeatTag(), I decided to take a chance and make one small change.  I changed line 587 above to '''if (count($Data) > 1)''' because I noticed that the actual result of '''count($Data)''' was '''1''' when ''$Data'' was an empty row from an empty table.  This change made ALL of the Warnings about the foreach() invalid argument go away.
 +
 +
I still saw the bogus lines printed from using the <nowiki><repeat></nowiki> tag, so I moved the '''if (count(...''' check out of the pWikiDB_ExpandVariables() function and into the wfWikiDB_RepeatTag() function right where it would be about to call the pWikiDB_ExpandVariables() function so that it is not even called if the ''$Row'' variable is found to have a ''count($Row)'' equal to '''1'''.  Here is the new snippet of code.  Notice that the '''if (count(...''' is on line '''575''' now instead of line '''586''', where it is now commented-out.
 +
 +
<pre>
 +
...
 +
567        $Data = $Table->PerformQuery($Where, $Sort);
 +
568        if (trim($Input) == "")
 +
569            $Output = $Table->FormatTableData($Data);
 +
570        else {
 +
571            $Output = "";
 +
572            $Data = $Table->NormaliseData($Data);
 +
573 //            print_r($Data);
 +
574            foreach ($Data as $Row)
 +
575            if (count($Row) > 1) {
 +
576                $Output .= pWikiDB_ExpandVariables($Input, $Row);
 +
577            }
 +
578 //            print($Output);
 +
579        }
 +
580        return WikiDB_Parse($Output, $Parser);
 +
581    }
 +
582
 +
583 // TODO - this is VERY crude at the moment, and may well fail in certain edge cases
 +
584 // (or even, certain 'non-edge' cases!)
 +
585    function pWikiDB_ExpandVariables($Input, $Data) {
 +
586 //        if (count($Data) > 1) {
 +
587        // Substitute variables.  Note that we need to replace $ with \$ within $Value, to stop it being
 +
588        // interpreted as a PHP variable reference.
 +
589            foreach ($Data as $Key => $Value)
 +
590                $Input = preg_replace("/{{{" . $Key . "}}}/", str_replace('$', '\$', $Value), $Input);
 +
591 //        }
 +
592        return $Input;
 +
593    }
 +
</pre>
 +
 +
So, what do you think?  This fixes my problem, and my other <nowiki><repeat></nowiki> tag uses all seem to be working fine, but since I don't know the code base anywhere nearly as well as you do, could there be some other problems created by this hack?
 +
 +
-- [[User:Mdrayman|Mdrayman]] 07:44, 6 July 2007 (BST)
  
 
== Administration ==
 
== Administration ==

Revision as of 06:44, 6 July 2007

This is an (incomplete) list of features and bugs that need fixing for v1.

Bugs

  • On table display, the definition is currently above the page title and sitenotice. This only seemed to happen since I set 'MediaWiki:Sitenotice', so maybe has something to do with that - or maybe I just never noticed it for some reason? --HappyDog 00:00, 6 January 2007 (GMT)
    • It was to do with the site notice. ParserBeforeStrip is being called once for each interface message that uses wiki text (I assume - it's being called for that one, anyway!). I now use a flag so that the code in my hook only executes the first time it is called for a page. This will only work if (a) the hook is called for every page view (i.e. it is not avoided by caching) and (b) it is always called first. I don't know the answer to either of these, so if the bug resurfaces please let me know! --HappyDog 03:30, 23 January 2007 (GMT)
  • If a table doesn't have a definition, the 'data' tab is not displayed in 1.6.8 (it's ok in 1.5.6). --HappyDog 03:33, 23 January 2007 (GMT)
    • FIXED - In 1.5.6 the first tab is always 'nstab-main' - at some point this changed to include the namespace (e.g. nstab-table). Now I just check the first 6 characters equal "nstab-".
  • If I use a <repeat>-tag in a template and then use a template-parameter in the criteria-clause, then the parameter-value in the criteria-clause is not replaced by its value first. (Stephanvaningen, 23 May 2007)
  • Like this: <repeat table="handelaars" criteria="id={{{templateparametername}}}">{{{tablefield}}}</repeat>
  • After that, I tried this:
    • <repeat table="handelaars" criteria="id=50">{{template}}</repeat>, with in the template then direct use of the field names from the table. This seems not to work
    • This I tried because I wanted to prevent writing *all* the table fields each time when calling the template: are there any possibilities there?
      • Hey, I actually got this to work. I was able to use the <repeat> tag inside a Template using template parameters to feed the fields of the <repeat> tag using a MediaWiki Extension called ExpandAfter. Follow the link to learn how to install it. Once installed, use it like this in your template: {{#expandafter|repeat|table="handelaars" criteria="id={{{templateparametername}}}">{{{tablefield}}} }} What this extension does is parse out the template parameters BEFORE calling the WikiDB code for the <repeat> tag. What's really cool about this ExpandAfter extension is that it works for ANY <xxx> tag in MediaWiki, not just the <repeat> or <data> tags for WikiDB, but for others too, such as any tags introduced by other extensions. -- Mdrayman 04:35, 3 July 2007 (BST)

BUG: data tag still creates records inside of nowiki tags

If I put the following code in a wiki page, the table referenced in the <data> tag still makes a record show up in that table. Is there any way to illustrate to a user in a wiki page how to use the <data> tag without it adding records to the real table?

<data table="ServiceCustomerJobs">
job_num=
account_num=
description=
date_called=
date_serviced=
completion_time=
job_page=
satisfaction=
complaint_page=
</data>

I bet that if you go look at the Special:UndefinedTables special page, there will be a new table called "ServiceCustomerJobs" because the <nowiki> tags didn't prevent WikiDB from using the <data> tag as a new record declaration.

-- Mdrayman 04:35, 3 July 2007 (BST)

I just checked the Special:UndefinedTables on this wiki and sure enough, even though I used the <nowiki> tags, the table still showed up with the record defined above.

Any clues as to what's going on here?

-- Mdrayman 04:37, 3 July 2007 (BST)

Hmmm... well I'm not sure if there's anything I can do about that, but I will check. I may be doing something funky like manually parsing the page - in which case this is definitely something that needs fixing. If not, then it is a problem in MediaWiki itself. I suspect the former, however, and will look into it. In the meantime, an easy workaround is to use &gt; instead of > and &lt; instead of < when creating your example data tags. --HappyDog 13:28, 4 July 2007 (BST)
Thanks. I'll use those special codes in the meantime. Good luck figuring out what's really going on. -- Mdrayman 20:02, 5 July 2007 (BST)

BUG: data tag does not create new table entry when using ExpandAfter MW extension

I mentioned above about using the ExpandAfter MediaWiki extension to populate fields of the <repeat> tag inside of a template. Now, this extension works with the <repeat> tag, however, it does not work quite right with the <data> tag. Below are two attempts to add a table entry to a new table called ExpandAfterTable.

Not using ExpandAfter:


<data table="ExpandAfterTable" template="default">
value1=nothing
value2=something
value3=everything
</data>

Using ExpandAfter:


{{#expandafter:data|table="ExpandAfterTable" template="default"|
value1=nothing
value2=something
value3=everything
}}

Now, what happens when I use the <data> tag by itself in the above manner, the new table entry shows up in the Table:ExpandAfterTable page and in the page it was added shows the normal table printout of the single entry.

When I use the ExpandAfter extension to do the same thing, which, by the way, does work for the <repeat> tag, the table printout appears on the page containing the <data> tag instance, however, the Table:ExpandAfterTable page does not show a record for this entry.

My question is, given that the ExpandAfter extension works correctly to recreate the wiki text shown in the first example above AND WikiDB does print out the table entry in the page it was entered, why then does WikiDB not add this table entry to the table itself?

You may need to install the ExpandAfter extension to test this out. When you do, you'll see what I mean, and, I think you'll like using this extension since it will allow you to use the WikiDB tags inside of templates and pass template parameters to the WikiDB tags inside of those templates.

Thanks! -- Mdrayman 20:02, 5 July 2007 (BST)

BUG: Using Repeat tag on a Table where the filter results in Zero results yields ERROR

I have been making heavy use of WikiDB in the past couple of weeks and I found that with any WikiDB table that not even need be empty, when using the <repeat> tag to filter the table's contents for a particular value of one of the fields, WikiDB reports some bad errors when there are ZERO results returned from the use of the filtering via the <repeat> tag.

At the right is a screenshot of my Firefox browser window in the wiki page using the <repeat> tag to filter contents of the table when the filter returns zero results. If I then change a single table entry on another wiki page where it is defined to then make the filter results non-zero, these errors go away completely.

Any idea what's going on here? Let me know if you have any questions about what I'm trying to do.

-- Mdrayman 21:27, 5 July 2007 (BST)

Ooops, I guess I didn't know I couldn't upload images. Well, if you could enable me to upload images, that would be great, but in the meantime, I'll just describe what the error says in the wiki page. As soon as I refresh the wiki page in the browser, I see nine (9) of these error messages repeated down from the top of the page (I stripped out my wiki path from between the two slashes "/"):

Warning: Invalid argument supplied for foreach() in /.../WikiDB.php on line 587

Also, an OK box pops up and says "Error: Failed to derive URL prefix for Timeline API code files", I click OK and the rest of the page renders in the browser window. Further down the page where I actually use the <repeat> tag to generate filtered table results, nine (9) bogus lines are printed out instead of what I'd expect a zero results filter response would be: no lines printed, or a message that says "No results". These bogus lines look like this:

  1. {{{job_page}}} : {{{description}}}
  2. {{{job_page}}} : {{{description}}}
  3. {{{job_page}}} : {{{description}}}
  4. {{{job_page}}} : {{{description}}}
  5. {{{job_page}}} : {{{description}}}
  6. {{{job_page}}} : {{{description}}}
  7. {{{job_page}}} : {{{description}}}
  8. {{{job_page}}} : {{{description}}}
  9. {{{job_page}}} : {{{description}}}

Here, the job_page and description identifiers are fields in the table I defined using the WikiDB interface.

I'll take a look at the WikiDB.php file on line 587 to see if I can figure out what's happening. I'll post what I find, if anything.

-- Mdrayman 21:40, 5 July 2007 (BST)

After experimenting a bit with different tables trying to figure out this bug, I did notice that the number of bogus lines printed out from a <repeat> tag is exactly equal to the number of fields in the table specified in the <repeat> tag.

In the WikiDB.php file centered around line 587 was this php function:

581 // TODO - this is VERY crude at the moment, and may well fail in certain edge cases 
582 // (or even, certain 'non-edge' cases!)
583     function pWikiDB_ExpandVariables($Input, $Data) {
584         if (count($Data) > 0) {
585         // Substitute variables.  Note that we need to replace $ with \$ within $Value, to stop it being
586         // interpreted as a PHP variable reference.
587             foreach ($Data as $Key => $Value)
588                 $Input = preg_replace("/{{{" . $Key . "}}}/", str_replace('$', '\$', $Value), $Input);
589         }
590         return $Input;
591     }

I see this is commented as crude, but I'm wondering since the error above mentioned an invalid argument for the foreach() construct, why would the $Data variable contain ANYTHING if the table is completely EMPTY? Apparently, an EMPTY table is somehow passing the if (count($Data) > 0) test. Any ideas as to what's going on? -- Mdrayman 06:47, 6 July 2007 (BST)

After digging into the WikiDB.php PHP code and playing around with some print() statements inside of the pWikiDB_ExpandVariables() function as well as its calling function, wfWikiDB_RepeatTag(), I decided to take a chance and make one small change. I changed line 587 above to if (count($Data) > 1) because I noticed that the actual result of count($Data) was 1 when $Data was an empty row from an empty table. This change made ALL of the Warnings about the foreach() invalid argument go away.

I still saw the bogus lines printed from using the <repeat> tag, so I moved the if (count(... check out of the pWikiDB_ExpandVariables() function and into the wfWikiDB_RepeatTag() function right where it would be about to call the pWikiDB_ExpandVariables() function so that it is not even called if the $Row variable is found to have a count($Row) equal to 1. Here is the new snippet of code. Notice that the if (count(... is on line 575 now instead of line 586, where it is now commented-out.

...
567         $Data = $Table->PerformQuery($Where, $Sort);
568         if (trim($Input) == "")
569             $Output = $Table->FormatTableData($Data);
570         else {
571            $Output = "";
572            $Data = $Table->NormaliseData($Data);
573 //            print_r($Data);
574             foreach ($Data as $Row)
575             if (count($Row) > 1) {
576                 $Output .= pWikiDB_ExpandVariables($Input, $Row);
577             }
578 //            print($Output);
579         }
580         return WikiDB_Parse($Output, $Parser);
581     }
582 
583 // TODO - this is VERY crude at the moment, and may well fail in certain edge cases 
584 // (or even, certain 'non-edge' cases!)
585     function pWikiDB_ExpandVariables($Input, $Data) {
586 //        if (count($Data) > 1) {
587         // Substitute variables.  Note that we need to replace $ with \$ within $Value, to stop it being
588         // interpreted as a PHP variable reference.
589             foreach ($Data as $Key => $Value)
590                 $Input = preg_replace("/{{{" . $Key . "}}}/", str_replace('$', '\$', $Value), $Input);
591 //        }
592         return $Input;
593     }

So, what do you think? This fixes my problem, and my other <repeat> tag uses all seem to be working fine, but since I don't know the code base anywhere nearly as well as you do, could there be some other problems created by this hack?

-- Mdrayman 07:44, 6 July 2007 (BST)

Administration

  • Update tables when page is moved.
  • Update tables when page is deleted.
  • Update tables when page is undeleted.
  • (check that rollbacks work)
  • 'What links here' should show templates that are included via data tags.

Syntax

  • Better data syntax: Need to allow multiple rows in a single definition.
    • Also, data definitions in a DB namespace shouldn't require the 'table' attribute (default to current).
  • Repeat tag - some things to iron out (see below)
  • Field names need to be limited to 255 chars (DB limit)
  • Field names need to have leading underscores removed (Reserved for special uses)
  • Provide a mechanism for 'pass-through' parameters to the <data> tag (i.e. parameters that are supplied to the template, but which don't get added to the table). Suggestion is to prefix the name with an exclamation mark. Alternatively, an underscore could be used, as this is already reserved in table definitions, and therefore is not allowed in data tags (though this is not enforced) - may kill two birds with one stone...

Repeat tag

Still quite a way to go on this, but here's where I'm up to:

Done

  • Basic sort/filter is implemented. Only limited filter options available, due to a parsing issue with MW/HTML.
  • Contents of repeat tag are repeated for each row with variables expanded (see below for limitations).

To Do

  • Tag Contents
    • Aliases are currently expanding correctly, but not in a foolproof manner (e.g. aliases that point to aliases are not handled - this is a general issue, I think)
    • Undefined fields - is their current handling OK? Currently they will be expanded if defined for the row, and left as they are if not. This might be the best behaviour, but perhaps not.
    • Cannot currently supply default values for empty fields. In any case (given the above point) default values for undefined fields must, by definition, be ignored (I think).
    • Cannot handle nested vars. Not a problem at the moment, but if default values are allowed, the default should be allowed to be an alternative variable (e.g. {{{ShortName}}} should be valid syntax).
    • Should strip blank lines at start and end. Maybe restrict to stripping 1 line only (so <repeat> need not be on the same line as the start of it's content). Should _not_ use trim(), as we need to allow for space at start of line (wiki <pre> syntax).
  • Filtering
    • Can't use > in attribute values, e.g. criteria="Field>Value", as it cripples parser. Need to decide how to specify criteria.
    • In general - how detailed can we get?
  • Other
    • Ability to have header/footer? Ideally you put your header before the repeat tag, and the footer afterwards, but tables are screwed up by this (though I might be able to fix that somehow). There may also be other situations that cause problems.

Code Tidying

  • Ensure all user-visible text uses the message cache.
  • Add profiling statements.
  • Move datatype handlers out of WikiDB class (into separate module).

Misc

  • References to tables in table="..." attributes should follow redirects. This allows tables to be moved without screwing everything up!
  • Special:UnusedTables to list tables that don't have any data (opposite of Special:UndefinedTables).