Editing WikiDB/Repeat tag syntax/Archive

{{Archived|WikiDB/Repeat tag syntax}}

The more I think about the syntax of the &lt;repeat&gt; tag, the more I realise that there is probably not a 'perfect' solution, but instead a 'best compromise' between simplicity, brevity, flexibility and readability, so to that end I have created this page to discuss the requirements and the pros/cons of different syntax types.

This is very much a public consultation, and I welcome all input - particularly from people who are actively using the extension.  For general discussion about the issues raised on this page, please use the [[WikiDB/Repeat tag syntax|talk page]], though if an addition/amendment seems more appropriate on this page then feel free to make it here instead.

There is no fixed time-scale for this consultation - I would rather get it right than rush through something half-formed.

This initial draft is just my first bash at getting my thoughts down - it will be added to and refined as we go along.

== Current syntax ==

The current syntax is defined on the [[WikiDB/Syntax#Repeat_tag|syntax]] page.

== Planned enhancements ==

The following known bugs/planned enhancements are all, I think, uncontroversial, and don't seem to pose any syntax issues to me.  Please let me know if you disagree with any of these, or have any additional suggestions.

* Allow default values for missing fields (e.g. <code><nowiki>{{{Email|No e-mail supplied.}}}</nowiki></code>)
* Allow other fields to be used as defaults (e.g. <code><nowiki>{{{CompanyName | {{{ShortName}}} }}}</nowiki></code>)
* Include special meta-fields (e.g. <code><nowiki>{{{_SourceArticle}}}</nowiki></code> or <code><nowiki>{{{_RowNumber}}}</nowiki></code>).

== Further Requirements ==

The following are things that the repeat tag does not currently handle.  Please add to this list.

; Must be able to output a custom table.
: This is the biggest short-coming at the moment.  Due to the way/order MediaWiki parses custom tags, you can't use a repeat tag to just add rows within table mark-up, as the rows will not be parsed before the surrounding table, thus resulting in nonsense.  The solution to this is probably to include header/footer sections somehow (though any other suggestions for how to achieve this are welcome).

; Should allow &gt; in where clauses
: &gt; cannot be used within a criteria, as the MediaWiki parser reads it as closing the repeat tag.  A decision needs to be made as to whether this is something we can live with (by using a different symbol, e.g. gt/gte) or whether an alternative method (inside the body of the tag) should be used instead.

; Criteria syntax
: What other syntax elements/comparators do we need for querying the data?  For the time being, let's deal with essentials only, otherwise the list will get very long!

== Suggested syntax ==

=== Header/footer section ===

Ideally we would be able to use the repeat tag within a table, as follows:
<pre> 
== Software Companies ==
{| 
! Name || Address
<repeat table="Companies" criteria="ProductType=software" sort="CompanyName">
|-
| {{{CompanyName}}} || {{{CompanyAddress}}}
</repeat>
|}
</pre>

However, this results in the contents of the repeat tag being parsed first, and because it is not a valid table it is output as it is, and placed verbatim in the surrounding table wrapper.  If we could work round this so that it works as expected, then that is an obvious 'simple' option.  Assuming we can't, here are the two alternatives I have so far.

==== #1: HTML-style tags ====

Option 1 is to use HTML style tags:

<pre> 
== Software Companies ==
<repeat table="Companies" criteria="ProductType=software" sort="CompanyName">
<header>
{| 
! Name || Address
</header>
|-
| {{{CompanyName}}} || {{{CompanyAddress}}}
<footer>
|}
</footer>
</repeat>
</pre>

Pros:
* Standard syntax, easy to remember.

Cons:
* Tags within tags are messy (and they have no meaning outside the repeat tag).
* Quite a bit of extra mark-up.
* Not brilliantly readable.
* Criteria is still defined in an attribute, with all the restrictions that brings.  Could be fixed by optionally allowing a &lt;criteria&gt; tag to be used instead of the attribute, but that adds even more unwieldy mark-up.

==== #1b: HTML-style tags (alternative) ====

An alternative to wrapping header/footer blocks would be to wrap the repeated section.  This would reduce the number of tags used.  Also, if the inner tag was the &lt;repeat&gt; tag, and the outer tag was some kind of format wrapper, then both tags would work in isolation, which sits a little better.  In other words, &lt;repeat&gt; would work as before, but in the context of the outer wrapper it allows you to combine the header/footer parts more easily.  Under this method I think we negate the three negative points in the above example.

Note: In the example I have called the outer tag &lt;wikidb&gt;, but this is a rubbish name.  If we go down this route, I will think of something better before launch - suggestions welcome!

<pre> 
== Software Companies ==
<wikidb>
{| 
! Name || Address
<repeat table="Companies" criteria="ProductType=software" sort="CompanyName">
|-
| {{{CompanyName}}} || {{{CompanyAddress}}}
</repeat>
|}
</wikidb>
</pre>

Pros:
* Standard syntax, easy to remember.
* The &lt;wikidb&gt; tag is optional - you can use the inner repeat tag on its own, and it will function as it always has.

Cons:
* This method wouldn't allow you to include any aggregate fields in the outer parts (which might be useful, e.g. we could provide _RowCount field) because the query parameters are specified within the inner tag.  Perhaps this could be fixed by moving the parameters to the outer tag, but this might be inconsistent.  More thought required on this.
* Criteria is still defined in an attribute, with all the restrictions that brings.


==== #2: Custom markup ====

Some kind of custom markup could be used to divide the sections:

<pre> 
== Software Companies ==
<repeat table="Companies" criteria="ProductType=software" sort="CompanyName">
{| 
! Name || Address
>>>>>>
|-
| {{{CompanyName}}} || {{{CompanyAddress}}}
<<<<<<
|}
</repeat>
</pre>

In this example, any row containing just 4 or more > symbols means 'header end' and any containing 4 or more < symbols means footer start.  Either may be omitted to indicate no header/footer.  An alternative would be to use the MediaWiki horizontal rule syntax (4 or more - symbols), which might be more intuitive but could conflict with desired markup.  We could get round this by adding a 'divider' attribute to the opening tag, so users can use something else if the standard divider is used in the markup itself.

Pros:
* Easier to read/clearer
* Less markup

Cons:
* Non-standard (harder to remember)
* Might conflict with valid markup (depending on choice of dividers)
* Less expandable if we need other sections in the future (see next section!)

=== Criteria ===
If we can't place the criteria in an attribute due to limitations in the MW parser, we need to include it in the body.  If we do this, it might (or might not) make sense to also include the other attributes in the body (or make it optional where they are placed).

If we use syntax #1 (above), then wrapping them in a <code>&lt;criteria&gt;</code> tag might be a solution, but if not then we need some way to indicate that the code is a criteria string, rather than part of the header/body of the tag.  Some suggestions:

* Special HTML comments: <code><nowiki><!--#Criteria: ... --></nowiki></code>
** (or are these stripped by the MW parser before my tag handler gets them?)
* Character at start of line (like [[WikiDB/Syntax#Table_definitions|table syntax]]): 
<pre>
> Criteria: Founded > 1950
or
> Founded > 1950
or 
!!! Criteria: Founded > 1950
or
!!! Founded > 1950
or
(something else)
</pre>
* Some kind of ending tag (if not present, no criteria are defined):
<pre>
<repeat table="Companies" sort="CompanyName">
Founded > 1950
======
{| 
! Name || Address
>>>>>>
|-
| {{{CompanyName}}} || {{{CompanyAddress}}}
<<<<<<
|}
</repeat>
</pre>

Note that in all cases we need to ensure that you can include criteria and still output in the standard (default) format by omitting the body.


== Bugs ==
The following bugs are known to exist.  Some are also mentioned and discussed in more detail above, but for now I'm keeping the full list here so it is in one place.

* Tag Contents
** Aliases are currently expanding correctly, but not in a foolproof manner (e.g. aliases that point to aliases are not handled - this is a general issue, I think)
** Undefined fields - is their current handling OK?  Currently they will be expanded if defined for the row, and left as they are if not.  This might be the best behaviour, but perhaps not.
** Cannot currently supply default values for empty fields.  In any case (given the above point) default values for undefined fields must, by definition, be ignored (I think).
** Cannot handle nested vars.  Not a problem at the moment, but if default values are allowed, the default should be allowed to be an alternative variable (e.g. <code><nowiki>{{{DisplayName|{{{ShortName}}}|No name defined}}}</nowiki></code> should be valid syntax).
** Should strip blank lines at start and end.  Maybe restrict to stripping 1 line only (so &lt;repeat&gt; need not be on the same line as the start of it's content).  Should _not_ use trim(), as we need to allow for space at start of line (wiki &lt;pre&gt; syntax).
* Filtering
** Can't use &gt; in attribute values, e.g. criteria="Field>Value", as it cripples parser.  Need to decide how to specify criteria.
** In general - how detailed can we get?
* Other
** Ability to have header/footer? Ideally you put your header before the repeat tag, and the footer afterwards, but tables are screwed up by this (though I might be able to fix that somehow). There may also be other situations that cause problems.