Difference between revisions of "WikiDB/Feature requests"

From TestWiki
Jump to: navigation, search
m (Ignore link syntax in criteria when filtering on link fields: tweak wording)
(New suggestion - distinct values. Copied from Talk:WikiDB#Table Definition Oddities and Getting Unique Data Records.)
Line 150: Line 150:
 
== Rename repeat tag ==
 
== Rename repeat tag ==
 
Consider renaming &lt;repeat> as &lt;query>; it's more intuitive, since the defining feature isn't that it repeats for each entry found, but that it finds the entries in the first place. <small>—The preceding unsigned comment was added by [[User:69.233.95.193|69.233.95.193]] ([[User talk:69.233.95.193|talk]] • [[Special:Contributions/69.233.95.193|contribs]]) 09:18, 1 July 2007 (UTC)</small>
 
Consider renaming &lt;repeat> as &lt;query>; it's more intuitive, since the defining feature isn't that it repeats for each entry found, but that it finds the entries in the first place. <small>—The preceding unsigned comment was added by [[User:69.233.95.193|69.233.95.193]] ([[User talk:69.233.95.193|talk]] • [[Special:Contributions/69.233.95.193|contribs]]) 09:18, 1 July 2007 (UTC)</small>
 +
 +
== Distinct values ==
 +
 +
Is there a way to output only unique entries?
 +
 +
Like if I have:
 +
 +
* Name: Testing 1
 +
* Name: Testing 1
 +
* Name: Testing 2
 +
* Name: Testing 2
 +
 +
Is there some way I can get an output of:
 +
 +
Testing 1, Testing 2
 +
 +
Instead of
 +
 +
Testing 1, Testing 1, Testing 2, Testing 2
 +
 +
?
 +
 +
[[Special:Contributions/74.65.72.174|74.65.72.174]] 23:39, 23 May 2021 (BST)
 +
 +
: This is a good suggestion - would need a bit of work to figure out how that would interact with other functionality, but seems doable. --[[User:HappyDog|HappyDog]] ([[User talk:HappyDog|talk]]) 23:28, 22 June 2021 (BST)

Revision as of 22:28, 22 June 2021

Please add any feature requests which you may have for WikiDB to this page.

Before adding new suggestions, please check WikiDB/Features, WikiDB/ToDo and WikiDB/Bugs to ensure that it is not already covered by one of those.

Matching criteria by Regular expression

Some of the issues with criteria matching could be solved by using a regexp match. Here's a diff.

--- WikiDB/includes/classWikiDB_Query.php	2015-03-21 00:31:22.000000000 +0000
+++ WikiDB.new/includes/classWikiDB_Query.php	2015-05-05 12:50:24.000000000 +0100
@@ -146,7 +146,7 @@
 //    and NULL (field was omitted from the <data> definition).
 //
 	function pSetCriteriaFromString($CriteriaString) {
-		$ValidOperators = array("<>", ">=", "<=", "!=", ">", "<", "=");
+		$ValidOperators = array("<>", ">=", "<=", "!=", ">", "<", "=", "=~", "!~");
 
 	// Setup regular expression for matching individual criteria.
 		$Regex = "";
@@ -176,6 +176,13 @@
 				if ($Operator == "!=")
 					$Operator = "<>";
 
+			// Allow regular expression matches.
+				if ($Operator == "=~") 
+					$Operator = "REGEXP";
+
+				if ($Operator == "!~")
+					$Operator = "NOT REGEXP";
+
 			// Parse the field to get the proper table/field names.
 			// All aliases are resolved.
 				$FieldData = $this->pParseFieldName($Field);

-- Patch0 11:51, 5 May 2015 (UTC)

This would be a great addition :-). Unfortunately, it's not that simple :-(
Data is stored in the database in a format that is not necessarily representative of the value as actually entered. This makes the data easily sortable, and when we are doing exact matches (as is currently the case) we normalise the value we are searching for by the same rules, so the querying works properly.
However, if we need to do any kind of partial or wild-card matching, this system will return incorrect results in some cases.
I suspect the solution is to add another field to the database, which stores the unmodified string. This would open the door to more sophisticated matching, like regular expressions, and the code would just need to know which is the appropriate field to use, depending on the operator type.
There is another issue with regex support, which is that the queries are run directly on the database. I am very wary about the security implications of adding regex support as I'm not sure how easy they will be to sanitise. If we can't guarantee that they will always be safe (and I don't just mean quoting, which should be easy, but also fatal errors due to invalid regexes) then I won't be able to add this feature (at least not in a way that is performant). However, if that is the case then I suspect we could manage a LIKE operator, which would be a lot simpler to sanity-check.
Either way, thanks for the great suggestion - I will look into it.
--HappyDog 22:43, 27 May 2015 (UTC)

Ignore link syntax in criteria when filtering on link fields

I'd like to submit one feature request, though I'm not sure how plausible it is. I came across this issue: I have a value I'd like to display elsewhere which is identified by one field's data being "ES012". However, for display purposes, the value as it is entered is "[[ES012]]", as I need it to display in a wikilink. Thus, I've defined the column as "wikistring", as this appears neccisary to make the link display correctly. Here is where my issue comes in: to repeat the data, I must put

criteria="Description=[[ES012]]"

rather than the more intuitive

criteria="Description=ES012"

So again, I'm not sure if it's possible or plausible to strip out the wikilink symbols when searching the table with the repeat tag, but it's something I'd very much like to see.

-- (sent by User:InShaneee, via e-mail)

This should be solvable by changing the field type to 'link' and entering the value ES012. This means the value stored in the field (and therefore used in criteria) is ES012 but the field will still render as a link for display purposes, equivalent to [[ES012]].
This feature is therefore already implemented, as far as I can see!
--HappyDog (talk) 22:23, 7 May 2018 (BST)

Some feature suggestions

Afer some toying around with the extension I have several "Nice to have" features believe would be beneficial that I haven't seen mentioned

  • repeat tag syntax - besides being able to specify the table header, it would also be nice to have a possibility to add an index column, listing the number of the row in the resulting table
  • some form of display size control - for large table it would be nice to add a syntax for specifying a display range, or using some default size, where it would display rows 1-20 and a link for "next 20" etc. even for the data records in the table definition itself it would be useful - one cannot view all records in large tables, due to the long processing time. I have a table with several thousands of entries and I get a PHP error indicating time limit exceeded after 30 seconds

Osishkin 11:19, 22 July 2010 (BST)

That's definitely a good idea! It should be relatively simple to add paging functionality to the data tab, though I think doing this in-page might be tricky. Adding a 'limit' argument to the <repeat> tag should be simple enough, but allowing you to page through it in-page would involve designing and building the appropriate API handlers, which is not something I'm quite ready to embark upon yet.
However, thinking about the data tab, it occurs to me that there are other things that could be added here, e.g. the ability to sort or filter the data and to hide columns in the output. The back-end code for this is already in place, so it shouldn't be too hard to add the UI for it.
If that functionality were added it would be relatively trivial (I think) to add a 'more...' link to the bottom of the output for repeat tags that are not showing all available rows, which would link through to the data tab, with the appropriate view.
I am currently trying to get a new release out, so I'm not considering new features at this point, but once it's out I'll try and put together some form of road-map for WikiDB, and this will be on it! --HappyDog 21:44, 7 August 2010 (BST)

External queries

This extension is really cool. Do you have any plans concerning external (read-only) queries to the database? --Martin 23:51, 10 July 2006 (BST)

Can you please expand on this - I'm not sure what you mean. --HappyDog 23:55, 10 July 2006 (BST)
Sure. Suppose I have wiki site which is primarily a user-editable database. Each wiki page may provide several database records (let's say information on sequences of proteins of a given family). There is some text which gives some information about the link between these database entries, that's the traditional wiki part. This cannot be put into the database because it's changing often, it may become large, and it doesn't have a meaningful format--the only field name which would be appropriate would be "comments". But the database records are essential for let's say the community of molecular biologists, who use client software to query and analyze the protein from the database. There may be wikis which want to use the database records as illustration, or maybe group the database records differently and comment on them. The idea is to not duplicate the database, but use feeds which would be updated and propagated from the source. My point of view comes from what I did for inserting bibliographic records (PMIDs from Medline and ISBNs from isbndb.org), in wikis: see Biblio extension. Someone also pointed out the Wikicite project. I don't know where these folks are going exactly, but the idea is to build a database of bibliographic references. Have a look. --Martin 00:19, 11 July 2006 (BST)
I haven't really given this any thought yet, and I will take a look at the links shortly, but on the surface what you say sounds quite plausible. How I would implement this is to use a new action argument, e.g. $action=rawdata (or an existing action if a suitable one exists - raw?). So if you entered e.g. http://some.domain.org/w/index.php?title=Table:Companies&action=rawdata, you would get the data (and schema?) from the Companies table in a standard format, probably XML. Does this sound like it would do the trick? (Bear in mind that if the table definition is altered, the XML output will change) --HappyDog 00:58, 11 July 2006 (BST)
I was thinking of something like an SQL SELECT statement. More specifically, the only features that I think would be useful would be:
  • download the whole database for local use as you just mentioned.
  • select one or several records by giving one column name and passing a list of strings. All records for which one of those strings matches is selected.
I don't know if you can do this simply using MediaWiki. Basically the URL I would pass would be like http://some.domain.org/w/index.php?title=Table:Companies&action=rawdata&select_id=12,34,567 where id is a column name and 12, 34 and 567 are acceptable id values. Maybe it is easier to just write an independent PHP script which queries the database according to the CGI arguments which are found. --Martin 01:40, 11 July 2006 (BST)
I think that level of detail is outside the scope of v1, however you're right, it would be fairly trivial to write a PHP script to parse and manipulate the raw data dump, particularly if it is in a standard format such as XML (or, it just occurs to me, SQL - I'm sure that would be possible, and then you could simply import it straight into a traditional database... hmmm... interesting thought!) --HappyDog 02:45, 11 July 2006 (BST)

Count of records

Is it possible to get number of record definitions without listing all? --Neptus 21:34, 14 March 2007 (GMT)

Can you please elaborate. Do you mean via a tag (so it can be displayed in a page) or on the table definition. I'm sure it would be fairly easy to add in either case - please tell me what you would like to see and where, and I'll see what I can do. --HappyDog 23:15, 14 March 2007 (GMT)
I have 4000 definitions in my WikiDB dictonary now and it grows all the time. I can count it only using tag <repeat table...># ... </repeat> now. This listing is long.
It will be nice to have only one aktual number of positions declared in WikiDB on page (or a number of selected records), visual effect like a number of pages {{NUMBEROFARTICLES}} in wiki - simple base counter: how many records is inside. Nice gadget for information and for a boast. Then it will be a tag, I think. --Neptus 01:40, 17 March 2007 (GMT)
The problem is that it would be a single number for the whole wiki. If I added a {{NUMBEROFROWS}} magic word it would give the no. of rows in all tables in the wiki, combined. It might be possible to somehow supply an argument to a magic word (e.g. {{NUMBEROFROWS|Table:Companies}}, in which case it might be possible, but I'm not sure about that.
I can definitely add the info at the top of the table definition as a way of seeing it without having to count by hand, but that won't allow you to embed it in other articles.
I guess a parser function would be possible, e.g. {{#CountRows: Table:Companies}}. I haven't investigated them yet, but if that's the kind of thing that will do what you're after then I can look into it. --HappyDog 00:12, 27 March 2007 (BST)
Thanks. It isn't necessary for life, but if it will be not especially complicate work, it will be nice and usefull tool. --195.16.88.26 00:55, 30 March 2007 (BST)
OK - I've done a quick fix that gives the info on the 'data' tab when you're in the table namespace. I'll have a think about other ways of getting this information (suggestions welcome) but you can at least see it easily now. WikiDB.php is the only file that changed for this fix (though there may be other fixes elsewhere, depending how old your copy is). --HappyDog 17:26, 3 April 2007 (BST)
Thank you. Now I know, what big it is and what I can do with it. It's no good way for visitors, but for admin is good. --Neptus 16:04, 21 April 2007 (BST)

Create Forms to fill in Table data

I'd like to suggest to everyone watching/discussing/contributing to WikiDB that we investigate a way (or create our own way) to create forms inside wiki pages that will actually fill in the fields of a <data> tag statement so that data in the tables can be easily entered without manual editing of the wiki page that contains the record entry via the <data> tag.

I've come across several "forms" style extensions for MediaWik such as Simple Forms, FormTools, Semantic Forms, however, I haven't found one that will work, either because it still eludes me or I don't understand how to make use of one or more of the extensions I listed here.

Anybody have any ideas about how to create forms to fill fields in a <data> tag statement inside a wiki article?

Javascript, perhaps ???

An expansion of WikiDB functionality ???

-- Mdrayman 04:48, 3 July 2007 (BST)

I wasn't considering this for v1, but I was planning it for a later version. It should be relatively simple to use a form to create data, though I am not sure how easy it would be to edit data in this fashion. I'm sure it will be possible, but when editing it will be necessary to gracefully handle all sorts of cases where the existing data doesn't validate against the existing table def, as well as being able to write the data back to the page without screwing things up... Any suggestions, or even code, is welcome. --HappyDog 13:51, 4 July 2007 (BST)

Hyperlink reference to Table:Name when editing page

When editing a page in the main namespace that includes some <data> tags, thought it would be a nice idea to also include a hyperlink at the bottom to the corresponding Table: page (if it exists!). A bit like how the Template: links are provided. - Cocjh1 23:43, 27 November 2007 (GMT)

This was implemented in v4 r1376. --HappyDog (talk) 12:51, 10 October 2017 (BST)

Some display suggestions

HappyDog, this is a great extension. I have a couple of suggestions that may make it even more useful or adoptable:

  • Accessing a Table:... page would show the Data by default, vice the table schema. This would allow people to avoid using the repeat construct and just interlink to the table for a particular data item directly. Or the "article" view for a Table page could be what is currently the data view.
  • The Table data display could use the class wikitable sortable and then you would be less reliant again on the repeat tag for filtering, sorting. Furthermore, some AJAX wiz could implement the filtering on the client side for the Table data display.
  • Enable table data column headers to have wiki markup, so you could have a definition (or at least alt- text) for each column header so users could read more about a particular field. This could be specified in the field specification on the schema page.

Please let me know if you have any questions about these goofy requests! :) 13:08, 30 September 2008 (BST)

Relationships between tables

I have searched about it but I didn't found, so if it's possible already sorry the mistake. The thing is to make more than one table that have relations of one to one, more than one to more than one... to make some fancy things by using WikiDB. Also I want to congratulate you for the project, I think is the best approach to join wiki and db maintaining the simplicity of wiki. --howl

Ups, sorry I didn't read WikiDB/Features#Data_linking. If you implement a voting for features to be developed, put mine to this xD. --howl
*grin* No probs - glad you're enjoying the extension. btw, have you seen WikiDB/Compatibility? The minimum requirements for WikiDB may change soon (due to needing some more advanced features of MediaWiki) but I try and bear the usage requirements of existing users in mind when this happens, and this page is how I manage that. No problems if you don't want to divulge this information, however. --HappyDog 13:09, 5 October 2009 (BST)

Include parameters that don't come from DB

Suggestion: ability to not include a item in database but show it in infobox. e.g. !name = This is shown in infobox, but not included in database. —The preceding unsigned comment was added by 88.193.93.92 (talkcontribs) 02:44, 10 October 2007 (UTC)

Good idea. I will look into that. --HappyDog 13:29, 4 July 2007 (BST)

Rename repeat tag

Consider renaming <repeat> as <query>; it's more intuitive, since the defining feature isn't that it repeats for each entry found, but that it finds the entries in the first place. —The preceding unsigned comment was added by 69.233.95.193 (talkcontribs) 09:18, 1 July 2007 (UTC)

Distinct values

Is there a way to output only unique entries?

Like if I have:

  • Name: Testing 1
  • Name: Testing 1
  • Name: Testing 2
  • Name: Testing 2

Is there some way I can get an output of:

Testing 1, Testing 2

Instead of

Testing 1, Testing 1, Testing 2, Testing 2

?

74.65.72.174 23:39, 23 May 2021 (BST)

This is a good suggestion - would need a bit of work to figure out how that would interact with other functionality, but seems doable. --HappyDog (talk) 23:28, 22 June 2021 (BST)