Talk:WikiDB

This extension is really cool. Do you have any plans concerning external (read-only) queries to the database? --Martin 23:51, 10 July 2006 (BST)

Can you please expand on this - I'm not sure what you mean. --HappyDog 23:55, 10 July 2006 (BST)

Sure. Suppose I have wiki site which is primarily a user-editable database. Each wiki page may provide several database records (let's say information on sequences of proteins of a given family). There is some text which gives some information about the link between these database entries, that's the traditional wiki part. This cannot be put into the database because it's changing often, it may become large, and it doesn't have a meaningful format--the only field name which would be appropriate would be "comments". But the database records are essential for let's say the community of molecular biologists, who use client software to query and analyze the protein from the database. There may be wikis which want to use the database records as illustration, or maybe group the database records differently and comment on them. The idea is to not duplicate the database, but use feeds which would be updated and propagated from the source. My point of view comes from what I did for inserting bibliographic records (PMIDs from Medline and ISBNs from isbndb.org), in wikis: see Biblio extension. Someone also pointed out the Wikicite project. I don't know where these folks are going exactly, but the idea is to build a database of bibliographic references. Have a look. --Martin 00:19, 11 July 2006 (BST)

I haven't really given this any thought yet, and I will take a look at the links shortly, but on the surface what you say sounds quite plausible. How I would implement this is to use a new action argument, e.g. $action=rawdata (or an existing action if a suitable one exists - raw?). So if you entered e.g. http://some.domain.org/w/index.php?title=Table:Companies&action=rawdata, you would get the data (and schema?) from the Companies table in a standard format, probably XML. Does this sound like it would do the trick? (Bear in mind that if the table definition is altered, the XML output will change) --HappyDog 00:58, 11 July 2006 (BST)

I was thinking of something like an SQL SELECT statement. More specifically, the only features that I think would be useful would be:

• download the whole database for local use as you just mentioned.
• select one or several records by giving one column name and passing a list of strings. All records for which one of those strings matches is selected.

I don't know if you can do this simply using MediaWiki. Basically the URL I would pass would be like http://some.domain.org/w/index.php?title=Table:Companies&action=rawdata&select_id=12,34,567 where id is a column name and 12, 34 and 567 are acceptable id values. Maybe it is easier to just write an independent PHP script which queries the database according to the CGI arguments which are found. --Martin 01:40, 11 July 2006 (BST)

I think that level of detail is outside the scope of v1, however you're right, it would be fairly trivial to write a PHP script to parse and manipulate the raw data dump, particularly if it is in a standard format such as XML (or, it just occurs to me, SQL - I'm sure that would be possible, and then you could simply import it straight into a traditional database... hmmm... interesting thought!) --HappyDog 02:45, 11 July 2006 (BST)

Fetch and use elements of a table

Hi!

As you know, I expect this extension for a long time. Beside his earlier stage, it is a very important step, if we think to use MediaWiki seriously, in my point of view.

Well, my question is about fetch elements in the DB and use them in a infobox or elsewhere in a page. I think the <repeat> tag may be used for that, but I think this tag automatically shows a table with the data fetched.

What I need to do is something like:

 This are the companies at the DB:
 <repeat table="Companies3" criteria="Name!=Microsoft" sort="Logo desc">
   {{name}} is founded at {{founded}} and have a revenue of {{revenue}}
 </repeat>

which produces the next output:

Intel is founded at USA and have a revenue of #N/A     
Apple is founded at April 2005 and have a revenue of $7.42

Maybe we can even use the <repeat> tag with a additional template option, as in

<repeat table="Companies3" criteria="Name!=Microsoft" sort="Logo desc" template="Infobox company">
</repeat>

Best regards and again, thanks for your effort

Ivo Almeida

Hi Ivo - the first example you give is exactly how I plan the repeat tag to work! I have not coded it yet, but basically the contents of the repeat tag will be repeated for each row, with variable substitution working the same way as for templates (I may even harness the template code to do this, if that is possible...). If you want to send me your e-mail address I will let you know when this change is in place. Alternatively, just check back here from time to time. Thanks for your interest. --HappyDog 17:20, 20 September 2006 (BST)

I added this functionality a little while ago, but forgot to update this page. There are still a few bugs which need to be ironed out, but the general case works. --HappyDog 19:10, 1 December 2006 (GMT)

WikiDB hate math?

It's imposible to use math and WikiDB on one page. My wiki have a lot of math formulas. I try build special dictionary using WikiDB. If I put on page list of WikiDB records, then any math no works on this page. :(( I use MimeTeX as cgi. --195.16.88.26 16:03, 22 January 2007 (GMT)

Hi - Can you give me an example of the markup that you use so that I can investigate further. Thanks --HappyDog 16:54, 22 January 2007 (GMT)

Thanks for interest. WikiDB is very pleasant extension. I contrive now, how to do nice search for it and it will be beatifull then.

Test page with WikiDB (You can destroy this page, it's only test for you): http://www.astro.hg.pl/wiki/algorytmy/index.php?title=Test_WikiDB%26Math

The same original page without WikiDB: http://www.astro.hg.pl/wiki/algorytmy/index.php?title=Ascendent_i_Descendent

Link for separate page with DB list is no problem here. I only sygnalize you conflict.

Definition of DB: http://www.astro.hg.pl/wiki/algorytmy/index.php?title=Slownik:Slowka

Definitions of listed records: http://www.astro.hg.pl/wiki/algorytmy/index.php?title=S%C5%82ownik:Ascendent

It can be even :<math>a</math> and listing of one record of DB. Effect is identical like definition of data and listing on the same page: error type ?UNIQ35cb8610deb4633-math-4952b6c343f9c54300000002-QINU instead of formula.--195.16.88.26 21:04, 22 January 2007 (GMT)

Thanks for the examples - I will look into it and post my findings here. --HappyDog 21:29, 22 January 2007 (GMT)

OK - I have managed to get textvc installed on my local copy, so I can enable math tags (for a proper test). I tried it with MW1.6.8 (as installed on your server) and I can replicate the problem. Interestingly, it is not a problem in MW1.5.6, which is what I have been working with up to now. I will look further and see what's causing the problem... --HappyDog 02:07, 23 January 2007 (GMT)

Fixed it! Download the latest version of WikiDB.php and you should be fine.

The problem was the way that I was accessing the parser - it caused it to lose the information about tokens that had already been stripped (in this case the math tag). Using a new Parser instance avoids this. Let me know if you have any other problems or questions! --HappyDog 02:59, 23 January 2007 (GMT)

Oh, yes! Now is excellent! Thanks. :)) --Neptus (195.16.88.26) 09:02, 2 February 2007 (GMT)

WikiDB is too slow

If number of base records grows, it is great problem. Server generates timeout and pages with queries no works. I have about 900 simple record definitions now.

Complet list works:

http://www.astro.hg.pl/wiki/algorytmy/index.php?title=S%C5%82ownik:Wszystkie_has%C5%82a

When I use selection, i.e.:

  <repeat table="Slowka" criteria="kod=en" sort="tekst"># {{{tekst}}} = {{{definicja}}}
</repeat>

- it no works because of very long time needing for this operation, considerably higher above server time limits. http://www.astro.hg.pl/wiki/algorytmy/index.php?title=English

Server generates error(500). It means in short translation: script needs too much server processor time or server memory or in .htaccess file contains any illegal directiv. --Neptus 17:21, 11 February 2007 (GMT)

As it says at the top of the page, there is loads of optimisation that needs to be done to this to make it anywhere near usable for large data sets. I won't be working on the optimisation of this code until the core functionality is complete (although anyone else is free to try or to offer suggestions for optimisation). In the meantime, all I can suggest is to increase max_execution_time for PHP (see [1] for one method). This will fix the time-out issue but won't increase the rendering time - i.e. it will still take 30+ seconds, but you will get to view the result (eventually...). --HappyDog 22:16, 11 February 2007 (GMT)

It's only virtual server, not my own PHP server. :( In that case, I must rewrite it to other format... Thanks. --Neptus 22:50, 11 February 2007 (GMT)

All is correct. It was malicious server error - only this one account from a lot of thousends on server works bad. Sorry. WikiDB is really slow, but it works in reasonable time. All my pages are viewed in a few seconds now. I still like and use this extension. :) --Neptus 21:09, 13 February 2007 (GMT)

How to test the WikiDB installed correctly?

I have done everything directed in the instruction. I can find the table and table_talk name speces. But when I run the example on my sandbox page, as soon as I save the following

<data table="Companies" template="Infobox Company">
name=Microsoft 
founded=1492 
revenue=$8 
</data>

the browser address bar shows

http://mysiteurl.com/wiki/index.php?title=Sandbox&action=submit

and the sanbox become an empty blank page

Checking the undefined table spacial page, I do see the Table:Companies and check the data of the table, I got the following:

Defined Fields	Undefined Fields
Source Article	name	founded	revenue
Sandbox	Microsoft	1492	$8

After that, the sandbox page is not accessable any more. Or it shows as a blank page, I cannot edit/delete it

If I define the table first and then use "table" tag to save some data, the data will be saved to the table, but using "repeat" to retrive the data, the data cannot be shown on a page, again, the page shows blank.

What could be wrong? is there any log file that I can trouble shooting for this problem? Please help!

Thanks in advance!

simple way to do WikiDB faster

Add second single index row_id to table wikidb_fielddata. My WikiDB is like a rocket now! --Neptus 15:06, 21 February 2007 (GMT)</repeat>