Offline Wikipedia reader for the iPod Touch/iPhone makes progress
First of all, the baby has a name: iKi. Not very fantastic, but rather short. I hope nobody will get angry about the name.
But well, during the last couple of days I’ve made some progress:
- Every article is found now. This includes template. I’ve rewritten the indexer so Xapian now gets only utf8 coded keywords or article names. Every xml/html coding was removed. In the end, I was able to get article with titles like “template:„Movies“”, including the quotations. Or article with German “Umlaute” (ä, ö, etc.)
- Most of the templates are expanding well and will render fine. Missing are some like “#ifexp” which is hard stuff to do in C++. At least for me. But they are not often used.
- iKi has a nice start page which allows you to enter the desired article names or search phrases. Searching is working nice, even with multiple words.
- Added some style sheets so the result pages are looking nice now.
- Support for references, tables, enumerations, definitions, internal and external links (doesn’t makes sense without the internal ones), headlines, and so on…
What’s missing?
- A frontend for the Mac/PC to get the indexer do his job and afterwards upload the result to the device.
- Not tested with the English edition but it should work (never trust a developer when he says “should”). Some “keywords” must be translated.
- TOC and categories (the later must be prepared on the host computer).
- Images: The program tries to load them but if they aren’t there you get an “Image not found” image. I don’t know where I can get the images right now or how large they are. Anyway, the image titles are displayed and thats nice sometimes for background information.
- Some issues in rendering still existing but mostly for the really complicated articles (Deutschland, München, …)
I will post some pictures soon.
Stay tuned.
BTW: The wikipedia markup language is a crap. For instances, does ”’ means start of italic (”) and a quotation (’) or bold (”’)? It means bold! Ah ha. Looks like that it was designed before xml hits the scene. You can even mix it: ”Italic on and ”’bold also now ”italic of but bold still on”’. Uahh.
Dezember 30th, 2007 at 04:02
That’s great! I tried to do this as well, but couldn’t get Xapian to cross-compile. I’d be really interested in lending a hand. BTW, you only need the 2.9 gig current articles database, so you can still get some music on the phone :).
Dezember 30th, 2007 at 11:41
Ryan-
Help in the project is much appreciated. I’ll get in touch to you by email.
Yes, making Xapian cross compile for the arm was hard stuff. If I remember right I’ve set the host triplet to “arm-apple-darwin”. The “build” makefile was then created correctly. Ii took me a while. I’m not a GCC, OSX or even unix expert at all. This project is in fact my first one for that environment.
Second, and that is the reason why I’ve not released it, is that I’ve used a new version of libstd. Xapian needs some functions which are not inside the original one. I have now a version, twice the size as the original one, in /usr/local/lib (on the iphone/ipod touch). Nasty. I’ve not seen any problem with other programs but who knows.
Finally you’re right, 2.9 GB is the memory you need for English wikipedia edition. Plus around 1.7 GB for the Xapian index. I have it working on the Mac, but not uploaded it to the device. For the German edition it is 1 GB plus for the index 414 MB. Currently I’m not sure if Xapian is the right tool.
Anyway, the English edition is working fine also and the server automatically sets the right namespace so you can witch between editions. Installed editions are detected automatically and even internal links are working correct.
Still some rendering issues.
-Tom