Images!
I’ve spent the day to add better image support to the Wiki2Touch reader. The old style is still working (simply put the images you like to the “Images” folder). The new style is one file where all the images (only thumbs) are in. Just like the articles.
I’ve also written a program to download the images from Wikipedia. But that takes a while! It’s not done here.
In the end not every image will be available. Here’s why:
- Some images are to large, even as thumbs. A lot of animated gifs for example. Yes, we can try to reduce the size.
- A few number of images can’t be found at all. That’s because it’s unclear where the images are stored. “Commons” or other areas.
- Last but not least: For some images I can’t get the name without completely rendering the article. If an image is used only inside a template it’s not easy to find the image. Because there is no Wikipedia style link. This link is created after the template is expanded. Yes sure, there are the database dumps for the image database.
While we’re on templates: The “facts” are nearly all the time displayed using templates. Infoboxes like company information, cite information, plant information and so on are templates. Not having template support is more worse than not having pictures. Look at the screen shot below: The facts about Apple are not display without having template support. You simply don’t get the information you want.
Only the text but not the facts.
Februar 25th, 2008 at 12:43
Hey Tom,
i know you prefer to keep the conversation in the comments in english, but it’s pretty tough to find the right words in a forign language to phrase the admiration and gratefulness your “little” programm gave to me
Weiter so Tom! Ich und tausende andere Iphone und Ipod-Touch Besitzer wissen Deine harte Arbeit wirklich zu schätzen!
….but enough german…..it’s already awesome and if you continue improving it like this it will be soon a must have even for the iphone contract owners
May I ask, how far the development of the installer add is? Uwe seems to be really busy in real life
Remember my words…..as soon as it will be available for the installer….it will be the bomb!
Best regards from Münster, Germany
gulibamba
Februar 25th, 2008 at 16:00
Hi Tom
Thanks for the great job. I installed both Chinese and German wiki data files, but only the German one works. If I remove the German wiki data file, then the Chinese one is also working? Dose wiki2touch support multiple languages simultaneously? or is there any way to easily switch langues?
Februar 25th, 2008 at 18:18
Hui-
yes, the software supports multiple languages at once.
The default language is english (en). If this is not installed the first language which is found becomes default. I assume this is German (de) but this is not related to the alphabetical order.
You can use Chinese (zn) or any other language by simply adding the language prefix to the search phrase. Type for instance “zn:香港地鐵”.
Maybe I add a convinient switch to set the default language. But for most of the users this will not be necessary I assume.
Februar 25th, 2008 at 19:51
Gulibamba-
thanks a lot for your kind words.
Haven’t heard from Uwe a while so I’m not sure how far he is. If all goes from I can do the installer script by myself. I thought about doing that by I started to add image support. Maybe a bit selfish.
Installing Wiki2Touch with the installer is for sure much more convenient. But the problem with the large language database files will still exists. I really would love to offer them to download but I can’t.
A solution would be to used an application on the computer which download and prepare the database and then stores them to the iPhone using iphuc (or iTunes). But maybe the upcomming SDK will over a solution for that problem.
-Tom
Februar 25th, 2008 at 19:56
An update:
Images are working great although I’ve just downloaded app. 20% of them. I assume the German image package will will be around 1 - 1.2 GB in the end.
I will put the latest version to Google code for download tomorrow. Not because of the image support (the problem here is the application which downloads and packs them) but because of a fix in the new search code. And the improvement to get a list of hits right from the first letter. Great, fast and convenient.
-Tom
Februar 25th, 2008 at 20:13
Hi Tom,
Just wanted to say thanks for all of your hard work. This is a fantastic program!
I have come across a couple of missing article problems (that exist in the online Wikipedia) but this has been much less of an issue than an earlier version of the program.
Looking forward to getting images included!
Thanks again,
Jason.
Februar 25th, 2008 at 21:00
Jason-
may you tell be which language your using and which articles are not found? Another one reported a similar issue but when I checked it I could find the article and was able to display it. Strange problem. Maybe the indexer has a hickup. Under which OS you’ve run the indexer?
-Tom
Februar 25th, 2008 at 21:28
Hi Tom,
first i want to say thank you for putting so much effort and energy in this wonderful app
In my opinion it’s absolutely the best of the available offline wikis for the pod (or phone)!
So, my question: I don’t know how the search engine works, but maybe it is possible. I would LOVE to have a suggestion list with entries after the first few letters (I’m looking forward to tomorrow because of the 1 letter suggestion feature), which is not dependend on the “correct spelling”… I don’t know how to explain it right
I’ll give you an example of what I mean. Let’s say I’m looking for “Schloss Garvensburg”. So I have to enter “schl” to get the suggestions with “schl” at the beginning. But if I enter the search term “garvensburg”, the article could not be found. Do you think it would be possible to alter the engine in such way, that not only the results (article) are displayed if i enter the whole term?! So that i enter “garvensburg” and I’m presented with the Article to “Schloss Garvensburg” or a server generated suggestion list of Articles where “Garvensburg” is part of…
That would be awesome!
Thanks so far and greetings from “Nord-Hessen”
Chris
Februar 25th, 2008 at 22:22
Hi Tom,
One of the examples was ‘Manchester Central FC’ (a short-lived football club from the late 1920’s). I just get the article not found page, but online there is a short article. I use the english language version.
I can’t remember the other article that couldn’t be found under the current version. The ones that I remember didn’t work for me under the earlier release of your software seem fine now, and it seems very stable now. Maybe having the articles in a single file helps cut down the chances of this issue occurring.
Given you could find articles that didn’t work for someone else I guess it is just an indexing hiccup as you say. I am running Mac OS X (Leopard).
Jason.
Februar 25th, 2008 at 22:38
Hey Tom, first of all, thanks for your great work, for me, this is iPhone’s killer app!
Unfortunately, on your latest version, when indexing i get this:
repackaging enwiki-latest-pages-articles.xml.bz2
Processed: 18% (584704 articles)
Repackaging done:
2791243776 bytes before, 584926 articles written (2497162014 bytes)
330 articles skipped (130396 bytes)
2814 blocks
reading articles titles
lowering and indexing articles titles
sorting index
checking index: passed
writing index: 2339704 bytes
It just stops indexing at 18% with a file size of a little more than 600 Mb and calls it a day. I’ve tryed different times on different hard drives and always got the same result! I’m using mac osx leopard on a macbook 1.83Ghz, Thanks for any help anyone can give
Lourenço
Februar 25th, 2008 at 22:46
Jason-
Great! I did just a quick check: The article about the football club shows up in the “hit list” but I get an “article not found” as soon as I want to display it.
That means: The article is present and found but a bug prevents loading it. Should be easy to iron that out.
Thanks for helping out!
-Tom
Februar 25th, 2008 at 22:53
Lourenço-
before the release I’ve checked the indexer against the latest English (en) Wikipedia dump available. Worked fine fore me. And there are people around who also successfully used it.
But I don’t know what’s happening if a downloaded image is corrupt and cannot be decompressed.
Let me check this. Maybe your download is corrupted.
-Tom
Februar 25th, 2008 at 23:02
Thanks for the quick reply Tom! I’ll try downloading the file again, maybe it’s it!
Lourenço
Februar 26th, 2008 at 02:47
words such as “animal” says the article is not found
Februar 26th, 2008 at 08:10
Tim-
great, another example. Thanks. I will fix that issue today.
-Tom
Februar 26th, 2008 at 08:35
Ok found the bug than some articles are listed but not displayed. Happens only in the English edition because of the length of that file.
I’ll upload a fixed version soon. Creating a new index is not necessary, simply use the new codes.
Thanks for reporting thr bug again.
-Tom
Februar 26th, 2008 at 12:35
You where right tom, after i downloaded the file again, it indexed without any problems! Thanks!
Lourenço
Februar 26th, 2008 at 12:55
Hi again,
I came across something weird. I was using my example “garvensburg” and after the first 4 letters the suggestion list appears the way it should. So i opened all of the suggestions made (only 6 suggestions, so not that much to choose from). All of them worked fine except one. “Garvey’s Ghost” didn’t.
If you try to open that one, nothing will happen. But if I enter the whole term in the search field, the page will be displayed. It’s an entry about Marcus Garvey and “garvey’s ghost” only appears in the text of the article.
I’m using a german wiki dump, indexed on 10.5.2 MB.
(another small rendering issue is, that if you turn your pod and the text is being upscaled, it won’t scale down again after turning the pod back. but this is not that important.)
Maybe it’s helpful.
Chris
Februar 26th, 2008 at 12:57
maybe it’s because of the ‘
sry for double posting^^
Februar 26th, 2008 at 13:08
Chris-
thanks for reporting! That is a simply JavaScript bug because of the apostrophe sign (’) in the articles title. Fixing is easy, I will do that with the upcomming release.
-Tom
Februar 26th, 2008 at 14:20
glad if I can help
Do you think it is possible to add such searching like I mentioned above? that would make finding articles way easier. Cause i don’t always know the right spelling or the whole term of the article i’m looking for…
Like the example with “garvensburg” and “schloss garvensburg”. Cause “garvensburg” is a part of the article’s header, but it won’t be displayed if you don’t enter the whole term.
So that if I enter “garven” there should be “Schloss Garvensburg” suggested by the server. Or on the other side, that if i enter “schloss” –> “Türschloss” will be suggested. you get what i mean!?
And is it possible for me to change a configuration file, to expand the length of the suggestion list!? Some kind of entry in a xml file or sort of
Thanks again
P.S.: the up-/downscaling issue only exists in the suggestion list for me. Maybe theres just some small fixing necessary, cause upscaling works fine (dunno if its handled by safari or on your server eg ajax…)
My knowledge of programming is very little, so that are only suggestions by me…^^
Chris
Februar 26th, 2008 at 16:37
Helo
where can i download a finished image package?
thanks
Februar 26th, 2008 at 16:49
Auge-
actually nowhere because the feature isn’t ready. But once it is I can’t say where you can download it. I simply don’t have the bandwidth to host a file which will be about 1 GB.
Maybe you can help? Do you have a server which can stand the load?
-Tom
Februar 26th, 2008 at 16:53
Chris-
A full text index for the articles titles was once in but I’ve removed the feature for a couple of reasons. One is that the index itself was very hugh (150 MB for the German version).
Yes, the feature is possible but a lot slower than the current search engine. Actually I’m not sure if and when the feature will return. There are a couple of thinks I think are much more important. An installer package and image support.
-Tom
Februar 26th, 2008 at 17:17
Nice to hear that such a feature was already implemented. I would love to see its return
I don’t care if its slower ^^
The index which is used to generate the suggestion list would be bigger or the suggestion list’s cache? Till now i thought after the article.bin was created, all data in the suggestion list was created by the server on the fly. But now (if i get it right) i think that the whole article.bin would be bigger if this kind of search returned, right!? And I can imagine that the speed will decrease a lot, since you don’t check the titles only from the beginning but from every position in the term. so a lot more work for that little plastic box^^
Anyways I’m looking forward to your next release and the possibility to download and integrate images on the whole!
Chris