Intuition - Home of Wiki2Touch

It was a rainy day today but luckily Norbert Jaspers got in touch with me last Friday.

Norbert added support for mathematical and chemical symbols to Wiki2Touch. Wiki2Touch is an offline Wikipedia reader for the Apple iPhone/iPod Touch.

Background: If you’re editing articles in Wikipedia and like some symbols you add TeX commands into a <math>-tag. This is then rendered by the Wikipedia software using TeX. The result is finally displayed as an image. TeX is great in producing nice formulas or other special symbols.

Norbert did the add-on without any changes to the current version. He exchanged every <math>-section inside an article with a link to a picture. He let TeX created the proper image from the original tag contents. This picture is added with a certain id to the image database. This works quite well. But you had to use a modified “articles.bin” database.

Norbert told me that he used some code from Erik Zachte. So Erik should be credited, too.

Wiki2Touch is now modified and the ids of the math sections are now calculated on the fly, the sections are removed and the proper images are inserted (if installed of course). Works like a charm:

Wiki2Touch featuring TeX rendered symbols

Because the background of the images currently isn’t transparent you can easily spot what is a image and what is text.

A couple of things still have to get changed. But math support is now added to Wiki2Touch. While on that subject: I think there’s finally enough stuff together for a new version now. So stay tuned.

Thanks for all the support!

Tom

The “ImageGetter” is a Microsoft .NET framework application which downloads the images for Wiki2Touch. It’s not a cofortable or nice application, but that will be change now.

Here is the story:

I have a free day today. I’ve planed to start a new indexer for Wiki2Touch. Some reasons are to get more control over the articles contents and metadata. Further to join downloading and indexing article together with image retrieving into one application. Then taking the article categories into account in order to create packages only containing some but not all articles. Think of all medical related ones or all excellent ones. Support for some special namespaces (i.e. “Portal” or so) along with math support in the end needs a new indexer, too. And yes, uploading the results to the device.

I’ve decided to use the .NET framework from Microsoft. There were a couple of other option: C++, Java, PHP and so on. I like the .NET framework best and developing is much faster and less prone to errors than using C++. And also I love my Mac I still think the VS.NET developing environment from Microsoft is the best I can get.

When I got my Mac more than one year ago I first tried Mono. Mono is an open source project which allows you to run Microsoft .NET Framework based applications on non Microsoft operating system (like Linux, Mac OS X). I was disappointed in these days, it was not working for me, even not for very simple applications.

Anyway, because I only needed a few hours I’ve written the “ImageGetter” for Wiki2Touch in C# (.NET Framework 2.0). Now I was able to download the images for Wiki2Touch. But I was not completely happy with the result. The application didn’t runs directly on my Mac. I had to use Parallels which allows to run Windows on OS X.

But that has changed: Today I’ve compiled the “ImageGetter” using the latest version of Mono. And it worked right from the start. On Apple OS X. I only had to exchange the hard coded path separator char (”\” on Windows, “/” on Unix, now I’m using the value provided by the framework). Here is the result:

The “ImageGetter” running on OS X using the Mono framework.

Speed is fine: It downloads the images not slower than the version running under Windows. And for what reasons ever it crawls to more than 440.000 images I already downloaded in only 2 minutes. I don’t know why that is so fast on OS X, compared to Windows.

I see no reason why that shouldn’t work on Linux, too. The guys from the Mono project did a great job. The ImageGetter uses MD5 hashed, has a GUI, and downloads http content using threads. I’m impressed.

But it even getting better: The open source “SharpZipLib” library works without recompiling, too. This library is needed to decompress the Wikipedia article archives. The job the former “Indexer” did.

Conclusion: Everything is set up to create a nice and feature rich GUI based front end for Wiki2Touch which runs on Windows, OS X and Linux which is easy to use for everyone. I’ve just started.

I will upload the slightly modified sources to the Google code page during the day.

Although we had very nice weather the last couple of days I was able to make some more progress on Wiki2Touch, the offline Wikipedia reader for the Apple iPhone/iPod touch devices.

Over at Google code they are doing some maintenance work so the Wiki isn’t available. Usually I track changes over there.

Beside some bugfixes I’ve added support for <imagetag> tags (sometimes used as {{#tag:imagetag|…}} as well). Tapping on these image maps isn’t support yet but they are at least displayed. You can see some of these types of images at the bottom of this screen shot:

A sort of an “On screen display”

More interesting is for sure the shinny black menu in the middle of the screen. It appears and disappears once you tap somewhere on the screen (you must not hit a link). Now you can easily go to the TOC, the top or bottom of the article or back to the search page. For those who like to extend this “menu”: It’s a simple <div> tag so you may add what you like.

Any more suggestions what I should add to this panel?

Once you’ve jumped back to the search page it will now show you the last search result. That’s even true if you reached the page due to an icon on your home screen or a bookmark entry. The last search phrase is stored by the server, together with the language you have used.

And if you have more than once language installed you know see a drop down box to choose a language as the one you like to use:

GUI support for multiple languages

Sure, if you have only one language installed the select box is hidden.

If everything turns out fine I will release the updated version on the next weekend.

Skylar, the manager of the community source iPod Touch Fans, has listed Wiki2Touch. Thanks a lot!

You will find it under the “Misc” category. Maybe not the proper category, but the managers of the community sources are asked not to create categories on their own. So where else?

Skylar told me that he expects up to 150.000 downloads in the first month. That’s more than 10 times more as before.

I don’t expect that everybody will finally useit. The process of putting the necessary data file onto the device is still way to difficult and inconvenient for most of us. Guess what’s next on my list At least the build in help is a good start.

Yesterday night I’ve got a little bit crazy. Only to see how it looks I’ve uploaded the Chineses Wikipedia database. Here’s a screen shoot:

At least it looks nice to me.

So enough for now, back to work.

I’ve just released the latest version of Wiki2Touch, the offline WikiReader for the Apple iPod Touch or the iPhone. It was a long and sometimes stony way from the last version up to this. Trust me.

So whats new? As I’d said in the past, this version deals with rendering bugs. So it has no new features. Or decide for you’re own if you like to name one or the other change a new feature. Here is a brief summary:

• Template caching works without any change of rights now for firmware 1.1.3/1.1.4, too. If you’ve not modified anything in the past while using 0.60 together with firmware 1.1.3/1.1.4 you will see the articles now displayed much faster once the template is cached.
• Support for #expr/#ifexpr parser functions. Have a look at articles like “2008″ (the year 2008). Mainly all the links in the head area are computed by the use these new functions.
• Preliminary math support: At least the text from the math-tags is extracted and displayed with a gray background for better readability and the math symbol “Pi” is displayed as the Greek symbol. Nothing else.
• More than 30 changes for better rendering support. The changes covering smaller issues like to much spaces or line feeds, correct handling of headlines, reference lists and enumerations, up to the support of sometimes very complex templates (i.e. in the English edition articles like “Al Gore” or “George Washington”. And a bug which crashed the server was also removed.

You find a complete list over at Google Code (see the links on the right side).

Still one of my favorite screen shots from the tests

I’m glad to finally have this version out. I have lots of ideas for new features, starting from enhanced multiple language support, followed by making the server faster, adding a full text search index, support for certain Wikipedia namespaces (categories, portal, …) and so on.

But all this is useless without a stable basis. And, although I know that there are some minor issues still left, I think this version is it.

Installer support for the latest version was set up (thanks Sanford).

While on the subject “installing”: There are no major changes to the indexer or the articles database. You DON’T need to re-index anything or have to re-upload the “articles.bin” or the “images.bin”. No changes here! So as soon as the update shows up in the installer simply hit the button. A restart of the server after the installation is a good idea.

Have fun with the new version,

Tom

After the Easter break I’m going on to fix rendering bugs. In order to do that I have to finally add support for #expr and #ifexpr “templates”. Basically you can use them to do calculations. Samples are the pages dealing with years (i.e.e 2002). At the top of the pages is an area which list previous and next years, decades and centuries. Every link in this header is calculated based on the page’s name. Compare your screen layout and the one online and you will see what I mean. So Wiki2Touch is now able to render them as well.

Calculation are use somewhere else, too. I.e. the English article about New York uses them to display changes of the population in percent. The current version is not displaying anything here but the next will.

That was one of the last great challenges. Except for “math” support the rest is even simpler. I’m really glad to finally have that also implemented. I’m continuing iron out a few open rendering issues.

There is currently no new version available. The current is ten day “old” and still fine and stable. But you may have a look at my “work in process” list over at Google code if you like.

Tonight it’s 4 days (or 96 hours) after the release of 0.60. App. 3500 downloads according to Google’s download counter. 0.52 was slightly over 8000 in 2 weeks (or 336 hours) (every download includes on download by myself testing Sandfords installer script).

When I compare these numbers to the count of comments and posts here and on the forum I’m getting very very happy. The majority of free software using users don’t say anything unless there is a problem. Like me, too. But how much posts are there dealing with issues? Around 10 or so. If not less. Mostly installation problems. A good start.

That doesn’t means that there are no bugs. So the next version will more or less dedicated to fix rendering issues. But for the wast majority of articles there are simply no problems.

Anyway, it’s the weekend of Eastern and I will have a couple of free days. I will spent them with my family so don’t expect to much news in the next few days.

Thanks for all your support so far and have some nice days, too.

Maybe you already know Sanford Poon, the guy behind the installer support for Wiki2Touch. And maybe you already noticed that there is a second entry int the category “Temporary” once you’ve added “http://168weedon/i” to your installer sources (to install Wiki2Touch). The entry has the name “BossPrefs Wiki2Touch Control”.

“BossPrefs” is a separated tool which you may install to enable/disable certain services on your device. SSH, EDGE, Wifi and a lot of other things. And by adding Sanford script Wiki2Touch, too:

Great, thanks a lot!

(BTW: The screen shot above was done without any additional program. Only using the firmware 1.1.3 or 1.1.4. Have a look at http://www.iphone-ticker.de to see how to enable this feature. Sorry, this site is in German language only)

Yesterday I’ve updated my device to use the latest firmware 1.1.4. Using 1.1.2 before I really love the new features. Adding icons to the home screen and arranging them is great.

The good news is, that Wiki2Touch is running fine on 1.1.4. You only have to make sure that the articles.bin/images.bon and so on must go to “/var/mobile/…” instead of “/var/root…”. Same as in 1.1.3. If you’re using Sandfords installer source (recommended, the link is http://168weedon/i) you will be guide because it script created the folder structure below ~/Media/Wikipedia for you.

But there is one disadvantage. Wiki2Touch trys to create a folder named “cache” below every installed language folder (i.e. ~/Media/Wikipedia/en/cache) and stores preprocessed templates to that folder. That is way faster than processing them every time they are used. And that’s not working under 1.1.4. Wiki2Touch is actually not able to create the”cache” folder on it’s own. And that means it runs way slower than on 1.1.2 if you have templates enabled.

Workarounds:

• Disable templates (makes it faster but with a lot less information)
• Manually create a folder named “cache” below your language code folder and set the owner and/or rights in such a way that the user mobile is able to write file into that folder. “chmod -R mobile ~/var/mobile” may also work for you. You can tell if it’s working as soon as you see files inside that folder once you’ve displayed an article. You don’t need to stop/start the server, it will use the cache on the fly but be sure to have templates enabled.
• Live with it until the next version is out

Otherwise it’s working nice on 1.1.4.

If you like, over at Google Code, I’ve uploaded version 0.60. There are a lot of changes.

Diacritic aware suggestion list

Here is a list of the major changes:

• Diacritic aware search/support for simplified Chinese character conversion: I.E. “Ä” becomes “a”, hence the phrase “apfel” will find “Äpfel”. Chinese traditional characters are converted to simplified Chinese once. I don’t know if the later this is really working.
• Enhanced image support: The properties of images (size, placement, title and description) are take into account. Further, the smaller 120px thumbs are scaled up by the browser to 180px. Yes, they get blurry but that’s ok. I’m really impressed how fine it’s looking now.
• The indexer finds a lot more images. For the German edition over 440,000! If you install them all (I did that only on my Mac) you hardly never see “Image not found” anymore.
• Achim’s “uploader tool” was updated to reflect the new features.
• The suggestions list looks very nice now.
• By clicking the globe on the main page you will be taken to an arbitrary article.
• Much more smaller changes and bug fixes, for a complete list have a look at the Wiki at Google Code.

I’ve also compile a small list of screen shot. Have a look at here if you like.