A new version of the offline Wikipedia reader is out
I’ve just uploaded a new version. I think there are some great enhancements:
- Xapian is no longer needed. Indexing is a lot faster and the index itself is much smaller. Indexing the English edition took about 40 minutes, the German is prepared in less than 13. Still perl/grep is used so as soon as these are replace by a native C++ program I assume the English index is ready in less than half an hour.
- The basic server is now only one single file.
- The server starts as a daemon if you’re launching it with the “-v-” switch.
- There is a nice generic application for the iPod Touch/iPhone. It has a familiar icon and allows you to start/stop the server and launch the browser. Manually starting the server using ssh is not necessary anymore! (great if you need to reboot your toy)
I found out that many people were not able to install the software on their devices. The need to install Xapian was to difficult. So now, if you’re on a Mac, everything is ready in the new package. For the other OSs the source for the indexer is included.
Here is the new installation instruction (Mac OS X, see some annotations at the end):
- Download the package here: WikiServer v0.20
- Unzip the package (on the Mac this is done automatically and you will find a new folder in your downloads folder)
- Download the Wikipedia archive files:
en: http://download.wikimedia.org/enwiki/latest/enwiki-latest-pages-articles.xml.bz2
de: http://download.wikimedia.org/dewiki/latest/dewiki-latest-pages-articles.xml.bz2 - Place the downloaded archive file into the “Indexer” folder (inside the unzipped package)
- Open a terminal, navigate to the “Indexer” folder and enter “make data”. For the German version please first edit the “Makefile” and replace “en” with “de” at the start of the file.
- While the file is splitted and then the index is build copy the contents of the “Applications” and “private” folders from the “iPhone or iPod Touch” folder inside the package to you’re device. Use the provided directory structure below the “private” folder. Finally you should have the “WikiSrv.app” in your “Applications” folder and a folder name “Wikipedia” inside the “Media” directory.
- The new application icon didn’t shows up automatically on the home screen. Either reboot you’re device (press and hold the home and on/off button some seconds) or install/uninstall an application using the “Installer.app”. Sending a “SIGHUP” to the “SummerBoard.app” works also. Any better ideas?
- As soon as the index is done you have a folder called “en” (or “de”) inside the “Indexer” folder. Copy everything to the “~/MediaWikipedia/en” (or …/de) folder. Be sure not to delete the contents of the destination folder, especially for the German version the “language.config” file is needed. An also be sure not to end up with something like “Wikipedia/en/en/…” or so.
- Start the “WikiSrv.app”, enable the server with the slider and launch you’re Safari.
- Enter a search phrase. “test” is a good one for the first start.
As before: No warranty that you’re device is not crashing or so. Mine does it not but who knows?
Anyway: Have fun!
If you’re not using Mac OS X:
You need to compile the “indexarticles.cpp”. If your c++ tool chain (using gcc) is working from the command line simply enter “make indexarticles”. I will check if the source happily compiles using Visual C++ on Windows also. If not it should be easy to fix. No rocket science technology used. I promise. But probably you will get into trouble when it comes to use perl and grep. As soon as I find the time I will do a stand alone application for indexing.
Januar 21st, 2008 at 20:11
Hi,
I’ve followed the instruction above, and installed the DE wikipedia. But whenever I search for something it returns “Language not installed”
thank you
Januar 21st, 2008 at 20:26
I copied all the file to the right folder, and I didn’t delete anything. It return 404 page not found - Language not installed
It would be great if you can help, thank you
Januar 21st, 2008 at 20:36
Rizqi-
First: The message comes form the WikiSrv itself. So well, it’s working!
Do you see the “index.html” (this is the site with the Wikipedia globe and the search box below)? If yes, the folder “de” below “Wikipedia” isn’t right.
Further you can try to enter the url directly. Try this:
“http://127.0.0.1/wiki/de:test”
You should get the overview page for “Test”.
You can also start the server (using shh -l root directly. You will see some info messages, include the found (installed) languages and the one the server chooses as the default one.
You#re not far away from a working version. Double check the “Wikipedia” folder. Below it must be the “de” folder.
Hope you get it working soon!
-Tom
Januar 21st, 2008 at 20:38
Looks lie the software here has problems with brakes. The command for starting ssh is (hopefully now displayed right):
ssh -l root your_devices_ip
-Tom
Januar 21st, 2008 at 20:58
I don’t see the DE folder below the globe. direct access http://127.0.0.1/wiki/de:test doesnt work either.
the de folder is in : private/var/root/Media/Wikipedia/
inside de folder there is language.config and thousands of rec00*.bz2 files
Januar 21st, 2008 at 21:06
Do you use any relative location within your programm? because I moved the application folder to private/var/Applications and linking it to the original folder.
Januar 21st, 2008 at 21:42
Rizqi-
there are no links used. The server decides whether a language is present or not by checking for the presents of the “articles.dat” and “articles.idx” files. These are the indexes for the .bz2 archives. Looks like that these two files are not there.
Did the indexing process has produced these file? Have in mind, the executable which creates the index is for Mac OS X only. Otherwise you get an error and only the splitted archives.
-Tom
Januar 21st, 2008 at 21:45
ok, I think I know where the problem might lie: when I index the file, it succeeded but returns error 132 at the end. So Articles.dat is empty. Can you send me your article.dat?
thank you
Januar 21st, 2008 at 21:46
and articles.idx
Januar 21st, 2008 at 21:57
The “articles.dat” is 36 MB (mostly text, so zipping would be very effective but I assume not less than 10 MB) but the index is 5.4 MB. Binary, hardly compressable.
So sorry, I cannot send this. To large. That is the reason why the indexer is there.
But the source for the indexer is included. If you’re using Linux (or Mac OS X) build the indexer with “make indexarticles”.
I’m really sorry for the trouble.
-Tom
Januar 21st, 2008 at 22:30
No need to sorry, I am very grateful you published this tool!!! Das ding funzt jetzt wunderbar!!! Danke danke!!!!
What I did is recompile the tool. Do you use leopard?
Januar 21st, 2008 at 22:44
How can I change the port to 8080? because it interferes with my apache server
Januar 22nd, 2008 at 00:02
First of all, thank you for your work with this app!
I’ve been eagerly awaiting an offline wikipedia for the iphone!
Thing is, i’m a zero when it comes to terminal or linux commands. I’ve investigated a bit and managed to at least learn how to navigate (not very diferent from DOS), but i don’t have a clue what you mean with “enter “make data”"!
When i do that in terminal it says “-bash: make: command not found”
What am i doing wrong?
Thanks for all your work and help
Januar 22nd, 2008 at 01:47
I too am having the same problem that Rizqi was having, with the “404 page not found - Language not installed” error.
I’m not very good with computers, so how do I recompile the indexarticles.cpp so that it creates the articles.dat and articles.idx (with Mac OSX) ?
Also - does that mean I will have to reindex the articles?
And thanks for all the great work!
Januar 22nd, 2008 at 08:13
@ Lourenco
You need to install Xcode developer tool from the Mac install disk.
@ John
To recompile you just need to “make indexarticles”
Januar 22nd, 2008 at 09:31
Rizqi-
I’m glad that you’ve got it running. I hope you have a lot fun with it.
I’m a but currious why the indexer inside the archive isn’t working. It’s compile on Max OS X 10.5.1 and isn’t using special libraries or UI stuff. So why isn’t it running? Maybe something with the rights or x flag.
Using port 8080 is possible but currently not by using the UI on the device. In the past I though about adding the option to enter the port to the UI. Not a great thing so I will do that next.
In the meantime start the server from a command line using ssh. The command is wikisrvd -v- (without -p 80). Then 8080 is used by default. By using the -p switch you can use any port. If you ommit the -v- switch the server will not run as a daemon process and quits as soon as you exit ssh.
When you do that always check with “ps -ax” if the server already isn’t running-
-Tom
Januar 22nd, 2008 at 14:23
How long does the indexarticles take to run (on English Wikipedia)?
Thanks for this great program!
Januar 22nd, 2008 at 16:23
Yuri-
to split and index the English edition it takes app 40 minutes on my Mac Book Pro 2.33 GHz.
-Tom
Januar 22nd, 2008 at 20:25
Thank you! This works great!
Januar 22nd, 2008 at 20:40
hey there…. i have windows and i’m just downloading the DE file for wikipedia….
when i’m done, how shall i compile it? i have downloaded visual c++ now and don’t know what i’ll have to o then >
Januar 22nd, 2008 at 23:38
Works great thank you tom
sometimes a bit slow but other than that superb
Januar 23rd, 2008 at 01:53
Thnx Tom. Great, great, great job!!!!!
I have build the indexer with visual c++. Perfect, no modifications.
For the make data I have used the Cygwin package (it includes all we need: perl, grep, bzip, make, etc…).
I have now installed Spanish and English Wikipedias. I have modified the index.html to add a selector for /en: or /es:. With the default page all the queries were going to /en: ¿?¿?¿?
I have a problem: English Wikipedia works fine but Spanish Wikipedia returns some erroneos pages with a no-formated text. It’s really strage because it seems to affect only to the search of some Spanish cities (Barcelona, Sevilla, Salamanca,etc…). With the English Wiki this searchs are working without problems. ¿?¿?¿?¿?
Januar 23rd, 2008 at 03:24
Thanks Tom!! This is really cool.
Two questions or one question and an issue
1) The “launch browser” link goes to http://127.0.0.1/index.html but I think it should go to 127…/Wikipedia/index.html.
Do you this is correct? I am using Apache on the iPod Touch
2) In the index.html file it references /wiki/ — should this be /wikipedia/ ?
Januar 23rd, 2008 at 06:59
I forgot the most important thing— when I click Go I consistently have this returned:
http://127.0.0.1/Wikipedia/test - with a page not found. There is no language prefix at all e.g., en:test or de:test.
* I am using English
* I do have Apache installed on my iTouch
* I used the Mac to create the bzip2 splits and articles (index) files (articles.dat (183 MB) and articles.idx).
I used a Mac to run the bzip2 split and to create the index file. I checked the index.dat file and it seems correct:
Here is a subsection of the articles.dat file:
00001en:AmericanSamoa 00001en:AppliedEthics 00001en:AccessibleComputing 00001en:Anarchism 00001en:AfghanistanHistory 00001en:AfghanistanGeography 00001en:AfghanistanPeople 00001en:AfghanistanEconomy 00001en:AfghanistanCommunications 00001en:AfghanistanTransportations 00001en:AfghanistanMilitary 00001en:AfghanistanTransnationalIssues 00001en:AssistiveTechnology 00001en:AmoeboidTaxa 00001en:Autism 00001en:AlbaniaHistory 00001en:AlbaniaGeography 00001en:AlbaniaPeople 00001en:AsWeMayThink 00001en:AllSaints 00001en:AlbaniaGovernment
I used the app to start wikisrvd and I checked multiple ways and confirmed the process is running (using SSH ps -ax command and the Sysinfo tool).
I even killed the process via the app you provided and tried to start manually via SSH Terminal. I was unsuccessful– got an error message.
But I know the process is running because I can ps -ax and see it.
In any resolve the big issue is no information is returned with I search.
Any help you can provide is much appreciated.
Januar 23rd, 2008 at 08:27
One more thing:
A) I was able to start the wikisrvd process manually via SSH
B) The error message once I enter a phrase (e.g., “test”) is
“Not found”
The requested URL /Wikipedia/Test was not found on this server
Apache/1.3.37 xxxx
Even if I change the bzip2 and index.html file locations to wiki it still errors out with the same message.
Januar 23rd, 2008 at 08:35
and when the wikisrvd process is killed the following messages are displayed:
# ssh /Applications/WikiSrv.app/daemon/wikisrvd -v -
OpenSSH_4.6p1, OpenSSL 0.9.7i 14 Oct 2005
…
…
root@localhost’s password:
Wikpedia offline server
Copyright (c) 2007-2008 by Tom Haukap
This program uses the bzip2 library. Version 0.15
Base path set to ‘/var/root/Media/Wikipedia/’
Installed laguages are ‘en’, default is ‘en’
Januar 23rd, 2008 at 12:44
Henry-
looks like that both the Apache server and the wikiserv are listening to the same port 80. and the Apache webserver will not server wikipedia files.
BTW: The url is 127.0.0.1/wiki/Test, not 127.0.0.1/Wikipedia/Test
Try
wikisrvd -p 8080 -v-
to start the server on port 8080 and use
http://127.0.0.1:8080/wiki/test
I will soon add the ability to choose the port inside the app.
-Tom
Januar 23rd, 2008 at 12:47
Alfuncia-
great to hear that it’s working with the Spanish edition. Never checked that.
Did you’ve checked and edited the “language.config”? The problem with templates in Wikipedia is, that they need a prefix to get loaded. And this prefix is language depended:
en: Template:Name
de: Vorlage:Name
es: DonTKnowICannotSpeekSpanish:Name
You can change the language depended strings inside the “language.config”. The file is well documented and gous to the “es” folder. You have to stop/start the server so that the dll is loaded.
I’m glad to hear that you’ve got it working under Windows.
-Tom
Januar 23rd, 2008 at 14:07
Hi Tom,
Thanks for the help and quick response.
I executed the app using the 8080 port like so:
# ssh root@127.0.0.1 /var/Applications/WikiSrv.app/daemon/wikisrvd -p 8080 -v-
// put in password correctly
// and the following
# ps -ax
8695 ?? Ss 0:00.00 /var/Applications/WikiSrv.app/daemon/wikisrvd -p 8080 -v-
And still to no avail.
I also tried stopping and starting Apache.
And to your point– I am certain Apache is running on port 80 because I put :80 in the URL for index.html page and it worked fine.
e.g., http://127.0.0.1:80/Wikipedia/index.html
and the page loads fine.
Put another port in, say 8080 and it does not load.
So it seems the Wikisrv app and Apache are now on separate ports.
BUT it seems the WikiSrv app is not processing or being accessed at all.
I will try some more things.
One special note:
Because of how I understand Apache works my home web directory must be /Library/WebServer/Documents– which I have linked/aliased to /private/var/Documents because of the iPod Touch partition limitation.
In the ../Documents folder I have
Wikipedia
|- en
|- images
|- stylesheets
|- favicon.ico
|- index.html // this is the original html from Wikisrv_0_20 zip
|- wiki-en.png
In the en subdirectory there are about 16,000 “rec….xml.bz2 bzip2 files, the two articles files and language.config and readme.txt
Finally I link from /private/var/root/Media/Wikipedia directory that links to the above directory /private/var/Documents/Wikipedia
I also know the Wikisrvd daemon is picking up the Wikipedia directory because if I move the directory or rename it– the daemon comes back with no language and tries to create a Wikipedia directory.
Again I will try more stuff and report back
Feeling a little dumb now.
Januar 23rd, 2008 at 14:15
Hello,
i compiled the indexer with borland c++ builder and have an executive. Then i started the indexing process with Cygwin an he reads the archiv and splitted it but then after “splitting done” comes the error: ../indexarticles: ../indexarticles: cannot execute binary file
make: *** [index] Error 126
why does it not work?
AK
Januar 23rd, 2008 at 15:48
Hello trixxer ,
Try to change the makefile and call the ../indexarticles.exe instead of ../indexarticles
I think you are executing the mac binary included with the release.
I had the same problem!!!
Januar 23rd, 2008 at 15:53
Henry-
sorry for your trouble. But:
“http://127.0.0.1:80/Wikipedia/index.html”
is wrong. The daemon would serve the page if you have a folder called “Wikipedia” inside “~/Media/Wikipedia”, means “~/Media/Wikipedia/Wikipedia”. And there the index.html.
The daemon gets the url. The servername is stripped, leaving /Wikipedia/index.html” in your case. Then, if the first directory is not “wiki” (and that’s the case here) the server tried to find the path below “~/Media/Wikipedia” and servers everything what is found.
I don’t know which service serves you the index.html. Maybe Apache, maybe not.
“/private/var/Documents” is the folder for Apache. Apache will deliver you the index.html, but cannot search you wikipedia contents. For the wikisrvd to work put everything to
/private/var/root/Media/Wikipedia”. Nowhere else.
Not really true:
You can set the base path for the wikisrv: -b but you shouldn’t do that.
Another hint: Using ssh, start the server and do not add -v-. Then it’s started not as a daemon and you see some more information, i.e. the base path and the used port.
-Tom
Januar 23rd, 2008 at 16:12
hey, I dont have access to a mac and my of skills are not up to making the index, could someone upload the index files somewhere? Please! It would mean sooooo much to me! Thanks in advance if anyone does do this.
Januar 23rd, 2008 at 16:53
hello,
i manged to create the index files and copy all to my ipod. but when i click the wikisrv icon it shows only a white page and crashes the after a few seconds .. i have copied all files correctly and rebootet my device
Januar 23rd, 2008 at 17:07
Trixxer-
That is a commonn behaviour if the execution flag is not set. shh you device, naviagte to /Applications/WikiSrv.app and enter “chmod 755 wikisrv”. Then, navigate to /Applications/WikiSrv.app/daemon and enter “chmod 755 wikisrvd”.
-Tom
Januar 23rd, 2008 at 17:08
Trixxer-
BTW: What program did you used to copy the files? It look like that this resets the bit because it is set inside the zip.
-Tom
Januar 23rd, 2008 at 17:17
I used WInSCP for copying the files and yes the permission wasnt set to 755 .. now the wikisrvd programming is running! But now there is a conflict with my apache server: 127.0.0.1/index.html shows the start page of apache and 127.0.0.1 shows an page cannot be found page
Januar 23rd, 2008 at 17:36
re dhalilagma’s comment. I’m also having trouble creating the idx file, is it possible to use someone elses?
Januar 23rd, 2008 at 17:44
Robert-
If someone knows a server or service where we can put the index files I can provide mine. I have the German (17 MB, zipped) and English (83 MB, zipped) ones.
Sorry but my webspace can’t stand the load.
-Tom
Januar 23rd, 2008 at 17:51
i will upload the files after i managed to install it … can i open the wikisrvd under a diferent port with the term-vt100 app? or can i set another port for my apache server?
Januar 23rd, 2008 at 18:00
Trixxer:
Yes you can use a different port:
wikisrvd -v- ==> Starts the server on port 8080
wikisrvd -p 8085 -v- ==> Starts the server on port 8085
I think you got the idea. The -v- starts the program “verbose” as a daemon.
Januar 23rd, 2008 at 18:24
Hi,
@tom.haukap how can i do this command “wikisrvd -v-” always the message comes “command not found” i know i am not in the right dir but how to get these command work? when i try too change dir it doesnt work cd \var\appplications\wikisrv.app … always the # stays in front
here is the articles.dat and *idx for the german version:
http://rapidshare.com/files/85993643/ipod_wikipedia_de_articles.rar
i will manage to upload also the indexfiles (1gb) and the english version files.
Januar 23rd, 2008 at 18:26
oh sorry password for the file is “http://haukap.net”
Januar 23rd, 2008 at 18:35
Trixxer-
Use “pwd” to display the current path you’re in. Start an exec in Unix use ./wikisrvd if you’re in the right folder.
Tock me a while to figure that one when I switched to Mac OS X.
-Tom
Januar 23rd, 2008 at 20:51
Tom,
Thanks so much for helping– I am making progress. I feel very dumb– especially since I was once a coder and everyone else has their Wikipedia up and running.
I know I am missing something simple!!
Well the good news:
I moved all the Wikipedia files into /private/var/root/Media/Wikipedia. So now there is a path of /private/var/root/Media/Wikipedia/Wikipedia and in it is:
* index.html
* en subdirectory
With the construct when go into the WikiSrv app via Springboard/Summerboard and click ‘Launch Browser’ the page loads successfully.
The not so good news:
When I enter “test” and click “Go” the same error
Not Found
The requested URL /wiki/test was not found on this server.
From above I thought it first looked for /wiki/ and would not find it so it would then look in the Wikipedia directory.
What I have done
I have checked permissions — all is 755 so that looks good.
Ran SSH with various port and options
I will continue to debug via various SSH options. I tried bypassing the -v- option and the service ran but I did not see any information.
thanks again– your efforts and responses have been very helpful!
Januar 23rd, 2008 at 21:01
And I am using Apache to serve pages.
Januar 23rd, 2008 at 22:07
First of all! It is working !!! Mega Thx too You Tom!
its really difficult to work with the console .. the commands differ from dos!
but now a fine working wikipedia is working - the free offline encyclopedia!!
Januar 23rd, 2008 at 22:13
@ Henry
the files have to be in /private/var/root/Media/Wikipedia/ - there should be no double ../Wikipedia/Wikipedia in Media/Wikipedia/ are the dirs /de /en /Images and /stylesheets!
Apache and Wikiserver use the same Port 80 so it doesnt work simultanously. For me it works when u start the “wikisrvd” process with terminal, in the Application folder with the command “wikiservd -v-” and so u can reach the wiki at 127.0.0.1:8080/
i hope it works so
Januar 23rd, 2008 at 22:25
Trixxer-
I’m glad that you’ve made it and I hope that you have a lot of fun with it!
Yes, Unix and DOS/Windows are somewhat different. Took me, just like you, a while to get it.
-Tom
Januar 24th, 2008 at 00:40
hey guys, I tried running the terminal on a new mac (leopard) at work on my lunchbreak, but it seems the commands are not working, cd does not give me change directory so I couldn’t even navigate to the folder. Does OSx need anything additional installed in order for this process to work, or am I giving the wrong commands, if I do get it working I will write a noob readme and upload the whole thing as a torrent.
Januar 24th, 2008 at 01:14
>>trixxer Thanks! I must have read Tom’s post to me above incorrectly– I thought he said create the ~/Wikipedia/Wikipedia construct.
In any resolve I created the Media/Wikipedia only directory with the files. And SSH wikisrvd with port 8080 and still the same error. I am missing something very simple.
Januar 24th, 2008 at 01:16
>>Robert
Given all my troubles– not sure I can help you.
But I used Terminal right from Leopard without issue–> at least the basic UNIX commands (like cd) should work. I assume you are logged on as an administrator (if your Mac has the rights set)– that may not matter anyway.
If you are having problems connecting to the iTouch– well then I can help there — you have to SSH root@IPaddress and enter the password when prompted // assuming root is your userid.
Januar 24th, 2008 at 01:45
holy crap, didnt even know I needed ssh at all, crappers. Will be a mission as the mac I’m using is a display model in my store
at least I have the choice between 8 different models and we should have the air in a week and a bit. Maybe I should try and get a pre compiled version of the script and run it on my pc at home. Damn, I’m dying to get this working, will have to be patient!
Januar 24th, 2008 at 02:52
Robert>> I hope I am not confusing you or making this worse for you.
If you have a Mac there is no need for recompiling or anything like that
- You would just run “make data”
Running from a Mac will be/should be easier than a PC because the Tom has already compiled for the Mac for you.
There is one glitch– to run “make data” you will need xCode v3.0– it comes with Leopard but is an optional install– meaning the store may not have installed it –
one quick test– if you type “make data” per Tom’s instructions– you will receive a command not found message.
Hope this helps!!
Januar 25th, 2008 at 00:36
how would i compile the file using windows?
i have visual C++, but im not to sure how to do it
any help will be apreciated.
thanks
moeshou
Januar 25th, 2008 at 16:05
hey tom.haukap, pls add me on icq: 286772198 i can upload the index files on my server, it’ll be okay
Januar 25th, 2008 at 18:25
Hi, I would like to get this in my phone, but I dont know how to compile that indexer file (im using win xp). Can someone please upload compiled indexer for me ?
Januar 25th, 2008 at 22:18
i want to do, someone just has to give the file to me, se post above….
._.
Januar 26th, 2008 at 01:10
Please oh, please please please
Someone please compile this so I can use it. PLEASE!!
http://www.ipodtouchfans.com/forums/showthread.php?t=29624
Januar 26th, 2008 at 20:05
What free tool can I use to compile this? I would like to make the compiled windows version available to myself and others. Thanks!
Januar 26th, 2008 at 22:50
So, I’ve got C++ now from Microsoft. And I’m trying to compile this program but I am having so many problems. Please help me. Please point me in the right direction. I press F7 to compile, but it just gives me errors and won’t run.
Januar 27th, 2008 at 16:58
Ive used Dev-C++ and I´ve compiled it succesfully. But I dont think that program works because after I activate it it seems it doesnt make anything at all.
Januar 27th, 2008 at 17:29
@Dexter:
Did you set permissions to 755?
Regards Timo
Januar 27th, 2008 at 18:31
I uploaded the complete german Wikipedia incl. Indexes and Application.
Appx. 1,15 GB, no password needed
http://rapidshare.de/files/38420729/iPhone_or_iPod_Touch.part1.rar.html
http://rapidshare.de/files/38421442/iPhone_or_iPod_Touch.part2.rar.html
http://rapidshare.de/files/38422142/iPhone_or_iPod_Touch.part3.rar.html http://rapidshare.de/files/38422142/iPhone_or_iPod_Touch.part3.rar.html
Have fun
P.S. Thanks for the great app. It works perfect
Januar 27th, 2008 at 23:29
1a spitzenaktion timo!!!!!
aber zwei mal part 3? werde es mal probieren ob es trotzdem läuft.
Januar 27th, 2008 at 23:36
schade funktioniert nicht.. bitte unbedingt die richtigen archive nochmal neu hochladen!
Januar 28th, 2008 at 01:32
Sorry,
i posted part 3 twice and forgot part 4
Once again:
http://rapidshare.de/files/38420729/iPhone_or_iPod_Touch.part1.rar.html
http://rapidshare.de/files/38421442/iPhone_or_iPod_Touch.part2.rar.html
http://rapidshare.de/files/38422142/iPhone_or_iPod_Touch.part3.rar.html
http://rapidshare.de/files/38426606/iPhone_or_iPod_Touch.part4.rar.html
Regards Timo
Januar 28th, 2008 at 15:07
Timo, this is great! Wish I spoke German. Is there any way that you would be willing to upload the English Version as well? Thank you!
Januar 28th, 2008 at 21:37
hey, thanks for that new rar it works perfect now. but:
It return 404 page not found - Language not installed. ???????
please help me !
Januar 29th, 2008 at 00:10
Shame on me. I am not very concentrated and forgot to put the index-files inside the archive.
trixxer already uploaded them http://rapidshare.com/files/85993643/ipod_wikipedia_de_articles.rar
Just put these files to the same folder, where the thousands of bz2-files are.
Regards
Januar 29th, 2008 at 20:26
Super Arbeit, Timo!
Kann vielleicht jemand die Dateien von Timo mal auf rapidshare.com laden? Dort haben sicherlich die meisten einen Premium Account.
Januar 29th, 2008 at 21:09
http://rapidshare.com/files/87618879/indexarticles.rar.html
hf
Januar 29th, 2008 at 22:11
@bachelor: Das mag zwar sein, aber leider herrscht bei rapidshare.com noch das altbekannte 100MB filesize limit nicht wie bei .de das neue 300Mb. Das ist ein wenig praktischer wenn du mich fragst =)
Januar 29th, 2008 at 22:54
naja aber bei RS.com kann man halt downloadtools nutzen die alles vollautomatisch downloaden, bei .de gehts nicht so vollautomatisch
Januar 29th, 2008 at 23:14
hihi, da outen sich die Downloader…..
Perfekte Arbeit Timo!!!! Vielen Dank!
Vielleicht lad ichs später noch auf rs.com….
Januar 29th, 2008 at 23:31
In welchen Ordner sollen den die articles.dat und die .idx? In \private\var\root\Media\Wikipedia\de?
Januar 30th, 2008 at 01:59
Zum Runterladen von rs nutzt doch einfach CryptLoad:
http://cryptload.info/
Erkennt automatisch die Captchas
@Says: Ja, in den gleichen Ordner, in dem auch die hunderte *bz2-Dateien liegen
Gruß
Januar 30th, 2008 at 08:06
warum will mein winrar immer ein Passwort für die wikipedia_de_articles.rar???
Need help.
Januar 30th, 2008 at 13:19
password is: “http://haukap.net”
Januar 30th, 2008 at 15:44
make data results:
/bin/sh: ../indexarticles: Bad CPU type in executable
make: *** [index] Error 126
I have an iMac G5 with OS X 10.5.1
Is there a problem with the CPU type? Do you have an Intel Mac?
Thanks for your cool work.
I had been waiting months
Januar 30th, 2008 at 16:18
I get the error 126 as well but for me it says “Permission denied”… any ideas?
Januar 30th, 2008 at 16:45
hey thanks Timo. The password works and so I have now the possibility and the opportunity to work with wikipedia offline. Thumbs up, good job.
Januar 30th, 2008 at 17:05
@German-speaking: Huhu zusammen! Ich bin ehrlich gesagt relativ n00bish, hab ein gehacktes 1.1.1 iphone und frage mich ob der Wiki-Reader für mich in Frage kommt. Mir gehts vor allem um die deutsche Version. Kann ich einfach die Anleitung so befolgen wie sie ganz oben steht? Oder kann ich mir das Phone dabei zerhauen?
Januar 30th, 2008 at 18:50
wenn ich make file mache, kommt lediglich: -bash: make: command not found
benutze leopard…
wo liegt mein fehler?
Januar 30th, 2008 at 19:55
the upload to my touch works fine, every single file is complete and in the right directory. when i start the app the screen gets white and after a while it does go back to the home screen. can you help me??? I appreciate every comment because i really want to have wikipedia on my ipod.
Januar 30th, 2008 at 20:01
ups, i should better read the comments above
the rights management was the conflict and after the apps have 755 everything works great, sry for that and thank you very much for this app, it’s the best one that currently exists.
Januar 30th, 2008 at 21:47
Bei mir rutscht der enabled slider immer wieder zurücl. das ist nicht normal, oder? wo in safari muss man den suchbegriff eingeben?
Januar 30th, 2008 at 21:52
hat sich erledigt.
jetzt kann safari die seite nicht öffnen. was soll ich machen?
Januar 31st, 2008 at 02:27
For anybody else having problem with building the indexes, I put up a copy of the English (from 20080130) articles.dat and articles.idx at:
http://www.in7ane.com/iphone/wiki/articles.dat (184MB)
http://www.in7ane.com/iphone/wiki/articles.idx (24MB)
However, it is probably easier to just “make indexarticles” (to recompile) and try again if you are getting an error during the indexing.
The whole /en/ directory is 3.45GB, so unless someone really wants it I’m not going to upload it just yet (maybe a torrent is a better idea).
Thanks for an amazing app by the way!
Januar 31st, 2008 at 02:58
And the two files ziped:
http://www.in7ane.com/iphone/wiki/articles.dat.zip (62MB)
http://www.in7ane.com/iphone/wiki/articles.idx.zip (22MB)
Januar 31st, 2008 at 13:33
Hello
I tried to use cygwin to do the splitting and indexing. Splitting is working fine so far.
When it comes to indexing i got an error:
../indexarticles: ../indexarticles: cannot execute binary file
make: *** [index] Error 1
The g++ compiler is installed and path is set
Would be great if someone could help.
cu
teefix
Januar 31st, 2008 at 16:40
to in7ane, if i download the files, and upload them to the iphone, will it work, im trying to say, is this all i need? other then the xml dump, to make the wikiserver work?
if yes THANK YOU SO MUCH, i have a pc, and have no clue how to compile and stuff…
if not, can you point me in the right directions?
Januar 31st, 2008 at 17:41
Hello, is there any chance to use this app with a dump from http://wikistats.wikia.com/dbdumps/dbdumps.html ? I tried one and got an error, even when using a *bz2 version. =)
Januar 31st, 2008 at 18:52
at in7ane, again sorry , if you can can you somehow upload the /en/ directory, i would love to have wikipedia and im dying to get it
thanks so much
Januar 31st, 2008 at 19:05
Ok, here are the Rapidshare.com Links for the german Wiki Edition:
http://rapidshare.com/files/87939487/Wikipedia_to_iPhone_or_iPod_Touch.part01.rar
http://rapidshare.com/files/87943719/Wikipedia_to_iPhone_or_iPod_Touch.part02.rar
http://rapidshare.com/files/87947758/Wikipedia_to_iPhone_or_iPod_Touch.part03.rar
http://rapidshare.com/files/87951374/Wikipedia_to_iPhone_or_iPod_Touch.part04.rar
http://rapidshare.com/files/87955064/Wikipedia_to_iPhone_or_iPod_Touch.part05.rar
http://rapidshare.com/files/87911372/Wikipedia_to_iPhone_or_iPod_Touch.part06.rar
http://rapidshare.com/files/87917299/Wikipedia_to_iPhone_or_iPod_Touch.part07.rar
http://rapidshare.com/files/87923547/Wikipedia_to_iPhone_or_iPod_Touch.part08.rar
http://rapidshare.com/files/87929348/Wikipedia_to_iPhone_or_iPod_Touch.part09.rar
http://rapidshare.com/files/87934725/Wikipedia_to_iPhone_or_iPod_Touch.part10.rar
http://rapidshare.com/files/87896219/Wikipedia_to_iPhone_or_iPod_Touch.part11.rar
incl. articles.idx and dat
Just copy them to your IPhone and change the Attributes of the WikiSrv.app!
Enjoy,
…
Januar 31st, 2008 at 19:46
moeshou: will upload it and post back when done - may take a while though.
Februar 1st, 2008 at 02:47
…, those rapid share links .. what do you mean with “change attributes?” you mean FTP write-rights? or what ?
Februar 1st, 2008 at 16:09
yeah, Found…, the FTP rights…
But remember, this ist the german Version!
Februar 2nd, 2008 at 05:38
in7ane, I REALLY want the /en/ directory. I have the bz2 file, but since it’s not split, the indexes won’t work. Please oh PLease upload this for me!! THank you!!
Februar 2nd, 2008 at 06:24
Hey onesilentlight: I am uploading the EN version now. There are 12 files and it is halfway through #8 so it should be done in 5 hours or so. The files are named en.part01.rar to en.part12.rar. When unpacked there should be 16520 files in the /en/ directory and be 3,670,215,801 bytes.
http://windisgood.com/bwd1
Februar 2nd, 2008 at 17:52
Just go here instead: http://windisgood.com/bwd1/
My server probably can’t handle too many downloads. A few a day is fine, but hundreds won’t be. I’ll also try rapidshare but it’s so slow to u/l (2hrs per file and only one at a time).
Februar 2nd, 2008 at 23:25
try the rapidshare remote upload function…
should be much faster…
Februar 3rd, 2008 at 04:38
how big is the file going on the iphone?
Februar 3rd, 2008 at 04:50
i mean, how big is the en/ directory supose to be?
bwd,i downloaded all your files, the 12 .part files , am i supose to extract them all, i know it does it all at once, but winrar is saying its at 0% and the folder is already 2.4 gigs?
so how big is the en/ directory going on the iphone?
Februar 3rd, 2008 at 05:47
It’s a 12 part rar archive. Just open up the first file and it should load all the rest. Then extract it once and you should have a en/ directory that has: 16518 .bz2 files, one articles.dat file and one articles.idx file = 16520 total files = 3,670,215,801 bytes. The entire en/ directory gets uploaded, as per Tom’s instructions above.
Februar 3rd, 2008 at 12:33
Kleiner Hinweis: im Artikel steht ““~/MediaWikipedia/en”” statt:”“~/Media/Wikipedia/en””.
Vielen Dank für die Arbeit. Wenn Du Hilfe brauchst, daraus ein Paket für den Installer zu machen (eventuell mit direkten Herunterladen der xml.bz2 direkt auf das iPhone, ich helfe gerne.
Februar 3rd, 2008 at 12:46
so, bei mir klappt auch alles prima! nur ne kleine unregelmäßgkeit ist mir aufgefallen, aber evt wird das einfach noch nicht unterstützt.
und zwar mal als beispiel: wenn man (fragt mich nicht warum ich ausgerechnet bei dem wort drauf gekommen bin) nach dem wort “brotzeit” sucht, dann kriegt man beim onlinewiki nen verweis auf mehrere artikel. z.b. “zwischenmahlzeit”. bei dem offlinewiki hingegen kommt man auf eine artikelseite, die auch “brotzeit” heisst, aber ansonsten komplett leer ist. da steht nur noch drunter “redirected from brotzeit”. bei “herzfinfarkt” klappt es hingegen, da landet man sofort auf der seite “myokardinfarkt”.
einer ne ahnung woran das liegt? ich hab gestern abend in abwesenheit die ganzen files auf das iphone übertragen, kann da evt was schief gelaufen sein?
gruss und danke und fettes lob an die macher!!!
saru
Februar 3rd, 2008 at 13:44
Hi!
At first a big thanks for implementing such a useful tool!
I am a windows user. The “make data” command ist not known by the system.
Which tool I have to install, in order to solve this problem?
Thanks & best regards,
Dirk
Februar 3rd, 2008 at 13:59
@Dirk: If you’re using Mac OS, you have to install xCode. I found it on the Leopard-CD, it’s an optional feature. If you do so, “make data” will work.
Greetings
Saru
Februar 3rd, 2008 at 14:00
Oh sorry, i just overlooked that you’re an windows user. So you have to compile or download the prepared files (look ahead, they where posted by another user (german version)).
Februar 3rd, 2008 at 23:14
English Wikipedia from 20080130:
http://www.in7ane.com/iphone/wiki/en.7z.001 (500MB)
http://www.in7ane.com/iphone/wiki/en.7z.002 (500MB)
http://www.in7ane.com/iphone/wiki/en.7z.003 (500MB)
http://www.in7ane.com/iphone/wiki/en.7z.004 (500MB)
http://www.in7ane.com/iphone/wiki/en.7z.005 (500MB)
http://www.in7ane.com/iphone/wiki/en.7z.006 (500MB)
http://www.in7ane.com/iphone/wiki/en.7z.007 (393MB)
This will take up roughly 3.4GB on your iPhone
This replaces steps 3, 4 and 5 of the installation instructions.
For step 8 you can replace the /en/ directory with the one you downloaded (it already contains language.config)
Februar 4th, 2008 at 04:41
is there supose to be picture? because all im getting are IMAGE NOT FOUND icons?
Februar 4th, 2008 at 20:29
Moeshou-
There are no pictures in the dump nor Wikipedia is offering them online. So actually no images. But the WikiServer is able to deliver them. Put them into the “Images” folder below “Wikipedia” and name them the right way. I did it myself for some of my favorite articles.
-Tom
Februar 5th, 2008 at 21:05
tom.haukap-
With images, some of the (iPhone wiki) pages have leading spaces in the links, for example:
http://en.wikipedia.org/wiki/Munster
Links to ./Image: Flag of Munster.svg (with a space between : and Flag…) while the image name is Flag_of_Munster.svg with no leading _ or space
Is there any logic to the leading spaces that I could replicate?
(I download the thumbnails and rename them to the image name while replacing _ with %20, and using the extension of the main image which seems to work fine, is there anything else to keep in mind?)
Februar 7th, 2008 at 02:46
the picture thing dosent seem to work, for example,
i took the picture of the iranian flag, and put it in the images folder
the file’s called Flag_of_Iran.svg, just like the wikipedia link, but i still see the no image found picture
what am i doing wrong?
Februar 7th, 2008 at 08:11
Moeshou-
it is working, trust me. The problem is, that the name of the picture is not the one you’re seeing in Wikipedia. Strange enought, spaces are replace by %20 in text and %20 will not be converted back to space.
Some tips: Try to download a page from your device with a browser from your desktop/notebook. You can get the ip from the settings/wireless page. After that, have a look in the source and try to find out the right name.
Also, the image maybe have a different prefix. Actually only “Image:” and “Bild:” are delivered, anything else is ignored.
If your able to start the server from the command line omit the “-v-” switch. Then you can see for what the browser is asking for.
-Tom
Februar 8th, 2008 at 19:47
Image Dump
http://www.in7ane.com/iphone/wiki/Images-Wikipedia05_20080207.zip
(691MB 43,152 files)
/Images/ with all the thumbnails from the pages listed on:
http://en.wikipedia.org/wiki/Wikipedia:Version_0.5/Alphabetical
(Each image is duplicated, once with a leading %20, since some pages link like that. Some images will not show up since the pages have changed between the dump and when the thumbnails were downloaded, a few others because there is the odd image that does have _’s rather than %20’s)
That page locally: http://www.in7ane.com/iphone/wiki/05.html
And a smaller example of “Oblasts of Russia” 16MB 1,321 files:
http://www.in7ane.com/iphone/wiki/Images-Oblasts_of_Russia_20080207.zip
(also, a very rough guess as to the size of all thumbnails on Wikipedia is about 16.5GB (876K * 812.7MB / 43,152) so it does not seem possible to fit all of them on an iPhone even if there was a dump or if it was possible to crawl all of Wikipedia in a reasonable amount of time)
And the source code for downloading thumbnails from Wikipedia if someone else wants to play about with it: http://www.in7ane.com/iphone/wiki/GetWikipediaThumbnails.zip
Februar 10th, 2008 at 06:42
Where do i have to put the Images?
simply under the Image Folder??
does it works with the german Wikipedia?
thanks
Februar 13th, 2008 at 18:56
I’ve got an error trying to compile with borland c++ builder 2007.
I’t says something about the make-command and then the error occurs:
FIND:wrong parameters
***error 2 **deleting split
thnx for your help
CU
XBert
Februar 14th, 2008 at 19:17
In7ane. Thanks so much for uploading these files and utilities. Especially the thumbnail crawler app. This is working perfectly for me. Thanks so much!!
Februar 14th, 2008 at 19:22
After I downloaded in7anes 691MB image pack, I used a free program that will take the whole batch of images and resize/recompress them to recompress the images at 25% quality. The heavy compression is barely noticable on the touch’s smaller screen, and it cut the file size down to about 35% of the original.
Februar 14th, 2008 at 19:51
In7ane-
Great! Currently downloading the image pack. For the problem with the %20 at the start: That can be easily automatically removed from the code. So theres no need to have to images of the contents.
Thanks a lot!
-Tom
Februar 15th, 2008 at 15:15
Some elements don’t work on the wikipedia. The Periodic Table for instance. It just says in red letters: Template::periodic table (standard) where the periodic table is supposed to go. How can I fix this?
Februar 18th, 2008 at 23:04
tag, ich hab mal ein kleinen video tutorial erstellt aus den ganzen sachen über mir
wems hilft:
http://www.metacafe.com/watch/1107749/offline_wikipedia_on_ipod_touch_tutorial_how_to_install_the_be/
Februar 19th, 2008 at 07:15
In7ane - when i try to download your image pack i get a message that says
“Service Temporarily Unavailable
The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.”
just wondering when i will be back up.
Februar 22nd, 2008 at 00:43
When I try to install your wikipedia app, I cant seem to get the web files to work. I downloaded the wikipedia rec00*.bz2 files already extracted, and i put them in the right folders. Could anyone upload their web files for me, or tell me what to do
thanks
chris
Februar 26th, 2008 at 01:13
Is it possible to split wikipedia by argouments, i’d like to make a dump just about science, biology, chemistry, mathematics..all scientific subjects.
Thanks for you app…is huge!!!!
März 21st, 2008 at 17:17
Mhh, sag mal ist der folgender Link down?
http://www.haukap.com/WikiServer/WikiServer_0_20.zip
Ich versuche bereits seit Tagen, deinen WikiServer zu laden, erhalte aber immer die gleiche “Error 404 - Not found” Meldung. Gibt es einen Mirror?
März 21st, 2008 at 17:47
Mont-
You really should use the current version which can be found at Google Code. The link to Google Code is on the right hand site of the main windows of this blog.
Version 0.20 is very very old, the current one is 0.60. Go to the main page of this blog which is http://haukap.net.And because 0.20 is so old I’ve removed it.
-Tom
März 22nd, 2008 at 19:40
Upps, das hatte ich wohl übersehen. Danke für den Hinweis. Dann werd ich mein Glück mal mit der 0.60 probieren.
for our english-spreakers:
Upps, you are right. Thanks a lot for your helpfull hint. So I will try it with the current version wiki2touch 0.60.
April 21st, 2008 at 04:53
T H A N K Y O U
^^