Talk:Wikipedia

Creating the files with images

How are the files which include images created? I can see making the smaller ones with the export tool, but the 40GB one with all the images seems harder. I found An old thread listing some of the tools used, but the default way to use those tools seems to hit Wikipedia's servers fairly hard, at least if everything is working correctly and all the downloads work. Is there a way to do this from the official dumps, which should be both faster and less likely to overload the Wikipedia servers? I think I can do this by hosting a Wikimedia instance with Parsoid myself and setting the script's parsoidUrl to "http://localhost:8000/localhost/" and hostUrl to "http://localhost/". Would this actually work? Would it have any subtle issues to be aware of? Is there a better solution at the moment? --DanielH (talk) 09:07, 8 March 2014 (CET)