8,236
edits
Difference between revisions of "Project Gutenberg"
Jump to navigation
Jump to search
→See also
| (14 intermediate revisions by 4 users not shown) | |||
| Line 3: | Line 3: | ||
== Goals == | == Goals == | ||
* A script (python/perl/nodejs) able to create quickly a ZIM file with all books in all languages. | * A script (python/perl/nodejs) able to create quickly a ZIM file with all books in all languages. | ||
* The data should be scraped from www. | * The data should be scraped from www.gutenberg.org. | ||
* The texts should be available in HTML and EPUB. | * The texts should be available in HTML and EPUB. | ||
* The ZIM should provide a simple filtering/search solution to find content (by author, language, title, ....) | * The ZIM should provide a simple filtering/search solution to find content (by author, language, title, ....) | ||
| Line 10: | Line 10: | ||
# Retrieve the list of books is published by the Gutenberg project in [http://www.gutenberg.org/cache/epub/feeds/rdf-files.tar.bz2 XML/RDF format] | # Retrieve the list of books is published by the Gutenberg project in [http://www.gutenberg.org/cache/epub/feeds/rdf-files.tar.bz2 XML/RDF format] | ||
# Parse the XML/RDF and put the data in a structured manner (memory or local DB) | # Parse the XML/RDF and put the data in a structured manner (memory or local DB) | ||
# Download the necessary HTML+EPUB data from | # Download the necessary HTML+EPUB data from Gutenberg.org based on the XML/RDF Catalog in a target directory | ||
# Create the necessary templates of the index web pages (For the search/filter feature, a javascript client side solution should be tried) | # Create the necessary templates of the index web pages (For the search/filter feature, a javascript client side solution should be tried) | ||
# Fill the HTML templates with the data from the XML/RDF and write the index pages in a target directory | # Fill the HTML templates with the data from the XML/RDF and write the index pages in a target directory | ||
| Line 83: | Line 83: | ||
copy-data-to-outputdir | copy-data-to-outputdir | ||
build-index | build-index | ||
== Scripting Stages == | |||
# [one-time only for mirror] rsync all data to zimfarm.kiwix.org/gutenberg | |||
# Download & Extract rdf-files.tar.bz2 | |||
# Loop through folder/files and parse RDF | |||
## Fill the Database with all data | |||
# Query the database to reflect filters and get list of books | |||
# Download the books based on filters (formats, languages) | |||
# [MLB] Generate a static folder repository of all ePUB files | |||
# Generate zimwriterfs-friendly folder of static HTML files based on templates and list of books. | |||
# Generate zim file from static folder | |||
* Prepare the templates | |||
** Article template | |||
** HomePage template | |||
** Index template? | |||
== Next steps == | |||
One of the problem is that even on Gutenberg, we don't have all the most important books of the French litterature. We should help to fix this. Here is the [[TOP 100 French ebooks to create|coordination page]]. | |||
== Others == | |||
* http://www.ebooksgratuits.com/opds/index.php | |||
* http://noslivres.net | |||
* http://fadedpage.com/ | |||
== List of books == | |||
* http://www.alalettre.com/fluctuat-10-livres-parfaits.php | |||
* https://fr.wikipedia.org/wiki/Biblioth%C3%A8que_id%C3%A9ale | |||
== See also == | |||
* [http://lite4.framapad.org/p/hackathon-kiwix-lyon Framapad with some notes] | |||
* [[Project Gutenberg/description_fr|Grant detail]] | |||
* [https://github.com/kiwix/gutenberg Github repository] | |||
* Photos: [[:commons:File:Kiwix hackathon Lyon juillet 2014 extérieur.jpg|outdoor]], [[:commons:File:Kiwix hackathon Lyon juillet 2014 intérieur.jpg|geeking]] | |||