Tell us your story
How has offline Wikipedia affected you? The Wikimedia Foundation (the non-profit that supports Wikipedia) is looking for personal, diverse and inspiring stories about how offline Wikipedia affects the world. If you have a personal story that you would like to share, please contact: stories@kiwix.org. Thank you!
Difference between revisions of "Hackathon Wikimania 2019"
(Hop) |
(→Budget) |
||
| (56 intermediate revisions by 10 users not shown) | |||
| Line 2: | Line 2: | ||
== Goals == | == Goals == | ||
| + | |||
===Kiwix=== | ===Kiwix=== | ||
| + | |||
| + | ====Build==== | ||
| + | * Explain all the build system and its integration in the CI. | ||
| + | |||
| + | ====Lib==== | ||
| + | * Discuss (and hopefully decide) about tag/name management of zim file (https://github.com/kiwix/kiwix-lib/issues/131 and associated issues) | ||
| + | |||
| + | * Discuss about the server integration (if not done since) | ||
| + | |||
====Tools==== | ====Tools==== | ||
| + | * Look the best tool to analyze activity of all kiwix's Github projects (try to use [https://chaoss.github.io/grimoirelab/ GrimoireLab]) and deploy a first version to test this - Florent | ||
| + | |||
====Desktop==== | ====Desktop==== | ||
====Android==== | ====Android==== | ||
| − | + | * See about the wrapper about the Library/Server of kiwix-lib. | |
| + | |||
====Hotspot==== | ====Hotspot==== | ||
| − | + | * Make Kiwix-Hotspot running in RPI0 - Renaud | |
| + | * Prepare presentation - Kelson, Renaud | ||
| + | |||
====MWOffliner==== | ====MWOffliner==== | ||
====WP1 Bot==== | ====WP1 Bot==== | ||
| + | * Make a Docker image of the update part - Kelson | ||
| + | * Try to convert Update -> Releases to Python - Travis | ||
| + | * Set up travis-ci for the bot's github project - Travis | ||
| + | * Develop scheduling infrastructure for update part - Travis | ||
| + | * Convert upload part to Python with tests - Travis | ||
| + | ** Refactor/reorganize project code to maintain logical organization - Travis | ||
| + | * Develop plan for web app - Travis | ||
| + | |||
| + | ====Other scrapers==== | ||
| + | * Planning of FO requests - Kelson, Renaud, Joe | ||
| + | * Fixing many problems in zip2zim - Joe | ||
| + | * Revamping phets scraper - Joe | ||
| + | |||
| + | ===Other=== | ||
| + | * Cleaning of Kiwix wiki - Kelson | ||
| + | * Managing the logistic of the hackathon - Kelson | ||
==Achievements== | ==Achievements== | ||
| + | ===Kiwix=== | ||
| + | |||
| + | ==== Android ==== | ||
| + | * Worked on integrating kiwix-lib changes to allow stack overflow ZIMs and others with redirects to work on Android. - Isaac | ||
| + | * Fixed various bugs relating to the bottom navigation bar. - Isaac | ||
| + | * Worked out a new way to build with local kiwix-lib. - Isaac | ||
| + | * Fixed various bugs with the home screen. - [[User:Siddharth2010|Siddharth]] | ||
| + | * Worked on integrating the wifi hotspot of kiwix-android with kiwix-serve. - Abdul and [[User:Siddharth2010|Siddharth]] | ||
| + | * Fixed bugs in the tab switcher. - Abdul | ||
| + | * Enabled Kiwix+LeakCanary to be tested using Android Robot (which already exposed some crashes in the app) - Julian | ||
| + | * Documented, tested and slightly improved the Custom Android App build process - Julian | ||
| + | * Started fixing some flaws in the app when it's run as a custom app. - Julian | ||
| + | * Found several bugs with the Notes feature - in custom apps it crashed, and on older devices the text is almost white on a white background - Julian | ||
| + | * Worked with Abdul to find the causes for 2 of the top crashes as reported in Android Vitals - Julian | ||
| + | * Did some testing using Android Monkey, too early to provide definitive analysis, I'm continuing to test using Monkey post hackathon and will report findings separately - Julian | ||
| + | * Prepare 2.5.3 release - Abdul | ||
| + | * Fixed duplicate zims on home screen and device tab - Abdul | ||
| + | * Fixed position of home screen download button for RTL layouts - Abdul | ||
| + | |||
| + | ==== libkiwix ==== | ||
| + | * Make a decision about how to test cutting-edge libkiwix against kiwix-android, [https://github.com/kiwix/kiwix-build/issues/345 here] and [https://github.com/kiwix/kiwix-build/issues/346 here] [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]) | ||
| + | * Introduce HTTP service in the libkiwix (from kiwix-serve) - Matthieu | ||
| + | |||
| + | ==== Kiwix-hotspot ==== | ||
| + | * Fixed homepage icon for some ZIMs (work around zimwriterfs bug) – renaud | ||
| + | * Released version 2.0.8 | ||
| + | * Updated Support Files wiki (build instructions for pi0-4 on with newer kernel) - renaud | ||
| + | * Switched to latest raspbian image (buster) - renaud | ||
| + | * Added support for Pi4 | ||
| + | |||
| + | ==== Maintenance ==== | ||
| + | * Fixing zimquarantine - Florent | ||
| + | |||
| + | ===Openzim=== | ||
| + | |||
| + | ==== libzim ==== | ||
| + | * Discuss a few possible paths of optimization of the libzim and open tickets: [https://github.com/openzim/libzim/issues/264 here] and [https://github.com/openzim/libzim/issues/264 here]. [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]) | ||
| + | * Introduce support of new Flavour Metadata [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]) | ||
| + | |||
| + | ==== zim-tools ==== | ||
| + | * Release of version 1.0.4 [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]) 14:11, 10 August 2019 (UTC) | ||
| + | |||
| + | ==== zimwriterfs ==== | ||
| + | * Fix regression related to favicon redirect writing [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]), renaud | ||
| + | * Introduce flavour Medata writing [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]), renaud | ||
| + | * Introduced scraper parameter to set metadata - renaud | ||
| + | * Rewrite the Dockerfile to allowing copying the src code directly from the repository [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]), renaud | ||
| + | * New release 1.3.5 [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]) | ||
| + | * Fixed how relative URLs in CSS are rewritten | ||
| + | |||
| + | ==== MWoffliner ==== | ||
| + | * Fix problem with Wrong description [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]) | ||
| + | * Add support of new ZIM metadata Flavour [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]) | ||
| + | |||
| + | ==== WP1 engine ==== | ||
| + | * Many changes to make the project more newcomers friendly. Emmanuel | ||
| + | * Setup Docker & deploy it. Travis, Emmanuel | ||
| + | * Setup Codefactor & Codecoverage, Emmanuel | ||
| + | * Setup CI with automated testing, Travis | ||
| + | * Wrote upload tables code, with tests. Travis | ||
| + | * Setup rq job queueing/worker code for update/upload portion. Travis | ||
| + | * Implement update logs. Travis | ||
| + | * Setupe infrastructure for Web service (milestone 3). Travis | ||
| + | |||
| + | ==== Gutenberg scraper ==== | ||
| + | * Fixed scraper and make it zimfarm compatible | ||
| + | |||
| + | ==== Zimfarm ==== | ||
| + | * Added support for gutenberg scraper | ||
| + | * rescheduled everything with new aliased format | ||
| + | |||
| + | ==== Others ==== | ||
| + | * Make decision on Categories & Tags approach Renaud, Matthieu, Emmanuel | ||
== Organization == | == Organization == | ||
| − | The hackathon | + | The hackathon took place at Stockholm University as part of [https://wikimania.wikimedia.org Wikimania 2019]. |
=== Agenda === | === Agenda === | ||
| Line 24: | Line 128: | ||
===Transportation=== | ===Transportation=== | ||
===Hosting=== | ===Hosting=== | ||
| + | |||
| + | We will be hosted at [https://www.theredboat.com/ The Red Boat] in Stockholm ([https://www.google.com/maps/search/Södermälarstrand+Kajplats+10118+20+Stockholm/@59.3209064,18.0462453,17z map]). | ||
| + | |||
| + | Room are reserved: | ||
| + | * Abdul + Siddharth: From Thursday 8th to Tuesday 20th | ||
| + | * Matthieu: From Friday 9th to Friday 16th | ||
| + | * Emmanuel + Renaud, Isaac + Joe, Florent: From Friday 9th to Sunday 18th | ||
| + | |||
| + | Julian and Travis have external accommodations. | ||
| + | |||
=== Budget === | === Budget === | ||
| − | + | * Hosting: CHF 3'651.74 | |
| + | * Transportation: CHF 4'749.23 | ||
| + | * Food: CHF 1'987.56 | ||
| + | * Varia (unplanned, incidentals, goodies, conference cost): CHF 2'254.- | ||
| + | ;Total: '''CHF 12'642.53''' | ||
| + | |||
== Invitees / Attendees == | == Invitees / Attendees == | ||
* [https://github.com/Kelson42 Emmanuel] | * [https://github.com/Kelson42 Emmanuel] | ||
* [https://github.com/mgautierfr Matthieu] | * [https://github.com/mgautierfr Matthieu] | ||
* [https://github.com/rgaudin Renaud] | * [https://github.com/rgaudin Renaud] | ||
| − | |||
* [https://github.com/mhutti1 Isaac] | * [https://github.com/mhutti1 Isaac] | ||
| − | |||
* [https://github.com/abdulwd Abdul] | * [https://github.com/abdulwd Abdul] | ||
| − | * [https://github.com/ | + | * [https://github.com/siddharth2010 Siddharth] |
* [https://github.com/isnit0 Joe] | * [https://github.com/isnit0 Joe] | ||
| − | * [https://github.com | + | * [https://github.com/audiodude Travis] |
* [https://github.com/julianharty Julian] | * [https://github.com/julianharty Julian] | ||
| − | * [https://github.com/ | + | * [https://github.com/florentk Florent] |
| − | |||
| − | |||
| − | |||
| − | [[Category: | + | [[Category:Hackathon|W]] |
Latest revision as of 15:32, 3 September 2019
This page summarizes the action plan for the Kiwix Wikimania 2019 Hackathon.
Goals
Kiwix
Build
- Explain all the build system and its integration in the CI.
Lib
- Discuss (and hopefully decide) about tag/name management of zim file (https://github.com/kiwix/kiwix-lib/issues/131 and associated issues)
- Discuss about the server integration (if not done since)
Tools
- Look the best tool to analyze activity of all kiwix's Github projects (try to use GrimoireLab) and deploy a first version to test this - Florent
Desktop
Android
- See about the wrapper about the Library/Server of kiwix-lib.
Hotspot
- Make Kiwix-Hotspot running in RPI0 - Renaud
- Prepare presentation - Kelson, Renaud
MWOffliner
WP1 Bot
- Make a Docker image of the update part - Kelson
- Try to convert Update -> Releases to Python - Travis
- Set up travis-ci for the bot's github project - Travis
- Develop scheduling infrastructure for update part - Travis
- Convert upload part to Python with tests - Travis
- Refactor/reorganize project code to maintain logical organization - Travis
- Develop plan for web app - Travis
Other scrapers
- Planning of FO requests - Kelson, Renaud, Joe
- Fixing many problems in zip2zim - Joe
- Revamping phets scraper - Joe
Other
- Cleaning of Kiwix wiki - Kelson
- Managing the logistic of the hackathon - Kelson
Achievements
Kiwix
Android
- Worked on integrating kiwix-lib changes to allow stack overflow ZIMs and others with redirects to work on Android. - Isaac
- Fixed various bugs relating to the bottom navigation bar. - Isaac
- Worked out a new way to build with local kiwix-lib. - Isaac
- Fixed various bugs with the home screen. - Siddharth
- Worked on integrating the wifi hotspot of kiwix-android with kiwix-serve. - Abdul and Siddharth
- Fixed bugs in the tab switcher. - Abdul
- Enabled Kiwix+LeakCanary to be tested using Android Robot (which already exposed some crashes in the app) - Julian
- Documented, tested and slightly improved the Custom Android App build process - Julian
- Started fixing some flaws in the app when it's run as a custom app. - Julian
- Found several bugs with the Notes feature - in custom apps it crashed, and on older devices the text is almost white on a white background - Julian
- Worked with Abdul to find the causes for 2 of the top crashes as reported in Android Vitals - Julian
- Did some testing using Android Monkey, too early to provide definitive analysis, I'm continuing to test using Monkey post hackathon and will report findings separately - Julian
- Prepare 2.5.3 release - Abdul
- Fixed duplicate zims on home screen and device tab - Abdul
- Fixed position of home screen download button for RTL layouts - Abdul
libkiwix
- Make a decision about how to test cutting-edge libkiwix against kiwix-android, here and here Kelson (talk)
- Introduce HTTP service in the libkiwix (from kiwix-serve) - Matthieu
Kiwix-hotspot
- Fixed homepage icon for some ZIMs (work around zimwriterfs bug) – renaud
- Released version 2.0.8
- Updated Support Files wiki (build instructions for pi0-4 on with newer kernel) - renaud
- Switched to latest raspbian image (buster) - renaud
- Added support for Pi4
Maintenance
- Fixing zimquarantine - Florent
Openzim
libzim
- Discuss a few possible paths of optimization of the libzim and open tickets: here and here. Kelson (talk)
- Introduce support of new Flavour Metadata Kelson (talk)
zim-tools
zimwriterfs
- Fix regression related to favicon redirect writing Kelson (talk), renaud
- Introduce flavour Medata writing Kelson (talk), renaud
- Introduced scraper parameter to set metadata - renaud
- Rewrite the Dockerfile to allowing copying the src code directly from the repository Kelson (talk), renaud
- New release 1.3.5 Kelson (talk)
- Fixed how relative URLs in CSS are rewritten
MWoffliner
- Fix problem with Wrong description Kelson (talk)
- Add support of new ZIM metadata Flavour Kelson (talk)
WP1 engine
- Many changes to make the project more newcomers friendly. Emmanuel
- Setup Docker & deploy it. Travis, Emmanuel
- Setup Codefactor & Codecoverage, Emmanuel
- Setup CI with automated testing, Travis
- Wrote upload tables code, with tests. Travis
- Setup rq job queueing/worker code for update/upload portion. Travis
- Implement update logs. Travis
- Setupe infrastructure for Web service (milestone 3). Travis
Gutenberg scraper
- Fixed scraper and make it zimfarm compatible
Zimfarm
- Added support for gutenberg scraper
- rescheduled everything with new aliased format
Others
- Make decision on Categories & Tags approach Renaud, Matthieu, Emmanuel
Organization
The hackathon took place at Stockholm University as part of Wikimania 2019.
Agenda
Arrival: Friday 9 August afternoon
Departure: Sunday 18 August.
Transportation
Hosting
We will be hosted at The Red Boat in Stockholm (map).
Room are reserved:
- Abdul + Siddharth: From Thursday 8th to Tuesday 20th
- Matthieu: From Friday 9th to Friday 16th
- Emmanuel + Renaud, Isaac + Joe, Florent: From Friday 9th to Sunday 18th
Julian and Travis have external accommodations.
Budget
- Hosting: CHF 3'651.74
- Transportation: CHF 4'749.23
- Food: CHF 1'987.56
- Varia (unplanned, incidentals, goodies, conference cost): CHF 2'254.-
- Total
- CHF 12'642.53
