Tell us your story
Tell us your story
How has offline Wikipedia affected you? The Wikimedia Foundation (the non-profit that supports Wikipedia) is looking for personal, diverse and inspiring stories about how offline Wikipedia affects the world. If you have a personal story that you would like to share, please contact: stories@kiwix.org. Thank you!

Difference between revisions of "Hackathon Wikimania 2019"

From Kiwix
Jump to navigation Jump to search
(Update WP1 Bot with plans for Travis, and update link in people section to avoid 404)
 
(34 intermediate revisions by 10 users not shown)
Line 4: Line 4:


===Kiwix===
===Kiwix===
====Build====
* Explain all the build system and its integration in the CI.
====Lib====
* Discuss (and hopefully decide) about tag/name management of zim file (https://github.com/kiwix/kiwix-lib/issues/131 and associated issues)
* Discuss about the server integration (if not done since)
====Tools====
====Tools====
* Look the best tool to analyze activity of all kiwix's Github projects (try to use [https://chaoss.github.io/grimoirelab/ GrimoireLab]) and deploy a first version to test this - Florent
====Desktop====
====Desktop====
====Android====
====Android====
* See about the wrapper about the Library/Server of kiwix-lib.


====Hotspot====
====Hotspot====
Line 12: Line 24:
* Prepare presentation - Kelson, Renaud
* Prepare presentation - Kelson, Renaud


===Openzim===
====MWOffliner====
====MWOffliner====
====WP1 Bot====
====WP1 Bot====
* Make a Docker image of the update part - Kelson
* Make a Docker image of the update part - Kelson
* Try to convert Update -> Releases to Python - Travis
* Set up travis-ci for the bot's github project - Travis
* Develop scheduling infrastructure for update part - Travis
* Develop scheduling infrastructure for update part - Travis
* Convert upload part to Python with tests - Travis
* Convert upload part to Python with tests - Travis
Line 31: Line 44:


==Achievements==
==Achievements==
''To be completed...''
 
===Kiwix===
 
==== Android ====
* Worked on integrating kiwix-lib changes to allow stack overflow ZIMs and others with redirects to work on Android. - Isaac
* Fixed various bugs relating to the bottom navigation bar. - Isaac
* Worked out a new way to build with local kiwix-lib. - Isaac
* Fixed various bugs with the home screen. - [[User:Siddharth2010|Siddharth]]
* Worked on integrating the wifi hotspot of kiwix-android with kiwix-serve. - Abdul and [[User:Siddharth2010|Siddharth]]
* Fixed bugs in the tab switcher. - Abdul
* Enabled Kiwix+LeakCanary to be tested using Android Robot (which already exposed some crashes in the app) - Julian
* Documented, tested and slightly improved the Custom Android App build process - Julian
* Started fixing some flaws in the app when it's run as a custom app. - Julian
* Found several bugs with the Notes feature - in custom apps it crashed, and on older devices the text is almost white on a white background - Julian
* Worked with Abdul to find the causes for 2 of the top crashes as reported in Android Vitals - Julian
* Did some testing using Android Monkey, too early to provide definitive analysis, I'm continuing to test using Monkey post hackathon and will report findings separately - Julian
* Prepare 2.5.3 release - Abdul
* Fixed duplicate zims on home screen and device tab - Abdul
* Fixed position of home screen download button for RTL layouts - Abdul
 
==== libkiwix ====
* Make a decision about how to test cutting-edge libkiwix against kiwix-android, [https://github.com/kiwix/kiwix-build/issues/345 here] and [https://github.com/kiwix/kiwix-build/issues/346 here] [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
* Introduce HTTP service in the libkiwix (from kiwix-serve) - Matthieu
 
==== Kiwix-hotspot ====
* Fixed homepage icon for some ZIMs (work around zimwriterfs bug) – renaud
* Released version 2.0.8
* Updated Support Files wiki (build instructions for pi0-4 on with newer kernel) - renaud
* Switched to latest raspbian image (buster) - renaud
* Added support for Pi4
 
==== Maintenance ====
* Fixing zimquarantine - Florent
 
===Openzim===
 
==== libzim ====
* Discuss a few possible paths of optimization of the libzim and open tickets: [https://github.com/openzim/libzim/issues/264 here] and [https://github.com/openzim/libzim/issues/264 here]. [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
* Introduce support of new Flavour Metadata [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
 
==== zim-tools ====
* Release of version 1.0.4 [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]) 14:11, 10 August 2019 (UTC)
 
==== zimwriterfs ====
* Fix regression related to favicon redirect writing [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]), renaud
* Introduce flavour Medata writing [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]), renaud
* Introduced scraper parameter to set metadata - renaud
* Rewrite the Dockerfile to allowing copying the src code directly from the repository [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]), renaud
* New release 1.3.5 [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
* Fixed how relative URLs in CSS are rewritten
 
==== MWoffliner ====
* Fix problem with Wrong description [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
* Add support of new ZIM metadata Flavour [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
 
==== WP1 engine ====
* Many changes to make the project more newcomers friendly. Emmanuel
* Setup Docker & deploy it. Travis, Emmanuel
* Setup Codefactor & Codecoverage, Emmanuel
* Setup CI with automated testing, Travis
* Wrote upload tables code, with tests. Travis
* Setup rq job queueing/worker code for update/upload portion. Travis
* Implement update logs. Travis
* Setupe infrastructure for Web service (milestone 3). Travis
 
==== Gutenberg scraper ====
* Fixed scraper and make it zimfarm compatible
 
==== Zimfarm ====
* Added support for gutenberg scraper
* rescheduled everything with new aliased format
 
==== Others ====
* Make decision on Categories & Tags approach Renaud, Matthieu, Emmanuel


== Organization ==
== Organization ==
The hackathon will take place at Stockholm University as part of [https://wikimania.wikimedia.org Wikimania 2019].
The hackathon took place at Stockholm University as part of [https://wikimania.wikimedia.org Wikimania 2019].


=== Agenda ===
=== Agenda ===
Line 53: Line 139:


=== Budget ===
=== Budget ===
* Hosting: CHF 6'500.- <!---- ca. 35'000 SEK with 1 CHF = 10 SEK + 2750 USD for Wikimania--->
* Hosting: CHF 3'651.74
* Transportation: CHF 6'000.- <!---ca. 5'000 CHF (plane) + 9'000 SEK (Bus, incl. airport) ---->
* Transportation: CHF 4'749.23
* Food: CHF 5'000.- <!---- 11(Fri) + 11x3x3 (Sat-Mon) + 11x2x2 (Tue-Thu) + 8x2x3 (Fri-Sun)= 159 meals with 1 meal = 30 CHF avge --->
* Food: CHF 1'987.56
* Varia (unplanned, incidentals, goodies) 20%: CHF 2'500.-
* Varia (unplanned, incidentals, goodies, conference cost): CHF 2'254.-
;Total: '''CHF 20'000.-'''
;Total: '''CHF 12'642.53'''


== Invitees / Attendees ==
== Invitees / Attendees ==

Latest revision as of 15:32, 3 September 2019

This page summarizes the action plan for the Kiwix Wikimania 2019 Hackathon.

Goals

Kiwix

Build

  • Explain all the build system and its integration in the CI.

Lib

  • Discuss about the server integration (if not done since)

Tools

  • Look the best tool to analyze activity of all kiwix's Github projects (try to use GrimoireLab) and deploy a first version to test this - Florent

Desktop

Android

  • See about the wrapper about the Library/Server of kiwix-lib.

Hotspot

  • Make Kiwix-Hotspot running in RPI0 - Renaud
  • Prepare presentation - Kelson, Renaud

MWOffliner

WP1 Bot

  • Make a Docker image of the update part - Kelson
  • Try to convert Update -> Releases to Python - Travis
  • Set up travis-ci for the bot's github project - Travis
  • Develop scheduling infrastructure for update part - Travis
  • Convert upload part to Python with tests - Travis
    • Refactor/reorganize project code to maintain logical organization - Travis
  • Develop plan for web app - Travis

Other scrapers

  • Planning of FO requests - Kelson, Renaud, Joe
  • Fixing many problems in zip2zim - Joe
  • Revamping phets scraper - Joe

Other

  • Cleaning of Kiwix wiki - Kelson
  • Managing the logistic of the hackathon - Kelson

Achievements

Kiwix

Android

  • Worked on integrating kiwix-lib changes to allow stack overflow ZIMs and others with redirects to work on Android. - Isaac
  • Fixed various bugs relating to the bottom navigation bar. - Isaac
  • Worked out a new way to build with local kiwix-lib. - Isaac
  • Fixed various bugs with the home screen. - Siddharth
  • Worked on integrating the wifi hotspot of kiwix-android with kiwix-serve. - Abdul and Siddharth
  • Fixed bugs in the tab switcher. - Abdul
  • Enabled Kiwix+LeakCanary to be tested using Android Robot (which already exposed some crashes in the app) - Julian
  • Documented, tested and slightly improved the Custom Android App build process - Julian
  • Started fixing some flaws in the app when it's run as a custom app. - Julian
  • Found several bugs with the Notes feature - in custom apps it crashed, and on older devices the text is almost white on a white background - Julian
  • Worked with Abdul to find the causes for 2 of the top crashes as reported in Android Vitals - Julian
  • Did some testing using Android Monkey, too early to provide definitive analysis, I'm continuing to test using Monkey post hackathon and will report findings separately - Julian
  • Prepare 2.5.3 release - Abdul
  • Fixed duplicate zims on home screen and device tab - Abdul
  • Fixed position of home screen download button for RTL layouts - Abdul

libkiwix

  • Make a decision about how to test cutting-edge libkiwix against kiwix-android, here and here Kelson (talk)
  • Introduce HTTP service in the libkiwix (from kiwix-serve) - Matthieu

Kiwix-hotspot

  • Fixed homepage icon for some ZIMs (work around zimwriterfs bug) – renaud
  • Released version 2.0.8
  • Updated Support Files wiki (build instructions for pi0-4 on with newer kernel) - renaud
  • Switched to latest raspbian image (buster) - renaud
  • Added support for Pi4

Maintenance

  • Fixing zimquarantine - Florent

Openzim

libzim

  • Discuss a few possible paths of optimization of the libzim and open tickets: here and here. Kelson (talk)
  • Introduce support of new Flavour Metadata Kelson (talk)

zim-tools

  • Release of version 1.0.4 Kelson (talk) 14:11, 10 August 2019 (UTC)

zimwriterfs

  • Fix regression related to favicon redirect writing Kelson (talk), renaud
  • Introduce flavour Medata writing Kelson (talk), renaud
  • Introduced scraper parameter to set metadata - renaud
  • Rewrite the Dockerfile to allowing copying the src code directly from the repository Kelson (talk), renaud
  • New release 1.3.5 Kelson (talk)
  • Fixed how relative URLs in CSS are rewritten

MWoffliner

  • Fix problem with Wrong description Kelson (talk)
  • Add support of new ZIM metadata Flavour Kelson (talk)

WP1 engine

  • Many changes to make the project more newcomers friendly. Emmanuel
  • Setup Docker & deploy it. Travis, Emmanuel
  • Setup Codefactor & Codecoverage, Emmanuel
  • Setup CI with automated testing, Travis
  • Wrote upload tables code, with tests. Travis
  • Setup rq job queueing/worker code for update/upload portion. Travis
  • Implement update logs. Travis
  • Setupe infrastructure for Web service (milestone 3). Travis

Gutenberg scraper

  • Fixed scraper and make it zimfarm compatible

Zimfarm

  • Added support for gutenberg scraper
  • rescheduled everything with new aliased format

Others

  • Make decision on Categories & Tags approach Renaud, Matthieu, Emmanuel

Organization

The hackathon took place at Stockholm University as part of Wikimania 2019.

Agenda

Arrival: Friday 9 August afternoon

Departure: Sunday 18 August.

Transportation

Hosting

We will be hosted at The Red Boat in Stockholm (map).

Room are reserved:

  • Abdul + Siddharth: From Thursday 8th to Tuesday 20th
  • Matthieu: From Friday 9th to Friday 16th
  • Emmanuel + Renaud, Isaac + Joe, Florent: From Friday 9th to Sunday 18th

Julian and Travis have external accommodations.

Budget

  • Hosting: CHF 3'651.74
  • Transportation: CHF 4'749.23
  • Food: CHF 1'987.56
  • Varia (unplanned, incidentals, goodies, conference cost): CHF 2'254.-
Total
CHF 12'642.53

Invitees / Attendees