Tell us your story
Tell us your story
How has offline Wikipedia affected you? The Wikimedia Foundation (the non-profit that supports Wikipedia) is looking for personal, diverse and inspiring stories about how offline Wikipedia affects the world. If you have a personal story that you would like to share, please contact: stories@kiwix.org. Thank you!

Difference between revisions of "Hackathon Wikimania 2019"

From Kiwix
Jump to: navigation, search
(Openzim)
(Budget)
 
(30 intermediate revisions by 9 users not shown)
Line 9: Line 9:
  
 
====Lib====
 
====Lib====
 +
* Discuss (and hopefully decide) about tag/name management of zim file (https://github.com/kiwix/kiwix-lib/issues/131 and associated issues)
 +
 
* Discuss about the server integration (if not done since)
 
* Discuss about the server integration (if not done since)
  
 
====Tools====
 
====Tools====
* Look the best tool to analyze activity of all kiwix's Github projects (try to use [https://cauldron.io/ Cauldron]) and deploy a first version to test this - Florent
+
* Look the best tool to analyze activity of all kiwix's Github projects (try to use [https://chaoss.github.io/grimoirelab/ GrimoireLab]) and deploy a first version to test this - Florent
  
 
====Desktop====
 
====Desktop====
 
====Android====
 
====Android====
 
* See about the wrapper about the Library/Server of kiwix-lib.
 
* See about the wrapper about the Library/Server of kiwix-lib.
 
  
 
====Hotspot====
 
====Hotspot====
 
* Make Kiwix-Hotspot running in RPI0 - Renaud
 
* Make Kiwix-Hotspot running in RPI0 - Renaud
 
* Prepare presentation - Kelson, Renaud
 
* Prepare presentation - Kelson, Renaud
 
===Openzim===
 
 
* Discuss a few possible paths of optimization of the libzim and open tickets: [https://github.com/openzim/libzim/issues/264 here] and [https://github.com/openzim/libzim/issues/264 here]. [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
 
* Discuss (and hopefully decide) about tag/name management of zim file (https://github.com/kiwix/kiwix-lib/issues/131 and associated issues)
 
  
 
====MWOffliner====
 
====MWOffliner====
 
====WP1 Bot====
 
====WP1 Bot====
 
* Make a Docker image of the update part - Kelson
 
* Make a Docker image of the update part - Kelson
 +
* Try to convert Update -> Releases to Python - Travis
 +
* Set up travis-ci for the bot's github project - Travis
 
* Develop scheduling infrastructure for update part - Travis
 
* Develop scheduling infrastructure for update part - Travis
 
* Convert upload part to Python with tests - Travis
 
* Convert upload part to Python with tests - Travis
Line 46: Line 44:
  
 
==Achievements==
 
==Achievements==
''To be completed...''
+
 
 +
===Kiwix===
 +
 
 +
==== Android ====
 +
* Worked on integrating kiwix-lib changes to allow stack overflow ZIMs and others with redirects to work on Android. - Isaac
 +
* Fixed various bugs relating to the bottom navigation bar. - Isaac
 +
* Worked out a new way to build with local kiwix-lib. - Isaac
 +
* Fixed various bugs with the home screen. - [[User:Siddharth2010|Siddharth]]
 +
* Worked on integrating the wifi hotspot of kiwix-android with kiwix-serve. - Abdul and [[User:Siddharth2010|Siddharth]]
 +
* Fixed bugs in the tab switcher. - Abdul
 +
* Enabled Kiwix+LeakCanary to be tested using Android Robot (which already exposed some crashes in the app) - Julian
 +
* Documented, tested and slightly improved the Custom Android App build process - Julian
 +
* Started fixing some flaws in the app when it's run as a custom app. - Julian
 +
* Found several bugs with the Notes feature - in custom apps it crashed, and on older devices the text is almost white on a white background - Julian
 +
* Worked with Abdul to find the causes for 2 of the top crashes as reported in Android Vitals - Julian
 +
* Did some testing using Android Monkey, too early to provide definitive analysis, I'm continuing to test using Monkey post hackathon and will report findings separately - Julian
 +
* Prepare 2.5.3 release - Abdul
 +
* Fixed duplicate zims on home screen and device tab - Abdul
 +
* Fixed position of home screen download button for RTL layouts - Abdul
 +
 
 +
==== libkiwix ====
 +
* Make a decision about how to test cutting-edge libkiwix against kiwix-android, [https://github.com/kiwix/kiwix-build/issues/345 here] and [https://github.com/kiwix/kiwix-build/issues/346 here] [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
 +
* Introduce HTTP service in the libkiwix (from kiwix-serve) - Matthieu
 +
 
 +
==== Kiwix-hotspot ====
 +
* Fixed homepage icon for some ZIMs (work around zimwriterfs bug) – renaud
 +
* Released version 2.0.8
 +
* Updated Support Files wiki (build instructions for pi0-4 on with newer kernel) - renaud
 +
* Switched to latest raspbian image (buster) - renaud
 +
* Added support for Pi4
 +
 
 +
==== Maintenance ====
 +
* Fixing zimquarantine - Florent
 +
 
 +
===Openzim===
 +
 
 +
==== libzim ====
 +
* Discuss a few possible paths of optimization of the libzim and open tickets: [https://github.com/openzim/libzim/issues/264 here] and [https://github.com/openzim/libzim/issues/264 here]. [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
 +
* Introduce support of new Flavour Metadata [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
 +
 
 +
==== zim-tools ====
 +
* Release of version 1.0.4 [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]) 14:11, 10 August 2019 (UTC)
 +
 
 +
==== zimwriterfs ====
 +
* Fix regression related to favicon redirect writing [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]), renaud
 +
* Introduce flavour Medata writing [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]), renaud
 +
* Introduced scraper parameter to set metadata - renaud
 +
* Rewrite the Dockerfile to allowing copying the src code directly from the repository [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]]), renaud
 +
* New release 1.3.5 [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
 +
* Fixed how relative URLs in CSS are rewritten
 +
 
 +
==== MWoffliner ====
 +
* Fix problem with Wrong description [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
 +
* Add support of new ZIM metadata Flavour [[User:Kelson|Kelson]] ([[User talk:Kelson|talk]])
 +
 
 +
==== WP1 engine ====
 +
* Many changes to make the project more newcomers friendly. Emmanuel
 +
* Setup Docker & deploy it. Travis, Emmanuel
 +
* Setup Codefactor & Codecoverage, Emmanuel
 +
* Setup CI with automated testing, Travis
 +
* Wrote upload tables code, with tests. Travis
 +
* Setup rq job queueing/worker code for update/upload portion. Travis
 +
* Implement update logs. Travis
 +
* Setupe infrastructure for Web service (milestone 3). Travis
 +
 
 +
==== Gutenberg scraper ====
 +
* Fixed scraper and make it zimfarm compatible
 +
 
 +
==== Zimfarm ====
 +
* Added support for gutenberg scraper
 +
* rescheduled everything with new aliased format
 +
 
 +
==== Others ====
 +
* Make decision on Categories & Tags approach Renaud, Matthieu, Emmanuel
  
 
== Organization ==
 
== Organization ==
The hackathon will take place at Stockholm University as part of [https://wikimania.wikimedia.org Wikimania 2019].
+
The hackathon took place at Stockholm University as part of [https://wikimania.wikimedia.org Wikimania 2019].
  
 
=== Agenda ===
 
=== Agenda ===
Line 68: Line 139:
  
 
=== Budget ===
 
=== Budget ===
* Hosting: CHF 6'500.- <!---- ca. 35'000 SEK with 1 CHF = 10 SEK + 2750 USD for Wikimania--->
+
* Hosting: CHF 3'651.74
* Transportation: CHF 6'000.- <!---ca. 5'000 CHF (plane) + 9'000 SEK (Bus, incl. airport) ---->
+
* Transportation: CHF 4'749.23
* Food: CHF 5'000.- <!---- 11(Fri) + 11x3x3 (Sat-Mon) + 11x2x2 (Tue-Thu) + 8x2x3 (Fri-Sun)= 159 meals with 1 meal = 30 CHF avge --->
+
* Food: CHF 1'987.56
* Varia (unplanned, incidentals, goodies) 20%: CHF 2'500.-
+
* Varia (unplanned, incidentals, goodies, conference cost): CHF 2'254.-
;Total: '''CHF 20'000.-'''
+
;Total: '''CHF 12'642.53'''
  
 
== Invitees / Attendees ==
 
== Invitees / Attendees ==

Latest revision as of 15:32, 3 September 2019

This page summarizes the action plan for the Kiwix Wikimania 2019 Hackathon.

Goals

Kiwix

Build

  • Explain all the build system and its integration in the CI.

Lib

  • Discuss about the server integration (if not done since)

Tools

  • Look the best tool to analyze activity of all kiwix's Github projects (try to use GrimoireLab) and deploy a first version to test this - Florent

Desktop

Android

  • See about the wrapper about the Library/Server of kiwix-lib.

Hotspot

  • Make Kiwix-Hotspot running in RPI0 - Renaud
  • Prepare presentation - Kelson, Renaud

MWOffliner

WP1 Bot

  • Make a Docker image of the update part - Kelson
  • Try to convert Update -> Releases to Python - Travis
  • Set up travis-ci for the bot's github project - Travis
  • Develop scheduling infrastructure for update part - Travis
  • Convert upload part to Python with tests - Travis
    • Refactor/reorganize project code to maintain logical organization - Travis
  • Develop plan for web app - Travis

Other scrapers

  • Planning of FO requests - Kelson, Renaud, Joe
  • Fixing many problems in zip2zim - Joe
  • Revamping phets scraper - Joe

Other

  • Cleaning of Kiwix wiki - Kelson
  • Managing the logistic of the hackathon - Kelson

Achievements

Kiwix

Android

  • Worked on integrating kiwix-lib changes to allow stack overflow ZIMs and others with redirects to work on Android. - Isaac
  • Fixed various bugs relating to the bottom navigation bar. - Isaac
  • Worked out a new way to build with local kiwix-lib. - Isaac
  • Fixed various bugs with the home screen. - Siddharth
  • Worked on integrating the wifi hotspot of kiwix-android with kiwix-serve. - Abdul and Siddharth
  • Fixed bugs in the tab switcher. - Abdul
  • Enabled Kiwix+LeakCanary to be tested using Android Robot (which already exposed some crashes in the app) - Julian
  • Documented, tested and slightly improved the Custom Android App build process - Julian
  • Started fixing some flaws in the app when it's run as a custom app. - Julian
  • Found several bugs with the Notes feature - in custom apps it crashed, and on older devices the text is almost white on a white background - Julian
  • Worked with Abdul to find the causes for 2 of the top crashes as reported in Android Vitals - Julian
  • Did some testing using Android Monkey, too early to provide definitive analysis, I'm continuing to test using Monkey post hackathon and will report findings separately - Julian
  • Prepare 2.5.3 release - Abdul
  • Fixed duplicate zims on home screen and device tab - Abdul
  • Fixed position of home screen download button for RTL layouts - Abdul

libkiwix

  • Make a decision about how to test cutting-edge libkiwix against kiwix-android, here and here Kelson (talk)
  • Introduce HTTP service in the libkiwix (from kiwix-serve) - Matthieu

Kiwix-hotspot

  • Fixed homepage icon for some ZIMs (work around zimwriterfs bug) – renaud
  • Released version 2.0.8
  • Updated Support Files wiki (build instructions for pi0-4 on with newer kernel) - renaud
  • Switched to latest raspbian image (buster) - renaud
  • Added support for Pi4

Maintenance

  • Fixing zimquarantine - Florent

Openzim

libzim

  • Discuss a few possible paths of optimization of the libzim and open tickets: here and here. Kelson (talk)
  • Introduce support of new Flavour Metadata Kelson (talk)

zim-tools

  • Release of version 1.0.4 Kelson (talk) 14:11, 10 August 2019 (UTC)

zimwriterfs

  • Fix regression related to favicon redirect writing Kelson (talk), renaud
  • Introduce flavour Medata writing Kelson (talk), renaud
  • Introduced scraper parameter to set metadata - renaud
  • Rewrite the Dockerfile to allowing copying the src code directly from the repository Kelson (talk), renaud
  • New release 1.3.5 Kelson (talk)
  • Fixed how relative URLs in CSS are rewritten

MWoffliner

  • Fix problem with Wrong description Kelson (talk)
  • Add support of new ZIM metadata Flavour Kelson (talk)

WP1 engine

  • Many changes to make the project more newcomers friendly. Emmanuel
  • Setup Docker & deploy it. Travis, Emmanuel
  • Setup Codefactor & Codecoverage, Emmanuel
  • Setup CI with automated testing, Travis
  • Wrote upload tables code, with tests. Travis
  • Setup rq job queueing/worker code for update/upload portion. Travis
  • Implement update logs. Travis
  • Setupe infrastructure for Web service (milestone 3). Travis

Gutenberg scraper

  • Fixed scraper and make it zimfarm compatible

Zimfarm

  • Added support for gutenberg scraper
  • rescheduled everything with new aliased format

Others

  • Make decision on Categories & Tags approach Renaud, Matthieu, Emmanuel

Organization

The hackathon took place at Stockholm University as part of Wikimania 2019.

Agenda

Arrival: Friday 9 August afternoon

Departure: Sunday 18 August.

Transportation

Hosting

We will be hosted at The Red Boat in Stockholm (map).

Room are reserved:

  • Abdul + Siddharth: From Thursday 8th to Tuesday 20th
  • Matthieu: From Friday 9th to Friday 16th
  • Emmanuel + Renaud, Isaac + Joe, Florent: From Friday 9th to Sunday 18th

Julian and Travis have external accommodations.

Budget

  • Hosting: CHF 3'651.74
  • Transportation: CHF 4'749.23
  • Food: CHF 1'987.56
  • Varia (unplanned, incidentals, goodies, conference cost): CHF 2'254.-
Total
CHF 12'642.53

Invitees / Attendees