Zurich 2018

This page summarizes the action plan for the Kiwix Google Zürich Hackathon 2018.

Goals

Mossroy

In Kiwix-js:

• Stabilize and optimize the ServiceWorker mode, to try to make it the default mode (on platforms that support it). This mode should be much more sustainable in the long term. Some help would be welcome on how to use the browser cache in this context
• Improve performance by using WebAssembly (on platforms that support it) for the bzip2 decompression. Preliminary tests let us hope around 30% performance improvement.
• Investigate on compiling the libzim to WebAssembly. If it's technically possible, it would allow to benefit from all the features and optimizations of this library, without the need of re-implementing them. If the people from Google know this technology, it might be a topic where they would be helpful.

Isaac

In Kiwix-Android:

• Work on 3.0
• Download improvements (improve stability)
• Search Improvements / Discussions

Julian

In Kiwix-Android:

• Implement test code coverage report in Travis see https://github.com/kiwix/kiwix-android/issues/836
• Help to reduce the number of crashes occurring for our users. Isaac agree the download manager code is the epicentre of many of the problems and he's planning to revise it during the hackathon. I'm happy to help with testing the changes.

Emmanuel

• Conclude preparation of projects for full automation and mwoffliner improvements
• Fix https://github.com/resin-io/etcher/issues/2274 if possible
• Migrate our wiki to Docker

Renaud

• Release Kiwix-plug installer 2.0.
• Setup master builds more frequently and tie it to nigtly builds (#171).
• Replace dependencies that are too exotic and problematic (pygobjects to PyQT?).
• Revamp UI to fix UI-speed issues.
• Update dependencies to latest stable.
• Update travis/appveyor to latest stable image.
• Draft pibox Documentation.
• Harmonize coding style (black?).
• Prepare pibox for localization (#43).
• Have an alpha cardshop
• Prepare the next project

Joe Reeve (ISNIT0)

• Become more familiar with the MWOffliner codebase
• Overhaul PhET app UI/UX
• Assist with Kiwix-Android testing/development

Magnus

• Get familiarised with the OpenZim codebase
• Successfully compile all OpenZim tools
• Resolve various bugs, build issues, and crashes

Achievements

Kiwix build

• Move on to Xapian 1.4.7

MWoffliner

• Release of MWoffliner 1.5 with following improvements
• Creation of additional new ZIM files of https://wikiwel.com/, Wikipedia Ray Charles, https://artofproblemsolving.com, KLexikon, http://awoiaf.westeros.org
• Various fixes in pre-existing ZIM file scrapping (around 20 tickets closed)
• Create "zim-requests" dedicated repo for new scrapping requests and move all ZIM creation tickets to it.
• Updated Documentation
• Added security and code quality checkers

Kiwix JS

• Release of Kiwix JS 2.3.1, Kiwix JS Windows and corresponding custom apps for Windows
• Big performance improvement on decompression of contents : it's now more than twice faster -> Release of 2.4.0
• The ServiceWorker mode is now much more stable
• The libzim has been compiled in WebAssembly. A prototype that uses it from javascript gives hope that libzim (or kiwix-lib) might be used by kiwix-js in the future (instead of a separate javascript implementation)

Kiwix Android

• Update of all Wikimed and Wikivoyage and others custom apps
• Generated Code Coverage for the automated tests locally, still some issues to resolve on some devices and there may be a clash with debugging the app when code coverage is enabled. Detail available https://github.com/kiwix/kiwix-android/issues/836 The changes have now been accepted and merged. We have asked bitbar if they can help us run coverage and obtain the results from their cloud testing service.
• Worked on revamped download mechanism on branch: https://github.com/kiwix/kiwix-android/tree/mhutti1/better-downloads . Adding improvements to download stability.

Kiwix Hotspot

• #228 (SSH timeout) - merged fix attempt (uncomfirmed)
• #231: OPEN, CLOSED - build_dir UI issue
• #232: CLOSE - setup password issue
• #230: OPEN, CLOSED - password issue
• #229: OPEN, CLOSED - Verify `previous_loop_mode`
• New WikiFundi dumps: updated content and new password rules in config
• released 2.0-rc9
• CLOSED openzim/wikifundi/issues/89
• CLOSED openzim/wikifundi/issues/90
• Recreated Wikifundi dumps to include fixed config
• CLOSED #232
• CLOSED #236 - Redirect kiwix-serve home to ideascube
• More verbose and safer `test_mount_procedure`
• released 2.0-rc10
• #234: CLOSED - File size entry as a selection
• rebranded as Kiwix Hotspot
• Better SSH to QEMU connections
• Fixed UAC regression on Windows (dropped metadata)
• released 2.0-rc11
• #239: OPEN, CLOSED - Renamed macOS App
• #240: OPEN, CLOSED - Use safer image sizes
• #241: OPEN - cache cleanup tool
• WIP: Attempt to get it work with RPI0, so far seems possible.

Cardshop

• Deployed docker containers on *demo server*:
  • rabbitMQ with SSL
  • mongo
  • basic scheduler (authentications, user mgmt)
  • warehouse
  • phony monitor

Zimwriterfs

• Added version arguments with meson build system
• Fixed dead symlink reference error
• Added macOS building to Travis

Lib Zim

• Refactored internal file/class name for (zim)creator
• Removed subprojects directory and updated README

Node Lib Zim

• Added security and code quality checkers
• Updated to support Node 10 and 8

Wikifundi

• Sort out last tickets and ready to publish WikiFundi 2.0

Others

• Decommissioning of old http://input.kiwix.org (replaced by the contact formular on http://www.kiwix.org)
• Creating new repo zim-requests to gather all ZIM requests and clean the "MWoffliner" one (move ticket from one to the other)
• Bootstrap work for version 2.0 of https://github.com/openzim/wikimedia_wp1_bot
• New recompiled ZIM file (with FT index) of "Wikipedia for schools"
• Preparing requirements document for two contracts Q3/Q4
• Dockerize and redeploy Kiwix Watcherbot in the Cloud.
• Bootstrap work for Google Code-in 2018

Method

7 developers meet during a week to make a few steps forward.

Place

The hackathon will take place in Zürich, Switzerland: 3 days @Google offices, the rest at the residency.

Agenda

• September 16th afternoon: arrivals (suggested arrival time 5PM)
• September 17th -> 23th: Kiwix hackathon (17th -> 19th @Google)
• September 23th afternoon: departures (suggested departure time 3PM)

People

• Emmanuel (in the field)
• Isaac (Sunday - Sunday /w one day break in the middle)
• Joe (Monday - Friday)
• Julian (Monday - Friday)
• Mossroy (Sunday - Sunday)
• Renaud (Saturday - Sunday)
• Magnus (Sunday - Thursday)

• Dattaz
• Rashiq
• Matthieu
• Guillaume

Costs

• Accommodation: ~4500 CHF
• Travels: ~700 CHF (local)
• F&B: ~ 2500 CHF

Total

approx. 7'700 CHF