Difference between pages "Lyon 2017" and "Kiwix-serve"

From Kiwix
(Difference between pages)
Jump to navigation Jump to search
 
(Remove things about external index)
 
Line 1: Line 1:
This page summarizes the action plan for the '''Kiwix Spring Hackathon 2017'''.
{{translations}}
[[File:Kiwix_0.9_rc2_article_served_en_screenshot.png|right|thumb|<translate><!--T:1-->
Kiwix-serve, embedded in Kiwix, on a Microsoft Windows computer</translate>]]
[[File:Two_kiwix-plugs.jpg|right|thumb|<translate><!--T:2-->
With [[Kiwix-plug]], kiwix-serve runs on pretty small hardwares</translate>]]


== Goals ==
<translate><!--T:3-->
'''Kiwix-serve is a ZIM compatible Web server.''' With its help you can deliver ZIM file content over your network.</translate>


; Android
<translate><!--T:4-->
* Setup the Android automated testing up and make a proof of concept
Based on the well known HTTP protocol, kiwix-serve allows you to share the [[Special:MyLanguage/Wikipedia_in_all_languages|ZIM files available]] on your computer easily with others connected to your network. You simply need to start kiwix-serve and your content will be available for anybody through their own Web browser.</translate>
* Release a new version of Kiwix for Android (maybe a first version with multizim support?)
* Implement Wikimed Feedback solution
* Implement a solution to allow non-tech to launch/update Android custom apps
* More sophisticated bookmark
* Work on 2.3 release
* Consider possible ways to enable users to provide device-info such as storage locations to help us improve support for their devices.


; Scrappers (Python)
<translate><!--T:5-->
* Fix Gutenberg scrapper and generate new ZIM files
Kiwix-serve is a good solution if you have a LAN network and don't want to install Kiwix on all computers. It's a good alternative if you don't have a shared network disk from which everybody can start Kiwix.</translate>
* Fix KALite ZIM export (for a first release)
* Create a fully automatized solution for Stackexchange projects ZIM files


; Scrapper Mediawiki (Nodejs)
== <translate><!--T:6-->
* Implement Wikihow scrapper (non-parsoid Mediawiki scraper)
Features</translate> ==
* Scrape categories
<translate><!--T:7-->
* Support videos (if Parsoid allows it)
Kiwix-serve has not so many features like Kiwix, but provides the most important ones:
* Full text search engine
* Search suggestions
* Really small and efficient
* Compatible with almost all browsers
* Available on all platforms
* Available as command line executable
* Embedded in Kiwix UI
* Able to deal with one ZIM file or XML library files</translate>


; Phet
== <translate><!--T:8-->
* Release a new version of Phet Simulation with an improved UX
Get it</translate> ==
* Add category filtering
<translate><!--T:9-->
On Microsoft Windows, Apple Mac OS X and GNU/Linux, kiwix-serve is packaged and distributed with Kiwix. You simply need to [[Special:MyLanguage/Software|download the Kiwix installer corresponding to your Operating System]].</translate>


; HTML5 (Javascript)
<translate><!--T:10-->
* Launch the first pure HTML ZIM read browser extensions (FF+Chrome)
We newly provide kiwix-serve as part of the Kiwix tools and [https://download.kiwix.org/release/kiwix-tools/ pre-compiled binaries for many systems are available].</translate>


; Search
<translate><!--T:11-->
* Fix fulltext search/indexes problems
For other systems, you will have to [[compilation|download and compile kiwix-serve source code on your own]].</translate>


; ZIM building
== <translate><!--T:12-->
* Setup a ZIM building scheduler (with Celery?)
Use it</translate> ==
<translate><!--T:13-->
Kiwix-serve is directly available in the Kiwix menu "Tools > server", but for people wanting to use it from the command line, here is how it works.</translate>


; Ideascube builder
<translate><!--T:14-->
* Get a Qemu Raspbian image ready in Travis
Prerequisites:
* kiwix-serve binary, available in the Kiwix application directory
* A ZIM file or a library file (you can use the library.xml file available in your Kiwix data directory).</translate>


==== Apache ====
<translate><!--T:15-->
* Create a zim module for Apache (based on [http://stackoverflow.com/questions/42605138/how-can-i-write-an-apache-module-in-c http://stackoverflow.com/questions/42605138/how-can-i-write-an-apache-module-in-c) Julian]
Unix command line (just use kiwix-serve.exe on Microsoft Windows) example to start kiwix-serve with an XML library file:
<source lang="bash">
kiwix-serve --library ~/.www.kiwix.org/kiwix/efrnsupg.default/data/library/library.xml
</source></translate>


=== zip2zim (zimwriterfs as a service) ===
<translate><!--T:16-->
Create an online tool for converting zip files of HTML into deployable zim files
Unix command line example to start kiwix-serve with a ZIM file:
<source lang="bash">
kiwix-serve --port=8080 wikipedia.zim
</source></translate>


== Group Discussion Topics ==
<translate><!--T:17-->
During the hackathon we'd like to discuss various topics collectively. These include:
Once kiwix-serve is started, anyone with access to this computer over the network, will be able to connect to it. You need to retrieve the IP (xxx.xxx.xxx.xxx) of your computer and remember the port (per default 80) you have used and then people will only have to type in their browser address bar:
* UX and particularly Onboarding
<source lang="bash">
* Desktop apps
http://xxx.xxx.xxx.xxx:80/
* Incorporating analytics
</source></translate>


We aim to agree a topic with about a day's notice so those who wish can do some preparation, e.g. reading up, sharing ideas and notes, etc.
== <translate><!--T:18-->
Options</translate> ==
<translate><!--T:19-->
Kiwix-serve provides a few options:
* --port=PORT to specify the port you want to open for listening (per default 80)
* --daemon to execute the software in background
* --attachToProcess=PID to attach kiwix-serve process to other process
* --verbose to get a few logs</translate>


=== UX & Onboarding ===
== <translate><!--T:21-->
When: 14:00 ish Tuesday 4th April 2017
See also</translate> ==
==== Intro ====
<translate><!--T:22-->
We believe large numbers of users only use the app briefly before abandoning it. There are various ideas why this might be so. Perhaps they don't understand what the app can do for them, or they may find it hard to get started. There may be problems downloading content, etc. Also, with our growth in custom apps, including non-Wikipedia content such as PhET simulations, perhaps the user interface, menu options, and some of the functionality is a poor fit for the content and capabilities. We'd like to discuss the UX, including onboarding (getting users from first starting the app to using it productively), and consider ways we might usefully improve the UX.
* [[kiwix-manage]]
 
* [[kiwix-plug]]</translate>
Sum up of the meeting : [[UX and Onboarding]]
* [http://kv5r.com/computers/offline-dictionary-server/ How-to setup kiwix.exe as a Windows service]
 
=== Automatic testing ===
When: Wednesday 5th April 2017
==== Intro ====
Some repos already have automatic testing (unit tests and/or UI tests). Some other ones plan to add it.
 
We should discuss our experience/ideas on that topic, and see if we could do something in common.
 
Sum up of the meeting : [[Automatic testing meeting on 2017-04-05]]
 
== Achievements ==
 
=== Browser extensions ===
A basic browser extension has been commited.
 
Before it is available from Google and Mozilla extensions system, it can be installed manually :
* For Firefox :
** download the .xpi file from https://framadrop.org/r/Sq5oZm0Uv0#pcLwtwvzsSuFzCeSFsqNDgXf8tzo4riA56teTIX7YVE=
** open a new tab with URL "about:debugging"
** click on "load a temporary module" and choose the xpi file
** It's expected that only the jQuery mode will work under Firefox extension (Firefox limitation for now)
** This jQuery mode works well on wikipedia ZIM files, but has not been tested on other kind of ZIM files, where it should have big limitations
* For Chromium/Chrome :
** download the .crx file from https://framadrop.org/r/HOhudv1lr8#+ITMyqfwcCcLzMz3UG0Kw+YfOeksy2zl1J28wTDV1Ak=
** open a new tab with URL "chrome://extensions/"
** drag and drop the crxf file into this tab
* On both browsers, a Kiwix icon should appear on the right of the address bar
* If you click on this icon, it should open a new tab with the Kiwix application. You must then choose your ZIM file(s), and you should be able to browse it
 
=== Gutenberg scraper ===
* Updated dependencies and made it python3 compatible
* Parallelized long-running operations: download, parsing and export
* Fixed 3 bugs related to the scraping process
 
=== Sotoki ===
* Release first version
* Correct last small mobile bug
* Package to python package : https://pypi.python.org/pypi?:action=display&name=sotoki
* Add downloader for dump
* Add intern link between questions, answers and users
* Add docker image and [https://hub.docker.com/r/openzim/sotoki/ CI integration in Docker-hub]
 
=== youtube scrapper ===
* update code and package to python package : https://pypi.python.org/pypi?:action=display&name=youtube2zim
* Add docker file and [https://hub.docker.com/r/openzim/youtube/ CI integration in Docker-hub]
 
=== Android ===
* Restructured project to allow for easier testing and readability.
* Various bug fixes
* Introduced new developers to tooling and processes
* Travis CI
 
=== zimwriterfs ===
* Add Docker file and [https://hub.docker.com/r/openzim/zimwriterfs/ CI integration in Docker-hub]
* Fix small bug in usage()
 
=== Ideascube builder ===
* Communication between python script and QEMU process
** establish ssh connection
** enable ssh server via QEMU process input/output
* Setting up environment for [https://github.com/ideascube/ansiblecube/tree/oneUpdateFile ansiblecube] (as buildMyCube.sh does)
* Script configuring and compiling linux for QEMU armhf emulation (does not have options for ansiblecube captive portal yet)
 
=== zip2zim ===
* Created working prototype
* Add docker image and [https://hub.docker.com/r/openzim/zip2zim/ CI integration in Docker-hub]
 
=== PhET ===
* Category Filtering
* UI/UX improvements
 
== Impact ==
 
This hackathon gathers different kind of tasks which all in one will help to implement the Wikimed Projet and reach our goals in term of new content for 2017.
 
== Method ==
 
~ 10 developers meet during a week to make this steps forward.
 
== Place ==
 
The hackathon will take place in Lyon, France
 
== Agenda ==
 
* April 2th afternoon: arrivals (suggested arrival time 5PM)
* April 3th -> 9th: Kiwix hackathon
* April 9th afternoon: departures (suggested departure time 3PM)
 
== People ==
; Travel + Accommodation
* Adrian
* Elad
* Emmanuel
* Joe
* Julian
* Rashiq
* Renaud
 
; Sponsored
* Hayley
 
; Only Travel
* Isaac
 
; Located in Lyon
* Matthieu
* Mossroy
* Guillaume
 
== Costs ==
 
* Accommodation: ~4.000 CHF
* Travels: ~2.000 CHF
* F&B: ~3000 CHF
-> ~9.000 CHF

Revision as of 18:12, 18 April 2019

Other languages:
Bahasa Indonesia • ‎Bahasa Melayu • ‎Deutsch • ‎English • ‎Nederlands • ‎Türkçe • ‎azərbaycanca • ‎català • ‎español • ‎français • ‎galego • ‎italiano • ‎lietuvių • ‎magyar • ‎polski • ‎português • ‎русский • ‎فارسی • ‎বাংলা • ‎日本語
Kiwix-serve, embedded in Kiwix, on a Microsoft Windows computer
With Kiwix-plug, kiwix-serve runs on pretty small hardwares

Kiwix-serve is a ZIM compatible Web server. With its help you can deliver ZIM file content over your network.

Based on the well known HTTP protocol, kiwix-serve allows you to share the ZIM files available on your computer easily with others connected to your network. You simply need to start kiwix-serve and your content will be available for anybody through their own Web browser.

Kiwix-serve is a good solution if you have a LAN network and don't want to install Kiwix on all computers. It's a good alternative if you don't have a shared network disk from which everybody can start Kiwix.

Features

Kiwix-serve has not so many features like Kiwix, but provides the most important ones:

  • Full text search engine
  • Search suggestions
  • Really small and efficient
  • Compatible with almost all browsers
  • Available on all platforms
  • Available as command line executable
  • Embedded in Kiwix UI
  • Able to deal with one ZIM file or XML library files

Get it

On Microsoft Windows, Apple Mac OS X and GNU/Linux, kiwix-serve is packaged and distributed with Kiwix. You simply need to download the Kiwix installer corresponding to your Operating System.

We newly provide kiwix-serve as part of the Kiwix tools and pre-compiled binaries for many systems are available.

For other systems, you will have to download and compile kiwix-serve source code on your own.

Use it

Kiwix-serve is directly available in the Kiwix menu "Tools > server", but for people wanting to use it from the command line, here is how it works.

Prerequisites:

  • kiwix-serve binary, available in the Kiwix application directory
  • A ZIM file or a library file (you can use the library.xml file available in your Kiwix data directory).

Unix command line (just use kiwix-serve.exe on Microsoft Windows) example to start kiwix-serve with an XML library file:

kiwix-serve --library ~/.www.kiwix.org/kiwix/efrnsupg.default/data/library/library.xml

Unix command line example to start kiwix-serve with a ZIM file:

kiwix-serve --port=8080 wikipedia.zim

Once kiwix-serve is started, anyone with access to this computer over the network, will be able to connect to it. You need to retrieve the IP (xxx.xxx.xxx.xxx) of your computer and remember the port (per default 80) you have used and then people will only have to type in their browser address bar:

http://xxx.xxx.xxx.xxx:80/

Options

Kiwix-serve provides a few options:

  • --port=PORT to specify the port you want to open for listening (per default 80)
  • --daemon to execute the software in background
  • --attachToProcess=PID to attach kiwix-serve process to other process
  • --verbose to get a few logs

See also