Tell us your story
Tell us your story
How has offline Wikipedia affected you? The Wikimedia Foundation (the non-profit that supports Wikipedia) is looking for personal, diverse and inspiring stories about how offline Wikipedia affects the world. If you have a personal story that you would like to share, please contact: Thank you!


Jump to: navigation, search

Automatic testing meeting on 2017-04-05

148 bytes added, 2 years ago
Minor changes, make the page more readable
Automatic testing''These are my raw notes of the meeting :''
On Mossroy : Currently on kiwix-html5 repo : travis + saucelabs
Julian : saucelabs provides pre-built hardware and software with browser/mobile versions etc.bitbar : android devices test remotely + iOS
bitbar android devices test remotely + iOS Guillaume : would like to test the installer on Windows/Mac/Linux. Travis only does Linux
Dattaz : curiosity
Julian :
Differences between what Matthieu and Isaac do
automate the most common user journeys : open the app, open a page, etc
BDD frameworks : given, when, then (natural language)
Opinion : little point to have common code between all the repos.
Java unit tests works well, but do not allow to interact with the system (permissions, SD card etc)
Image-based testing : not recommended for us
Which content to use for the tests? Ray Charles archive allows to test some aspects, but not all of them.
Careful about the multi-lingual aspect.
Step by step : start with something simple to test, and see what needs to be improved.
Android pains : different storage locations, onboarding
Levels at Google :
* 0- no tests* 1- at least one test, and CI that runs it* 2- policy : team that agree that, at every change, a procedure must be * 3- coverage : minimum 40% 
The code coverage is not a primary measure, but can indicate a trend on code quality attention by developers
But too much unit tests can be complicated to maintain.
Feature coverage can be another measure.
What we're not going to do in the short term for Android testing :
- * language testing : only English at first- * multi-device testing- * BDD testing- * search (case-sensitivity, etc) : it must be tested by unit tests on the ZIM lib- * network connectivity : slow networks can be simulated, but it becomes complicated. We assume the network is working correctly- * test permissions of the system- * test available storage- * test custom apps 
For the short term : use preferably Espresso and/or UIOtomato for Android testing.
Choice for Android testing (by the end of the week) :
- * work on Travis automation (continuous building)- * or work on local testing automation, that could be run manually by all Android developers. This one is chosen. For kiwix-html5 : the tooling seems good enough for now. About saucelabs : create a "kiwix" account and use this credential.
No tests on kiwix desktop for now. Maybe for kiwix-serve?

Navigation menu