26
edits
(raw paste of my notes) |
(Minor changes, make the page more readable) |
||
Line 1: | Line 1: | ||
''These are my raw notes of the meeting :'' | |||
Mossroy : Currently on kiwix-html5 repo : travis + saucelabs | |||
saucelabs provides pre-built hardware and software with browser/mobile versions etc. | Julian : saucelabs provides pre-built hardware and software with browser/mobile versions etc. bitbar : android devices test remotely + iOS | ||
Guillaume : would like to test the installer on Windows/Mac/Linux. Travis only does Linux | |||
Guillaume : test installer on Windows/Mac/Linux | |||
Travis only does Linux | |||
Dattaz : curiosity | Dattaz : curiosity | ||
Line 19: | Line 16: | ||
Julian : | Julian : | ||
Differences between what Matthieu and Isaac do | Differences between what Matthieu and Isaac do | ||
automate the most common user journeys : open the app, open a page, etc | automate the most common user journeys : open the app, open a page, etc | ||
BDD frameworks : given, when, then (natural language) | BDD frameworks : given, when, then (natural language) | ||
Opinion : little point to have common code between all the repos. | Opinion : little point to have common code between all the repos. | ||
Java unit tests works well, but do not allow to interact with the system (permissions, SD card etc) | Java unit tests works well, but do not allow to interact with the system (permissions, SD card etc) | ||
UIOtomato | UIOtomato | ||
Image-based testing : not recommended for us | Image-based testing : not recommended for us | ||
Which content to use for the tests? Ray Charles archive allows to test some aspects, but not all of them. | Which content to use for the tests? Ray Charles archive allows to test some aspects, but not all of them. | ||
Careful about the multi-lingual aspect. | Careful about the multi-lingual aspect. | ||
Step by step : start with something simple to test, and see what needs to be improved. | Step by step : start with something simple to test, and see what needs to be improved. | ||
Android pains : different storage locations, onboarding | Android pains : different storage locations, onboarding | ||
Line 36: | Line 42: | ||
Levels at Google : | Levels at Google : | ||
0- no tests | * 0- no tests | ||
1- at least one test, and CI that runs it | * 1- at least one test, and CI that runs it | ||
2- policy : team that agree that, at every change, a procedure must be | * 2- policy : team that agree that, at every change, a procedure must be | ||
3- coverage : minimum 40% | * 3- coverage : minimum 40% | ||
The code coverage is not a primary measure, but can indicate a trend on code quality attention by developers | The code coverage is not a primary measure, but can indicate a trend on code quality attention by developers | ||
But too much unit tests can be complicated to maintain. | But too much unit tests can be complicated to maintain. | ||
Feature coverage can be another measure. | Feature coverage can be another measure. | ||
What we're not going to do in the short term for Android testing : | What we're not going to do in the short term for Android testing : | ||
* language testing : only English at first | |||
* multi-device testing | |||
* BDD testing | |||
* search (case-sensitivity, etc) : it must be tested by unit tests on the ZIM lib | |||
* network connectivity : slow networks can be simulated, but it becomes complicated. We assume the network is working correctly | |||
* test permissions of the system | |||
* test available storage | |||
* test custom apps | |||
For the short term : use preferably Espresso and/or UIOtomato for Android testing. | For the short term : use preferably Espresso and/or UIOtomato for Android testing. | ||
Line 62: | Line 67: | ||
Choice for Android testing (by the end of the week) : | Choice for Android testing (by the end of the week) : | ||
* work on Travis automation (continuous building) | |||
* or work on local testing automation, that could be run manually by all Android developers. This one is chosen. | |||
For kiwix-html5 : the tooling seems good enough for now. About saucelabs : create a "kiwix" account and use this credential. | |||
For saucelabs : create a "kiwix" account and use this credential. | |||
No tests on kiwix desktop for now. Maybe for kiwix-serve? | No tests on kiwix desktop for now. Maybe for kiwix-serve? |
edits