UX and Onboarding

UX and Onboarding

Note: These are raw notes made during our discussion. Most of the people at the hackathon participated, including Stephane who was visiting for the day. They're available to be refined and corrected, as I'll be doing.""

Scope: Android App Source: Installs and Uninstalls tracked by Google Play Proxy measures: download data of zim files, piwix tracks downloads stats.kiwix.org

Do we set the user-agent of the Android app. OkHttp library. A topic to expand further, perhaps beyond this meeting. We’d like to also consider aspects such as tracking users, asking for permission, etc.

Can we define indicators as measures to use to assess UX Android Google Play supports A/B

Can we prioritise which measures are important.

We need data to know where we are going. How TBD

Say to users we’re tracking their use. Perhaps some users don’t want to be tracked.

Getting data to make a decision: In-app analytics:

Need data, 7, 6, 6, 7, 7, 7, 7, 4, 7, 7, 7, 7 No opinion - in appl 4, 3, 7

On measurements: (Hayley) Currently we only know what Google Play Store provides. 1. Important to know what the user feels, rather than what they do. Several ways we could do this e.g. UX is not only about how you use it, how you feel? Ketchup Bottles: Heinz 2 versions: Glass bottle and Squeezy bottle. Glad - premium, nice to look at, classic, always driven to go towards the nice, shiny glass bottles. Not attracted to the squeezy plastic bottle - better to serve the ketchup.

2. Also important to capture how the user interacts: If we’re breaking the user’s expectations of navigation perhaps that’s a reason we’re losing users. We don’t necessarily need to use intrusive methods. We can use user-testing, paid services, recording, or simply us using the app as a user e.g. using goal-based exercises. Hayley’s not looked at the Kiwix app yet, only the PhET app. Do we want to buy from an ugly store, do we trust it. We perceive they’ve not done enough to convince users we’re worth their trust. And because we don’t know why we’re losing users we don’t know how to improve the app. There are many ways we can get feedback.

What can we measure about use of the app. Feeling can come from many things e.g. a home button at the bottom-right of a screen - it’s not where user’s expect it to be. Thumb reach is important in the design. Apple design the iPhone so users can reach content on the screen. We need to look at where the user is tapping, where they’re focusing on in the screen, where they’re sitting on a page, and the bounce rate.

Joe’s example of trying to work out how to switch zim files. User journeys are important (from A to B). If we haven’t designed the app for user-journeys and considered ways to minimise the user’s steps needed to perform an action. Examples of Amazon payment process, etc. Frustration in the number of steps affects the UX.

Renaud: from the start of the Android app it was based on the desktop app and had only a simple basic HTML page. Isaac, there have been improvements to the welcome page but it’s still text heavy and not necessarily the best. Hayley: In large organisations, the designs are done by teams before the work comes to the developers to implement. Can we go back a step and find out about the users as a team. Renaud: the app was originally designed around technical features. Hayley: it’s good to start with designing an initial app based on our ideas. Then it’s important to revisit the app to revise the design of the app, otherwise we struggle to make effective progress in terms of improving the app from our users’ perspective.

Can we now come together and think of the users? Emmanuel: the Android app started as a one-week hack to display Wikipedia in English. They’d implemented a pure Java reader but it was too slow so they implemented the JNI. We had feature driven development. The most recent release has a rating of 4.7 (better). Isaac: However we’re stuck at 50,000 users with similar numbers of installs and uninstalls. c.f. custom apps which have a much higher retention rate. We don’t know why this is - perhaps because users have already downloaded the content. Hayley: perhaps users who download custom apps are more focused and invested in using the app. Maybe users used to look for anything free, but now are more discerning.

Various approaches: Can we use a mix?

Can we get data back to us: How much data to collect.

Joe: Can we start with something low-touch? Mattieu: strongly believes it’s vital for the data to be anonymous. Renaud: Can we measure data that’s valuable for us? e.g. why people are leaving the app. Hayley: what about an in-app survey. Julian: how about 4 of us who haven’t used the app in the last month: Mattieu, Hayley, Guillieme, Renaud. Emmanuel: Privacy problem is a complex problem: we don’t need to address it now. Perhaps we could add an instrumented version of Kiwix and pay users to provide feedback. Isaac: biases. Emmanuel: Consider scaling once we’ve tried something simple.

Can we run surveys?

Isaac: devise hypothesis so we can test it. What about the size of downloads: What about querying device characteristics such as the amount of space available? Isaac: the list of ZIM files is 6MB.

Cost, complexity/easiness, privacy

Obvious problems: e.g. the welcome page is seldom available translated. It’s possible to translate with translation wiki.

Actions? 1. Informal local test using kiwix for android. Boris, Hayley, Guillieme, Mattieu, Renaud. Julian to manage. Today. 2. User-agent to correlate the downloads. Compare with active use by version. Ideas on the wiki. Review what to collect and that we can see it in the next release. Perhaps on the Beta channel. 3. A custom version of the app with instrumentation. Perhaps for in-place trusted evaluators.

Boris: what about the iOS app? Emmanuel: there’s only 1 developer, he’s not here. Several of the team will meet him in Northern New York right after Wikimania. Chris is a very sensitive developer and much interested in usability feedback. However volumes are not similar so let us stick to Android for the time being.