Usability Evaluation of a mobile application that allows real-time tracking of shuttle services


We performed Usability Evaluation to assess the overall effectiveness and experience of the DoubleMap application for its target users. In this project, we identified pain points where users get confused and tasks which they were able to perform with ease. Evaluation methods like cognitive walkthroughs, think aloud observations, and heuristic evaluation was applied to measure the usability of the system. Design recommendations were suggested for all the usability issues discovered throughout the evaluation.


DoubleMap is the web and mobile application designed to track the shuttle services and municipal transit systems across various campuses. Campuses include universities, corporate complexes, hospitals, and airports. The application is available on both Android and iOS platforms.


The goals of our evaluation study were as follows:

  1. Assess the overall effectiveness of DoubleMap for different types of users
  2. Identify features of the application which users easily learn
  3. Identify obstacles in the application where users struggle
  4. Determine opportunities for design improvements



Target Users

The primary users of the application include students/staff, corporate/company employees, Hospital staff, Airports and Municipal transit users. Bus drivers and campus administrators who manage new and current routes or buses are secondary users of the system. We designed a persona that would sufficiently meet DoubleMap’s majority user base.

Fig. Susan’s Persona

Common Application Tasks

  1. Finding the nearest bus stop
  2. Add favorite stops
  3. Searching for relevant bus arrival timings
  4. Set up push notifications to get notified about bus arrival
  5. Checking for service announcements or any delay in bus arrival due to unforeseen conditions
  6. Provide feedback to the system (if any)


Fig. Typical user task flow

Competitive Analysis

Competitive analysis was conducted with the below competitors of DoubleMap

  1. Transloc Rider: It is a similar app that provides real-time bus updates and announcements. It is used by many transit systems and university campuses
  2. RideCell: It is a company that provides ride/car sharing applications. It also provides services for fixed routes fleet automation. Works similar to Transloc and DoubleMap


Contextual inquiry sessions were conducted with some students on the IUPUI campus and some were conducted in their individual apartment. The sessions followed a method where the user performed the tasks and the observer only interrupted in case the user was confused. The following things were recorded:

  1. Amount of time taken to perform a task
  2. Patterns of behavior, pain points, and places where users face difficulty
  3. Alternate approaches to performing a certain task

Fig. Contextual Inquiry sessions

The focus of the cognitive walkthrough session was on understanding DoubleMap’s learnability for novice users. In this session, we went through all the sequence of tasks involved and asked questions from the perspective of the user.
Fig. Cognitive Walkthrough session

For the Heuristic Evaluation, we judged the application based on Jakob Neilson’s ten usability heuristics of interface design. The heuristics can be found here.


Ten participants were recruited for the usability study out of which five were novice users and others were experienced users of DoubleMap.


The time taken by users performing basic application tasks were recorded and plotted on graphs



The overall success rates of novice vs experienced users was measured.


After completion of all the tasks,  we asked users a couple questions regarding DoubleMap in the post-task questionnaire. We received the following responses.


Does DoubleMap solve the purpose of real-time tracking of buses?



85% participants agreed that DoubleMap solves the purpose of real-time tracking of the bus locations while 15% disagreed with it.


Did DoubleMap meet your expectations?



85% of the participants agreed that DoubleMap meets their expectation level whereas only 15% of the participants disagreed on this.


Quantitative Insights

The following insights were discovered:

  1. Novice users completed 84% of all the tasks whereas experienced users completed 96% of all the tasks
  2. Finding the estimated arrival time of the bus (ETA) had the lowest completion rate of 70% as compared to all other tasks
  3. On average, new users took 20 seconds longer than experienced users to find the nearest bus stop from their location & selecting a bus route to the desired destination stop
  4. 100% success rate was observed for all the participants for finding the IUPUI campus system on the application, finding the nearest bus stop from the current location & tracking the location of their bus

Qualitative Insights

DoubleMap’s overall design was leveraging Google Maps APIs, so most of the users experienced resemblance with google maps quite useful. The following comments were noted:

  1. “App is good. Especially real-time bus tracking”
  2. “Bus Tracking feature is accurate”
  3. “Toggle buttons for selecting/unselecting routes  are clear”
  4. “Changing routes was easy”

System Usability Score(SUS)

Based on the calculations, the SUS for the application was determined as 81.25 where individual scores varied from 75 to 87.5. Overall, this score is a good usability score. DoubleMap is usable but has definite scope for improvement.


GPS enable prompt

Existing System Design

When GPS is disabled, system does not provide any prompt to enable it

Recommended System Design

GPS prompt to be shown to the user

Search Bar & Destination Stop

Existing System Design

In the current Home screen view, user cannot search for exact stop location

Recommended System Design

We introduce a search bar to search for stops and a new location to destination finder

Estimated Time of Arrival

Existing System Design

Confusing Routes and ETA information

Recommended System Design

A table based ETA and Bus route information for quick reference


Existing System Design

The current feedback system pops up a bunch of options which confuses the user

Recommended System Design

A rating based feedback system for quick response


Overall DoubleMap does an effective job of creating a good impression on users by doing the following things:

  1. Real-time display of bus locations
  2. Provide timely announcements about the service to the users
  3. Display accurate estimated arrival times of the buses for each route
  4. Intuitive and minimal design – It’s easier for user to remember the task flow