Product Feature Usage Analysis

Overview

KittyCarts is a fictional eCommerce company that sells monthly subscription boxes containing feline care products. They have recently expanded their business by offering a line of branded webcams on their website.

The “KittyCam” allows users to monitor their cat’s behavior throughout the day. KittyCarts had an existing app for box subscribers, and are expanding it to support KittyCam functionality for both subscribers and non-subscribers. In the app, users are able to livestream their KittyCam feed. Users may also create clips from their feed to save or share on social media.

KittyCarts wants to analyze the onboarding flow for users connecting their KittyCams, specifically the step in their flow that displays an orientation screen. The user can choose to skip this step, and KittyCarts wants to know how this affects onboarding progression. Additionally, they are conducting an A/B test on copy for one of the buttons in their onboarding process. They want to see which group of users was more likely to complete the full onboarding flow.

Questions:

  • How does interaction with the orientation screen affect onboarding flow conversion?
  • What is the impact of the A/B test on onboarding flow conversion?

_____________________________________________________________________________________________________________

Question 1: How does interaction with the orientation screen affect onboarding flow conversion?

 

Open the Funnel Tool: Let’s click the Funnel icon in the Navigation Bar.

x9.gif

 

Create Funnel Visualization: To create our funnel, let’s begin dragging the events in our onboarding flow. Funnel measures user progression from event to event, so let’s drag in our onboarding flow events in the following order:

  1. Click “Register KittyCam”: Within the app, users click this button to begin the flow.
  2. Skip Orientation Screen: Users have the option to skip the orientation screen if they wish.
  3. Create Camera Access PIN: Users create a PIN for accessing their camera feed.
  4. Grant App Permissions: Users grant necessary app permissions .
  5. Connect Webcam: Users’ webcams connect and begin streaming.

x10.gif

Executing this query gives us our onboarding flow  visualization. In the blue circles, we can see the number of users who progressed to that event. In the webbings, we can see the percentage of users who proceeded to the next step.

 

Make a Step Optional: The event “Skip Orientation Screen” is an optional action by the user. A user may choose to skip the orientation screen or go through the orientation, before progressing through the flow. However, our current funnel requires users to have completed the event “Skip Orientation Screen”.

To also see what our numbers look like for users who chose not to “Skip Orientation Screen”, we can utilize Indicative’s optional step functionality.

x12.gif

  • Let’s click the pushpin next to “Skip Orientation Screen”
  • Next, let’s execute the query to update it

Above, we can see a Funnel visualization with multiple paths. It is currently showing the onboarding flow with the “Skip Orientation Screen” event included. As we can see, the numbers are the same as in our initial funnel.

x13.gif

  • Let’s click on the path above “Skip Orientation Screen” to look at the flow that excludes this event.

 

Observation 1

  • 14.81% of users who did “Skip Orientation Screen” completed the onboarding flow
  • 29.16% of users who did not do “Skip Orientation Screen” completed the onboarding flow

Users who did “Skip Orientation Screen” were less likely to complete the onboarding flow. In light of these results, KittyCarts should consider deleting the option to skip the orientation screen.

 


_____________________________________________________________________________________________________________

Question 2: What is the impact of the A/B test on full flow completion?

Based on the results of our first analysis, KittyCart chose to delete the option to “Skip Orientation Screen”. They now want to run an A/B test of the copy on the “Register Kitty Cam” button to measure the effect on flow completion. 

  • Group A sees “Connect KittyCam”
  • Group B sees “Register KittyCam”

 

Create Funnel Visualization: To create our funnel, let’s drag in the events in our new onboarding flow.

  1. Click “Register KittyCam”: Within the app, users click this button to begin the flow.
  2. Create Camera Access PIN: Users create a PIN for accessing their camera feed.
  3. Grant App Permissions: Users grant necessary app permissions
  4. Connect Webcam: Users’ webcams connect and begin streaming.

x14.gif


Create a Shared Breakout: The Shared Breakout feature in Funnel allows us to segment a specific step using a property and follow those users throughout the flow.  

x15.gif 

  • In the Settings dropdown, let's change the funnel's Breakout Mode from "Per Step" to "Shared"
  • Let’s select the event property “AB Group” to begin our split. Drag “AB Group” into the circle representing “Click ‘Register Kitty Cam’”
  • Select “Segment Step By”

This segments  the users who did “Click ‘Register Kitty Cam’” by their AB Group and follows them through the rest of the funnel. We can then measure and compare conversion rates and drop off.

 

Compare AB Groups: By clicking on a wedge representing either Group A or Group B, we can highlight this group. After a group has been highlighted, the percentages in the webbings will only reflect step-to-step conversion for that group.

Above we can see that 45.45% of Group A users progressed from “Create Camera Access PIN” to “Grant App Permissions”. For Group B, this number was 64.06%.

 

Observation 2

  • 12.85% of Group A users completed the onboarding flow
  • 39.08% of Group B users completed the onboarding flow

Users from Group B showed a significantly higher likelihood of completing the onboarding flow. This shows that from the copy options tested, Group B’s copy would be the best to implement.

0 users found this helpful