Cloud-server console research

01Context

The console where customers run their servers

Arvan Cloud sells cloud servers, which it calls “abraks”, alongside its CDN, DNS and storage. A customer creates a server on one page of the console and runs it from a list on another: the server list, with a menu of operations on each row.

In February 2020 Sina, a product designer at Arvan, began research on that part of the console. It ran until January 2021 and asked what customers did there, what they asked for and what other clouds showed. This study follows the work in the order it happened. Arvan has updated the console since, so what is online today is not what is described here.

02Approach

Five ways of listening, one board

Most of the work lives on one Trello board, made on 19 February 2020 and filled mainly through November. Each kind of evidence got its own lists: analytics, a poll of customers, team surveys, competitors’ pages and a page-by-page review. A smaller board of 11 February holds the recordings that came first.

Each method answers a different question. Recordings show what a customer does and what the console says back. Analytics show how many do it. The poll and the surveys show what people ask for. The competitors show what else is possible. The reviews show what is wrong, page by page.

A count says how often; an answer says why. The study needed both.

The year in order

01
Watch recordings11–22 Feb 2020: 12 findings on a first board
02
Review the old console19–29 Feb: 32 findings, page by page
03
Read the analyticsFeb–Mar: page views, devices, the ranking of 15 Mar
04
Ask and compareMar: eight surveys, a customer poll, four competitors
05
Count the clicksFrom April: click tracking on the create page and the list
06
Code the poll17 May: 131 tags on 62 answers
07
Review the new console28 Oct–26 Nov: 22 findings; weekly counts to 17 Jan 2021

03Recordings

Most problems were in what the console said back

The research began on a smaller board, made on 11 February 2020 from recordings of real customers using the console. Sina watched them, froze the frame where something went wrong, and made one card for each problem, with the minute mark of the moment in a comment.

The twelve findings came from the CDN, tickets, payment, notifications and DNS parts of the console, not from servers. Ten of them are about what the console says back after someone acts: a message that contradicts itself or repeats, or that leaves out the amount, the reason, the next step or a way back.

Ten of the twelve problems were about what the console said back, not about what it let people do.

Six ways the console answered badly

01
Clashing messagesTwo notices that disagree, or one that repeats
- “Disabled” and “saved” on screen together
- One raw English error, stacked four times
- The same wallet notice, three times
02
Missing factsA result with the number or the reason left out
- A top-up with no amount shown
- A switch that does nothing, with no reason
- An error that says to upgrade, with no link
03
No next stepA success that leaves the person stranded
- A payment succeeded, and nothing to do next

04Visits

Seven in ten visits went to one page

Google Analytics gave the first count. Between 1 May 2019 and 14 March 2020 the console’s eleven main pages had 82,027 page views. The server list took 71.9% of them and the create page 9.8%; no other page reached 5%.

Time on page told a different story. People stayed about five minutes on the instant-report, SSH-key and firewall pages, and about three and a half on the create page and the server list.

On 15 March 2020 Sina ranked the eleven pages by hand, one to eleven, with the figures for two periods beside each: ten and a half months, and the last three and a half.

A bar chart of the eleven main pages of the console by page views between 1 May 2019 and 14 March 2020, 82,027 in all. The server list leads with 58,963 views, 71.9 percent. The create page follows with 8,037, 9.8 percent. Firewall settings, instant report, network and server images come next at 4.1 to 2.5 percent, and the last five pages are each under 2 percent. — Figure 02Page views of the console’s eleven main pages, 1 May 2019 to 14 March 2020, from Google Analytics: 82,027 in all.

A table of Sina’s ranking of the console’s eleven pages, one to eleven: server list, create, firewall settings, network, server images, volumes, SSH keys, own operating system, floating IPs, limits and instant report, with the page views of two periods beside each: 1 May 2019 to 14 March 2020, and 1 December 2019 to 14 March 2020. — Figure 03Sina’s ranking of the console’s eleven pages, typed from a handwritten note of 15 March 2020, with the page views of two periods beside each.

05Clicks

Six in ten clicks on the create page were buttons and menus

Click tracking started on the console in the first days of April 2020, and Sina set it up. On the create page it counted 49,674 clicks between 1 and 24 April: six in ten on buttons and menus, and about one in eight each on the choice of data center and on the two tabs, standard and marketplace. Windows was the operating system clicked most, 1,456 times, a third of all operating-system clicks.

On the server list, 65,066 clicks went to the menu on each row. Just over half of them only opened the menu. Then came the server console (21.2%), delete (7.0%), turn off, resize, restart, rebuild and snapshot, the last with 1.3%.

Help links told a third story. Of 716 clicks on them in April, a quarter were on the create page (25.3%) and about one in a hundred on the server list (1.3%).

A chart of the 49,674 clicks tracked on the create page between 1 and 24 April 2020, in six groups: buttons and menus 60.4 percent, data centers 13.0, tabs 11.8, operating system picks 8.7, marketplace apps 5.2 and other 0.9. Beside it, the ten most clicked labels, led by “click on new abrak” with 13,637 clicks. Windows is the most clicked operating system, with 1,456. — Figure 04Clicks on the create page, 1–24 April 2020: 49,674 in six groups, and the ten labels clicked most. From Google Analytics events.

A bar chart of 65,066 menu clicks on the server list in April 2020: opening the dropdown 35,214 (54.1 percent), server console 13,764 (21.2), delete 4,568 (7.0), turn off 3,854 (5.9), resize 3,685 (5.7), restart 1,615 (2.5), rebuild 1,531 (2.4) and snapshot 835 (1.3). — Figure 05Clicks on the server list’s menu, April 2020: 65,066 in eight kinds. From Google Analytics events.

06Poll

Features first, then finding things

A Hotjar poll on the create page asked how easy the page was, and whether the customer had needed something they could not find. Sina captured 62 of the answers that came with a comment, out of at least 466 received by 29 March 2020.

On 17 May 2020 Sina coded them into ten categories with 131 tags in all, since one answer can carry several. Requests for features came first (38), then not finding things (22) and no explanation or tutorial (16). Technical and financial wording followed (13 and 12), then slowness (11).

A bar chart of the 131 tags Sina coded on 62 customer poll answers, in ten categories: feature requests 38, not finding things 22, no explanations 16, technical wording 13, financial wording 12, slowness 11, errors 8, visual design 5, console bugs 4 and development problems 2. — Figure 06What 62 customer answers were about: 131 tags in ten categories, coded by Sina on 17 May 2020.

07Surveys

What the team ranked first

Eight short Google Forms surveys of the team asked one question each: what matters more when creating a server, which size counts most, and which list features and filters people would use. Five had 8 answers, one had 10 and two had 19.

The answers were small but clear. Of 8, 5 put the data center above the operating system and 6 put price above CPU. For the list, 7 of 8 picked search; among 19, the filters picked most were status (16), operating system (14) and tag or label (12).

Eight to nineteen answers can’t settle a question, but they show where to look first.

Four small bar charts of team surveys. Creating a server, which matters more: data centers 5 of 8. Which size counts most: price 6 of 8. List features, three picks each: search 7 of 8. List filters, from a survey of 19: status 16, operating system 14, tag or label 12 and IP address 9. — Figure 07Four of the eight team surveys: how many people picked each option. The answers number 8, 8, 8 and 19; the last two allowed several picks.

08Rivals

Four rivals, four ways to create a server

In late February and March 2020 Sina captured how Vultr, DigitalOcean, Linode and AWS EC2 let a customer create a server. Read side by side, three keep everything on one page and AWS spreads it over steps.

The captures are the companies’ own screens, so they are not shown here; the comparison is in words. What stands out is the price: Vultr, DigitalOcean and Linode show it by the month and by the hour while a person is still choosing, and Vultr and Linode add a running total.

What each shows when you create a server

01
VultrOne long page
- Server type, location, image, size
- Monthly and hourly price on every size
- A running total at the bottom
- SSH keys as tiles
- A server page of seven tabs
02
DigitalOceanOne long page
- Image, plan, region
- Monthly and hourly price on every plan
- Backups priced as a share of the plan
- SSH key as the default way in
- A server page with thirteen sections
03

09Reviews

Fifty-four findings, in two reviews

Between 19 and 29 February 2020 Sina went through the old console page by page and made 32 cards: 8 on the server list, 6 on networks, 5 on the console’s home page, 3 each on the firewall and on notices and errors, 2 each on creating a server and on images, and 1 each on volumes, limits and NOC. Among them: the server console was hard to find, a list could show each server’s CPU, RAM and disk, a form had no cancel button, and “album” should have said “snapshot”.

From 28 October to 26 November 2020 Sina reviewed the new console as it was built: 22 cards, 12 on the server list, 9 on the create wizard and 1 on volumes. Their labels: visual design (10), information architecture (7), usability (3) and copywriting (2), with four marked “undone”.

Where the findings fell

01
Server list8 in February · 12 in November
- The wording of the delete dialog
- Uptime in the list
- An unclear status
- A copy icon for the IP address
- A group restart that looks wrong
02
Creating a server2 in February · 9 in November
- Back does not change the address
- No error when the credit is too low
- The order of the operating systems
- “What is vCore?”
- “Is it SSD?”
03

10Usage

Delete led every full week

The board also holds weekly counts for the new server list, from 18 October 2020 to 17 January 2021: 20,175 operations in 14 weeks, the first and last of them partial. Delete led in every full week, 7,802 in all, almost four in ten; resize (3,849) and restart (3,427) followed.

Rename and rescue were used less. Tag, a feature the surveys had put near the top, first shows in the week of 27 December, and 80 times by the end.

These are counts of the new list’s operations; the April counts were of the old menu’s clicks, so the two are not comparable.

A stacked column chart of operations on the new server list for each week from 18 October 2020 to 17 January 2021, 20,175 in all. Weekly totals rise from 855 to about 2,000 in early December. Delete is the largest part of every full week, and tag first appears in the week of 27 December. — Figure 08Operations on the new server list, by week, 18 October 2020 to 17 January 2021: 20,175 in all. The first and last weeks are partial.

11My role

Sina’s part

Sina ran this research as a product designer at Arvan Cloud, from February 2020 to January 2021. The board holds no design files, so this study shows research and review, not screens.

Owned

The recordings review and its twelve findings
The click tracking, set up in April 2020, and the counts read from it
The ranking of the console’s eleven pages
The coding of the customer poll
The team surveys
The comparison with four competitors
Both reviews of the console, 54 findings

Co-owned

Collaborated

12Outcomes

What the research left behind

This research led to a redesign of the whole console. The board holds the research and the reviews, not the redesign work, so this study lists what the research produced, not what the redesign contained.

cards on one research board: 190; Analytics, a customer poll, team surveys, competitor captures and two reviews, February 2020 to January 2021; Delivered output · Research board
tags on 62 customer answers: 131; In ten categories; requests for features came first, with 38; Delivered output · Coded chart, 17 May 2020
pages ranked: 11; By hand, from 82,027 page views over ten and a half months; the server list took 71.9%; Delivered output · Google Analytics, 1 May 2019–14 Mar 2020
review findings: 54; 32 on the old console in February 2020 and 22 on the new console in October and November; Delivered output · Review cards

What was delivered

A research board of 190 cards, and a first board of 12 findings from recordings
Click tracking on the console from April 2020

13What I learned

What I’d change

01
Start the click tracking first
Page views went back to May 2019, but click tracking began only in April 2020, after the review was under way. The menu and create-page counts cover a few weeks. Switching it on with the first card would have given months.
02
Count what I watch
The recordings gave 12 findings, but nothing on the board says how many recordings I watched or how often each problem came up. A tally beside each card would have shown which problems were common and which were one-offs.
03
Keep the poll’s totals
I kept 62 answers out of at least 466, and my coded chart rests on those. Without the full totals, I can’t say how far the 62 stand for the rest.
04
Write down what changed
The board records findings and counts, not what changed in the console afterwards. A line on each card, changed or not, and when, would have tied the research to its result.