GetGood

UnderSTAND

Team

Our team comprised of 3 members, with Ken Soh (pictured on the right), Desmond Yeo (in the middle), and myself.

Role

I took up the role of Research Lead due to my prior experience and background.

However, we decided early on as a small team, we wanted to be involved in every aspect of the UX process. As far as possible, decisions were all made collaboratively, with agreement from all parties.

I led the team in discussing our expectations and working styles before we dove into the project.
Aligning our goals from the beginning shaped and facilitated all our subsequent interactions.

Project

Client

GetGood (getgoodapp.com) was co-founded by Du Guo Xin and Niklas Krause in early 2019. Guo Xin is currently based in Singapore, whereas Niklas works from Germany. GetGood approached our team through a partnership with General Assembly, Singapore.

Platform

GetGood is a web and mobile platform for e-sports gamers. The gap they are trying to fill is a way for players to advertise and look for both team recruitment opportunities, as well as matches.

The GetGood website was launched in July 2019 which scrapes requests from multiple Discord (a chat and VoIP application) channels into a consolidated, searchable list. This was very well-received by the community and GetGood aims to continue growing their platform, adding user-requested features, and expanding to other games.

The platform is currently under heavy development in early-alpha stage, and there are plans to launch and monetise the business in 2020.

Primary Goals

To design and test the usability of 3 product concepts/features :
• Recruitment: Automated recommendations for players and managers to form e-sports teams
• Auto Scrim: Scheduling tool for existing teams to find practice matches, also with automated recommendations
• Ranked Scrim: ‘Live’ matchmaking for matches based on a GetGood ranking score instead of Blizzard’s Skill Rating (SR)

Secondary Goals

• Validate the above 3 concepts to ensure they are aligned to user needs
• Investigate user attitudes towards monetization and willingness to pay

Scope

• Limited to only 1 game: Overwatch, and on web platform only

Our client was relatively new to UX and entered the first meeting with an open mind. Because our team adequately prepared for the meeting, we were able to confidently advocate the value of UX for their business. In addition, we were aware of our strengths and limitations, negotiating primary goals (which were more aligned to UX), and secondary goals (which the client felt was important, but fell more in the realm of market research, outside our expertise).

Round 1 - Research

Research Plan

Primary Goal & Methodology

To design and test the usability of 3 new features, integrated within the GetGood website

• As there was no existing design or concept for the recruitment feature, we decided to hold a formal stakeholder interview with the client to find out their objectives for this feature.
• For the remaining features, the client was able to provide some basic designs and wireframes which they had created on Google Slides. Hence, we proceeded directly to conducting a heuristic evaluation and competitive analysis before prototyping.
• With a functional prototype, we proceeded to do usability test and iterate on our design

Secondary Goals & Methodology

• Investigate user attitudes towards monetized premium features and willingness to pay
• Examine and validate the needs of the target audience
• Examine perceived usefulness of proposed

• We did a short user interview in combination with the usability test, and also included a survey at the end to reach a wider audience

Shortly after the first meeting, we realised that it would add more value to the client if we were to work on the 3 new features in conjunction with core website features (e.g., registration, navigation) instead of in isolation as earlier discussed. We established a clear, on-going, and professional line of communication to propose this change in direction to the client, and the rationale for doing so.

While we originally planned for at least 2 rounds of usability tests, we adapted and revised this to only 1 round, even though this meant we would be unable to compare metrics or validate our design changes. The team felt strongly that it would ultimately be more useful to the client, and we took extra effort to communicate and negotiate with the client to fight for this.

Stakeholder Interview

Objective

• To find out about the client's objectives and direction for Recruitment feature as there was no existing design or concept

Mode

VoIP call over Discord as one of the co-founders, Niklas is based overseas in Germany

Key Findings

• Marketplace a key feature (already in early development) for players to find teams, and vice versa
• To explore ‘Tinder-style’ matching to make searching less tedious and more visual
• Provide ‘automated’ recommendations based on needs

During the interview, we found out that the client had many ideas for the recruitment feature (e.g., 'tinder-style' matchmaking). We probed deeper to find out what were the underlying objectives so that we can ideate more solutions to meet said objective and not be overly fixated on any single solution.

At this stage, we also grappled with the subject matter as we were unfamiliar with many of concepts, systems, and jargon used in the e-sports and Overwatch scene (e.g., LFG/LFT/LTR/LFR/LFS/LFC, leagues, tiers). We had to quickly adapt to ensure that we had the prerequisite knowledge to understand and communicate effectively our clients and users.

Heuristic Evaluation

Primary Goal

To quickly evaluate the usability of the current website and existing auto scrim/ranked scrim designs, based on recognized usability principles.

Methodology

• Nielsen’s 10 usability heuristics was used as a basis for evaluation
• The team independently evaluated each feature before coming together to collate and debrief our findings

Key findings

• A key issue in the client's designs was that proposed interactions did not match what was expected from a web-based, desktop platform. For instance, the use of 'press and hold' interactions, without any indication of affordances or onboarding.
• Other issues were mainly the lack of consistency and visibility of system status in the interface design.

The heuristics evaluation was useful in helping us quickly identify major usability problems in before committing to a prototype and user testing. However, we did face some limitations as we were basing our evaluation off mid-fi wireframes, and were unable to identify more complex flow or interaction issues. Furthermore, we were relatively inexperienced and our personal experiences may not be representative of actual users.

Competitive Analysis

The analysis revealed a market gap, with multiple competitor sites offering recruitment and match finding features, but geared towards casual gamers instead of professional e-sports players. Furthermore, we also identified key features that GetGood should implement or may wish to look into implementing as part of their expansion plans.

We also examined how competitors approached design problems. For instance, players may be part of multiple teams, or they may play duo roles as both player and manager of their team. We wanted to find out how the information architecture and interface would cater to these different use cases.

Round 2 - RESEARCH

Usability Test & User Interview

Usability Test Objective

• To test the usability (ease of use) of the proposed recruitment, auto scrim and ranked scrim features, as well as base website features

User Interview Objective

• To investigate the challenges & needs current Overwatch e-sports players face
• To examine user attitudes towards usefulness of GetGood's proposed features
• To examine user attitudes towards monetization and their willingess to pay

Recruitment strategy

• Via Overwatch community servers on Discord, as well as personal contacts

Incentive

• All participants would receive 1 month of premium service when GetGood launches

Key Tasks

• Registration(All): registerand set up profile
• Recruitment(All): reject ‘spam’ requests'; send invites on listings and via player/team profiles; create own listing; use matching feature, request to add/join team
• AutoScrim (Managers only): create scrim listing; find recommendations; request and confirm scrim
• RankedScrim (Players only): find a ranked match

Mode

Moderated remote testing via Discord, with the share screen feature

Participant Demographics

• n = 10
• 3 players, 4 managers, 3 players & managers (duo role)
• 5 Europeans, 2 North Americans, 2 Singaporeans, 1 Australian

The usability test was challenging for a number of reasons. Firstly, as the Overwatch e-sports scene in Singapore is still young, our target audience was mostly in the EU and North America where the scene is more mature. Hence, we had to conduct remote testing across 16 time zones. We were aware that participants' attention spans may be shorter than what we were used to, and tried to reduce the burden on participants by keeping the session short, without compromising on our research. Initial recruitment was slow, so we tried to solve for this by requesting for incentives from our client. and to also fairly compensate participants for their time.

For a majority of our participants, English was also not their native language. This was an added challenge towards communication; we had to pay extra attention to express ourselves clearly and simply, and also probe patiently to find out more about our users' thoughts & emotions.

Quantitative Results

Analyses revealed that SEQ (usability) scores were average, with room for improvement especially for the Recruitment task. Hence, we paid extra attention to the recruitment pages in the next design round.

In general, players also found the 3 proposed features very useful.

Qualitative Results:
Rainbow Spreadsheet

The usability test revealed over a hundred issues. We tried to analyse issues using a rainbow spreadsheet. This helped us identify whether multiple issues stemmed from a similar underlying problem. We also rated each issue on the impact on the user (severity & frequency), as well as effort required to address it to prioritise which to work on.

Qualitative Results:
Affinity Map

We identified 5 key challenges that current players face:
1. Lack of consolidation
(have to go to multiple mediums to find scrims/players/teams)
2. Difficulty finding quality players/scrims
3. Difficulty finding teams/players with specific criterias
4. Difficulty finding ringers and scrims last minute
5. Difficulty reviewing games/performance

The 3 proposed features that GetGood wants to implement meets the first 3 needs on this list. This validates GetGood's current plans and direction.

The remaining 2 challenges are opportunities for GetGood to step in and offer solutions for in the future.

Personas

As our stint with GetGood would only last 3 weeks, we crafted personas that our client could continue to use to guide both their design and business decisions going forward. We found some data supporting the notion that players from regions outside of EU, have their own distinct needs and motivations. However, we lacked data points to include them as full-fledged personas. . Here, they have been included as proto-personas, and we felt that these would be extremely useful as GetGood looks to expand globally

e-Sports Player/Manager Journey Maps

Investigating the needs of the users was one of the secondary goals as our client wanted to know what additional features to prioritise in their expansion plans. With that in mind, we crafted a series of journey maps which current players and managers go through. This is useful in helping identify future opportunities for GetGood to step in and offer a service.