This Summer, I joined Atlassian on their mission to unleash the potential of every team through Help and Support Design.
Responsibilities: UX Research, Product Design, Service Design
Role: Product + Service Design Intern, Summer 2018
Collaborators: Bryce Hays, Kevan Lin, John Collins, Yisi Lin, Kathryn Taylor
Help and Support Design
I had the opportunity to work with Help and Support Design where we designed service channels to help users understand how to use Atlassian products.
Project: Atlassian Community
Community is a platform where users can find answers, support, and inspiration from other Atlassian users. Anyone can engage with Q&A and Discussion posts on product or non-product related topics.
Users on Community helped generate great content.
With great content comes great discoverability challenges.
Our team wanted to figure out how Community could help users find great and relevant content. I ran an experience canvas workshop with Bryce and Kevan to help clarify what problem our project is trying to solve, the customer(s) we're solving it for, and what success looks like.
From that, we realized that much of this great content that users have created are found on Q&A and Discussion posts.
It became more clear to us why that was the case from a Community member:
We looked up statistics to prove this quote. (We didn't have a product manager assigned to this project, so we got to be our own PMs!)
We found that the answers on Q&A posts had a 25% acceptance rate in a given month, leaving us unsure if users are having their questions accurately answered.
We also saw that Discussion posts only had an average of 4 replies per post. With over a million members on Community, we expected to see more replies.
I was also interested in hearing from Community experts and listen to any observations they had.
I summarized the three main observations that Community Managers noticed about Community. Most importantly, they believed in the importance of accessing great content and the difficulties of finding that content on Q&A and Discussion forums.
Benefits of Accessing Great Content
What happens when users can easily access existing great content?
Especially because I was new to Atlassian and Community, I wanted to learn more about the motivations behind why users go to Community.
After talking with Community members and managers, I narrowed the motivations down to three values and aligned them with the values of the company.
From those conversations, this quote from a Community manager stood out to me about why Community benefitted both the users and company.
According to Community’s 2018 reflections, a majority of Community members liked it when Atlassians interacted with them on the Q&A and Discussion posts. It helped humanize products that they often felt disconnected from.
Talking to Atlassians through Community and seeing how receptive Atlassians were to users’ feedback made users feel more engaged with the product because Atlassians were taking their feedback seriously and translated their feedback into actionable items.
I distinguished two types of goals for Q&A and Discussion forums.
I considered Q&A to be goal-based, where a questioner is concerned with finding the most helpful and relevant answer(s).
I considered Discussion to be non goal-based, where a poster is concerned with finding interesting content.
Having reactions can help emphasize great content. Reactions are an interactive way for users to leave quick and meaningful feedback on content that can help others find great content.
I also thought of reactions as an actionable method for the majority of users, who may not have anything to add via a comment or reply.
How are online communities allowing users to give meaningful feedback?
I explored ways to give meaningful feedback and looked to how other places were answering the same question of how online communities were using these methods such as reactions.
I compared how various online communities were taking note of users’ interactions with posts and comments.
Based on my analysis, I used Microsoft Answers as my primary example because I noticed how Microsoft Answers was addressing a similar question to what Atlassian Community was addressing, which was: “How to help users find, access, and identify great content on product Q&A and Discussions?”
Microsoft Answers were using statements, like “I have the same question” and “Did this solve your problem? Yes/No,” for users to answer on Q&A posts. To Microsoft, having specific feedback to certain questions will lead to a higher rate of accepted answers.
Same goes for Discussion posts, where Microsoft Answers were using statements, like “I recommend this discussion” and “Up vote (# of users who have up voted),” to elicit specific feedback to lead to interesting content.
Microsoft Answers had confirmed my hypotheses for Atlassian Community by using goal-based reactions to lead to accepted answers on Q&As, and non goal-based reactions to lead to interesting content on Discussions.
Testing with Users: Goal-based Reactions (Q&A)
I tested 2 different methods of goal-based reactions for Q&A posts. I used a question to test if users preferred a limited method (that elicited specific feedback) or an upvote reaction to test if users preferred an open ended method (that elicited a more ambiguous feedback).
Based on the results, users preferred having upvotes as a way to react because:
1) It was clear that pressing on an upvote reaction meant recommending a content.
2) The location of the upvote made it easy to skim and see how many users recommended a content.
3) It was clear that by recommending a content, the user’s upvote will affect the order of the content on a post.
Testing with Users: Non goal-based Reactions (Discussion)
I tested 2 different methods of non goal-based reactions for Discussion posts. I created a written text field to test if users preferred an open ended method (that elicited unique feedback) or a pre-defined set of reactions to test if users preferred a limited method (that elicited specific feedback).
Based on the results, users preferred having a pre-defined set of reactions to give feedback because:
1) The pre-defined set of reactions were clear and meaningful, and it was understood by users what the feedback meant.
2) Having pre-defined options was a quick and efficient way to give feedback.
UI Button Explorations and Testing with Community Champions
For Q&A posts, I explored how the voting system can help lead to a higher rate of accepted answers, ultimately leading to helpful and relevant answers.
The arrow buttons were clear because they indicated that a recommendation by an upvote would affect the shift the order of the content, while the +/- buttons were not as clear that a shift would occur.
The upvote itself was clear because users can measure how helpful and relevant an answer is by the number of positive recommendations. When I tested a voting system with upvotes and downvotes with new Community members, they said that they preferred seeing a positive and negative ratio of recommendations. However, with Community champions, who were seasoned Community members, they emphasized that having only upvotes was preferred because a ratio of positive and negative recommendations would make it more confusing to decide which content is helpful and relevant, and downvotes did not positively contribute to the “Community culture.”
Based on user feedback, I decided on having only the up-arrows (right option on image).
I wanted to test the ‘+1’ button because I noticed that Community members adopted a trend of commenting ‘+1’ if they agreed or liked a comment.
However, from the Community champions and new Community members I talked to, they preferred the ‘thumbs up’ button because the icon contributed to the “Community culture,” while the +1 had an ambiguous meaning.
I assigned the word ‘recommendation’ with the icon to ensure that users would use the button to recommend the content. At first, I tested with the words ‘liked,’ ‘supported,’ and ‘agreed’; However, the word ‘recommended’ applied to most use cases and implied a conscious way of giving feedback that benefitted others. I wanted users to support content that they thought was worthy enough to recommend to others.
Based on user feedback, I decided on having the ‘thumbs up’ recommendation button (bottom option on image).
By adding goal-based and non goal-based reactions, these reactions will allow users to: 1) find great, relevant content, 2) improve moderation on posts, 3) help others find great content), and 4) engage more on posts.
Overall, many users believed that adding reactions was seen as an improvement from the current experience.