Answering questions and setting expectations
On Udemy, if you enroll in a course, you can post questions in the course Q&A. As Udemy was growing, instructors were seeing an explosion of questions (sometimes hundreds a week), and often repetitive questions. This posed a problem for student as well as instructor satisfaction.
Team
Myself (primary product designer), 1 product manager, 2 engineers
My role
User interviews, user research synthesis, user journey mapping, competitive analysis, wireframing, prototyping, presentation
Tools
Sketch, Figma, Whimsical
Metrics
Instructor perception of repetitive questions; instructor and student sentiment
Outcome
Launched October 2019 to 60K instructors and 40M students, with over 30% of instructors noticing fewer repetitive questions, and positive instructor sentiment overall
Design process
As you’ll see, the questions guided a lot of the work. I conducted user interviews with Udemy instructors, synthesized existing research, and gathered feedback from multiple teams to develop frameworks for conversation and direction.
What were instructors struggling with?
I created a journey map (below) and summarized the big picture:
Pretend you are an instructor, and yay, your course is growing! But the flow of student questions—nice at first—is gradually becoming a big, expensive burden.
How are instructors answering all the students’ questions? Instructors have a variety of tactics
Who’s responsible for answering questions? Instructors have different viewpoints
Why are students asking so many questions in the Q&A?
Asking questions is a natural part of the learning experience. But courses exploding with thousands of questions (many of them repetitive) were taking a toll on instructors. We had outgrown the expert-based Q&A model.
HMW ease the burden on instructors, while making sure that students are still able to find the answers to their questions?
Our hypothesis is that by nudging from an expert-based platform to a community-based platform, we will alleviate the burden on instructors.
We studied the factors that make up successful community-based Q&A platforms:
Exploratory sketching
How might we use community dynamics to alleviate the burden on instructors?
Concept testing
After a round of internal stakeholder and user feedback with instructors, we learned and iterated based on the following insights:
Instructors, already busy with designing their courses, preferred as little additional work on their end as possible
As a product organization, we prioritized low cost, low risk updates that made foundational improvements for our users
Launch and reflection
In December 2019, the Q&A improvements were launched to the courses of Udemy’s 60K instructors, impacting over 40M students. After launching, 32% of instructors could immediately tell that they were receiving fewer repetitive questions after the update. Positive feedback from instructors continued to follow. Today, upvotes in the Q&A continue to enable additional product efforts across the company.
Looking back, if I had more time, I would have spent more time with students. I would have led a roundtable discussion with students and instructors, to better understand student and instructor mindsets. I also would have mapped out a timeline of our updates within the context of how we might imagine future projects, toward some future north star vision.
Yes! I love the idea of “Availability Statuses” and had requested it in the Facebook forum. Thank you so much “Question Upvoting” is also a great feature.
-Udemy instructor
Hello - thank for the Q&A update - I love that I can mark both my answer time availability and when I am away. I have built up my following by being available for “my tribe” and think it is great that they now have a clear expectation about when they will be answered.
-Udemy instructor