Yellowdig was entirely rebuilt starting in 2018. It provided the opportunity to more easily meet modern accessibility standards while keeping what worked well, removing what did not, and taming technical debt.
I advised on many aspects of the redesign and managed the roll out to clients, which required a reintegration with each partner organization handled mostly during 2019 and early-2020.
It was built on a more modern tech stack enabling faster development, more stable releases, snappier and prioritized loading, more control over interface designs, and easier client customization and support.
The main single-feed concept was retained and spruced up while much of the interface and supporting pages were significantly redesigned to address a slew of user complaints and usability challenges I had identified through my user experience research, analysis of support tickets, and other voice-of-the-customer data.
Many of the following were part of Engage from the beginning, but following Agile principles, many were also added or iteratively improved shortly after release of the MVP version of Engage using actual user data and feedback.
Updating the look and feel (e.g., users disliked the yellow and it created contrast issues).
Ensuring we were exceeding increasingly strict accessibility and privacy standards.
From conversations in 2018 and 2019, it was clear there was a growing sentiment these were vitally important and procurement offices were becoming vocal about edtech companies not keeping pace.
Instructors had occassionally posted things in social media about Yellowdig that were bad for the brand, regardless of their absolute truth, and had started positively mentioning brands that they knew were leading in these areas.
Based on obvious moral and legal arguments and these market sentiments, I continuously surfaced accessibility and privacy information to ensure it was a positive differentiator of the new platform.
Improving LTI integrations and visibility into Learning Management System (LMS) setups (e.g., to avoid or quickly address grade sync issues) and creating and improving documentation for integration pathways like APIs.
Responding to usability challenges like the student experience starting in a universal feed that combined posts from multiple communities in one space.
Most students participated for individual classes and did not understand how their activity and points tracked to their courses, which often also had different assignments and point setups.
Faster and better content finding from incorporating an Elasticsearch database, more filtering functionality, and the inclusion of Topic tags to allow alignment of content with course designs.
New activity dashboards and point logs for students.
Helped them easier understand their point earning and progress, reducing confusion and resulting support tickets.
Instructors were given more ability to see and understand individual user behavior.
Enabled them to faster detect and intervene for disengaging students, empowered them to monitor subpar behavior, and understand point earning patterns.
Empowered instructors increased engagement metrics. Visibility also significantly reduced support inquiries to address student questions or understand concerning behavior patterns.
Gamified community health dashboards for instructors replaced raw statistics.
Used percentile scores of per student behavior and clear indicators that helped communicate whether their community participation was good, mediocre, or bad relative to what other instructors produced.
I had learned instructor activity strongly predicted student engagement and that instructors often thought what I knew were mediocre communities were as good as they could hope to get, often "blaming" busy, lazy, or disinterested students.
The dashboard motivated improvement. Instructors are a group who are not very used to "failing" grades, so low percentile health scores encouraged community presence and experimentation with more suggested practices and settings.
In Classic, there was a drop-down at the top where Learners and Instructors could move between "Boards" (i.e., content feed). It was possible to create a Board per course assignment and to integrate each Board so that students went directly from a course assignment to a dedicated area just for discussing that assignment.
For a normal course this created 10-15x as many content feeds in our system and a need for users and the company to move, manage, organize, and combine content, user data, and points across those spaces. For a course setup, Educators and Designers were also having to configure 10-15x as many integrations between the LMS and Yellowdig.
I had figured out Learners would almost never go to a prior space to respond to or add to conversations from previous weeks, new conversations had to start completely cold each week, and even if there were ok "Discussions" there was little building of supportive relationships or a sense of ongoing community. The topic-specific Boards had been entirely single-purpose, transactional, unnatural conversations.
It was clear from client conversations about the removal of this feature that those using it did not fully understand or believe the totality of the negative impacts (e.g., ~33% less student engagement; 10x as many support tickets; more LMS integration and LMS course copying issues leading to point sync problems, etc.).
I knew this choice would have many positive impacts for Educators and Designers, but those stakeholders saw it as disruptive for focusing Learners on very specific topics and assessing their learning about those topics. Those two goals may be important, but they are unique goals of Educators and Designers and I had plenty of evidence that highly focused "discussions" within their own spaces created an experience that was far less engaging.
Unfortunately, despite our inclusion of Topic Tags to address those needs, this decision was very unpopular among the Designers, Educators, and clients who were most heavily using the integration functionality.
Some Institutional Leaders also thought the integration pathway was indispensible enough functionality to their Educators and Designers that they delayed adopting YD Engage despite its many improvements. That also delayed our sunset of YD Classic. Some amount of client frustration with our choice was because these were already the clients least happy with Yellowdig's outcomes and seeing the least value from using it; among our users they had the least reason to trust Yellowdig's technology or our judgment as a company.
It seemed clear that those most insistent on needing this function would just continue to abandon the product at higher rates even if we decided to provide it.
Most Educators and clients ultimately made the switch and became much more satisfied. Most importantly, this choice safeguarded new adopters from making a sub-optimal choice. Going forward most came away with a much stronger sense that Yellowdig "worked."
Learners who had multiple courses with multiple conversation topics had less clutter in their navigation interfaces. If a Learner ever wanted to revist a prior week's information they would not need to change spaces to a completely new feed or understand how points associated with different spaces rolled up to their grade for a course. It was significantly easier for them to think about each course having one community containing many topics that appeared in an endless feed they could easily search and filter.
For Educators, Designers, and IT Professionals, executing one integration and checking it was far easier than setting up or maintaining and checking 10-15x more integrations per course launch. Errors in this process were creating a huge source of support tickets and frustration, especially at already-busy term starts. It also created high-tension support situations, requiring Yellowdig to help instructors with frustrated students move data and points toward grades for those who accidentally misdirected their LMS links to the wrong Yellowdig feed.
In natural data I had found designers and instructors who were using only one feed for the whole course. They and their students produced almost no support tickets and even though many were still using discussion board designs, Learner participation was also over 50% higher on average.
When one of our largest clients was not happy with our "One Course, One Community" choice and wanted us to add it back, I convinced them to experimentally test the implementation difference by showing them my prior analysis and new data showing people who had switched to Engage got about 25% better engagement. The client was a major player in online education that we wanted to expand our partnership with, so we definitely could not just ignore them, but we did not feel we could deliver on their demand because my analyses were making us increasingly confident the choice was the right one.
When the experiment with six sections integrated from only one LTI link produced >60% more student engagement, it experimentally verified my prior correlational analyses and broke our stalemate. They went from being modestly "happy" to legitimately thrilled with the student engagement, especially after they later verified better term-to-term retention for those sections. They bought heavily into community as the primary driver of engagement and retention and experimented more with our other best practices. From that work, they saw Learner retention benefits that more than justified Yellowdig's costs and they reworked their entire existing course catalogue to remove discussions and include Yellowdig communities. They are now Yellowdig's largest revenue enterprise contract.
Rather than enable multiple integrations directly to specific course topics, we added Topic tags to communities to "house" conversations within a single, more constantly active space.
They were bright clickable tags affixed to every post.
Multi-selectable by Learners while creating a post, to enable more accurate tagging and encourage Topic blending.
Displayed at the top of posts in the feed and in a module to the right, where they were clickable as feed filters that worked essentially as channels would in Slack. A list of Topics also appeared in the feed filtering interface.
Educators had options and control:
Topic names and colors customizable by Educators or Designers.
Educators could require a Topic be added to each post, use a system reminder for Learners at post submission, or make them completely optional.
Later, No-Point Topics were added for instructors who wanted to allow social interaction that did not earn course grades (e.g., "Lounge" or "Coffee House"). These were indicated with desaturated tags (see below).