Toniq Research
- Role - Product Designer
- Project Type - Research for internal data fluency web platform
- Duration - Ongoing
- Team - Full-stack Developers, Data and Machine Learning Scientists, Project Managers
Toniq is an end-to-end MLOps platform for healthcare designed for both data professionals and external stakeholders like clinicians and medical professionals. It aims to be a data fluency platform that aims to accelerate the innovation lifecycle in the healthcare industry by providing tools for data discovery, model prototyping and operationalizing actionable insights.
This part of the project focuses on design-led user research that propose recommendations to improve the current prototype that align with business objectives, product needs, and user goals
The Myasthenia Gravis use case
The concept of Toniq was inspired by an earlier study conducted with the team about Myasthenia Gravis (MG) patients. 90% of respondents of the study reported that they would use an AI model to predict their flares, while 78% would use an app to monitor their symptoms. As a result, the main problem was determining the feasibility of collecting voice and video data to build clinically validated AI models for objective detection of MG symptoms and identifying changes before, during and after flares.
Doing this would require an internal MLOps platform that allows us to build our own AI models with voice and video data from our own studies, predict the severity of MG symptoms and identify subtypes of MG. Identifying changes before, during, and after flares based on passively collected data, would allow us to build clinically relevant AI models for detecting MG.
There is a need for user research to better understand the challenges, assumptions of creating such a product, and generating recommendations for the next iteration of the platform.
Objectives
We started that by drafting a user research plan delineating the project's context, objectives of the research, our research questions and assumptions, and research and recruitment methods. This document ensures that the research was aligned with the needs and concerns of the product manager and stakeholders while giving design guidelines to move forward with.
These were the objectives:
- Examining the MLOps industry and challenges to applying an MLOps framework in a healthcare context.
- Understanding the challenges research for internal data fluency web platform
- Challenging assumptions about the target users and problem-to-be-solved.
- Developing proposals for product offerings that may appeal to the needs of the target user.
The MLOps Industry
Given that design had formal education in AI or machine learning, I conducted a deep literature review to gain a better understanding. The review was documented and provides a broad overview of the AI/ML domain, including analyses of emerging technologies, market trends and capabilities. It serves as a resource guide for designers, to understand the impact, players, and vocabulary surrounding AI and MLOps to help them contextualize design in this domain and develop a sense of fluency and curiosity about this rapidly evolving industry.
We learned in this research that the global artificial intelligence market size was valued at $62.35 billion in 2020 and is expected to expand at a compound annual growth rate of 40.2% from 2021 to 2028. Continuous research and innovation are driving the adoption of AI in industry verticals, such as healthcare, driving the need for a more seamless approach to the machine learning process.
MLOps marries ML model development and operations, aiming to accelerate the entire model life cycle process. MLOps drives business value by fast-tracking the experimentation process and development pipeline, improving the quality of model production—and makes it easier to monitor and maintain production models and manage regulatory requirements. The MLOps market is expected to expand to nearly $4 billion by 2025.
A major challenge with MLOps is that organizations are constrained by artisanal development and deployment techniques, dependent on singular data scientists. These models are developed and deployed using manual, customized processes that aren’t scalable.
Though I wrote dozens of research questions inspired by the review, here are a few that were most critical to understand:
- What are key differentiating features a health MLOps platform has over more traditional platforms?
- What are some usability gaps in the existing Toniq platform?
- How can users who are not as data fluent or technical best contribute and add value to the MLOps workflow?
Challenges and Assumptions
Based on the existing framework and needs of the platform, the team knew the main target users would be both data scientists and clinicians. I wanted to learn how their need for an MLOps platform and other challenges and limitations they see. After conducting some user interviews with our target users, I identified the following core challenges:
- Building models for healthcare requires aligning with specialists such as clinicians and caregivers, who may not have the required data fluency to validate models.
- Communication is crucial in healthcare. Data scientists and clinical researchers may not understand each other’s approach and needs and need a way to facilitate conversations cross-functionally
- There’s a lack of model explainability and erodes the trust of stakeholders that want to apply the models.
I also aligned with some assumptions that need to be further validated in later usability testing, including:
- Stakeholders want an end-to-end platform for MLops in health
- The platform does not need to match the full MLOps process in order to maximize conversions and return users.
- Non-technical users may not want to be involved in every aspect of the MLOps workflow, but including them throughout is important in maintaining trust.
- Users prefer to rely on just one MLOps platform.
- Novice users may want a “quick” way to ease into the product
This diagram illustrates the complexity of the challenge of incorporating key stakeholders, including non-technical clinicians, into a framework that's inherently technical and optimized for data scientists:
Competitive Analysis
Next, I needed to understand how the current Toniq platform follows the typical MLOps funnel. Doing this allowed me to dive deep into the product and understand how our data scientists use the product.
I also conducted a competitive analysis of the major MLOps competitors in the market and created flows to compare and and more deeply understand how the products differentiate from one another.
Based on the competitors, I narrowed down relevant features and identified key opportunities for the initial concept that also align with the needs of stakeholders: no-code visualization, collaboration and bolstering data fluency.
Heuristic Evaluation
In the short term, I had to prioritize evolving the current developer concept as the proposals generated from user research would need to be scoped out strategized in the long term. A heuristics evaluation identified many usability issues and “quick win” opportunities to improve the current platform.
