Mediary

OVERVIEW

Mediary is an AI-based medical support mobile app that uses advanced transcription technology to convert medical consultations into clear, personalized, and actionable guidance that is tailored to each patient's capabilities and needs. It is designed to help you maximize the benefit you receive from each and every medical consultation.

ROLE

Product Designer

CONTENT

Research, Interaction, Visual Design, Prototyping & Testing, Information Architecture

October 2023 - April 2024

BACKGROUND

After a few years away from UX design, I updated my skills in 2023-2024 by completing the UX & Visual Interface Design Certificate at the University of Washington. The course covered Design Thinking, Prototype Modeling, Usability Testing, Research Techniques, and Accessibility.

I designed Mediary, a medical support mobile app, for my capstone project. Despite many existing apps that each address a small component of a patient’s needs, my goal with Mediary was to create a tool that genuinely addresses patients' real world needs and challenges, providing more than just basic navigation through their health journey.

THE PROBLEM

Imagine sitting in an exam room; it's cold, and it's intimidating. You're already feeling nervous and overwhelmed just being there. Eventually the doctor comes in, looks you over and then gives you the results of your blood tests... it's not good. The doctor continues to talk, but you feel panic crash over you, and a loud buzzing sound has filled your ears. You can't make sense of anything the doctor is saying. Though you know the information is important, you can't absorb it.

Even a routine checkup can feel this way for many people. The inability to process this information is exacerbated by stress, information overload, cognitive difficulties, language barriers, and emotional and/or mental conditions.

A study in 2023 conducted by the Agency for Healthcare Research found that patients, in general, will immediately forget 40 – 80% of the information they receive during a medical consultation. And of what is remembered, only about 50% is recalled correctly. They also found that only 12% of Adults demonstrate adequate health literacy skills. That means that 88% of adults DO NOT have the ability to effectively obtain, understand and use health information to make informed decisions. [The Cardiology Advisor]

Being able to effectively process, retain, and apply information and make decisions based on the information received during a medical consultation can prove critically important to a patient’s medical outcome.

PROBLEM STATEMENT

“How can we help the user effectively process, understand, recall, and apply the information received during a medical consultation?”

THE RESEARCH

Working with a team of four peers, we started this project by conducting market research and pinpointing available design opportunities.

EXECUTIVE SUMMARY

SITUATION

Despite having multiple tools available to help them, individuals dealing with the stress of a new medical diagnosis often feel completely overwhelmed and lost in the medical process.

TENSION

Navigating the complexities of the medical system can be stressful and confusing under the best of circumstances, but facing additional strain from an unforeseen medical diagnosis can make patients’ situations and care extremely difficult to manage by themselves.

OPPORTUNITY

Those facing unforeseen medical diagnoses can benefit greatly from clear medical guidance and targeted support, creating the necessary environment for reaching a full medical recovery or resolution.

COMARATIVE/COMPETITIVE ANALYSIS

Together, we explored and evaluated several existing medical apps, while looking into other areas of medical care including awareness campaigns, forums, competitions, incentives, etc., and even law.

USER INTERVIEWS

From the information gathered, we put together a user interview discussion guide focusing on the three following objectives:

To learn about the user’s experience, goals, needs, and pain points as they pertain to how we might respectfully empower patients and their friends and family to navigate and support their diagnosis while honoring the wishes of the patient, reducing stress on the community, and maximizing the effectiveness of everyone's contribution.
To learn about support systems for those who struggle with diagnoses and those who care for them.
To learn about different angles of care - caretakers, loved ones, and those who experience illness themselves.

In total, we conducted 18 user interviews divided amongst the team.

Summary: Our main goal was to identify jobs to be done, pain points, behaviors, and solution opportunities.

ARCHETYPES

JOURNEY MAP

THE CONCEPT

At this point our group branched off to work on our own concepts. My goal was to create a tool that would help the user remember, understand, and effectively apply the information they received despite some emotional or cognitive difficulties.

After several sketching sessions, mapping out various user flows, and creating a series of user stories, I put together a list of 4 simple, must have, key features:

KEY FEATURES

Consultation Recording: Must enable the user to record (from smart device) their medical consultations effectively.
Accessible Consultation Content: Recorded content must be accessible in various formats (i.e. audio, text, summarized, translated, etc..) to ensure the user can process and apply the information to the best of their ability.
Shareability: Provide the user with multiple options for saving, reviewing, printing, and sharing their medical consultation information with family or friends for added support and understanding.
Translation Services: Minimize the language barrier and knowledge limitations in receiving effective medical care and direction through language translation services including AI generated medical terminology definitions.

In support of these features, I wanted to include a set of Integrated Tools that would effectively enhance understanding, improve retention, and support the application. Tool ideas: Medication Reminders, Appointment Specific Question/Concern List, General Reminders, Action Items list, etc..”

THE TESTING

User testing was an iterative process that was conducted at regular intervals throughout the project to identify pain points and areas for improvement. Below is a summary of the main take-aways.

INITIAL CONCEPT TESTING - Wireframes

For the initial concept testing we put together four wireframes, each illustrating a main component of the design. The initial concept was overall well received, particularly the recording/transcribing and translating capabilities. The Diary and Appts & Reminders features were not valued as highly.

What was loved about my concept:

Recording/Transcription/Notes Tool
Translation Services
Appointment Review Page

What they wish they had:

Suggested Question feature
- Include a suggested AI generated question list during an active appointment.

INTERACTIVE IDEATION - White Models

Moving into the interactive side of things, I put together more defined user flows and a detailed site map. I also quickly explored different layouts and transitions by constructing several interactive paper models which I tested out on my cohorts.

USEABILITY TESTING - LoFi Prototypes

Using the insights gained, I created several lo-fi prototypes in Figma for useability testing. I enlisted 5 individuals to test the prototypes by accomplishing several pre-decided tasks within the app.

Feedback gleaned from useability testing:

Add Notes section: Users expected to be able to add notes during an active recording. Initially I assumed users wouldn't want to add notes while it was being actively recorded, I was wrong in this assumption. Solution: Based on further detailed feedback, I decided to remove this feature entirely as it added to much confusion.
TRY IT OUT button on the welcome page was too ambiguous. To create soft onboarding, I used a TRY IT OUT button on the Welcome screen to allow the user to choose how they proceeded through the app. Unfortunately, they were confused as to the intent of the button. Solution: Added a SETUP ACCOUNT button
The nav styling and page identification was inconsistent: This caused some users to feel disoriented in the app. Solution: Create a refined navigation method that orients the user and allows for easy navigation.
Questions & Concerns tool is confusing: Though extremely well received, users were concerned that the Q&C Checklist did not offer a way to add notes or answers to each list item. Solution: Revision of design. Adding the ability to add answers would streamline this content and make it a bit more intuitive.

THE SOLUTION | Hi-Fi PROTOTYPE

RECORD CONSULTATION - Primary feature

Many people struggle to process the information given to them at a medical consultation. Whether due to stress, complexity of information, or other factors, it is extremely hard to absorb everything presented to you, let alone remember it and apply it correctly. How do we capture the information received during a medical consultation and format it in a way that helps the user understand it better?

The Record Consultation feature allows the user to record the raw audio from their medical consultation. Once captured, the audio is transcribed and processed, with the help of AI, into various formats that enable the user to choose how and when they want to access the information, and how the information is formatted, so that they can get the most benefit from the information given.

To remove language barrier issues, I have also included a translation option that is available in multiple configurations and adjustable in the profile settings.

AUDIO RECORDING & TRANSCRIPTION

Includes:

Raw Audio
Timestamps
Speaker Names
Highlighted Medical Terms

SUMMARY OF TRANSCRIPTION

The consultation contents are consolidated and organized into main points, making the information easier to manage and absorb. A sampling of these sections include:

Health Update Overview
Next Steps
Pinned Medical Terminology: Mediary will highlight medical terminology in the transcription. If you would like to see an explanation/definition you can tap on the highlighted term. This will open a window with the explanation/definition. To include it in the Summary, tap the pin icon at the top and it will be added to the Summary. It can be translated into your primary language for better clarity.
Personal Action Items List: This is a list of action items that were pulled from the transcription. This list would include things discussed during the consult like, "Increase cardio workouts to x3 - 30 minute sessions a week." or "Drink 120oz of fluids a day."

Screens: Transcription Summary, Summary with Action Items shown, and Summary in Spanish

AUDIO PLAYBACK

This feature allows you to playback the audio from your medical consultation, pause, and move forward or backward. This is particularly helpful for auditory learners or those with reading difficulties. As the audio is played, the transcript will highlight the words as they are spoken so that you can easily follow along with the text if desired. You can also playback in original audio, your primary language (translated), OR both.

MEDICAL TERM SEARCH

A lot of us are baffled by confusing medical terms or jargon. AI will actively search the transcript for medical terminology and highlight the terms for you. Tap on the highlighted term to open a window containing a definition/explanation. For future ease of reference, you can pin the definition to the transcript, and it will appear in the summary.

Medical Term Definitions can be accessed during an active recording or in the transcription. You can also ASK MEDIARY (see the ASK MEDIARY section) to explain something to you at any time.

ASK MEDIARY

Ask Mediary is an interactive AI assistant that can help you in many ways including:

Ask a medical question
Set a new reminder OR change an existing one
Add a question or concern to the Q & C list for an upcoming appointment

Once you have made a request, Ask Mediary will have you confirm your request, complete it, and then show you the changes where applicable.

Optional: When privacy is needed, or writing out your request is preferred, you can click on the speech bubble and open a text interface.

PRIMARY & SECONDARY PROFILES

Home Page

Many of us, now or in the future, will be responsible for the healthcare of a dependent, whether it be an aged parent or minor child. How do we effectively incorporate dependents’ healthcare into Mediary?

Primary and Secondary Profiles allow the user to track and organize their own medical care and the care of minor or adult dependents.

A secondary profile can have varying levels of autonomy. It can be fully controlled by a primary profile, for fully dependent minors and adults. OR the secondary can operate as an independent account while allowing primary access. Which would be ideal for teenagers (13+) or independent adults who want a primary to help manage their healthcare.

SUMMARY

MEDIARY REVIEW & MARKET POTENTIAL

The prototype received enthusiastic feedback from several potential users, indicating a promising market for a tool like Mediary. However, the necessary financial support to advance it to production is currently lacking.

KEY CONCERNS

Privacy: Privacy is a major concern for many potential users, particularly when it involves sharing detailed medical information with an AI system. To maintain user trust in data security, we must obtain informed user consent and be fully transparent about the data we collect and how it is managed. This includes secure data storage with multi-level encryption and designing user accounts to ensure anonymity.

AI Accuracy: As AI becomes more integrated into society, its logical pitfalls become more apparent. This can be particularly dangerous for users seeking medical advice or when information is transcribed incorrectly. To mitigate these risks, we must include clear disclaimers and ensure our AI model is as accurate as possible.

NEXT STEPS

This product was initially designed for educational purposes, but I intend to continue refining it and exploring further advancements in AI technology and integration. For example, developing an AI-driven "questions to ask" feature that can be generated during a medical consultation to help users know what to ask, thereby becoming better informed about their medical condition.

Additionally, I aim to consult with individuals who have diverse cognitive issues, such as ASD, TBI, and Dementia, to gain a comprehensive understanding of how to create a more robust and accessible user experience.