{"id":556,"date":"2023-01-26T06:18:45","date_gmt":"2023-01-26T06:18:45","guid":{"rendered":"http:\/\/michelle-le.com\/?page_id=556"},"modified":"2025-01-31T00:09:18","modified_gmt":"2025-01-31T00:09:18","slug":"error-handling-interview-concept-test-google","status":"publish","type":"page","link":"https:\/\/michelle-le.com\/?page_id=556","title":{"rendered":"Google | Error Handling | User Interviews & Usability Testing Concept Testing"},"content":{"rendered":"\n
\"\"
Note: the above images were taken from a public Google site<\/mark><\/span><\/a> and does not contain confidential images<\/figcaption><\/figure>\n\n\n\n
<\/div>\n\n\n\n

Error Handling @ Google<\/h1>\n\n\n\n
<\/div>\n\n\n\n

Challenge: combined functionality = combined errors<\/h4>\n\n\n\n
<\/div>\n\n\n\n

Ads Creative Studio (ACS)<\/span><\/a> was Google\u2019s new advertising platform. It acted as a unified home that enabled creative users to build video, display, and audio ads in a single place. As a result, it consolidated features previously spread across different products. <\/p>\n\n\n\n

<\/div>\n\n\n\n

However, it not only brought together all this great functionality, it also consolidated all error handling. So the errors previously spread across different products\u2026 are now smashed into one place. As a result, the current designs for showing errors (meant for one product) looked overwhelming when used in ACS.<\/p>\n\n\n\n

<\/div>\n\n\n\n

Opportunity: new error handling designs needed support and guidance<\/h4>\n\n\n\n
<\/div>\n\n\n\n

To help troubleshoot this, an interaction designer created new error handling designs. Error handling, for reference, is how systems identify what error was made, where it is located, and how to fix it. The new designs were made to consolidate errors in a way that wasn\u2019t overwhelming.<\/p>\n\n\n\n

<\/div>\n\n\n\n
<\/div>\n\n\n\n

Please note that these designs are NDA-protected<\/strong>, so I don’t show any pictures of the current and proposed designs. Instead, this case study will focus on the approach to research and the impact<\/strong> afterwards.<\/p>\n\n\n\n

<\/div>\n\n\n\n

Role: independently executed and managed the study<\/h4>\n\n\n\n
<\/div>\n\n\n\n

As a UX Researcher, I managed and led the study under the guidance of a senior UX Researcher. I scoped the research, shared the study with stakeholders, created the research plan and script, and implemented the study. My manager would be available to help and support me if I needed help, but I owned the project.<\/p>\n\n\n\n

<\/div>\n\n\n\n

Research planning: scoped research from a design review<\/h4>\n\n\n\n
<\/div>\n\n\n\n

My UXR managers said that a good senior researcher keeps an ear out for future projects. By actively listening to stakeholder discussions, they can anticipate what topics will have higher priority and impact – and thus, senior UXR\u2019s should be able to source projects that matter. My managers had heard about a new error handling design that was coming down the pipeline. So, they told me that this would likely be my project and to create my own study.<\/p>\n\n\n\n

<\/div>\n\n\n\n

When the designer introduced the new error handling designs as part of a weekly design review, I took careful notes of what was important to all stakeholders: the engineers, product managers, senior designers, and UX researchers. After compiling their questions, comments, and concerns – I grouped them by similarity, ranked them by how often those questions came up, prioritized them by how often those questions came up. This created the foundation for my research plan and the research questions.<\/p>\n\n\n\n

<\/div>\n\n\n\n
\n
<\/div>\n\n\n\n
\n
\n
<\/div>\n\n\n\n

Research goal: learn users’ workflow & tailor designs accordingly<\/h3>\n\n\n\n
<\/div>\n\n\n\n
\n
\n
\n

Learn how users handle errors<\/h4>\n\n\n\n
\n
<\/path><\/svg><\/span>How do users collaborate to troubleshoot errors?<\/span><\/div>\n\n\n\n
<\/path><\/svg><\/span>What types of errors should be blocking, where it needs to be fixed to move on? <\/span><\/div>\n\n\n\n
<\/path><\/svg><\/span>When do users want to see critical errors? Before export or real-time? Both?<\/span><\/div>\n<\/div><\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n

Gauge how current designs matches user workflow & get feedback<\/h4>\n\n\n\n
\n
<\/path><\/svg><\/span>How should the errors be grouped?<\/span><\/div>\n\n\n\n
<\/path><\/svg><\/span>Do the new design elements help users resolve the errors? Are too many callouts overwhelming?<\/span><\/div>\n<\/div><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
<\/div>\n<\/div>\n\n\n\n
<\/div>\n\n\n\n

Methods: semi-structured interviews & usability\/concept test<\/h4>\n\n\n\n
<\/div>\n\n\n\n
\n
\n
\n
Goal: learn how users handle error<\/h5>\n\n\n\n
<\/div>\n\n\n\n
\n
<\/path><\/svg><\/span>How do users collaborate to troubleshoot errors?<\/span><\/div>\n\n\n\n
<\/path><\/svg><\/span>What types of errors should be blocking, where it needs to be fixed to move on?<\/span><\/div>\n\n\n\n
<\/path><\/svg><\/span>When do users want to see critical errors? Before export or real-time? Both?<\/span><\/div>\n<\/div><\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
Semi-structured interview <\/strong><\/h5>\n\n\n\n
<\/div>\n\n\n\n

For the first portion of the study, I chose to conduct a semi-structured interview to understand participant background and product expectations. ACS hadn\u2019t launched at the time, so my team was still conducting foundational research to further understand the roles of each user profile. There hadn\u2019t been any research conducted on who exactly was involved in error handling or what the current process looked like.<\/p>\n\n\n\n

<\/div>\n\n\n\n

I wanted to be able to guide users through a series of questions about their current role and processes. However, I also wanted to be flexible and adapt if the participant shed light on a perspective or scenario that I didn\u2019t know about. This was especially important since the team was still learning about these processes. I didn\u2019t want to pigeonhole myself into something before I really truly understood the situation.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n

<\/div>\n\n\n\n
\n
\n
\n
Goal: Gauge how current designs matches user workflow & get feedback<\/h5>\n\n\n\n
<\/div>\n\n\n\n
\n
<\/path><\/svg><\/span>How should the errors be grouped?<\/span><\/div>\n\n\n\n
<\/path><\/svg><\/span>Do the new design elements help users resolve the errors? Are too many callouts overwhelming?<\/span><\/div>\n<\/div><\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
Walkthrough the mocks (not quite usability testing, but not exactly concept testing either)<\/h5>\n\n\n\n
<\/div>\n\n\n\n

Designers were mainly focused on the error handling’s tactical feedback. Since ACS had not launched (and engineers were still building the product), this was a completely new platform that participants hadn’t seen before. As such, we didn\u2019t know what users needed from a consolidated platform. <\/p>\n\n\n\n

<\/div>\n\n\n\n

To understand their workflow when tackling different types of errors across different stages<\/strong>, we needed to conduct a usability test<\/strong> and gauge how certain design elements functioned. We had static design mocks drawn up for this purpose. <\/p>\n\n\n\n

<\/div>\n\n\n\n

Since the designer wasn’t sure which combination of design elements<\/strong> functioned the best, we implemented a concept test<\/strong>. There were 4 variations anticipated, but since multiple variations had overlapping elements (e.g. variant 1 & 2 had XXX design element, variant 3 & 4 had YYYY design element) – it wasn’t a clear cut experiment. We still moved forward with the concept test, but I would focus on the specific design elements across variants (instead of declaring a specific design variant as the winner).<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n

<\/div>\n\n\n\n

Data analysis – affinity mapping<\/h3>\n\n\n\n
<\/div>\n\n\n\n

Although I take thorough notes during the session, I prefer to review the recording to make sure I didn\u2019t miss anything. During this time, I start to compile my raw notes with powerful quotes, pain points and feature requests, and design feedback. I\u2019ll write everything because I don\u2019t know what is a trend or a theme until I get more participants. <\/p>\n\n\n\n

<\/div>\n\n\n\n

Once I have all of my raw notes, I\u2019ll use affinity mapping to group similar themes together. (E.g. These are all the pain points – within those pain points, these two tend to come up – those were driven by XYZ \u2192 overall need\/solution is XYZ.) After I\u2019ve gathered the themes, I\u2019ll determine the severity based on how many times something came up and how important it is to the participants. (The importance is determined from participant feedback. I\u2019ve learned to always ask the participants and dig deeper: why is this important? How does this affect you? How often do you come across this? My manager mentioned that if I\u2019m advocating for something new, I\u2019ll need concrete proof to back it up.)<\/p>\n\n\n\n

<\/div>\n\n\n\n

Impact: informed future designs + generated engineering buy-in and support to leadership<\/h3>\n\n\n\n
<\/div>\n\n\n\n

Informed future designs<\/h4>\n\n\n\n
<\/div>\n\n\n\n

Detailed findings are NDA-protected<\/strong>, so I won’t list specific insights. Instead, I’ll share my general approach and why it’s important. <\/strong><\/p>\n\n\n\n

<\/div>\n\n\n\n
\n
\n

When sharing design insights, I prefer to show information top-down (where I start with bigger concepts before diving into the details).<\/p>\n\n\n\n

<\/div>\n\n\n\n

From the tactical findings, I created mental models<\/strong> for error handling. For instance, participants wanted to see errors ASAP and avoid a large list of accumulated errors. These mental models shed light on user behavior, enabling the design and engineering team to tailor the platform notifications accordingly. This helped us ensure that ACS would fit more seamlessly into their workflow.<\/p>\n<\/div>\n\n\n\n

\n
\"\"<\/figure>\n<\/div>\n<\/div>\n\n\n\n
<\/div>\n\n\n\n
\n
\n

Towards the end of my presentation, I shared the more detailed tactical feedback<\/strong>. I compile my issues into themes during the data analysis. When putting together the presentation, I take screenshots of the product or the recording to show where the issues exist. This provides them with examples for them to refer to and get a better understanding of what participants struggled with. I try to include this when compiling tactical feedback, since designers care deeply about addressing issues. Also, its dedicated section means designers can skip directly to the end for future reference. E.g. designers and engineers are anticipated to review this post-launch, so having everything in one place and with pictures means they remember this information a few months (or even a year) later.<\/p>\n<\/div>\n\n\n\n

\n
\"\"<\/figure>\n<\/div>\n<\/div>\n\n\n\n
<\/div>\n\n\n\n

Generated engineering buy-in & leadership support for design overhaul<\/h4>\n\n\n\n
<\/div>\n\n\n\n

Before the study kicked off, there were initial pushback from engineers on changing error handling. First, the platform was an engineering Frankenstein, so certain elements were built from other teams\u2019 code and it was difficult to add\/remove. Second, these were not consistent with Google Ads design guidelines, which was meant to give all Google Ads products the same feel. So changing it in one ad product meant changing it across all ad products. This needed leadership approval before engineers could work on this. <\/p>\n\n\n\n

<\/div>\n\n\n\n

To help overcome that, I emphasized with engineers to step in their shoes. I know they\u2019re busy working on building out the product and there\u2019s a tight deadline. These error-handling designs won\u2019t be considered until post-launch, so it\u2019s for later stages. The main goal is to save them time later on. To ask that they make sure the code is flexible for potential later on. This research is only meant to determine whether or not these changes are even needed. Also, the designer will be handling leadership persuasion. They won\u2019t be asked to build this unless it\u2019s approved. <\/p>\n\n\n\n

<\/div>\n\n\n\n

The results from the study showed that current designs were overwhelming and not helpful. The engineers agreed with us to revisit this post-launch. The study would also be used to support design changes toward leadership. Since Google has a consistent design guideline across all advertising products, making this change for ACS would require changing it for all of Google Ads. So, these insights would go on to advocate for these design changes.<\/p>\n\n\n\n

<\/div>\n","protected":false},"excerpt":{"rendered":"

Error Handling @ Google Challenge: combined functionality = combined errors Ads Creative Studio (ACS) was Google\u2019s new advertising platform. It acted as a unified home that enabled creative users to build video, display, and audio ads in a single place. As a result, it consolidated features previously spread across different products.  However, it not only […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"no-title","meta":{"_uag_custom_page_level_css":"","footnotes":""},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"adminmichelle","author_link":"https:\/\/michelle-le.com\/?author=1"},"uagb_comment_info":0,"uagb_excerpt":"Error Handling @ Google Challenge: combined functionality = combined errors Ads Creative Studio (ACS) was Google\u2019s new advertising platform. It acted as a unified home that enabled creative users to build video, display, and audio ads in a single place. As a result, it consolidated features previously spread across different products.  However, it not only…","_links":{"self":[{"href":"https:\/\/michelle-le.com\/index.php?rest_route=\/wp\/v2\/pages\/556"}],"collection":[{"href":"https:\/\/michelle-le.com\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/michelle-le.com\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/michelle-le.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/michelle-le.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=556"}],"version-history":[{"count":93,"href":"https:\/\/michelle-le.com\/index.php?rest_route=\/wp\/v2\/pages\/556\/revisions"}],"predecessor-version":[{"id":1239,"href":"https:\/\/michelle-le.com\/index.php?rest_route=\/wp\/v2\/pages\/556\/revisions\/1239"}],"wp:attachment":[{"href":"https:\/\/michelle-le.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=556"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}