Associate Tasking.

Project for Research Methods for HCI @ Gatech
Client: The Home Depot

AR, B2B, Accessibility, Internal Management, Retail

Aug. '18 - Dec '18

Yizhou Liu, Xi Chen, Taylor Stillman, Yannu Lee

Pen + Paper, Sketch, Invision, Principle, Adobe CC

My Contributions

1. I took part in all research methods and worked with teammates to analyze our data and distilled the user requirements.
2. I actively contributed in the ideation, sketch and wireframe phases. The general user flow and visual style were revised and finalized by me. I also worked as facilitator and notetaker in both heuristic evaluation and usability testing.
3. I was the contact person for our industrial partner. I negotiated and scheduled several meetings, also continuously communicated our needs with Home Depot.

A better workflow with AR technology for associates' work.


Design Problems & Outcome

Our target users are Home Depot associates who need to do bay checking and in-store projects every day. We have 3 main challenges based on our research result.

Therefore, we presented a new APP for MET associates. We incorporated AR technology to help users check the shelves in a more efficient way and also, provide an expert user flow to help associates have more freedom in their work. Chick to see our prototype.



The employees in the Home Depot store are responsible for different kinds of work. Therefore, our design prompt is to optimize the working efficiency and experience of ‘MET associates’ who are the employees dealing with the product arrangement and display.

Expert Interview with Industrial Partner

At the very beginning, we interviewed our industrial partner: Home Depot. We tried to figure out their business objectives and expectations to our project. Also, with zero background knowledge, we designed questions to grasp a general understanding of the user group. Below are some main concepts for your information.


  • On-shelf availability is the most fundamental element to the sales.
  • The company want more data of their stores and associates for future optimization.
  • Quality should be more important than quantity in their associates' work.


To familiarize ourselves to our target users’ working flow and working pace, we did an observation in the store. We seperated into 2 groups: one followed the associates closely for detailed tasks, another followed in a distance to know their working pace and normal behavior (because staffs might act differently when feeling that they’re being observed).

Pictures taken during observation


1. What are their work flow and tricks to finish the tasks.
2. How associates work and communicate with each other.
3. How supervisors check the quality and give guidance to the associates.

Task Analysis

After observations and interviews, we created hierarchical task flows of MET associates’ and main stakeholder: supervisors’ daily work. We highlighted the difference of standard workflows and actual flows to find potential problems and design opportunities.
For associates, their work are mainly regular bay checking and special in-store projects(shelf and rearrange the products).

Contextual Inquiry

After the general task analysis, we intended to know more about users’ thoughts and motivations during the process. Also we found some places that associates’ actual working process are different from what the experts’ expectation. That is why we started contextual inquiry to understand why this occured and know more about their challenges, pain points, and expectations when finishing bay services and projects. When they were working and conducting tasks, we encouraged them to use think aloud protocol and we observed their behavior, asked clarifying questions, and took notes.

We conducted contextual inquiry for 3 associates working with bay checking at 2 Home Depot stores separately.

Notes taken during contextual inquiry


1. Expert associates have already fully memorize the whole flow so they feel troublesome and unnecessary to click on many checkboxs every time.
2. The tasks about planogram are most troublesome and time-consuming for associates.
3. The communication between supervisors and associates are weak and inefficient.

of Findings

Task efficiency, Flexibility
and Communication.

Affinity Diagram

An affinity diagram is made to conclude out the data from former researches especially contextual inquiry.
Click here to check the details of #Affinity Diagram.

When the project went here, we just learned about a UX technique called Jobs-to-be-done. It focuses more on users’ desired outcomes and is more helpful to assess real user needs rather than the superficial needs encumbered by existing products.
Therefore, we decided to use it to summarize our insights from affinity diagram and criticize whether current activities of associates can reach the outcomes that users and higher leadership are seeking.

Wrap-up: Jobs-to-be-Done

We incorporated it to walk through the entire user experience. Building empathy from both task-based and cognitive perspective, we thought more about the desired outcomes of associate tasking to finalize the problems that we defined.



After defining the desired outcomes and pain points, we conducted 2 rounds of brainstorming and generate 50+ possible ideas. At first round, we individually come up with ideas and then discussed together to group them into 12 categories. Based on these, we brainstormed a second round and get more ideas and created a mindmap to wrap it up.
Click here to check the details of #Mindmap.

Idea Evaluation

We evaluated our ideas in 2 dimensions: Creativity and Feasibility. Since it’s a B2B product, feasibility is more important for the business side. Therefore, we picked all ideas with high feasibility. Among these ideas, those with higher creativity is suitable to dig deeper into and those with lower priority need a verification of their effectiveness in the following stages.


We merged the picked ideas and eventually got three potential design directions. Two of them are further exploration of creative ideas and one is from the feasible ideas with low creativity. Since associate tasking is a complex problem, the 2 creative ones does not cover the supervisors’ side.

Design Alternatives

We built 3 design alternatives based on the design directions. We referred to our JTBD result when building these. Therefore the alternatives focused on the outcomes of each job to provide different methods rather than just optimizing the current user flow that they're using now.

#AR + AI Assisted Tasking

Associates could use the camera to scan the bay areas and shelves and AI could recognize the potential availability and shoppability problems. Associates could just need to follow the guidance and finish the checking.

#Voice Assisted Tasking

Associates will wear earphones and use voice to ask about instructions, report the working progress and problems and communicating with supervisors for help. Using NLP, system will generate related options, actions and information to assist users' work.

#'Freestyle' bay checking and in-store project assistance

Detailed tasks are categorized and users can choose their own tasking sequence. We named it as 'freestyle' checking. Also, for in-store projects, they can set alarms to remind of their own schedule and do report by uploading photos.

Feedback Session

We created paper prototypes first and ran an feedback session to 2 experts in Home Depot. Our design alternatives changed the regular associate tasking methods. Therefore, in this stage we didn't go to the users, but asked experts to evaluate the value of each feature. We created 3 evaluation dimensions and got feedbacks about thier pros and cons as well as suggestions.

Paper prototypes

    #Evaluation Dimensions

  • Feature effectiveness: Do you think it can improve the current efficiency and experience?
  • Feasibility: Does it have any difficulty to implement in reality?
  • Business objective satisfaction: Does this feature meet their business value?


  • AI Accuracy. Poor accuracy of machine learning algorithm may influence the automatic planogram checking.
  • Voice accessibility. Voice is a good interaction method, but wearing earphones may disable them to notice customers nearby. Also, the noise in the store is an important accessibility problem.
  • Unnecessary photo quality checking for each bay. From the business side, they don’t want to build strong AI capacity to just check the quality of every bay tasking by photo and supervisors’ are also too busy to do that.

Based on these ideas, we decided our final design directions.



After deciding the design directions, we refined our design alternatives and combine the features. Then, we went to the home depot store and ran a quick feedback session and went through it with 3 associates. The feedback we got helped us gain more insight to make the prototype.


  • Generally speaking, users like the AR-facilitated feature, but also mention that hardware improvement for the is needed for these features.
  • Accessibility issues: Buttons on AR interfaces and report keyboards may be not so accessible to click on.
  • For AR checking, they still want to see the outline of the tasks as a reminder.

Hifi interfaces

We created high fidelity interfaces and make modification based on user feecbacks. I firstyly made a templete with Material Design, then created all the interfaces together with Xi Chen.

Test and iteration.

We made a prototype in Invision. It can be seen here.
We conducted heuristic evaluation and task-based user tests to identify usability problems and collect feedbacks on design concepts and interfaces.

Heuristic Evaluation

The goal for heuristic evaluation is to check if the overall experience of the system is seamless and fluent and whether our design language match their guidelines.To do this, we can quickly get inexpensive feedback from Home Depot UX team and identify important usability issues.
We conducted it with 5 UX designers in Home Depot. Before that, we identified 7 evaluation dimensions based on the pain points we identified for end-users.

#Evaluation Matrix

Me explaining our design to Alan, Home Depot UX designer.


1. Briefly introduce user contexts and our system to the designers. Provide them evaluation matrix for reference.
2.Walk-through the whole system for the expert, ask them to think aloud.
3.Let experts freely use the prototype and gain a feel for interactions and the scope. Ask them to talk about the issues that they identifies.

We categorized our findings in the heurstic evalution in 7 evaluation matrix that we identified before. The details of the usability issues we obtained from them can be viewed here.

Task-based Usability Testing

After heursitic evaluation, we conducted task-basked usability tests with 3 associates. Our goal was to understand if our designs followed user’s mental models and to find out if they perceived our solutions as better than the current solution.
We created 3 main tasks for them to do.

#Tasks for test takers

1.Get an overview of the bays you need to finish today and then finish the tasks in the first bay including AR POG checking and shoppability checking..
2.Complete the tasks using freestyle mode and use voice to report an issue.
3.Get an overview of the projects, complete the “Reset installation” project, and set an alarm.

Testing Session with a participant


1. Introduce the project goal, and put users at ease.
2. Describe the 3 tasks and their scenarios, let users use the prototype while thinking aloud.
3. Ask users to finish ASQ questionnaire everytime they finish a benchmark task.
4.Let them fill up SUS questionnaire and ask follow-up questions.

We got a avergae SUS score of 85.83, which indicates that that our system has adequate usability. Also, we summerized the ASQ scores and insights from follow-up questions, and found out some detailed issues of the interfaces. They can be viewed here.

Then we dealt with the usability issues that identified in the former tests and iterated our design. Below are the modifations for 2 main issues that we identified.

Iteraction 1: Learnability Issues

In the tests, one big problem we identified is that the system is not easy for users to learn. Users are confused about what a certain button for or why this notificatin shows up here. Even though the training for APP usage can be a solution, we conducted iteration to reduce users' unnecessary learning cost.

Iteraction 2: Accessibility Issues

The working contexts are complex in Home Depot stores, also the some associates may have slight disabilities. Therefore, in the tests, we identified several pages with low accessibility, especially in AR part. And then I modified them.

What's next.

Future steps

#Supervisors' experience

As the most important stakeholder for associates, supervisors' current experience cannot satisfy their needs too. Based on our research, we found that they are difficult to have an overall grasp of his team situation and usually need to spend extra time for communication and allocation. It would be great if we can design something for him.

#Better test methods for AR

For AR part in the project, we just used stastic pages and a selected bay area to let associates try the prototype. If we can really prototype something out, users may be able to have a better understanding and we can get a deeper insight about how AR works better for them.


#Be practical but always think of something big.

This is the problem that we have always faced. In this project, we came up with the AR assistant to help associates improve their efficiency. Then, after talking to Alan, we started to think about how AR could be applied to the whole retailing area, such as to the customers’ side, to the whole associates’ working flow, etc. But in reality there are many constraints in the store and the cost often outweighs the benefit. So a lot of our innovative ideas have not been prototyped for practical reasons. However, we think these ideas are awesome and we envision that in the future, AR technology could be applied to different interesting scenarios as we thought.

Your ideas, especially the AR one really give me many inspirations about how our products can be improved.

——— Quote from Alan, UX manager of Home Depot

#To go further, go together.

I have an amazing team for this project! We collaborated so well and supported each others in the whole process. One important lesson that I learned is that keeping on the same page and trying to achieve consensus are so important for a team, especially for 4 people from different backgrounds like us. Take a look at the portfolios of my talented teammates: Xi Chen, Taylor Stillman, Yannu Lee.