Future Home

Recruiting

The main recruiting session took a week. However, since there are participants who opted out, the recruiting was actually spread across the whole process.

The Screening Survey

Bosch provided the screening with general criteria, screen-out criteria, and a specific screening survey with 18 questions.

Translate the Material. I translated the general screening from English to Chinese.

Modify the Material. In translation, I figured out some room to improve the screening to make it more fitted for the Chinese market. For example, one of the screening survey questions asks to screen out those who have never used the oven. However, in the Chinese market, the oven is still pretty rare, while appliances like steam cookers are more popular. I identified some of the issues and reached out to the clients to communicate the issues.

Develop the Survey. Further, I developed the screening survey using the Tencent surveying tool, a popular surveying platform in China.

Circulate the Survey. My colleague and I circulated the survey on social media and interest groups, anonymously, to avoid recruiting biases.

The Screening Call with the Qualified Participants

Select the Qualified Participants. Based on the survey responses, I selected ~20 qualified participants based on their responses and considering Bosch’s target group in China.

Leverage the Screening Call. To make sure that the participants are eloquent enough, I scheduled a 10 min call with the qualified participants before the main study.

#2 Study: Evaluative Research

Four months after conducting the foundational study, we heard back from the client that they are conducting a follow-up project. That is, based on the insights from the foundational study, they have developed 8 concepts, and they would like to have the concepts tested by a small set of the same participants.

The goal of the evaluative research is to see how would people respond to the concepts, so that Bosch could further develop its innovation strategy and its specific products.

Recruiting

Reach out to the Participants. Bosch intended to conduct the concept testing with the same group of participants. I randomly selected 6 of the original 10 participants, shared what was going on, and asked if they would be willing to participate in the follow-up study. All 6 participants agreed to join. In fact, they are pretty happy to see that Bosch was following up based on their previous responses.

Concept Testing

Develop the Materials. Bosch provided the to-be-tested concepts, and suggested the test consist of a survey and a one-hundred-dollar test. The test is conducted in the form of a moderated interview.

My co-worker translated the material. Further, we created the Figma board combined with the 8 concepts and the survey.

Streamline the Process, and Visualize the Responses. As included in the concept testing, each participant is expected to read the image and the descriptions of each design, and then respond to 5 questions (e.g., “do you like it”, “is it clear“, “is it novel”, etc., on a 5 Likert scale) of each design, one by one. Then, from the 8 concepts, each participant is expected to pick the top 3 concepts and invest their 1 million USD. If we go that way without much preparation, it would be difficult for me to document the responses, and it would be difficult for the participants to recall their thoughts.

So, the Figma board is created to streamline the process. My co-worker and I developed 6 boards for each participant. The 8 concepts were randomized on each board to avoid bias. The survey was attached to each concept, so that I as the moderator could document the responses on live. In that visualized way, it is also easier for us to communicate and share insights with the clients.

Ask the Follow-up Questions. As participants respond to the survey and the one-hundred-dollar test, what matters is not their short answers, but the reasons behind their answers. I leveraged several techniques to ask follow-up questions, to make sure that we have dived deep into their rationale.

Engage Stakeholders. As planning the concept testing sessions, I shared the research calendar with my stakeholders. The client sat in on several sessions. In each session, before wrapping up, I asked if the client had further questions. After each live session, the client and I had a debrief session, where we discussed our observations. Further, as we identified some unclear parts during the session, the debrief session also enabled us to quickly discussed and aligned on the next steps.

Delivery

Again, Delegate the Work Wisely. Same as the first foundational study, we delegated the transcription and translation work to an agency. During the whole process, I had to make sure that the work was done with quality and speed.