So diary studies, a quite common method to collect data over a longer period of time. Experience sampling method (ESM) is a form of diary study, but the respondent get’s to record what is happening in the moment with the support of some tool, instead of recording the diary entry retroactive. One problem with diary studies can be that the respondent needs to recall what happened during a day or a set period of time, and memory can most definitely fail even the best. ESM is all about the respondent being able to log the activity right in the moment when it happens. In this article I will write shortly about my experience using ESM, why diary studies can be so good, but why I believe the biggest strength is the post-diary interview.
What is a diary study
Diary studies are mostly used when observation can’t be done, if the study spans over a long period of time, or if the respondent might feel too exposed by someone observing them. Observation will of course give you a more accurate representation over the phenomena you’re studying. The respondent might still act un-natural, but you will be able to draw conclusions over what’s happening and not only the things the respondent tells you in a diary entry.
Pro’s and con’s with diary studies
It takes time, that is the biggest con. It’s not a quick and dirty method. Of course you can scale your diary studies, but it demands planning, it gives you a lot of data to analyse, quite often only qualitative data which takes A LOT of time to analyse, and then you have to find a good way to visualise the data you gather, and makes sense of it and make it into something tangible. It also demands a lot from the respondents. The incentive you give to them needs to be carefully thought through, you want them to engage but you want them to engage with a fairly genuine agenda, so it can’t be too much and it can’t be too little. But a diary study is also an amazing method to really get to know your users. It gives you much richer data, with better nuances than an interview when they have to recall everything.
When to choose a diary study? When you have the time and money to do it, it’s worth it. If you need insight quickly on a specific feature or function it might be better to go for a more goal-driven usability approach.
Case study
I recently did a diary study with the ESM approach, I wanted the respondent to log the activity in the moment. I also wanted to mix the in-data with both qualitative and quantitative data. So it would make it faster and more effective to analyse and to compare the different activities towards each other. The diary study kept on going for seven days with ten respondents. The respondents were recruited by an agency. It was important for us to get a good spread over the sample, ages, genders, where they lived, living-situations and so on.
The research question at hand was simply to know more about the journey of buying a new home and selling an old one. What activities they set out to do, how they felt about it and general emotions during the entire process itself. Former studies have shown that how you feel about this process is depending on why you chose to step into it in the first place. Buying a home usually starts from either a dream or a need, they can of course overlap. Even though you go into the process with the motivation of buying a dream home most studies points towards that it sucks. It’s the biggest affair you do in a lifetime, it feels complicated, it feels scary. And even if you have done it before you often feel like a first time buyer all over again.
The diary study itself gave ok data, we got an understanding for their life situations, what services they used, how they used them and what they felt about them. I believe some of them did not log the activity while they were doing them (ESM style) but decided to recall them instead afterwards. But that is just something you have to consider while analysing the material. But what surprised us was how emotional they became and how much they started to open up to the personal aspects of why they were moving, and how they felt about these life changing events. We finished up the diary study with post diary interviews, 60min asking them more about their reasons to buy a new home, their life situations and they got to talk more about their activities and how they felt about them. We did expect them to be fairly open, but we did not expect it to be as emotional as it was. I won’t go into detail because it feels like it’s too personal for it to exist online. I have never been in interviews which have been this personal before, which absolutely adds a lot of great insight into the emotional rollercoaster it can be buying a new home and everything around it.
We had contact with the respondents everyday for seven days, they gave us insight into their private lives and therefore I think they felt comfortable enough to become so personal in the last interview. And if they dared to act that way because we did it remote or if it would have become more intimate doing it face to face is hard to determine. But as an insight into the method I would say, plan the post diary interview with care, prepare for them to be quite personal and do not see it only as a wrap up activity to end the diary study. Act like it is as important as the diary study itself.
