There is a scene in the first season of Six Feet Under which has remained with me. In it, Brenda Chenoweth, a smart but fractured character whose childhood was scrutinised by psychologists and then documented for all to see, argues with her boyfriend that the findings of researchers are not always accurate. “It is a fucking law of physics that the very act of observation changes that which is being observed”, she says.

This law – named ‘The Observer Rule’ – feels intuitively true, and is a  constant burden of good research. Because we are worried about research findings which are simply spin, or play to the crowd, or which are in some manner manipulated, the neutral observer might tend to impulsively favour methods which make invisible the act of observation.

The candid camera approach to human behaviour, first created for laughs in 1960s TV and then exploited ad nauseum in the late 90s with the dawn of Reality TV, holds as its premise that life is more inherently insightful when it is most free of affectation.

Much ink has been invested on the search for truth about people. This is the reason trash journalists sieve through the rubbish of famous people; the reason off-mic moments are given such sway (I enjoyed Ken Clarke referring to Britain’s new Prime Minister May as a “bloody difficult woman”); the reason we tend to believe someone when they speak off-the-cuff, rather than with carefully prepared comments.

As a practitioner of qualitative research – the kind where you sit down with 6-8 people and seek to understand their lives in an inherently unreal setting – I take very seriously the natural blocks to insight which the research experience creates. Along the way, I have discovered ways to alleviate these problems:

For example, a high-energy conversation which is challenging, tends to illicit more real contributions, as people naturally seek to assert their opinion and defend their point of view. By bringing focus to content, the context is forgotten.

Or, I have found the act of co-creation – where I share in a full and honest way the problem I am trying to solve – enables my participants to inhabit their own personal point-of-view as well as the point-of-view of my client. This is inherently respectful and empowering, treating people as peers, and not ‘subjects’.

By way of further example, I frequently choose informal research environments where people feel they are in a group of peers, rather than risk their feeling like quasi-lab rats. That is to say, I’ll often talk to guys about beer in pubs, to women about kids in their homes. I’ve once talked  to dog owners about their dogs, while walking in the park. [It reminds me of that well trod advice for parents, to have tough conversations with their teenage boys whilst in the car. Talking without eye contact is calming, apparently. Situation is everything.]

Yes, we know, at some intuitive level, that we should not accept as unquestionably true the words that tumble out of someone’s mouth. Depending on the topic, there are often many reasons why participants may want to hide or gild the truth.

American writer, Gay Talese, sees the use of tape recording, which emerged in the 1970s, as the harbinger of doom for journalists seeking the truth. Talese’s argument is a compelling one: that the arrival of a journalist to record a conversation compresses the interchange of views into an ever-tightening time band. What results, he claims, is not insight. It is merely soundbites. His preferred method of journalistic research is to simply bring along a native curiosity about people and hang out with his subjects. His job is to observe them, to shadow them, to be outside as well as inside with them. Importantly, Gay Talese does not frame his stories around verbatims, but around meaning.

The rise of research practices such as safaris, in-store observations and ethnography all have, as their root, Brenda Chenoweth’s key concern that the very act of observation changes that which is being observed. Each method has its role. But, to half-borrow from that anthropologist of the heart, W.B. Yeats, the truth comes dropping slowly. It is a thing discovered – an accretion of evidence which points to something incontrovertible. When the same smoke signal arrives from five different mountain tops, it’s time to embrace its tidings.

Across a well-structured and fluid discussion with consumers, I find that core issues can be addressed from many angles. The Observer Rule does not disappear, but it can be carefully managed. One ends up carefully tracking the coherence of what is being said as well as the content. I recall, years back, a young man who argued passionately against the role of brands. It was an elegant argument which had merit. But it was also tempered, in my mind, by the fact that his sweatshirt was emblazoned with one word: Guinness.

The act of building evidence for a hypothesis is my core analogy for qualitative research: it is detective work, seeking to solve a puzzle with evidence and judgment. The tools we are given are imperfect, but they are also powerful. To my delight, this was neatly summarised in a recent New Yorker cartoon send-up, which depicts an intrepid investigator, bit by bit making connections and coming to a Gestalt realisation that, indeed, something is up.

Words are words, and sometimes they are cheap. Meaning, however, is valuable. In attempting to bring qualitative research to its most productive and insightful place, managing the act itself is a constant issue. On this point, as on many others, Brenda Chenoweth and I agree.