Broadening perspectives in health service assessment

8 Sep
Vicky Serra-Sutton
Vicky Serra-Sutton, sociologist PhD

What lies behind a significant volume of hospital readmissions? What makes a service present a good healthcare praxis? What obstacles are there when changing to a healthcare model such as in major out-patient surgery which encourages patients to go home on the same day as their operation? Do managers and nursing staff have the same opinion about what efficiency is in an operating theatre? What is the perception of professionals of the possible benefits of people-centred attention?

Do we all see a dragon?


Reality is complex and therefore approaches are needed which facilitate the interpretation and understanding of that reality. With qualitative research, places can be reached otherwise unattainable when using other methodological aproximations. When answering questions like those we asked ourselves previously, a truly qualitative approach is required. We need to make the approach using an adequate and credible technique to validate the process of all those involved and  to ensure precision in results as is done in quantitative research but not forgetting that we need to be critical and independent in the analysis made.

We will briefly outline the evolution of the qualitative approaches in the context of the assessment of health services. A reflection on the usefulness of qualitative techniques  in the assessment of health services or medical technologies is not a new one and you can find a series on this subject in the British Medical Journal of 1995 and in the Health Technology Assessment report of 1998.


Health assessment agencies have given great importance to questions about the scientific evidence available when talking about the efficiency and safety of treatments and biomedical interventions of a clinical nature. Randomized controlled trials and systematic reviews are considered to be the reference standards for causal atributions of the benefits of an intervention for the improvement in the health status of patients.

Society has evolved and the needs of the system adapt to this. We formulate new questions related to the preferences and expectations of users facing treatment and how different professionals contribute to providing better results in patient health care. One must bear in mind that when assessing the benefits and results of attention given,  many factors come into play.

In this context, the paradigm of evidence based medicine and the supposed superiority of quantitative approaches and of some study designs above others, have created obstacles in the application of qualitative research. In this sense, the letter to the editors of the British Medical Journal signed by more than 70 researchers of reference for giving their support to qualitative research is clear proof of this remaining obstacle.


Questioning the efficacy of a medical drug cannot be answered using a qualitative approach but we can broaden the scope of questions that we pose ourselves.

For example, we can consider asking ourselves questions, among others, about the preferences of patients, the perception of the benefits of a medical drug, the expectations or opinion of professionals that prescribe it or the possible reasons for a low adherence of the medical drug.

Another scenario could be that of a patient with osteoarthritis who has undergone a knee replacement (arthroplasty) and who is being attended by several professionals such as the primary care doctor, the traumatologist surgeon, the anaesthetist, the nurse, the physiotherapist and other professionals if the patient has other comorbidities. That patient has certain preferences and expectations which need to be understood and then give the health care to cover those needs, which can go beyond the mere surgical procedure.

With qualitative research we develop a discourse, texts, opinions and perceptions of people, communities, with images, perspectives, ideologies and complexities. We must guarantee rigour and that the photograph and interpretation of reality that we make remain valid and coherent for the research group and the populaton or group of people that we are assessing.

The application of qualitative techniques has been on the rise using interviews, semi-structured questionnaires, field notes, focus or discussion groups to gather the opinion of different groups of professionals and users.

From my point of view, there are three examples which can be of great use to know the approach and the process in carrying out an assessment of services with a qualitative approach:

  1. Opinions, experiences and perceptions of citizens regarding waiting lists
  2. Job satisfaction or productivity, a study exploring the opinions of different professional profiles regarding the efficiency of operating theatres
  3. What opinion do professionals have of the benefits of an integrated attention in the United Kingdom?

Avoiding the classic metrics means being able to measure in an alternative or complementary way by combining different approaches be they qualitative or quantitative. I find the introduction to qualitative research we find in René Brown’s TED talk the power of vulnerability. This qualitative researcher recommends we measure that which is apparently unmeasurable and go more in depth into the complex phenomenon of vulnerabilty.

We broaden perspectives by understanding the reality from within, by bearing in mind the multiple existing points of view to improve that which is disfunctional or by identifying better practices to spread them. We can measure what we want to measure. It will be necessary to adapt the approach to the context and audiences and to continue progressing to show with rigour and practice the usefulness of qualitative approaches.

We continue learning. This time, it has been at the Congrés Iberoamericà de Recerca Qualitativa en Salut (in Twitter #IICS2016) held in Barcelona, 5-7 September. The Agència de Qualitat i Avaluació Sanitàries de Catalunya (AQuAS) and the Agència de Salut Pública de Catalunya (ASPCAT) shared the stand to explain their experiences.

2016 Congreso Iberoamericano de Investigación Cualitativa en Salud
Santi Gómez Santos (AQuAS/ASPCAT), Dolors Rodríguez Arjona (ASPCAT), Mireia Espallargues (AQuAS), Vicky Serra-Sutton (AQuAS)

Post written by Vicky Serra-Sutton (@vserrasutton), sociologist PhD in AQuAS.

First commandment: do not make assumptions about the preferences of people who suffer from a disease

16 Jun
Joan Escarrabill
Joan Escarrabill

One of the things that we humans do quite often is to assume a fact, a situation, or what someone else thinks. We assume that our circumstance is representative of the general circumstance, that a person, by the mere fact of having university studies, has the capacity of global understanding (including that of diseases, diagnostic procedures, or about treatments that this person has never heard anything about) or that all professionals, doctors or nursing staff, think in a standardized way.   This often leads us to generalize. We do this daily. Who hasn’t said a sentence like this at one time or another: “all men are…”, “the residents are not like they used to be…”, “the Poles are…”? Assumptions, on the other hand, are not very far away from prejudices. They say that once Winston Churchill (1874-1965) was asked what he thought about the French. “I don’t know”, he replied, “I don’t know all of them”.

It is not surprising then, that this trend to make assumptions also occurs at the moment of evaluating the needs or the values of people who are ill. In general, the functioning of health organizations revolves around professionals (adding criteria of efficiency, which are not always contrasted). In this context, we trust professional experience to identify and clarify the needs of the sick people too much. But, do we really know what truly interests a person when he/she is ill?

Satisfaction surveys are a first step to get closer to the perspective of those who use a service. Satisfaction is a very comprehensive construction that comes from the user of a service and it is highly related to the user’s expectations. In addition, quite often what gives the most satisfaction is not necessarily related to the key elements of the service received or with the actual quality of the service received. The communication skills of the professional who offers a service may be related to a high degree of satisfaction, without having a direct relationship with the results obtained. Friendly incompetent people are dangerous precisely because the patina of their ways of being can conceal the consequences.

If we only use the satisfaction surveys to assess the perspective of the person who uses the service, it is possible we have little capacity for discrimination. In practice, it is still odd that in the area of health in the satisfaction surveys the same problems are consistently identified, but no action is ever taken. The key element is to be sure of the relevance of the feedback of the people who come in contact with health services.

One of the barriers in appreciating the value of the patients’ feedback is the professionals’ skepticism of just how knowledgeable the patients are about the organization as well as the technical aspects. Well now, what we must not confuse is that it is one thing to “not know what you want” and another, which is very different, “to know perfectly what you don’t like”.

Sick people (and people who care for them) know perfectly well “what they do not like”. These people identify the touch points, or points of contact, which can generate unpleasant, problematic or critical situations (in this case they would be “pain points”, or “trigger points “) perfectly. The best way to identify these situations is to ask explicit questions and to not make assumptions.

There is not one unique methodology to collect the perspective of those who suffer from diseases. You can use quantitative methods (surveys of all kinds: face-to-face, online, by telephone) or qualitative methods (interviews, focus groups and/or analysis of complaints and suggestions).

Being interested in the feedback about the care offered is very good, but it is not enough. Angela Coulter affirms that it is unethical to collect data on the experience of the patient and then later ignore this information. That’s why, more and more every day, talking about the patient’s evaluation of the experience has a triple dimension: a key element in the evaluation of quality, a very powerful leverage and a strategy to improve the empowerment of the patient.

Doyle et al expounds that the patient’s experience, the effectiveness and clinical safety are closely linked and suggests that we must consider the patient’s experience as one of the pillars of the quality of health care.

The sick person’s perspective is very important for identifying opportunities for improvement in the provision of the service. And the step into action must be done involving everyone who plays a role in the care process. The concept of “co-design” in the framework of health care refers to cooperation between professionals and people assisted in the design of the transformations of the service, from the beginning of any improvement process and in all directions. The “co-design” represents a radical reconceptualization of the role of patients in the process of innovation in the provision of services.

Lastly, the evaluation of the patient’s experience cannot be separated from the information or from the therapeutic education that, in the end, should contribute decisively to the activation of sick people caring for themselves, through a deliberative process. This activation to address the disease within a framework of shared decision making improves clinical outcomes and has an impact on the cost.

The video about empathy from the Cleveland Clinic is an example that can be used to avoid “assumptions”: Empathy: The Human Connection to Patient Care. The solution is very simple. Instead of making assumptions one must observe and ask questions.

Post written by Joan Escarrabill, (@jescarrabill), director del Programa de Malalties Cròniques de l’Hospital Clínic de Barcelona.