4 principles for assessing the real benefits and harms of treatment

By seeking medical attention, we hope to solve our health problem, or at least get more benefit than harm. However, how can one understand how much benefit and how much harm the treatment can bring? How do you decide if you should accept the proposed treatment or undergo tests, and how to save you money and time?

There are no simple and short answers to these questions. However, the principles outlined in this article will help you better understand the real benefits and risks of health care services and help you make better health decisions.

1. Don't forget the denominator

Consider the following phrase:

Studies have shown that treatment X reduces the risk of severe illness by 50%.

Similar messages often circulate on television and other media outlets. Mainstream medicine offers patients a range of services and drugs that can be described in this way.

Would you like to take this kind of treatment? It seems that the answer should be "definitely yes", but not everything is so simple.

A 50% reduction in morbidity in people taking drug X seems to be compelling evidence of its effectiveness. In fact, this message says almost nothing about the real value of such treatment and whether or not you should take it. We cannot correctly understand this message, as it does not say how often the disease develops without treatment.

How it works

Imagine the following situation:

In a group of 1,000 people without treatment, severe illness develops in all people. If all people take medicine X, half of them manage to avoid developing a dangerous disease.

500 / 1 000 × 100% = 50%.

In such a situation, there is no doubt that drug X is very valuable. Only relatively few of the currently available medical interventions are so effective.

Now imagine a different situation, closer to reality. In a group of 1,000 people without treatment, only two people develop the disease. When all people (one thousand) are on treatment, the incidence is halved, from two to one in 1,000.

While we also end up with a 50% relative reduction in incidence (1/2 × 100% = 50%) as a result, due to the low incidence of illness in people who do not take treatment (denominator), the drug is no longer as attractive.

What is useful

If your doctor or pharmacist suggests that you take preventive treatment or get a preventive examination, ask him:

Why do you think I'm at risk?
How likely is it that I will get sick if I don't take treatment or get tested?
How exactly can this medication (test) help me?
How likely is the treatment (examination) to be beneficial and how likely is it to be harmful?

2. Try to find indicators expressed in absolute values

Now in public and private clinics, patients are offered many services with little benefit: screening for breast cancer, prostate cancer, aortic aneurysm and others. Unfortunately, there are often cases when, instead of adequately informing, patients are frightened of the possible consequences or are ashamed for an inattentive attitude to their health.

To protect yourself from such manipulations, it is important to learn how to understand how great the real benefits and real harm of services. Even if we are able to understand percentages and statistics with enough attention and training, our minds are ill-equipped to process such information. For almost the entire history of mankind, people have not had to deal with this type of information, and therefore it easily causes cognitive distortions in us.

Much more familiar and therefore much more understandable for us is information presented in the form of absolute values or the natural frequency of events.

How it works

Example No. 1

Let's translate an example already known to us with the effectiveness of drug X into this format:

Without treatment, the disease develops in two out of 1,000 people. This is a natural incidence of the disease.

When 1,000 people take treatment:

one person, thanks to the treatment, manages to avoid the development of a serious illness;
one person gets sick despite treatment;
998 people take treatment in vain, because even without treatment, they would never develop the disease.

This presentation of information is more transparent and clearly shows all the important outcomes: how many people were helped by the treatment and how many people were taking the medicine in vain.

The benefits of many medical services are great and obvious. It is difficult to overestimate the value of trauma treatment, certain dental services, vaccinations, treatment for acute infections, and so on. At the same time, many other medical services have only marginal utility. Some modern possibilities for early diagnosis of cancer benefit only one or a few people out of 1,000–2,000 patients.

Example No. 2

Results from large randomized trials show that preventive mammography reduces the risk of dying from breast cancer by 15–29%. This is not to say that breast cancer screening is an absolute choice for all women and that women who do not get it take their health carelessly.

Since in a group of 1,000 women in their 50s, about six die of breast cancer over the next 10 years, the real benefits of testing are as follows:

For 10 years, it helps to prolong the life of one or two women out of 2,000 by starting treatment earlier.
The remaining 1,998 women will have no use, and some of them will suffer from imperfect mammography.

When you consider the transparent data on the effectiveness and negative consequences of preventive mammography, it becomes clear that the decision to screen for breast cancer is not at all straightforward. If women do not see the benefit of this survey, they have every right to refuse it, and no one has factual reasons to call them irresponsible for such a decision.

Example No. 3

The situation is similar with screening for prostate cancer in men. Systematic implementation of this examination in men aged 54 to 69 years for 13 years is associated with a 30% reduction in the risk of death from prostate cancer.

But aggressive forms of prostate cancer are relatively rare, and when converted to a more transparent form, this indicator means the following:

If 1,000 men aged 54–69 years undergo a PSA test every few years for 13 years, this test will significantly prolong the life of one or two men due to the earlier detection of an aggressive form of the disease. It is impossible to predict in advance which of the 1,000 men will benefit from it.
For the remaining 999–998 men in this group, the screening will be useless, and some men will suffer from PSA screening.

Thus, in the case of prostate cancer screening, the final decision is also not obvious, it can only be made by the man himself.

Example No. 4

A correct understanding of statistical indicators is necessary in other situations as well. For example, when patients are afraid to use drugs that bring significant benefits with relatively small risks.

In the medical literature on the interpretation of statistical indicators, an incident that occurred in England in 1995 is often considered. After the U. K. Committee on Safety of Medicines reported that “the use of third-generation combined contraceptives increases the risk of deep vein thrombosis in the legs by 100%,” many women were frightened and stopped taking these contraceptives.

Thrombosis can be dangerous, as the migration of a clot can lead to blockage of important blood vessels (thromboembolism) and death. However, how justifiable was the panic really justified, and did giving up combined contraceptives help women take better care of themselves?

The results of studies in which an increased risk of thrombosis was observed were as follows:

Women who took second-generation combined contraceptives developed thrombosis with a frequency of one in 7,000 women.
Women who took third-generation contraceptives developed thrombosis with a frequency of two in 7,000 women.

Thus, in the group using third-generation combined contraceptives, the relative risk of thrombosis did increase by 100% (two times), but the absolute increase was one additional case per 7,000 women.

The ensuing wave of combined contraceptive abandonment has resulted in nearly 13,000 unwanted pregnancies, including among adolescents. And most importantly, women who became pregnant after refusing contraceptives not only did not reduce their risk of thrombosis and thromboembolism, but also increased it. The fact is that during pregnancy, the risk of developing thromboembolism is almost three times higher (about 29 cases per 10,000 women) than when using combined oral contraceptives.

This example shows that information presented in the form of a natural frequency of events makes it possible to more adequately assess the real benefits and real harms of drugs and other medical services.

What is useful

To be able to choose the services that truly interest you and to create realistic expectations for health care, you need to learn to ask your doctors the right questions:

What happens if you refuse examination or treatment?
How urgent is the examination or treatment needed?
What scientific evidence supports the feasibility of the services offered?
What harm can these interventions do?
Is it possible to solve the problem in some other way, including a cheaper or more secure one?

The doctor must provide reasoned answers to these questions. For more detailed advice on medical decision making, see.

3. Make sure the message uses the same comparison groups

When you are offered a treatment, especially under the guise of an innovative method, inquire about the risks and make sure that information about different outcomes is expressed using the same comparison groups.

How it works

Consider the following message:

The treatment works for 10 out of 1,000 patients, but it causes serious side effects in 2 out of 100 patients.

At first, it may seem that many more patients benefit from treatment than harm. In reality, this is not the case. Due to the use of different comparison groups and our natural tendency to ignore the denominators, the message creates a strong cognitive illusion.

Everything becomes clear if you bring the indicators of benefit and harm to one denominator, for example, to 1,000:

The treatment helps 10 out of 1,000 patients, but causes serious side effects in 20 out of 1,000 patients.

It turns out that the actual risk of treatment is twice its benefit.

To make it easier to compare indicators presented as fractions with different denominators, you can also convert the fraction to a percentage.

For example, let's compare the fractions 1/5 and 1/9:

1/5 × 100 = 20% (20 people out of 100);
1/9 × 100 = 11% (about 11 people out of 100).

What is useful

Fortunately, only a few medical problems require truly urgent action. If the solution to the problem can be delayed for a while, it can be quite helpful:

Explore it in more detail by comparing information from different sources.
Compare the advantages and disadvantages of different capabilities.
Get a second opinion.

4. Pay attention to the emotional frame of the message and try to change it

Imagine this situation:

The patient is asked to choose between surgery and rehabilitation treatment. At the consultation, the doctor informs that during the operation one out of 100 patients dies due to complications.

How do you feel about such an operation?

Now imagine that the doctor says: “The safety of the operation is 99%; out of 100 patients who undergo surgery, 99 patients go well."

It may seem that in the second case we are talking about some other operation, but from a mathematical point of view, both messages are equivalent. Only their emotional setting is different.

How it works

We take messages that are formulated in a negative emotional frame much more seriously, especially when it comes to the possibility of catastrophic losses. In the prehistoric past, such an adaptation probably helped people to be more careful and survive, but in the present, we increasingly need to reconsider how useful such an attitude is.

When faced with a one-sided message, try reformulating it to include all the important outcomes:

Out of 100 patients who undergo surgery, one patient dies, and in 99 everything goes well.

Negative emotional wording is often used by anti-vaccine advocates. To justify their position, in addition to pseudoscientific conclusions, they also use emotional manipulation. They focus the audience's attention on the extremely rare cases of children affected by vaccinations, and ignore another, positive part of the story - the huge number of children who were vaccinated normally and who, thanks to it, received protection from dangerous infections.

What is useful

When you need to make a medical decision, try to shift your focus from emotions to numbers and facts. To learn this, practice yourself in different ways of presenting information.

Outcome

The benefits of these principles are not in finding the only correct solution (in fact, it does not exist), but in making the decision that will suit you the most, based on your attitude to risk and the goals that you set. before medicine.

Of course, this is not a complete list of what is required for better medical decision-making, but possessing these skills will already allow you to better navigate the mass of medical messages and services.