Equine Ownership

Interpreting Research for Horse Owners, Part 2

The second part of this series on making sense of scientific research findings looks at bias, blinding, and the placebo effect.

A researcher observing a horse in a stall.

It is not only the caregiver of the horse who could affect the outcome of a research study - the researcher themselves could be experiencing 'observer bias.'

By: Tracy Bye/Saddle Research Trust | December 15, 2022

Listen to this Article

0:00

-:--

Playback Speed

you're listening to an article from horse sport.com. interpreting research for horse owners, part 2 by: tracy bye/saddle research trust. in the first instalment of this series we reviewed some of the basic principles that scientists use when designing an intervention-based study. we talked about how we can test the effects of an intervention such as an item of tack or a supplement, and we discussed the importance of having a ‘control group’ which doesn’t have the intervention, so that we have something to compare the intervention group to. another important consideration at this point is the placebo effect. the placebo effect is the observed phenomenon of inert or ‘sham’ treatments having a therapeutic effect. as the treatment itself cannot possibly be effective, this is attributed to the psychological effect of the patient believing that they have had a treatment. obviously if a horse is taking part in a research study, they do not know that they have had an intervention or treatment. however, the caregiver placebo effect has been reported in studies of domesticated animals. this is where owners report an improvement in symptoms in animals who are not receiving treatment. advertisement scroll to continue with content an example of this can be seen in headshaking study, where 32 horses were given either a supplement developed to alleviate headshaking or an inert placebo in a crossover design [1]. owners were not told which condition their horse was in, and they were asked to fill out a questionnaire about their horse’s headshaking behaviours. owners reported improvement in headshaking on both the supplement and the placebo, whereas professional assessment of headshaking symptoms showed no significant improvement in either condition. similarly, a study into osteoarthritis treatment in cats also found a significant improvement in caregiver ratings for mobility and activity for animals on a placebo treatment with no active ingredients [2]. the caregiver placebo effect is one reason why evidence based on owners or riders’ observations of their horses, such as product testimonials, is not as useful as the results of well-designed scientific studies. headshaking can be problematic for horses and their owners, but a study showed that the caregiver placebo effect was most likely responsible for any owner reported improvement in headshaking in a group horses fed a supplement designed to alleviate it. owner reported results are not usually considered as outcome measures in intervention studies, but how owners or riders interact with their horses could have an effect on other outcome measures. the is known as placebo by proxy effect. this idea was originally described in the context of humans taking part in drug trials [4]. the authors explain that if a patient’s doctors and family members feel empowered and optimistic about a disease, the patient’s environment can become less stressful and more supportive, which can lead to changes in the patient’s own mental state. the placebo by proxy effect has yet to be thoroughly investigated in horses, but it is reasonable to assume that it would be present in animals in the same way as seen in humans, and to try and control for it. for example, if a rider believes that their usually tense horse has received a calming supplement, they may be more relaxed when riding them, leading to the horse displaying less of the tension associated behaviours which are being used as the outcome measure. to avoid the placebo by proxy effect impacting on our research, the best approach is to make sure the control group take part in the same testing process as the intervention group, but based on a placebo or sham condition, so they would be the placebo group. the important thing here is that all conditions look the same to the caregiver or rider of the horse involved. this is referred to as blinding. so, in the example of the calming supplement study, the horses would receive the real supplement in the intervention condition, but in the control condition they would receive a placebo which looks the same as the real supplement but has no active ingredients. when the person that is involved with riding or caring for the horses is unaware of which condition is which, this is known as a single-blind study. this means the caregiver cannot know whether their horse is getting the treatment or the placebo, so if the placebo by proxy effect is present it should be the same for both groups, and the difference between the groups is the true effect of the intervention. it is not only the caregiver of the horse who could affect the outcome of a research study. the researcher themselves could be experiencing observer bias. this is a type of unconscious bias, meaning the researchers themselves are not aware of it. an example of this was shown in a study of veterinary students who were asked to score the behaviour of groups of hens from a video. the students scored more positive and fewer negative behaviours when told that the hens were from an organic instead of a conventional farm, even though the videos were the same [4]. the students will likely have had a more positive view of organic farming compared to conventional farming, which led to them unconsciously looking for more positive welfare and behaviours in the ‘organic’ videos and more negative welfare and behaviours in the ‘conventional’ videos. to alleviate the potential impact of this observer bias we can take the blinding one step further and also make sure that the researchers who are testing for the outcome measure don’t know which condition is which. this is then referred to as a double-blind study. double blinding can be achieved by having an independent person randomly allocate the horses to the intervention and control (placebo) conditions. when the researcher is conducting their assessment, they do not know which horses are in which condition, so the possible effect of observer bias is removed. the independent person keeps a list of which horses were in which condition, which they provide to the researchers after the study is over so that the results can be grouped and analysed. in a truly randomised study, this sorting into two groups is done in a way that is unsystematic and unstructured, so that neither the caregivers nor the researchers would not be able to work out which horses were in which group. the randomised, double-blind, placebo-controlled trial is therefore considered the gold standard for intervention-based research studies, and this is the approach used in human medical research to investigate the effectiveness of new treatments. there are some types of study where it is not possible for a rider, caregiver, or researcher to be blinded as to which condition a horse is in, such as when testing new items of tack for example, especially when there is a clearly visible difference between the two conditions. if this is the case, researchers will usually think very carefully about the type of outcome measures they use, making sure to use objective measures (e.g. biomechanical analysis) as opposed to subjective measures (e.g. behavioural observations). in the next instalment we will investigate these ideas a little more, as we delve into the different ways we can measure the outcome of a research study, and how we can ensure that the results are valid and reliable. ***. references. 1. talbot wa, pinchbeck gl, knottenbelt dc, graham h, mckane sa. a randomised, blinded, crossover study to assess the efficacy of a feed supplement in alleviating the clinical signs of headshaking in 32 horses. equine veterinary journal. 2013 may;45(3):293-7. 2. gruen me, dorman dc, lascelles bd. caregiver placebo effect in analgesic clinical trials for cats with naturally occurring degenerative joint disease‐associated pain. veterinary record. 2017 may;180(19):473-. 3. grelotti dj, kaptchuk tj. placebo by proxy. bmj. 2011 aug 11;343. 4. tuyttens fa, de graaf s, heerkens jl, jacobs l, nalon e, ott s, stadig l, van laer e, ampe b. observer bias in animal behaviour research: can we believe what we score, if we score what we believe? animal behaviour. 2014 apr 1;90:273-80. &&>

In the first instalment of this series we reviewed some of the basic principles that scientists use when designing an intervention-based study. We talked about how we can test the effects of an intervention such as an item of tack or a supplement, and we discussed the importance of having a ‘control group’ which doesn’t have the intervention, so that we have something to compare the intervention group to.

Another important consideration at this point is the placebo effect. The placebo effect is the observed phenomenon of inert or ‘sham’ treatments having a therapeutic effect. As the treatment itself cannot possibly be effective, this is attributed to the psychological effect of the patient believing that they have had a treatment.

Obviously if a horse is taking part in a research study, they do not know that they have had an intervention or treatment. However, the caregiver placebo effect has been reported in studies of domesticated animals. This is where owners report an improvement in symptoms in animals who are not receiving treatment.

Scroll to continue with content

An example of this can be seen in headshaking study, where 32 horses were given either a supplement developed to alleviate headshaking or an inert placebo in a crossover design [1]. Owners were not told which condition their horse was in, and they were asked to fill out a questionnaire about their horse’s headshaking behaviours. Owners reported improvement in headshaking on both the supplement and the placebo, whereas professional assessment of headshaking symptoms showed no significant improvement in either condition.

Similarly, a study into osteoarthritis treatment in cats also found a significant improvement in caregiver ratings for mobility and activity for animals on a placebo treatment with no active ingredients [2]. The caregiver placebo effect is one reason why evidence based on owners or riders’ observations of their horses, such as product testimonials, is not as useful as the results of well-designed scientific studies.

Headshaking can be problematic for horses and their owners, but a study showed that the caregiver placebo effect was most likely responsible for any owner reported improvement in headshaking in a group horses fed a supplement designed to alleviate it.

Owner reported results are not usually considered as outcome measures in intervention studies, but how owners or riders interact with their horses could have an effect on other outcome measures. The is known as placebo by proxy effect. This idea was originally described in the context of humans taking part in drug trials [4]. The authors explain that if a patient’s doctors and family members feel empowered and optimistic about a disease, the patient’s environment can become less stressful and more supportive, which can lead to changes in the patient’s own mental state. The placebo by proxy effect has yet to be thoroughly investigated in horses, but it is reasonable to assume that it would be present in animals in the same way as seen in humans, and to try and control for it. For example, if a rider believes that their usually tense horse has received a calming supplement, they may be more relaxed when riding them, leading to the horse displaying less of the tension associated behaviours which are being used as the outcome measure.

To avoid the placebo by proxy effect impacting on our research, the best approach is to make sure the control group take part in the same testing process as the intervention group, but based on a placebo or sham condition, so they would be the placebo group. The important thing here is that all conditions look the same to the caregiver or rider of the horse involved. This is referred to as blinding.

So, in the example of the calming supplement study, the horses would receive the real supplement in the intervention condition, but in the control condition they would receive a placebo which looks the same as the real supplement but has no active ingredients. When the person that is involved with riding or caring for the horses is unaware of which condition is which, this is known as a single-blind study. This means the caregiver cannot know whether their horse is getting the treatment or the placebo, so if the placebo by proxy effect is present it should be the same for both groups, and the difference between the groups is the true effect of the intervention.

It is not only the caregiver of the horse who could affect the outcome of a research study. The researcher themselves could be experiencing observer bias. This is a type of unconscious bias, meaning the researchers themselves are not aware of it. An example of this was shown in a study of veterinary students who were asked to score the behaviour of groups of hens from a video.

The students scored more positive and fewer negative behaviours when told that the hens were from an organic instead of a conventional farm, even though the videos were the same [4]. The students will likely have had a more positive view of organic farming compared to conventional farming, which led to them unconsciously looking for more positive welfare and behaviours in the ‘organic’ videos and more negative welfare and behaviours in the ‘conventional’ videos.

To alleviate the potential impact of this observer bias we can take the blinding one step further and also make sure that the researchers who are testing for the outcome measure don’t know which condition is which. This is then referred to as a double-blind study.

Double blinding can be achieved by having an independent person randomly allocate the horses to the intervention and control (placebo) conditions. When the researcher is conducting their assessment, they do not know which horses are in which condition, so the possible effect of observer bias is removed. The independent person keeps a list of which horses were in which condition, which they provide to the researchers after the study is over so that the results can be grouped and analysed.

In a truly randomised study, this sorting into two groups is done in a way that is unsystematic and unstructured, so that neither the caregivers nor the researchers would not be able to work out which horses were in which group. The randomised, double-blind, placebo-controlled trial is therefore considered the gold standard for intervention-based research studies, and this is the approach used in human medical research to investigate the effectiveness of new treatments.

There are some types of study where it is not possible for a rider, caregiver, or researcher to be blinded as to which condition a horse is in, such as when testing new items of tack for example, especially when there is a clearly visible difference between the two conditions. If this is the case, researchers will usually think very carefully about the type of outcome measures they use, making sure to use objective measures (e.g. biomechanical analysis) as opposed to subjective measures (e.g. behavioural observations).

In the next instalment we will investigate these ideas a little more, as we delve into the different ways we can measure the outcome of a research study, and how we can ensure that the results are valid and reliable.

***

References

1. Talbot WA, Pinchbeck GL, Knottenbelt DC, Graham H, McKane SA. A randomised, blinded, crossover study to assess the efficacy of a feed supplement in alleviating the clinical signs of headshaking in 32 horses. Equine Veterinary Journal. 2013 May;45(3):293-7.

2. Gruen ME, Dorman DC, Lascelles BD. Caregiver placebo effect in analgesic clinical trials for cats with naturally occurring degenerative joint disease‐associated pain. Veterinary Record. 2017 May;180(19):473-.

3. Grelotti DJ, Kaptchuk TJ. Placebo by proxy. BMJ. 2011 Aug 11;343.

4. Tuyttens FA, de Graaf S, Heerkens JL, Jacobs L, Nalon E, Ott S, Stadig L, Van Laer E, Ampe B. Observer bias in animal behaviour research: can we believe what we score, if we score what we believe?. Animal Behaviour. 2014 Apr 1;90:273-80.