The Evidence Pyramid Revisited

This is the 9th anniversary of my “Best of Health” articles. It’s hard to believe I’ve been writing these for 9 years! Thank you to everyone who reads them and to those who correspond with me following their publication. You can find a complete archive of my articles on my blog at: https://sunsongdachshunds.wordpress.com/health-welfare/our-dogs-best-of-health-articles/

Last month I wrote about the challenges of cherry-picking data from published research studies and how they can be used to generate click-bait headlines in the national press and on social media. I emphasised the importance of breed clubs collecting their own data with robust health surveys. Ideally, these should include gathering responses from owners of dogs that aren’t part of the show community or that aren’t KC registered. These non-show and non-KC data have the potential to demonstrate whether or not there are differences in the health of these different sub-populations of our breeds.

My caveat at the end of last month’s article was that more data won’t improve dog health or longevity. There is little point in endlessly arguing with the published research or debating whether or not the sample in a survey is truly representative of what’s happening in a breed.

I was reminded of an article I wrote in 2017, following the Breed Health Coordinators’ Conference. One Of the presentations was by Dr Zoe Belshaw from the Centre for Evidence Based Veterinary Medicine at Nottingham University. Zoe talked about the so-called Trust Triangle which describes the different types of information you might come across and the levels of trust that can be associated with each.

A variation of this is the Evidence Pyramid which has expert opinion at the bottom, followed by Case Studies, Cohort Studies and Randomised Control Trials (RCTs). These latter 3 are unfiltered information which may be available as Open Access papers. Sitting above these are a series of filtered information sources such as Systematic Reviews (meta analysis). These publications dissect and critique a set of primary research papers in order to arrive at “the best evidence” to support a particular case (or to disprove it). This is the sort of work that Zoe’s colleagues do at the Nottingham Centre for EBVM and they then publish what can be considered to be best practice for vets and clinicians to adopt. As with all science, “best practice” today could well change if new research evidence emerges.

This all seems quite logical but, recently, I was intrigued to read an article by Dr Michael Putnam, an Associate Professor of Medicine in Wisconsin who argued that the Evidence Pyramid is flawed.

One of the points he makes is that, in the real world, when a medical professional needs an answer to some obscure clinical question, they rarely dig through published case reports. Their pragmatic approach is to ask a respected colleague for their expert opinion because they feel this carries more weight than reading some randomly published paper.

Putnam argues that systematic reviews are emphatically not the highest level of evidence. This is a good point because they are simply a view of the actual evidence collected from a range of RCTs and observational studies. Obviously, this then depends on the quality of the review process and the input papers and studies which are in the review pool. Many of the studies included in these reviews will involve small sample sizes and (sometimes) dubious methodologies particularly when it comes to statistical analysis. I have had several conversations recently about papers published on research into intervertebral disc disease where the statistical analyses were less than ideal and/or where the studies were underpowered due to small sample sizes. The reason I queried these papers was that the findings contradicted previous studies (both in dogs and humans). Luckily, I have some very capable statistician friends and am in contact with researchers to whom I can turn for a critical appraisal of new papers.

A paper published by Prof. John Ioannidis (Stanford University) in 2016 said that there is massive production of unnecessary, misleading, and conflicted systematic reviews and meta-analyses. Instead of promoting evidence-based medicine and health care, these instruments often serve mostly as easily produced publishable units or marketing tools. He concluded that “China has rapidly become the most prolific producer of English-language, PubMed-indexed meta-analyses. The most massive presence of Chinese meta-analyses is on genetic associations (63% of global production in 2014), where almost all results are misleading since they combine fragmented information from mostly abandoned era of candidate genes.” More shockingly, a 2022 paper by Ioannidis stated that simulations show that for most study designs and settings, it is more likely for a research claim to be false than true and that many research findings may simply be accurate measures of the prevailing bias.

What should be at the top of the pyramid?

Putnam argues that RCTs should be at the top of the evidence pyramid. However, he goes on to say that we should acknowledge that many RCTs are poorly designed, underpowered and subject to bias. Therefore, instead of the pyramid having discrete layers that differentiate between sources of evidence, the model should recognise that some trials are worse than some cohort studies. In other words, there is more of a blurred boundary between observational studies and RCTs. The case for this reengineering of the Evidence Pyramid was also made by Murad et al in the BMJ in 2016.

Does it matter in the real world?

We are encouraged to consider Evidence Based Veterinary Medicine as an underpinning principle for recommending approaches to diagnosis and treatment of canine health conditions. The quality of evidence should determine the confidence in recommendations.

In practice, there don’t seem to be that many published RCTs that are of relevance to us but there are numerous observational and cohort studies (retrospective and prospective). Putnam ended his article by saying that “good observational studies may be better than bad RCTs and that we should read and judge each paper by its individual merits, not by its strata on a colourful pyramid.”

So, in the real world, it might be worth reading the “Limitations” section of any research paper before you read the full paper. It’s also worth reading the “Conflicts of Interest” declaration to find out who funded the study. I think it was Sid Vicious who said “Today, everything’s a conflict of interest”! 

Advertisement

Who’s using the data and why?

It’s not unusual for research papers to get widely reported in the national press, particularly where it’s possible to spin a provocative headline that grabs readers’ attention. Lobby groups and campaigners are also skilled at selecting research that supports their cause. You’d be surprised if they didn’t do that; after all, they have a cause to promote. 

In March 2019, I wrote about the discussions on social media when new scientific papers on canine health and welfare matters are reported and when breed clubs publish their health survey results. Newly published research should prompt us to ask the question “why?” – why might a particular association have been identified and why might the results have turned out like they did.

Last year, the VetCompass project published a paper that generated a lot of publicity in the national press. Life tables of annual life expectancy and mortality for companion dogs in the

United Kingdom reported an approach to assessing canine longevity (Life Tables) which is well-established in human health studies. The paper’s findings that the French Bulldog had the shortest life expectancy of just 4.5 years compared with Jack Russell Terriers (12.7 years) inevitably made for some eye-catching headlines. The VetCompass paper also reported the average longevity of Japanese French Bulldogs to be 10.2 years. Perhaps unsurprisingly, that figure didn’t feature in the press headlines.

In contrast, the Kennel Club’s 2004 and 2014 breed health surveys showed the median age of mortality of French Bulldogs to be 9 and 5.9 respectively. This data is summarised in the French Bulldog Breed Health and Conservation Plan. When the KC’s 2014 survey results were published in 2016, the overall median age of death of pedigree dogs was reported as 10, down from 11.3 in 2004. This led to descriptions of an “apocalyptic drop in purebred dog longevity” despite the fact that no such (statistically valid) conclusion could be drawn from the data reported.

Why the differences in longevity?

The VetCompass paper describes some of the limitations associated with the results they published. The limitations included:

• The sample size for FBs was relatively small, resulting in life tables with reserved confidence
• The sample was biased towards younger dogs that contribute proportionately more deaths in younger ages

VetCompass reports are based on a particular sample population – dogs attending first opinion vet practices in the UK. This sample will, like every sample, have its own biases. We know, for example, that insured dogs are more likely to be taken to the vet. The Life Tables paper was derived from a sample of nearly 900,000 dogs that had at least one visit to a vet

(90% in England) during 2016. These visits would have been not only due to illness but also included routine vaccinations. It is, therefore, not as simple as arguing that the dogs in the study were all ill. The paper drew data from nearly 900 vet clinics and VetCompass claims to gather data from about a third of all practices in the UK.

It is always worth looking at a range of research papers and analyses in order to build a picture of a breed’s health and longevity. Another useful source of such information is the International Partnership for Dogs (www.dogwellnet.com). They have collated and published data for many breeds, including French Bulldogs, Pugs and Bulldogs (all 3 of which have been the subject of recent VetCompass studies). Their Globally Relevant Integrated Health Profiles (GRIHP) are particularly useful as they pull together data from multiple sources, including UK and Swedish insurance data, VetCompass, Breed Club surveys and more. They don’t, however, provide any specific longevity statistics for French Bulldogs, only data on relative risk mortality compared with an all-breed average. The Agria data show, for example, that French Bulldogs have at least twice the relative risk of mortality than all breeds, for 6 types of health condition including respiratory, eyes and neurological.

One thing VetCompass does really well is communications; it publishes open-access peer-reviewed papers that meet the needs of a technical audience, with supporting data available. It also publishes infographics; easily digestible summaries that are visually appealing and comprehensible to the “person on the Clapham Omnibus”. VetCompass seems to practise a very “agile” approach to the use of its data assets. It publishes useful chunks of information at frequent intervals that answer specific questions. 

Breed clubs need their own good quality research

If a breed has concerns about a particular health condition, age of death or cause of death, it should ensure it can collect sufficient good quality data from a representative sample of dogs. I have previously emphasised the value of clubs collecting data that can show any differences between sub-populations (e.g. show vs. working or show vs. non-show).

The point of Breed Health Surveys depends totally on “the exam question” you are setting out to answer. If the exam question is “what’s the biggest issue?”, most Breed Clubs’ surveys have been able to answer that. The KC’s 2004 and 2014 surveys did that too. If you want to know where to focus effort on improvement, asking about prevalence and age of diagnosis isn’t going to get you very far. You need to know about severity and welfare impact as well. Finally, if you want to know if there’s been any change (for the better or worse) over a period of time, you’d better make sure you’re measuring the same thing, in the same way, at the start and the end. Otherwise, you’ll end up with “Garbage in, Gospel out” with someone cherry-picking the results to suit their own argument!

VetCompass gives us “big data”; breed health surveys give us “small data”. We need both, but most importantly, we need to define the “exam questions” before rushing off to design surveys, or to see if we have data already available.

I believe that all our breed club communities should be supporting the development and use of The Kennel Club’s Breed Health and Conservation Plans (BHCP). These are the most comprehensive documents that summarise the breadth of research available. Breed club officers and health representatives need to be familiar with the content for their own breed so that they can present a balanced and evidence-based argument when challenged about health and/or longevity.

More data won’t improve dog health or longevity

In September 2020, I said that data are necessary but not sufficient. I went on to explain that without human behaviour change, the lives of dogs will not improve. The missing science is that of Human Behaviour Change. 

We do need to question the research that is being published and to understand its strengths and limitations. Every piece of research has the potential to nudge us towards actions that will improve the health and longevity of our dogs. We need to see the bigger picture and focus on the behaviours of owners, breeders and vets who can all actually make a positive impact on dog health.

Every day is a learning day

One of the potential barriers to improving the health of pedigree dogs is breeders’ lack of understanding of genetics. Most breeders are, by now, familiar with DNA tests for genetic mutations for health conditions such as PRA, CLAD, DM and many more (often also with 2 or 3 letter abbreviations!). The principles of recessive mutations with 3 genotypes; Clear, Carrier and Affected and what these mean in terms of clinically healthy or unhealthy dogs is generally known by breeders. This may be less clearly understood by buyers who may still think that Carriers are likely to be a problem and get ill. 

As we move into discussing which combinations of those 3 genotypes can safely be bred together, there are still a range of opinions on whether DNA Affected dogs should be bred from. As long as an affected dog is mated to a clear dog, any puppies will not be affected but will be carriers. In many breeds, where there are small gene pools, it is reasonable to breed with affected dogs (mating only to clears). While it might be argued that not using affected dogs is a quicker way to remove deleterious mutations, it also has the effect of removing all that dog’s genes from the population. Removing dogs from breeding on the basis of one genetic mutation alone is often not in the best interests of the breed.

When it comes to coats and colours, far fewer breeders seem to be educated or to make the effort to understand the genetics. All too often, discussion of colours seems to be based on historical urban myths or worse, on fashion and personal prejudices. It’s such an emotive topic, as we have seen with numerous discussions over many months about Colour Not Recognised (CNR) and Non Breed Standard (NBS) colours.

Emerging science

What I find really fascinating is the rate at which the science underpinning the genetics of coats and colours is developing. Some of the research is being enabled by Citizen Science where dog owners’ contributions help research teams by providing DNA samples and photographs. One such study published last year resulted in an improved understanding of one of the earliest coat colour mutations, designated as ancient red (eA). This genotype is associated with “domino” in Alaskan Malamute and other Spitz breeds, “grizzle” in Chihuahuas and “pied” in Beagles.

Another new paper (2021) explains variations in the PMEL gene which causes dapple (merle) in Dachshunds. A previous study of this gene in Australian Shepherds had correlated the length of an insertion into the PMEL gene with 4 broad phenotype clusters of merle, described as “cryptic”, “atypical”, “classic” and “harlequin”. This new paper reports on a similar study in Dachshunds and identified numerous cases of “hidden” merle in light red dogs. The paper suggests that the frequent identification of cryptic, hidden and mosaic variants of the merle pattern makes DNA testing critical to avoid producing puppies with serious health problems. Double-merles are known to be at risk of deafness, blindness and microphthalmia (small eyes) and are banned from registration by the KC.

Time for cocoa?

Another lesson I learned recently was the existence of a gene associated with the brown (liver/chocolate) colour. Variants of the B locus are the most common cause of the brown coat colour, with 5 known mutations of the TYRP1 gene that explain the majority of dogs with brown coats and noses. One exception has been the brown or chocolate French Bulldog which, when tested, is found to be BB (i.e. not chocolate as normally expressed). Recent research (2020) has identified a mutation on the HPS3 gene associated with brown in FBs and which has been called “cocoa”, for which a DNA test is now available. 2 recessive copies of this mutation (co/co) are required for the dog to be brown/chocolate.

5, not 4, basic coat colour patterns

The most recent paper I have been reading was published by Dannika Bannasch and her colleagues at the University of Bern. Professor Bannasch and one of her collaborators, Prof. Tosso Leeb, have both been winners of the prestigious International Canine Health Awards. The study clarified how coat colours and patterns are genetically controlled but also discovered that the light coat colour in many modern dog breeds is due to a mutation that originated in an extinct species more than 2 million years ago. 

Dogs can make 2 types of pigment; black (eumelanin) and yellow (pheomelanin). Production of these 2 pigments in the right place on a dog’s body results in very different coat colours and patterns. The agouti signalling protein (ASIP) is the main switch for the production of yellow. Without ASIP, black pigment is formed. 

In addition, there are 2 “promoters” which result in ASIP production on (a) the belly and (b) banded hairs. The study identified 2 versions of the ventral promoter and 3 versions of the hair cycle promoter, resulting in 5 possible combinations which cause different coat colour patterns in dogs. Previously, it had been thought that there were only 4 basic patterns.

Image source: https://www.nature.com/articles/s41559-021-01524-x.pdf

Within each of these 5 pattern types, there may be further variation due to other factors such as the position of the boundary between red and black areas, the shade of red (from dark to nearly white) and the presence of a black facial mask or white spotting caused by genes other than ASIP. 

Prof. Bannasch said: ‘While we think about all this variation in coat colour among dogs, some of it happened long before ‘dogs’ were dogs. The genetics turn out to be a lot more interesting because they tell us something about canid evolution.’

Learning from history

While a lot of the science and understanding of the genetics may be new, many breeds have a wealth of historical information on coats, colours and patterns. Much of that is held in Kennel Club registries or breed archives. These should be essential resources to inform any discussion about colours that can or cannot exist legitimately in any breed. I am aware of a study in one breed, looking at dogs from the early part of the last century, which clearly shows that colours that are not currently fashionable were around over 100 years ago. 

The fact that some of these colours are associated with recessive mutations should make it unsurprising that those colours can still crop up and be bred today. There are plenty of examples of breeds where breeders have specifically selected for a particular colour or pattern and that’s not something new. We should be very careful not to forget our breeds’ histories and their genetic origins, and not fall into the trap of altering Breed Standards simply on the basis of what is or isn’t fashionable today. That, of course, applies to conformation as well as to colours!

Remember, prejudice is a great time-saver; it enables you to form opinions without having to gather the facts.

Pandemic Puppies Survey: Did you buy a puppy in 2019 or 2020?

A team of researchers at the Royal Veterinary College (RVC) wants to improve the lives of puppies and dogs, but they need your help with this survey!

Puppy buying can be an exciting and emotional time for households, but buying and owning a new puppy can also present a range of challenges. The RVC team is interested in learning about the puppy buying process from owners who acquired puppies in 2019 and/or 2020. By understanding how and why people buy puppies, we can learn where things go ‘right’ or ‘wrong’ in that process. This will help future owners avoid pitfalls and help target advice to owners who have experienced problems purchasing their puppy.

Who can take part?

Dog owners who meet all of the following four criteria:

1. Are over 18 years of age.
2. Resident in the United Kingdom.
3. Brought a puppy of any breed or crossbreed home aged under 16 weeks during 2019 or 2020.
4. Purchased your puppy from a private seller (i.e. you did not breed them yourself, you paid for the puppy AND you did not obtain them from a formal rescue organisation, as we are interested in the process of buying a puppy).

N.B. Even if you no longer own your puppy/dog, we would still like to hear from you.

Take the survey here

https://www.surveymonkey.co.uk/r/pandemicpuppies

Canine anxiety and puppy-rearing

Earlier this year I analysed some data collected by one of our Dachshund Breed Rescues. We wanted to see whether the massive increase in popularity of Miniature Smooth Dachshunds was feeding through into a rehoming and rescue problem. Unsurprisingly, the 2 are linked and this particular rescue charity has seen a 4-fold increase in rehoming cases from 2017 to 2019. Of those, 70% were not Kennel Club registered and that figure mirrors what we know about the market for pedigree dogs. Far more are bred outside the KC registration system than within it.

The analysis of the rescue data showed that a quarter of all cases were associated with biting or aggression. That is a worryingly high proportion, especially when compared with the findings of one of our breed surveys. In 2012, our survey asked about behaviour and temperament, and just 1% of Mini Smooths were reported as being aggressive with people (5% were aggressive with other dogs).

My suspicion is that many of these rehoming cases are a result of badly bred dogs producing puppies that are badly reared and then sold to inexperienced owners who know very little about canine behaviour and can’t cope.

Last month, I wrote about the breeding recommendations in a recently published paper “Throwing the baby out with the bathwater” (Dawson et al). Firstly, the authors recommended that breeding choices and puppy-rearing processes should be based on knowledge of good practices. Secondly, they advocated that all dogs should be independently tested for suitability before being bred from. In addition to suitability from a health point of view, they suggested behavioural testing is important to check their suitability to be good companion animals. Dogs that are themselves good companions, are more likely to produce puppies that will be good companions as well.

Fearfulness and its causes

I’ve been re-reading another paper on behaviour: “Early life experiences and exercise associated with canine anxieties” published by Hannes Lohi and Katriina Tiira in 2015. It’s an Open Access paper so you can download the full version, yourself. The study collected data from a Finnish family dog population to identify environmental factors that might be associated with canine fearfulness, noise sensitivity and separation anxiety. I was particularly interested in the findings on fearfulness, in light of the aggression/biting data found in the rescue Dachshunds. The paper notes that aggressiveness is often motivated by fear and that bite injuries from human-directed aggression are an important public health concern. In 2017/18 there were just under 8000 NHS hospital admissions for dog bites and this figure has risen by almost 5% since 2015. However, a 2017 paper (Westgarth et al) suggests that the real burden of dog bites is considerably larger than those estimated from hospital records.

While fearfulness is known to have relatively high heritability, 2 major environmental factors are also known to affect this: lack of juvenile experiences and aversive learning. In the Lohi/Tiira paper, they found that a puppy’s maternal care and the amount of socialisation had the largest effects on fearfulness. Fearful dogs had received poorer maternal care and were less well socialised compared with less fearful dogs. Additionally, fearful dogs also lived in households with fewer other dogs and with more human adults. Bitches and younger dogs also tended to be more fearful. There was also a tendency for fearful dogs to get less exercise and they were more likely to live indoors, rather than spending their time indoors and outdoors.

It’s fireworks season

In our area, the firework season seems to have spread well beyond Bonfire Night and there will inevitably be another week of loud noises as we approach the New Year. Noise sensitivity was the second issue investigated by Lohi and Tiira. They found that dogs with noise sensitivity got significantly less daily exercise than dogs with no noise sensitivity. They were also more likely to have been neutered and were likely to be their owner’s first dog. The more dogs an owner had and the more dogs they had previously owned, the later the age of onset of noise sensitivity in their current dog. Overall, the evidence suggests that more socialised dogs were less likely to be noise sensitive.

I (don’t) want to be alone

Among social media discussion groups, there seem to be endless questions about Dachshunds with separation anxiety. It’s not just Dachshunds, of course. According to Dogs Trust, surveys have shown that between 13% and 18% of owners reported separation-related issues with their dogs. One study (albeit a small sample) found 85% of the sample had behavioural and psychological signs of stress when left alone.

The Lohi/Tiira study found that separation anxiety was more common in dogs that received less exercise. Other studies (Sargisson 2014) have shown that dogs tend to develop separation-related behaviour if they are male, sourced from rescues or puppy farms, and are separated from their littermates before 8 weeks. Protective factors include ensuring a wide range of experiences outside the home with other people from 5-10 months old, stable daily routines and the avoidance of punishments. 

No surprises!

It probably comes as no surprise that the largest explanatory factors associated with fearfulness were maternal care and the amount of early socialisation (up to 3 months old). However, it is important to note that comments on maternal care in the Lohi/Tiira paper were made by the owners, not the breeders. This reflects their recollection of what they had seen when they visited the breeder before taking the puppy home. The importance of the “See Mum” message cannot be overstated and, in practice, buyers should aim to see the puppies interacting with their mother at least once before the day they take their puppy home. It’s also worth reading the Puppy Plan (Kennel Club and Dogs Trust www.thepuppyplan.com) as a week-by-week checklist of experiences that well-reared puppies should have been exposed to. 

The findings on exercise also come as no surprise to me. Our dogs love to sniff when they are out, off the lead. This is an important aspect of their mental stimulation as well as them getting physical exercise. So many of the cases of separation anxiety and destructive behaviour that I read on social media are, I’m convinced, due to the dogs simply not getting enough exercise. The authors note that exercise may work as stress resilience, particularly as the resilient effect of exercise on anxiety and depression has been recognised in people. It is known that exercise increases serotonin production in animals and people, and this acts as an antidepressant. Interestingly, the study also found that dogs with less daily exercise were more aggressive to other dogs. The amount of daily exercise may be an indicator of the overall quality of dog management. 

In conclusion, I think buyers need to be much more aware of how their potential puppy has been socialised. They also need to be much clearer on their responsibilities for socialisation and exercise. Breeders probably need to exaggerate when explaining the amount of exercise a dog will need. Otherwise, we will continue to see dogs suffering from anxiety in their new homes and growing demand for rescue and rehoming services.

.

.

.

.

.

.

.

.

.

.