Clinically proven

My eye was caught the other day by this advert:


Quite a bold claim, I thought. “Defends against cold and flu” would indeed be impressive, if it were true. Though I also noticed the somewhat meaningless verb “defend”. What does that mean exactly? Does it stop you getting a cold or flu in the first place? Or does it just help you recover faster if you get a cold or flu?

I had a look at the relevant page on the Boots website to see if I could find out more. It told me

“Boots Pharmaceuticals Cold & Flu Defence Nasal Spray is an easy to use nasal spray with antiviral properties containing clinically proven Carragelose to defend against colds and flu, as well as help shorten the duration and severity of both colds and flu.”

It then went on to say

“Use three times a day to help prevent a cold or flu, or several times a day at the first signs helping reduce the severity and duration of both colds and flu.”

OK, so Boots obviously want us to think that it can do both: prevent colds and flu and help treat them.

So what is the evidence? Neither the advert nor the web page had any links to any of the evidence backing up the claim that these properties were “clinically proven”. So I tweeted to Boots to ask them.

To their credit, Boots did reply to me (oddly by direct message, in case you’re wondering why I’m not linking to their tweets) with 4 papers in peer reviewed journals.

So how does the evidence stack up?

Well, the first thing to note is that although there were 4 papers, there were only 3 clinical trials: one of the papers is a combined analysis of 2 of the others. The next thing to note is that all 3 trials were of patients in the early stages of a common cold. So right away we can see that we have no evidence whatsoever that the product can help prevent a cold or flu, and no evidence whatsoever that it can treat flu.

The “clinically proven” claim is starting to look at little shaky.

But can it at least treat a common cold? That would be pretty impressive if it could. The common cold has proved remarkably resilient to anything medical science can throw at it. A treatment that actually worked against the common cold would indeed be good news.

The first of the trials was published in 2010. It was an exploratory study in 35 patients who were in the first 48 hours of a cold, but otherwise healthy. It was randomised and double-blind, and as far as I can tell from the paper, seems to have been reasonably carefully conducted. The study showed a significant benefit of the nasal spray on the primary outcome measure, namely the average of a total symptom score on days 2 to 4 after the start of dosing.

Well, I say significant. It met the conventional level of statistical significance, but only just, at P = 0.046 (that means that there’s about a 1 in 20 chance you could have seen results like this if the product were in fact completely ineffective: not a particularly high bar). The size of the effect also wasn’t very impressive: the symptom score was 4.6 out of a possible 24 in the active treatment group and 6.3 in the placebo group. Not only that, but it seems symptom scores were higher in the placebo group at baseline as well, and no attempt was made to adjust for that.

So not wholly convincing, really. On the other hand, the study did show quite an impressive effect on the secondary outcome of viral load, with a 6-fold increase from baseline to day 3 or 4 in the placebo group, but a 92% decrease in the active group. This was statistically significant at P = 0.009.

So we have some preliminary evidence of efficacy, but with such a small study and such unconvincing results on the primary outcome of symptoms, I think we’re going to have to do a lot better.

The next study was published in 2012, and included children (ages 1 to 18 years) in the early stages of a common cold. It was also randomised and double blind. The study randomised 213 patients, but only reported efficacy data for 153 of them, so that’s not a good start. It also completely failed to show any difference between the active and placebo treatments on the primary outcome measure, the symptom score from days 2 to 7. Again, there was a significant effect on viral load, but given the lack of an effect on the symptom score, it’s probably fair to say the product doesn’t work very well, if at all, in children.

The final study was published in 2013. It was again randomised and double blind, and like the first study included otherwise healthy adults in the first 48 h of a common cold. The primary endpoint was different this time, and was the duration of disease. This was a larger study than the first one, and included 211 patients.

The results were far from impressive. One of the big problems with this study was that they restricted their efficacy analysis to the subset of 118 patients with laboratory confirmed viral infection. Losing half your patients from the analysis like this is a huge problem. If you have a cold and are tempted to buy this product, you won’t know whether you have laboratory confirmed viral infection, so the results of this study may not apply to you.

But even then, the results were distinctly underwhelming. The active and placebo treatments were only significantly different in the virus-positive per-protocol population, a set of just 103 patients: less than half the total number recruited. And even then, the results were only just statistically significant, at P = 0.037. The duration of disease was reduced from 13.7 days in the placebo group to 11.6 days in the active group.

So, do I think that Boots Cold and Flu Defence is “clinically proven”? Absolutely not. There is no evidence whatsoever that it prevents a cold. There is no evidence whatsoever that it either prevents or treats flu.

There is some evidence that it may help treat a cold. It’s really hard to know whether it does or not from the studies that have been done so far. Larger studies will be needed to confirm or refute the claims. If it does help to treat a cold, it probably doesn’t help very much.

The moral of this story is that if you see the words “clinically proven” in an advert, please be aware that that phrase is completely meaningless.

43 thoughts on “Clinically proven”

  1. Hello,

    From my own statistical sample of one (which i know is pretty meaningless) I’ve had a great result from both Vic first then Boots cold defense use. Not had a cold this year for the first time in around 20 years. I travel by train and tube every day and have been surrounded by people who’ve had the cold/flu in the last 5 months. Usually I’d have had it at leat 2 or 3 times so far. So it’s working pretty well for me. I’d be happy to participate in a decent trial for this stuff, but even withought that so far it’s done pretty well for me, or maybe i’ve just been lucky! I’m going to keep using it and see how long I can go before I get a cold.

    1. I completely endorse what you are saying. I have often used Vicks First Defence at the first signs of a cold and it does not get any worse. Initial early symptoms clearing completely within a couple of days. Colds always turn to chest infections for me, so glad to be able to ward them off from the start.

  2. “First Defence” as a cold preventative. Does it work?

    This paper reports the development and implementation of a unique research project designed to investigate the effects of “First Defence” as a preventative for colds.
    Previous studies investigating the effects of cold and flu nasal sprays employed participants already at the onset of an upper respiratory tract infection and their results have been inconclusive.
    This unique study investigates the effectiveness of a nasal spray used as a daily preventative.
    One female singing teacher (age: 42-44yrs) completed a 2-year longitudinal empirical study. Data regarding her lifetime experience was noted before the intervention where 4-6 colds a year were reported.
    “First Defence” was taken nightly after teaching, 3 squirts up each nostril. A slight tingly sensation was noticed.
    An ongoing diary method was employed to register day-to-day effects and the participant reported no side-effects or illness, including colds or flu, for the full 2 year experiment.
    The implication from this groundbreaking research is that vocal students, teachers of voice and singing performers at any level would benefit greatly from taking “First Defence” to avoid catching colds.
    Limitations of the research were the amount of participants and the fact the participant administered the research. There was no control, no randomization and no ethics approval was sought. No conflict of interest was reported.
    Further non-serious research is being conducted to investigate the effects on a wider demographic of singing students.


  3. I have had around 50 colds over the last 4 years and I have taken BOOTS cold & flu defence and each time, it has effectively cleared it up. 100% got rid of it. I took it when I had the flu and it cleared it within 2 days.

    So who gives a sh*t about written studies and evidence!!! If you just took it yourself when you had a cold or the flu, you would see for yourself that it WORKS!!!!! You have to take it 3 times a day, 2 squirts in each nostral for a week !!!

    1. How do you know your colds wouldn’t have got better if you didn’t take it?

      If the product really worked as well as you think it does, then that would have shown up in the clinical trials. But it didn’t.

      That, my friend, is why we have to do properly controlled clinical trials. It’s pretty much impossible to know whether something works otherwise.

  4. These products do not contain active ingredients and instead are salt solution with some polysaccharide which forms a barrier preventing the virus from penetrating the nasal mucosa- a process needed for systemic infection, spread and development of a cold. Not sure therefore if you can reliably do a clinical trial or state clinically proven. I use these products when I feel that pain in my nose that there is an inflammatory reaction occurring (a possible cold forming) and if dodged a few colds in the last few years, but this (like others) is very subjective evidence. I use the product as to me the principle of how it works makes sense and it appears to work- with the caveat that this is very subjective. If someone told me that other therapies worked I would go through the same exercise before regularly using them.

    1. Having read the packaging, I believe they also contain Eucalyptus, Zinc, and something Acidic. Zinc was found, by the common cold unit, to inhibit cold progression if taken in large enough quantities so I think there is, at least, some sort of experimental background to the ingredients.

  5. This stuff is the advertising industry’s dream: if you don’t get a cold, the medicine worked; if you do get a cold, you took it too late. The same argument could apply to all those hair-restoring medications such as Regaine.

      1. Maybe you were going to regrow it, but many studies have shown that minoxidil does regrow hair. Will you also say that antibiotics don’t work? How can we possibly know that you wouldn’t have got better anyway? Especially if clinical trial studies are meaningless to you…

  6. I would not be without Boots spray. I use it both when I have been with someone with a cold, or if I feel the very first symptoms. I now seldom if ever get a full blown cold. It is miraculous as far as I am concerned. (The Vick spray hurts my nose so I abandoned that) .

  7. I have non CF Bronchiectasis and suffer severe side effects to most antibiotics. It is really important that if I am going to stay out of hospital that I do not pick up bacterial/viral infections. This Spray was originally created by a company based in Vienna. To find out more about trials it is best to go to their website. There is a strong tradition in folk medicine in the UK that red seaweed- carrageen is good for upper respiratory tract and lung infections. Right now there are clinical trials going on in Wales using red seaweed as a new treatment for Cystic Fibrosis.
    Since the end of September 2015 I have had no Exacerbations. I am putting this down to using BOOTS Cold and Flu Defense before and during outings on public transport/public places, avoiding visits from people with colds, hand washing and relying on a face mask when I have to. This winter has been really bad for virus/ bacteria infections and right now at least half the population are coughing and sneezing. I would recommend this product and I strongly believe that due to the strategy I have adopted I have managed to stay in better health than many people without the chronic lung condition I have.

    1. So let me get this straight. You have been avoiding people with colds, been careful about washing your hands, and wearing a face mask, and you think it’s the Boots spray that’s the reason you’ve avoided getting a cold?

  8. The data you provided initially confirms it must work only 20 to 1 chance it doesnt From having colds/manflu no idea how many times best guess 2 to three times per year to no colds in 3 years I’m certain it works as are loads of people everyone I know who has tried it swears by it I don’t think it can cure manflu that sounds implausible +it tastes horrible /stings I always take a halls menthol lyptus drop to mask the taste I also drink whisky, stop all cold drinks, and dress warm when I feel a cold coming on none of which is clinically proven but why not who wants a cold

    1. When you say “only 20 to 1 chance it doesn’t”, you are making a very common error in the interpretation of P values. A P value of 0.05 does not mean that there is only a 1 in 20 chance that a product is ineffective. It means that there is a 1 in 20 chance that, if the product is ineffective, you would have seen the results that you did. Those are not the same thing.

      The wonderful XKCD explains why they are not the same thing better than I could:

  9. Thank you for this blog. Until now I had been blindly sniffing the stuff as I thought there was peer reviewed science behind it. What a joke.

    Until they actually conduct some proper studies, I guess I’ll just stick to tissues and a grumpy face.-

  10. It is confusing that there are not more studies being conducted to support the use of these sprays because my experience (and many other people i know) is that they have proven highly effective at halting the development of colds. Working in an environment with lots of unwell people was resulting in 1-3 colds per year for me, but since i bought my first bottle 3 years ago i have not had a single full blown cold following the early signs. I agree that the empirical evidence does not appear to support the claims, but my experiences absolutely do.

  11. Was it your complaint that took the product off the shelves for a period this year? Having worked well for me in the past, I tried to get it but was unable, I then had a full on cold over an important engagement that probably could have been prevented or made a lot less severe. Whether or not it was adequately clinically proven to be advertised as such, its not snake oil and does work. Did you spot the advert while raising your head to sneeze over everyone else in the carriage? Because that would infer a similar indifference to the conequences of your actions for other people.

    1. No, my complaint was only about the advert. Boots are still perfectly entitled to sell the product if they wish. If they are not selling it, you’ll have to ask them why not.

  12. I have used this product for the last 12mths and have not had a cold or respiratory infection since starting to use it. I suffer from bronchitis if I get a cold so I am anxious to prevent infection. I work with children and families so am always in contact with the cold virus.
    I use the product before and after contact with children and also if I feel any cold symptoms coming on . My husband also has used the product for the same amount of time. He is asthmatic and suffers from bronchitis for most of the winter. He has not had bronchitis for 12 mths.
    I am a Health proffesional and recommend this product to friends.

  13. I can only add my vote in favour of this product. I had several holidays being unwell after catching colds/respiratory viruses until my sister told me about this. Since then my husband and I have used it just before any flight (once or twice a year) and at the first sign of a cold or having been exposed to cuddles from streaming very young grandchildren. I used to go down like a sledgehammer with every cold – a week feeling dreadful. Since taking Vicks I’ve never had anything lasting longer than a couple of days with mild sniffles. There are countless similar reviews on Amazon and the negatives seem to relate to using it incorrectly directly up the nostrils. Maybe it is a placebo effect – but do I care? Definitely not!

  14. Interesting post (will read more, Statsguy) and comments, thanks all. Considering buying as boyfriend has had chemo. Suspect it will quickly start gathering dust in bathroom cabinet. Best tip: avoid rubbing/touching one’s eyes. Lesley praps referring to instruction “do not inhale deeply”.

  15. I am glad of blogs like yours trying to clarify woolly thinking and dodgy science. But this means you have to be very careful you are thinking clearly yourself. Some of your replies to comments sound a bit knee jerk.
    The trials you analysed seemed to deal with people in the early stages of a cold. Many of people writing in are talking about using it to prevent getting a cold in the first place – or at least getting infected when in situations of high risk – planes schools etc etc. The biology of a cold does involve the immune system fighting of virus and bacteria in the throat. It is at least possible that inhaling something through your nose can help or hinder this process. Whether it does or not in this particular case
    I don’t know. It seems it’s not clinically proven – or disproven. So read carefully what people are claiming before you tell them they are idiots.

  16. The RX antiviral medicine Tamiflu reduces symptoms by an average of 18 hours only. According to you the point estimate from the third study is a reduction of symptoms of 2 days for this nasal spray. It’s good enough! I think you are being too harsh, especially it costs just £8.99! I will pay £8.99 every time for feeling better a day earlier, not to mention 2.

    Plus, for colds the alternative choices are just water and honey, which are probably even less efficacious.

  17. Even if it does work, I’m wondering now what effect it might have on the immune system. Would you lose immunity that the body fought so hard to build up? Just a thought.

  18. It is truly an amazing feature of the human psyche that people believe their own N=1 uncontrolled non-experiment rather than clinical trial evidence. Makes you wonder why we bother! I am late to this party having only noticed the existence of this stuff today. The first thing I did was to look for the evidence, which led me to your page. Thank you for laying it out so clearly and saving me £9 every time I feel a tickle.

  19. I understand that it is the wording of the advert is what the blogger is getting uptight about? Indeed, let’s challenge such claims but we must be clear about all the issues involved in this…..and not jump to denigrate or mock those with other viewpoints.
    Let’s face it, the scientific world and its processes, however valid they are in their own way, are only one way of perceiving how the world and living things work. Science has endorsed many products over the years as effective and safe which are later, over decades in some cases, proven to be downright dangerous. So if we start from that premise maybe some balance may enter the argument.
    And science does take into account that there are such things as both false positives and false negatives, ie and a case in point, the blood test commonly used to identify if you have cancer.
    My own particular focus is on what you put into your mouth in the first place – if you eat food contaminated by heavy metals, pesticides, herbicides, fungicides, antibiotics, and now, more topically, the takeup and concentration by microplastics of toxic chemicals in the oceans, our immune systems will necessarily be compromised. And science seems to me to be blinkered in that it tests a single chemical and pronounces on that without considering the potential impact of that chemical in combination with all the others in the cocktail we call ‘food’ these days. And perhaps we should extend our definition of food to include what we breathe and what we touch. See ‘Phthalates’.
    The great flu epidemic of 1918 killed the millions of people it did by generating an out of balance and over the top immune reaction, so that people literally drowned in their own antibodies’ waste products….this uncomfortably reminds me of the allergic responses people seem to increasingly suffer from these days, which may mean that our immune systems are more and more out of balance in this way.
    Personally, I’ll go for all the food I can grow myself organically, with the addition of a cocktail of garlic, ginger, chilli and horseradish in cider vinegar as my winter booster.
    With respect to the efficacy or not of First Defence type sprays, it may well be that the purchase and use of such products if nothing else serves to raise awareness about immune systems and the host of other issues relating to them, and so, for a few pounds, it might be worth while even if there are no scientific results to support its claimed effects. And even if it no more than a placebo, well, placebos have their place….illness can be reduced/moderated – what you will – by the power of the belief that things work – or, rather, that your own body works.
    And to make a final comment to the originator of this blog, life is more than the sum of its parts and since science still only works with what it can see and measure, there are bound to be relationships and reactions between living things that are beyond its present scope.

  20. I noticed Vicks First Defence on a shelf in Boots about a month ago whilst I was waiting in a queue to pay for something else. I thought it would be a good idea to get some “just in case”. A few days ago, I started with a cough and sore throat so gave it a try. The symptoms didn’t get any worse and have now almost gone. I will keep using it for a couple more days as advised. I am impressed with this product and would advise others to try it.

  21. Fully agree with your thoughts.

    Another point to consider with these trials is that they don’t compare the actual viral illness being experienced, and hence any comparative symptom scores are useless. You can make the assumption that all common colds are the same, but as we all know, their severity does vary and that depends on not only your own bodily state, but also the particular virus and strain.

    You don’t know which people in which groups had milder or more severe colds, so comparisons are unhelpful.

  22. I notice that Vicks first defence has very different ingredients to Boots Dual Defence (I assume as still under patent). Have you investigated the similar claims for the Vicks product?

  23. Vicks First Defence is amazing stuff I don’t care about all this nonsense clinically proven or whatever, it works for me Iv used it for years and I can 100% tell you that if I do not use it my cold lasts a lot longer with much worse symptoms, if the difference between missing a days work and not and that is using it after I get full blown cold and again 100% if I get Vicks First Defence in during the early stages of a cold I will avoid the cold all together sometimes with a few minor symptoms, I would be really cross if this got taken of the shelves because someone was being padantic about phrasing used in they way the advertise, this stuff really works and I think if more people used it there would be far less cold circulating during the winter.

  24. Felt a cold coming on. Got Vicks First Defence. No effect whatsoever (used it immediately at the start of noticing symptoms four times a day as directed). Don’t waste your money.

  25. I developed a condition that means Iits very dangerous for me to catch a cold. I’ve been using first defence and it 100% makes all my colds to be extremely mild. My partner who always suffers from extreme colds uses it as well and she now just get mild colds.

    My partner is a Doctor so we understand health and this works for us. In fact I now actively seek to get a cold to build my immune system up because I’m confidant that First Defence (and I’ve also started to use ColdZyme) will stop the cold being of concern.

Leave a Reply

Your email address will not be published. Required fields are marked *