Possibly the worst “scientific” study EVER
August 21st, 2008
|
| Share |
I’ve seen some extremely bad scientific studies before. Some claim that there is a link between (insert name of scary chemical or radiation) and cancer by citing a tiny and statistically insignificant increase in a tiny study group. Some use a control group that is not properly demographically matched. Some are too small in scope to draw meaningful conclusions. Some do not account for possible incidental factors. A few may even obviously cherry pick their data.
Therefore, it takes a lot surprise me at this point, but when the British Homeopathic Association claimed a study had proven the effectiveness of homeopathy, I was actually surprised by how badly and blatantly they violated the most basic rules of conducting a scientific study. Of course I have absolutely no expectation that a homeopathy association would actually put out any kind good or well-controlled and rigorous study, but I would have expected they’d at least try to fake it a bit better.
Here’s their little press release:
Nearly 60% of patients who had received a series of homeopathy appointments reported an improvement in health that affected their daily lives. A total of 1,602 patients were seen at follow-up appointments in all five UK NHS homeopathic hospitals during one month in 2007. The study found that patients receiving homeopathic treatment often have more than one main medical problem.
The “study” was in “Homeopathy” which somehow managed to get on the website Science Direct. I guess it is “peer-reviewed” in the sense that it was reviewed by a group of frauds and idiots. Clearly the fact that something calls itself a “peer reviewed journal” and even shows up on a site along with actual journals does not mean that it can be trusted. According to the abstract:
A total of 51 medical practitioners took part in data collection over a 4-week period. Consecutive patient appointments were recorded under the headings: (1) date of first appointment in the current series; (2) appointment number; (3) age of patient; (4) sex of patient; (5) main medical complaint being treated; (6) whether other main medical complaint(s); (7) patient-reported change in health, using Outcome Related to Impact on Daily Living (ORIDL) and its derivative, the ORIDL Profile Score (ORIDL-PS; range, –4 to +4, where a score ≤−2 or ≥+2 indicates an effect on the quality of a patient’s daily life); (8) receipt of other complementary medicine for their main medical complaint.
So according to the study the actual effectiveness of the treatment is based entirely on subjective statements from the patient. The patients data was collected by the practitioner which makes it even worse! The patient is being put on the spot by their treater and asked to rate how they have faired with the treatment. The practitioner asking the question and collecting the answers has a clear vested interest in the result and the patient who goes to such a practitioner clearly would have faith in the treatment to begin with. The sample is not random but rather a sample of those who went to find homeopathic treatment.
Of course, the most glaring and absolutely inexcusable omission is that there is no control group. There is no placebo! The already biased group is given no context at all for the numbers! The numbers are meaningless as there is no comparison to what would be expected from a placebo, or for that matter, from doing absolutely nothing.
Given how badly this is, I’m surprised that the numbers for effectiveness are as low as 60% Normally placebos will give a better than 50% rate of subjective ratings for improvement, even independent of a biased group or vested interests.
Obviously nobody who knows the first thing about science or medicine would buy into this crap, so why do I care? Simply because extremely bad “studies” are a great tactic to defraud the public who generally won’t look into the background. Beyond that, the mainstream press has been known to buy into such studies and report them without the context or the skepticism they deserve.
This is about as low as fraud gets! This is really a non-news event, since homeopaths release fraudulent information and claim it proves something constantly, but hopefully if the chorus of voices calling them on it gets loud enough it’ll eventually make some difference.
This entry was posted on Thursday, August 21st, 2008 at 10:36 pm and is filed under Bad Science, Not Even Wrong, Obfuscation, Paranormal, Quackery. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
View blog reactions
August 21st, 2008 at 10:48 pm
Ya, that sets the bar on a new low, nevertheless I am confident that they will somehow manage to go under it in time. I wonder if that mouthy woman homeopath that was here several months ago that was outraged that her “profession” was being raked over the fire has to say about this?
I suspect we won’t find out.
Quote Comment
August 21st, 2008 at 11:54 pm
They probably just figured that the homeopathy itself was all the placebo their study needed.
Quote Comment
August 22nd, 2008 at 12:48 am
I, also, am no doctor.
I use “regular” pad scribblers when I must and rarely meet honest practitioners.
Look into zinc supplements for your hay fever.
Hope you feel better.
Something all “Doctors” fail to admit; nutrition has major effects regarding health.
They would rather sell patent medicines. Or, in the case of homeopaths, water.
Quote Comment
August 22nd, 2008 at 1:35 am
Ross said:
Oh I don’t believe that for a second. Everyone knows that their nutrition is a big issue in health. I think most doctors will be the first to tell their patients that the best way to stay healthy is to eat well. I also know that my father’s doctor pestered him to eat better and loose weight. (my father is a bit overweight)
I don’t think doctors have anything to fear for loosing business to people improving their diets. Most people know that eating a lot of junk food is not good for them and most people know they should eat better than they do but they have imperfect diets anyway because its there and it tastes good. I mean, I know my diet is far from perfect and I really should and can do better. I guess you can call me lazy.
Quote Comment
August 22nd, 2008 at 4:02 am
Mike said:
What does one use as a placebo for homeopathy? “Allotropic” medicine?
Ross said:
I presume you never have heard from the profession called dietiticians? There are even clinical dietiticians, and I don’t think there are many places with a higher number of physicians than hospitals.
Quote Comment
August 22nd, 2008 at 8:21 am
Now, that would be a “Specialist”.
Most HMOs require that “regular doctors” refer patients to “specialists” or for hospitalization.
Or, are you lucky enough to have socialized medicine?
Quote Comment
August 22nd, 2008 at 9:03 am
Ross said:
Firstly, I live in Belgium, so you would probably consider our system of social security to be socialized medicine.
Secondly, if regular doctors refer someone to a dietician, they acknowledge the importance of nutrition. If they didn’t know (however crudely) which effects nutrition has, they would be unable to refer someone to a dietician. Otherwise they would probably be unaware of their existence, and they certainly wouldn’t lose their time convincing patients to go to a dietician. If they were actively opposed to dieticians (possibly induced by money coming from the sales of drugs, as you seem to imply), they also wouldn’t refer their precious patients to them.
Not being a specialist in your HMO’s I’m not going to discuss their policies for dieticians.
Quote Comment
August 22nd, 2008 at 1:48 pm
My experience (mostly from dating doctors) is that they would prefer that their patients use diet, exercise and, other good health practices to solve their health problems. The reason they don’t push those solutions is that they get less than 1% compliance rate. If a doctor has a patient who has bad knees, for example, the correct solution might well be to lose 40 lbs, exercise regularly to improve the muscular support for the joint, stop smoking, and eat more vegitables. But the number of people who will actually do that is more or less zero. So the doctor just gets into the habit of recommending medicines, pain killers, and surgery. It is a pragmatic solution that mostly works.
If you ask your doctor for advice about diet and exercise, he will give it to you, and if you actually do something as bold and inovative as folowing the advice, you will give the poor guy a heart attack.
Quote Comment
August 22nd, 2008 at 2:38 pm
Chuck said:
Great point.
Quote Comment
August 22nd, 2008 at 5:15 pm
There are NHS homeopathic hospitals? God damnit! No wonder the public finances are in such a mess.
Quote Comment
August 22nd, 2008 at 10:15 pm
Josh said:
Yes, NHS recognizes and covers homeopathy and every time it is called into question the homeopathy lobby claims discrimination and that they are trying to help patients while mainstream doctors are trying to keep people bound to their medicine for money while denying them “complimentary” treatments.
They get a lot of sympathy.
It is a national disgrace, if you ask me. NHS does not necessarily treat a homeopathic hospital any different than any other kind. A long term care rehabilitation hospital, a mental health hospital, a neurological hospital, a cancer hospital or a homeopathic hospital. As far as NHS is concerned they are all equally valid varieties of care institutions.
Quote Comment
August 22nd, 2008 at 10:21 pm
RBR1978 said:
Yet you can’t get a dentist in the U.K. according to some posters on a previous thread?
Quote Comment
August 26th, 2008 at 1:37 am
Homeopathy is all about ignorance and missrepresentation. it’s said to be a concept that is traditional and thousands of years old. It’s said to be natural-based. It’s said to ‘treat the whole patient’
All of this is blatantly false and just obscures the simple facts about it and that it’s all based on diluting agents to the point of having a small chance that even one molecule remains. It’s a concept so stupid even at face value it reaks of scam and idiocy. This is why they try so hard to avoid that one core fact.
Quote Comment