As a reader of this blog you are probably interested in maintaining a healthy weight by following an eating plan that’s appropriate for your metabolic type, exercising intelligently, and if beneficial, bolstering your efforts with a little help from nutritional supplements. You want information that is based on sound principles of scientific research for all of these things, especially when it comes to nutritional supplements.
But how can you know that a health benefit claim of a supplement manufacturer is legitimate or not? After all, no one wants to waste their hard-earned money on products that don’t work as claimed or worse, are potentially dangerous. Let’s explore the issue of evaluating supplement research and claims for scientific validity by examining a very popular and highly-touted weight loss supplement: Green coffee bean extract (GCBE).
GCBE is an extract that is made from unroasted coffee beans. Coffee beans are naturally green in color prior to the roasting process. A cup of Joe made from green coffee beans would taste terrible as you might imagine. The active ingredient in GCBE is thought to be chlorogenic acid. The product we’ll examine is standardized to contain 46% chlorogenic acid.
In The Land of Oz
The GCBE craze started when America’s favorite TV doctor, Dr. Oz, promoted GCBE on his show as the new miracle weight loss supplement.
There was only one problem: the research was heavily flawed and Dr. Oz should have at least taken the time to evaluate it before making such fantastic claims. He remained steadfast, however, in these claims even when called before a Congressional hearing on weight loss supplements.
The first question that should always be asked about any study, regardless of its claims, is who sponsored the research? This little detail is vital when it comes to sniffing out any potential bias resulting from a vested financial interest based upon the outcome of the study whether it be positive, negative or neutral.
In the 1950s with increasing evidence that cigarette smoking caused lung cancer the tobacco giant, Phillip Morris, created an entity known as the Tobacco Institute Research Committee (TIRC) to sponsor research into this supposed link. PM hired a cancer researcher who believed cancer was entirely due to genetics and not to any extraneous factors such as harmless cigarette smoke. Research was funded with this biased focus. Needless to say no relevant data came out of this research [1].
The research for GCE was sponsored by a supplement company in Austin, TX, the maker of the GCBE evaluated in the study. This doesn’t automatically mean the study is flawed scientifically. It’s just a red flag that must be evaluated along with the rest of the methodologies employed in the study. Appropriate credit should be given to any supplement manufacturer that goes through the expense and trouble of sponsoring a legitimate scientific study of their product to show efficacy and safety.
Next, we need to consider where the study was conducted. Was it conducted at an institution that has integrity and reputation for quality scientific research? The GCE was conducted at the Health Sciences Clinic in Bangalore, India. While the name Health Sciences Clinic sounds impressive, an Internet search came up empty.
What about the researchers themselves? Are they recognized leaders in this field? The paper lists three authors: Joe A Vinson, Bryan R Burnham, and Mysore V Nagendran. Typically, the first author named is the lead scientist and has the most time invested in the study followed by the second author. The last author is usually the person in whose lab the study was conducted in.
Both Vinson and Burham are located at the University of Scranton in Pennsylvania; a small private Catholic and Jesuit school. Vinson is in the chemistry department. His homepage on the University’s website is a long rambling read that has very little to with his science background. In fact, I don’t think I’ve seen anything quite as bizarre for an academic’s homepage. He provides such interesting tidbits such as to how his mother went to high school with Pretty Boy Floyd and in his first play of his burgeoning amateur acting career; he was cast as the Fireman in Eugene Ionesco’s absurdist comedy The Bald Soprano. Said fireman goes around in pink long underwear putting out fires. A noble endeavor indeed, but how this is relevant to Vinson’s qualifications as a scientist seem tangential at best. (http://www.scranton.edu/faculty/vinson/). Burham is in the psychology department. A Google search of Nagendran turns up empty other than mentions of this study.
Another consideration is where were the results published? Prestigious journals like Science and Nature would really cause one to sit up and take notice. This study was posted in the journal Diabetes, Metabolic Syndrome and Obesity: Targets and Therapy. As impressive as this may sound, it turns out that the publication is an open access journal as opposed to a subscription-only journal.
Open access journals are a product of the Internet age. They enjoy the advantage of anyone being able to view the entire article without paying a fee with the attendant disadvantage (or advantage depending on which side you’re on) of potentially undermining the peer review process, which is vital to any sound research study. Some publishers will just accept a fee from the authors without adequate peer review. In fact, a staff writer for Science critical of these journals submitted a deliberately flawed study that in many cases was accepted to these types of journals. There are a few solid open-access publishers. The publisher of this particular journal is not one of them.
The house of cards the GCBE study was based on is beginning to come into focus at this point. Let’s now have a look at the study itself. You’ll hear Dr. Oz and his ilk spouting. “In a randomized, double-blind, placebo controlled study” to get you really juiced up about all the science behind the product they’re pushing. Yes, this is important, but neither the research subjects nor the persons conducting the study should know who is getting the supplement in order to satisfy the double-blind claim. In the case of this study, that would be knowing which subjects received the GCBE or a placebo pill containing no GCBE. Problems with this particular study can be discovered early on by simply reading the Materials section of the published report.
First of all, the study involved a small number of people: 16. This is an extremely small sample size and whatever statistically significant data can be derived from it is highly questionable. The subjects were then divided into three groups and all were given a high dose of GCBE for 6 weeks, low dose GCBE for 6 weeks, and a placebo for six weeks. There was a two week period between each treatment when nothing was administered.
The only difference between these three groups is the order in which they were administered the three different treatments. A crucial control group is missing: a group that got only the placebo throughout the study. The other problem is the high dose and the low dose treatments were readily apparent to both the research subject and the researcher since only two pills were taken per day for the low dose segment and three pills for the high dose segment. All groups lost weight in the initial six weeks of the study, even in the placebo group [2].
The 8 kg or 16 lb. lost after 22 weeks of the study is deceiving, and the data would suggest it had more to do with the daily monitoring of their diet [3]. If we look at the data for the high dose GCBE, it would be a more modest 2 kg or 4 lb. of weight lost after 22 weeks.
In summary a study should be designed to:
- Have adequate controls in place to eliminate results that are due to other factors. Admittedly, controls can become unwieldy especially in human studies. In this case, it would appear the effects of the study were more due to the daily monitoring of the subjects’ diet. The missing control group of receiving only placebo throughout the study would have flushed this out.
- Be double-blinded which means neither the researcher nor the subjects should know who is getting the supplement or a placebo until the end of the experiment.
- Be randomized which means that the subjects are randomly selected for each treatment group. There is something suspicious in this study since one of the treatment groups starts the trial at a considerably lower mean weight.
- Have an adequate sample size so statistically significant results can be reported with confidence.
It turns out that initially the paper was submitted to a journal that had some integrity but they rejected it for publication. It was only then that Vinson and Burham were recruited by the supplement company to lend their scientific gravitas as well as to rework the paper. While they noticed discrepancies and asked for clarification from India, they never reviewed the raw data or questioned the changed numbers.
The FTC then got involved and demanded to see the data and found that the Indian researcher had in some cases changed the final weights and even the duration of the study. Vinson and Burham caught “green-handed” had no choice but to send a retraction to the journal. The FTC fined the supplement company 3.5 million dollars.
The FTC deserves a great deal of credit going after these fraudsters. Obviously, some reforms need to be implemented by the supplement industry to protect the consumer against false claims. The FTC has suggested requiring two independent studies be conducted before claims can be made. I would suggest some method to provide more separation between the supplement companies and the researchers. Another added protection is to force the supplement companies to submit the trial parameters to clinicaltrials.gov before the start of the study so it can’t be altered after the fact [4]. Sometimes, government regulation is a necessary evil. The alternative is to let the wolves to roam free to prey on the unsuspecting and defenseless.
It looks like the good doctor has no shame. He’s still pushing the miracle cure du jour on his TV show.