Cow Commenter DaveD brings to my attention the latest shenanigans from the makers of Shoo!TAG, the ridiculous superstitious trinket that is supposed to keep insect pests away from your pet (and now from humans too).

Regular readers of The Cow will be familiar with my call to Energetic Solutions (the company behind Shoo!TAG) to show us the science (that they keep boasting they have) that substantiates the efficacy of their ludicrous little piece of plastic. Well, we still haven’t seen any results from the ‘European trials’ that they have bragged about in the past, ((These supposed trials were either a lie or they produced negative results that Energetic Solutions don’t want people to see. Otherwise, why not provide the data? I’m betting that the former proposition is true.)) but it appears my niggling has made them realise that no-one will take them even remotely seriously if they can’t provide some proper scientific results to back up their claims.

Only one small problem… they don’t actually understand what science is.

The Shoo!TAG site is now making the following announcement:

Texas A&M University monitors Field Trial
74% reduction in mosquito bites with shoo!TAG™!

How many of you understood this to mean that Texas A&M University had something to do with these tests? Well, it won’t surprise you, I’m sure, when I tell you that you’re wrong. How many understood it to mean there were successful, properly run scientific trials that showed some amazing results? Wrong again.

Let’s re-word the Shoo!TAG announcement in a more factually correct manner, shall we?

Poorly-constructed Shoo!TAG™ test prompts an independent observer to suggest (on Texas A&M University letterhead) that results are statistically meaningless!

Ah, yes. The actuality doesn’t sound nearly so impressive, does it? But this scenario is nothing more than we’ve come to expect from Shoo!TAG: don’t let a bit of truth get in the way of some duplicitous self-aggrandizing! What, you think that the Shoo!TAG people couldn’t possibly be that disingenuous? Hahaha! You haven’t been paying attention!

But, for the sake of science, let’s examine the Shoo!TAG experiment and the actual conclusions of Dr Rainer Fink, the independent observer who viewed the proceedings.

The full report of the Shoo!TAG trial written by Shoo!TAG CEO Carter McCrary is here. In a few sentences from the begining of the abstract we get to this:

The Purpose of this Initial Field Test is to verify the claim that the shoo!TAG® significantly reduces the number of mosquito bites to humans when worn as instructed.

Whoopsy. Oh well, he’s not a scientist I guess, so you expect that kind of thing. Did you spot it?

‘The Purpose of this Initial Field Test is to verify the claim…‘

Uh-uh, Mr McCrary. You’ve scuppered your scientific credibility ((OK, I know I’m being generous in allowing that these people have any scientific credibility in the first place…)) in the very first paragraph of your abstract. Science is not done like that. Scientific tests are not set up to endorse something you’ve already decided to be true. If you approach science like this, you’re already demonstrating something that real scientists go a long way to avoid: bias. I don’t suppose you have the faintest clue how this works though, so I’ll forge on to some of the more egregious problems with your trial.

The ‘Methods’ section of the abstract outlines the procedural method of the test:

The study consisted of six participants who were divided into two groups.

You what? SIX people? Surely you’re not going to tell us that you’re going to attempt to do meaningful statistical science with a group of six people? ((Divided into two groups? Can three people even be considered a group?)) Oh. You are. Right. But that’s going to be really difficult for a double-blind trial… Oh, what’s this… it’s not double-blind. OK, you can still do worthwhile science with a blind trial… Oh… lookit that. It’s not a blind trial either.

Um. OK. Do you Shootaggers know anything at all about science, other than what you’ve seen on the SyFy channel?

OK, Acowlytes, let’s take a look at some of the other howlers in this escapade. I’ll synopsize a bit, but I urge you to read the pdf of the trial yourselves in order that you might see that I, at least, am not playing fast and loose with facts.

The next thing that happened is that the test subjects, 3 of whom were wearing Shoo!TAGs, and 3 of them not, were put in separate tents with mosquitoes. Bites were counted. Then there was some baffling shuffling of tags and people in and out of tents during which time mosquitoes also apparently were free to come and go.

It must be noted that a portion of the mosquitoes in the Group 3 tent escaped during the change-out or had already bitten the participants, thus the number of available mosquitoes was estimated to be only 250 during the second set of testing – the data was corrected by an estimated x2 factor to compensate.

Hang on, surely that can’t be right. I’ll read it again. An estimated number of mosquitoes flew away, an estimated number were excluded due to satiation and an estimated factor of magnitude was added in to ‘compensate’ for these estimations? If there were worse things you could do in a science experiment, its hard to imagine them. Especially in an experiment with a subject sample size of six people. ((Humans are notoriously bad at estimating. Try to imagine, if you will, estimating the number of mosquitoes flying around in a tent… could you tell the difference between oh, a hundred and two hundred? Try five hundred and six hundred?))

Quite incredibly, the Shootaggers then go on to attempt a statistical analysis of all this spurious data.

A total of 362 bites were recorded. The mean number of bites experienced by participants with the shoo!TAG® was 18, with a standard deviation of 15.87. The mean number of bites experienced by participants without the shoo!TAG® was 67.5, with a standard deviation of 22.45. There is a significant difference between the mean number of bites of subjects with the shoo!TAG® present and those without the shoo!TAG® present. The P- Value for the two-sample unpooled t-test between the means of bites is approximately 0.00538.

Let me translate that into something that makes more sense:

Numbers; more numbers; some more numbers; some fancy statistical language that sounds impressive but means nothing in this instance; completely fanciful conclusion.

Or, in one single word: bullshit. If you know anything at all about statistical data correlation, this whole exercise is one laughable step after another. The waving around of a P-Value is completely berserk in this ridiculously small sample. If the point of this experiment is to gain scientific credibility for the effectiveness of ShooTAG, it is a piece of unparalleled buffoonery.

But we all know, of course, that the point of the experiment is nothing of the sort. The real purpose behind these farcical proceedings is to fool people who know nothing of science into thinking that science has been done.

The Shoo!TAG report goes on to fluff out the abstract by adding in all kinds of equations and tables – none of which have any real meaning given the experimental protocol – and then ends with the most entertaining bit of all: two ‘references’ that are contextually irrelevant, and three attached ‘exhibits’, the first two of which are the Shoo!TAG packaging. It is to laugh. They think this is science?

The third ‘exhibit’ is the letter from the ‘independent observer’ of the experiment, one Dr Rainer Fink, and it is here that we find the real meat in the sandwich of this whole exploit. Dr Fink appears to be a bona fide scientist. According to his credentials on the letter, he is an associate professor (of what it doesn’t say) and a ‘director’ in an engineering department of Texas A&M University. Let’s give his credentials and his independent status the benefit of the doubt – he hasn’t disgraced himself in our eyes yet. His full report, in the form of a letter on Texas A&M University letterhead, is here.

First of all, I want to point out that Dr Fink details his ‘independent’ status quite clearly at the end at the end of his letter:

I have no financial interest in and have not been promised any financial interest in Energetic Solutions LLC, or in the product Shoo!TAG. I received no payment or incentive for my participation. My motivation was purely scientific.

Of course, having no financial interest in something doesn’t guarantee you don’t have some other interest – you might want to see your pals do well in their business, for instance, or you might hold unusual beliefs of your own that you’d like to see substantiated. ((I’m not suggesting at all that this is the case for Dr Fink, just making it clear that there are many kinds of motivations other than money.)) Certainly, Dr Fink’s professed motivation of science seems quite peculiar when, in his first paragraph, he makes the same partisan mistake as Mr McCrary.

The object [of the Shoo!TAG Field Test Study] was to prove [my emphasis] Shoo!TAG’s ability to repel mosquitoes from humans…

Interesting language for a scientist. Shoo!TAG’s ability to repel mosquitoes has never, ever, been scientifically established, so, as you can see, Dr Fink is already demonstrating bias. Which is, again, quite interesting given that his #2 self-reported reason for his involvement in the proceedings is given as:

2. [To] Oversee the Field Test Study to ensure it remained unbiased such that independent results are obtained.

Well, of course, as we have seen, this ‘experiment’ fairly reeks of bias from all quarters, so Dr Fink is already on the back foot.

Dr Fink outlines the experimental progress, and several episodes are detailed that don’t appear to have made it into Mr McCrary’s relating of events. This one is particularly good:

…during the time interval between exiting from Control Group 1 and entering the tent as Control Group 3, participants were asked to use their cellular phones to attempt to dissipate any remaining frequency based interferences remaining from the time they were wearing the Shoo!TAG.

Excuse me? What? Cell phones were waved around to dissipate ‘frequencies’ that were… what… hovering in the air. Or something? As an independent scientific observer, Dr Fink is starting to look like a prize idiot. He reports this as if it was an acceptable – conventional, even – scientific procedure. ((Even if you were dumb enough to buy into the sheer daftness of these concepts, representatives from Shoo!TAG have said in their own words right here on Tetherd Cow Ahead that the ‘frequencies’ that Shoo!TAG uses have nothing to do with cell phone frequencies.))

Oh boy. A loon ‘independently’ verifying, on University stationery, the antics of other loons. I’m sure you’re getting a vivid picture here.

Dr Fink goes on to relate all manner of other things, including completely unsubstantiated personal speculations such as this:

Once the participants left the tents, mosquitoes that had either escaped through the tent opening or were physically attached to the study participants aggressively attacked all the study contributors and observers with a complete lack of interest in study participants still wearing the Shoo!TAG. Leading to a possible conclusion that the Shoo!TAG caused the mosquitoes to preferentially feed on unprotected or less protected individuals in the area before biting Shoo!TAG wearers.

Such terrible subjective observations and conclusions are no better than the vapid testimonials that Shoo!TAG has trumpeted on their web site as ‘evidence’ that the daft thing works. It is most profoundly not the language you’d expect from someone calling himself a scientist.

But probably the most damning thing about Rainer Fink’s analysis of the whole affair are his conclusions. No matter how predisposed he is toward helping the Shootaggers out, he is smart enough to know that there are some things you just can’t put on the letterhead of your employer without risking your job. After some tabling of the numbers gathered in the test, he writes:

‘…… it must be noted that the size of the study conducted was insufficient to evaluate the statistical significance of the results.’

…and, again later in the report:

… the scale of the test was insufficient to establish the efficacy of Shoo!TAG performance to be supported by statistical data analysis.’

In other words, any actual data gathered from the experiment (as questionable as it is), is, in Dr Rainer Fink’s opinion, completely useless. In fact, the only outcomes that Dr Fink consider affirmative are of the spurious subjective kind that Shoo!TAG has already promoted ad nauseum as ‘scientific’.

One must question once more, in that light, the Shoo!TAG website boast of ‘74% reduction in mosquito bites with Shoo!TAG’ and their attempt to promote statistical success in their own reading of the data. They have again, as they have done many times in the past, just pulled a completely fictitious ‘fact’ from their asses and are using it to promote their product. ((This 74% figure is remarkably close to the ‘75%’ figure that Energetic Solutions has already bandied about a year or more ago. It seems to me that they had already decided this number WELL before this experiment was carried out; a number big enough to be impressive, but not so big that it rules out the odd usual flea that someone spots on their pet.))

So, to succinctly recap the some of the numerous problems with Shoo!TAG Initial Field Test:

•The test is riddled with bias: the conductors of the test expected to see positive results before they commenced the experiment. In short, they had already made up their minds about what was going to happen – the experiment was not about gathering impartial data.

•The trial was completely unblinded: experimental blinding is specifically designed to counteract bias. A lack of blinding combined with evidence of bias (as above) are strong indications of corrupt procedure and would, by themselves, get any serious experiment kicked out on its ass.

•The subject sample size was insignificant: an experimental base of six subjects is preposterous (and we don’t even know how the subjects were chosen – my bet is that they are all friends of the experimenters)

•The ‘experimenters’ made subjective assessments in numerous areas: there was no rigorous control of most vectors of the experiment. Guesses were made of variables and then taken as fact. This is a scientific dog’s breakfast.

•The ‘experimenters’ made spurious subjective data alterations: data was altered by ‘guessing’ and making unguided assumptions. Way to screw up your dataset.

•There is NO endorsement by Texas A&M University of this trial, although that is heavily implied: Texas A&M University is not shown to be endorsing this test. The test was not carried out on the premises of Texas A&M University nor with Texas A&M University supplied protocols. It certainly does not have the imprimatur of Texas A&M University. The fact is that a person who works at Texas A&M University was called on to be an observer and provided his observations on a University letterhead.

•The scientific credentials and bias of ‘impartial’ observer are questionable: Dr Rainer Fink’s statement exhibits bias and subjectivity as well as irrational thinking.

•The scientific endorsement of the trial is equivocal: The only definite conclusion made by Dr Fink is that there isn’t enough data to make any kind of meaningful sense from the results. In any other scientific situation this would mean that the test was useless and a better experiment was required.

In Shoo!TAG’s world, though, this translates as an outstanding success worthy of trumpeting on their website.

Here on Tetherd Cow Ahead, the call for Shoo!TAG to ‘show us some science’ has been frequent and firm. The Shootaggers vocally insist, at every opportunity, that their product is based on scientific principles, and is not (as I contend) just a pseudoscientific trinket that smacks of magical thinking. Critics might say that this first Shoo!TAG ‘Field Trial’ is at least an effort by Energetic Solutions to attempt to gather some scientific data on their product. I say it’s nothing more than a publicity stunt designed with the express purpose of deceiving potential customers into believing that there is some science behind Shoo!TAG when there is none.

Setting the hopeless errors of procedure aside, the mere fact that Energetic Solutions is leading people to believe that this test has the endorsement of a university, or has produced data that shows anything at all is a testament to their world view. They don’t care whether or not their product works, they just want your money.

And they’re prepared to lie to get it.