« Was Enron a Ponzi Scheme? | Main | Capital First Fund - Almost a Ponzi Scheme? »

Testimonials, Covariation Tables and Magic Pills

plous.gif Scott Plous, in 1993, wrote an excellent text or compilation of the then current state of the art on the Psychology of Decision making, and included a number of test exercises at the beginning of the text. The tricks of the mind that psychologists have discovered are very ably covered by Plous. There is a general agreement amongst scientists is that there are heuristic tools that we use to break down a decision problem, which generally work well. But we overuse these heuristics and make predictable mistakes.

Here is an example about the use of testimonials to aid our decision making. If I am going to buy a new computer, I will generally read product reviews, especially from authorities. Some of these reviews provide important information. When the "discovery" of a Mac OSX virus was a hot topic, reading authoritative reviews helped my understand what the problem was and how to gain protection until Apple released its security update. In general, relying upon authorities is a good idea and trying to identify an authority is a worthwhile exercise.

Where is this a bad strategy is when the event you are seeking information about is highly unusual, rare, or wildly out of the norm. How important are authorities in these cases? Consider the following problem. You are a member of jury in personal injury lawsuit. The plaintiff has testified that the car that hit him was yellow. Let's make the plaintiff an "authority" on spotting cars: if a car is yellow, then the plaintiff will correctly identify it as yelllow 90s% of the time. Further suppose that there are only two colors of cars: yelllow and green. Our authority is just as good as spotting yellow cars as green cars. You also know the total number of cars is 200, and 180 are green. What are the chances that if the plaintiff says that the car was yellow, it was yellow, given that he is an authority. Well, let's count the cases.

A = number of cars that are yellow & are identified as yellow. C=number of cars that are green & are identified as green.
B = number of cars that are yellow & are misidentified as green

D= number of cars that are green & are misidentified as yellow.

From the population totals, we know that A + B + C + D = 200, and that A + B = 20, while C+ D = 180. We also know that our authority doesn't make mistakes very often, so that A/A+B = .9, or A= 18 and B =2. Similarily, we know that C/C+D = .9, so C = 162 and D =18. What is the universe or total collection of cars and identifications in which our authority says that the "car was yellow". It is the information state containing A and D, a total number of 36 different states. Only in half of those states, A, does our authority get it right despite his 90% ability. This is because the chances of getting a yellow car are small, 20/200. So when our expert identifies a yellow car, he has picked a relatively rare event to be right about. As David Hume said, and I paraphrase, when presented with testimony as to a very rare event, I have to judge whether the rare event happened or that testifier has made some mistake.

What does this have to do with magic gas pills?

Technorati Tags: hot topic, authoritative reviews, heuristic tools, test exercises, mac osx, gain protection, topic reading, psychologists, new computer, predictable, compilation, state of the art, scientists, psychology, testimonials, discovery, heuristics, product reviews, scott plous

At this site there are a series of "testimonials" from happy campers about the efficacy of Bioperformance. One individual goes so far as to draw an illusory correlation between posters who have purchased the produce and their satisfaction with it versus all the other posters who have panned the product on scientific grounds. The suggestion is that if you have not tried it, then you cannot comment on the efficacy of the magic pills.


But that is just wrong. Magic pills are a rare event, not non-existent because science does produce amazing results. But when judging, we are entitled to use the rarity of magic pills as valid information. Could the testimonials, assuming that they are even honest, be wrong in some detail? A very good statement of the minimal conditions for the testimonial to be correct was stated at the Quatloos site,


Post the mileage logs -- before and after using the magic pills.


Also, include comments as to any changes in your driving habits or routes and any service records other than normal minor maintenance.

This information should cover a statistically significant period and should show a comparison of the same season for the before and after.

Anything less is subject to too many variables to attribute it to an untested product.


Technorati Tags: hot topic, authoritative reviews, heuristic tools, test exercises, mac osx, gain protection, topic reading, psychologists, new computer, predictable, compilation, state of the art, scientists, psychology, testimonials, discovery, heuristics, product reviews, scott plous

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

TrackBack

TrackBack URL for this entry:
http://www.bizop.ca/mt/mt-tb.cgi/121

Listed below are links to weblogs that reference Testimonials, Covariation Tables and Magic Pills:

» What does a 18th Century Philosopher have to Offer the 21st? from Psychology of Compliance & Due Diligence Law
It is common to believe that our generation, has a monopoly of all the wisdom that is worth acquiring. But one of the great advances... [Read More]

Google Advertisements

Why are there Adsense advertisements on Bizop.ca?

Bizop.ca is a law blog about misleading advertising regarding the sale of franchises, business opportunities and network marketing.

But the ads that are placed here are done by Google.

Google Adsense will run any ad that it thinks is appropriate based on syntax.

Some of the ads may be for dubious opportunities.

So if you think that an ad is misleading, tell us in the Discussion Forum.

Tell us why the ad is misleading.

Use your own words.

Help with Bizop.ca's mission of providing quality information by analyzing ads.

Help us help others.

Archives

How to Subscribe

Privacy Policy

Subscribing allows you to be updated with either email or RSS, automatically and without having to return to the site. You will never have concerns about privacy or spam.

Enter your email address:

Delivered by FeedBurner

feed.jpg

Recommendations

These are ads for tools or programs, which I either use daily or are deserving public ads.

Even though I would recommend these tools or programs, I may receive compensation for doing so.

No compensation is received for the public ads.

Mediators Without Borders