The World according to DocBrain

Friday, August 29, 2008

Show me the

Elsewhere in this blog, I have argued that life is like the wheel of fortune: anything can happen, but what does happen usually depends upon the law of probability. Statistically rare events can and do occur, but only rarely. In compiling statistics, knowledge is vital. The less accurate the knowledge, the less accurate will be the wheel of fortune. Items that are rare may occur frequently because the statistical model is faulty or incomplete.

Medicine is moving towards the wheel of fortune model, often called evidence-based medicine. Studies are done, controlling for as many variables as possible and then seeing what the short term and long term outcomes are. Studies are compared and contrasted with similar studies. Diagnostic and treatment pathways are then based upon these studies.

Most studies do not show an absolute perfection of any diagnostic or treatment pathway, only the greatest good for the greatest number. So, it is possible to go through an evidence-based diagnostic pathway only to have your condition misdiagnosed and to go through an evidence-based treatment pathway only to fail treatment and/or have adverse reactions to the treatment. Most people just define this simply as "S**t happens".

Medical professionals do not argue the truth of evidence-based pathways, but do argue the applicability of these pathways in places where they were not tested and in changing times. So a procedure that was found too dangerous in 1970 may now be the best thing we have, and a treatment that was found perfect for a 50 year old may be wrong for an 80 year old.

In health care, we have lots of people, so we can do a lot of experiments (of course, with their informed consent).

There are other arenas where there are lots of people, and yet, little evidence-based outcome directed research. Politics, economics, and education come to mind.

The research in these fields does not appear to choose proper outcomes, are not done prospectively, do not run long enough to see all the fallout, and generalize too broadly from the available data. This leaves these areas wide open to opinion, speculation, partially founded beliefs, and passions to "fix things" without knowing whether the "fix" is worse than the problem.

One example: A study showed that reducing payroll taxes for workers for one year increased hours worked during that year. The evidence-based conclusion was that lowering taxes increases work. This type of generalized conclusion would be blown out of the water by any 2nd year medical student and yet was (and is) widely accepted by many economists and politicos. It generalizes into the future something that is unknown. Could people get used to the lower taxes and regress back to their old work habits? Actually, evidence in other studies does indicate that people tend to regress to their mean. What per cent would regress? What per cent would continue to work longer hours? What per cent would work less hours? With longer hours on the job, what increase would there be in work related injuries and disabilities?

Those of you in economics, politics, education, and government, you got some splaining to do! For now, all of these areas are more like art forms than true sciences. Art is what you like, not what is true.

0 Comments:

Post a Comment

<< Home