Site Index:
More
Information:
In This Section:
News and Info:
|
Top 10 projects
These are my favorites, in chronological order.
For bibliographies of these projects,
click here.
Stimulated by the BCBSA screening problem, I
wrote my PhD thesis on "A mathematical theory of intermittent
inspections". The general problem solved by the model was
as follows
 |
There is a process
that is changing continuously over time (e.g. a cancer in a
person, a crack in a dam) |
|
The are outcomes
that can occur as the processes progresses (e.g. pain,
disability, death from a cancer; breaking of a dam) |
|
The probability or
magnitude of the outcome is a function of the degree of
development of the process (e.g. the spread of the cancer, the
size of of the crack in the dam) |
|
There are tests
that can be used to detect the process |
|
The tests can be
uses in any order and at any times, not necessarily in lock step
at the same time |
|
The tests can make
mistakes. They can either indicate that the process is present
when it is not (a false positive) or they can miss a process
when it is there (a false negative) |
|
Both the false
positives and the false negatives can be a function of the
degree of development of the process (e.g. the size or amount of
calcium in a breast cancer, the size of the crack in the dam). |
|
The tests can also
be affected by random errors (e.g. the technician misreads the
test, the lab slip gets lost) |
|
The tests cost
money. The costs can vary as a function of the degree of
development of the process |
|
There are
"treatments" that can be undertaken to try to fix (e.g. stop,
repair) the process. |
|
The effectiveness
of the treatments depend on the degree of development of the
process at the time the treatments are given |
|
The treatments
cost money, and the cost is a function of the degree of
development of the process |
Although the model was motivated by the
medical screening problem, it is general and can be applied to any
problem that has any of the qualities just listed. It was picked
up by such diverse places as the FAA (cracks in airplane wings),
National Transportation Administration (to prevent railcars from
derailing), and the Department of Energy (inspection of nuclear
reactors). At that point I had to make a career decision: do I
pursue the model, which would lead me into a variety of different
fields; or do I stay with the applications of mathematical methods
in medicine? I chose the latter.
I published the theory as a book "Screening
for Cancer: Theory Analysis and Design". It won the 1980
Lanchester Prize, given by the Operations Research Society of
America and The Institute of Management Sciences for the most
important contribution to the field. That, in turn, got me an
immediate promotion to full professor at Stanford.
The model has been used by many organizations
and individuals. They include the BCBSA, American Cancer Society,
National Cancer Institute, World Health Organization, and at least
10 countries. The most recent use of it of which I am aware is the
analysis of
3
Methods to Enhance the Sensitivity of Pap Testing,
by Brown and Garber (Obstetrical
& Gynecological Survey. 54(5):305-306, May 1999).
(I stopped writing screening papers in 1990.)
Incidentally, the thesis was printed on the
world's first laser printer. It was at Xerox PARC in Palo Alto,
where I was a fellow at the time. The printer filled an entire
room.
(to top)
This is the most important application of the
screening model. In 1978 the American Cancer Society (ACS) asked
me to help them write guidelines for cancer screening. It took two
years to do the analyses and marshal the conclusions through the
ACS committees and Board. The resulting national guidelines
propelled the ideas of the three-year pap smear and the three- to
five-year sigmoidoscopic exams. It was the beachhead for what
ended up being a twenty year debate on mammography in women under
age 50. It was the paper that shut down screening for lung cancer
and bladder cancer. The report I wrote also laid out the
principles for designing guidelines, including the ideas of
evidence-based medicine. From the first page:
|
"In making these recommendations, the
Society has four main concerns: first, there must be good
evidence that each test or procedure recommended is medically
effective in reducing morbidity or mortality; second, the
medical benefits must outweigh the risks; third, the cost of
each test or procedure must be reasonable compared to its
expected benefits; and finally, the recommended actions must be
practical and reasonable." |
The guidelines made the front page of the NY
Times. I presented it as the keynote address at the second annual
meeting of the Society of Medical Decision Making. In the two
years following the Report I gave at least 150 speeches on cancer
screening. The ACS commissioned a Lieberman poll of physicians to
determine how many changed their behavior on the basis of the
report; more than half said they did. Cancer screening became a
tar baby for me. In 1990, I finally decided that I would
categorically refuse all future invitations for speeches, papers
and committees on cancer screening.
(to top)
In about 1984, Sue Gleeson of the Blue Cross Blue Shield
Association asked me if I would be the chief scientist for the
Association's Medical Advisory Panel (MAP). At that time, the
members were all plan medical directors. I was a great opportunity
to put into practice some ideas about the assessment of new tests
and treatments, in a very real setting. One of the first things I
did was to argue that we needed to write down specific criteria
that we would use -- for accuracy, consistency, accountability and
defensibility. Naomi Aronson was the staff person for that effort.
I believe that those criteria are the first occasion on which he
principles of evidence-based medicine were institutionalized --
made a formal part of the process by which every technology would
be assessed. Those criteria have been used as the role model for
many other organizations. The MAP itself has expanded greatly;
currently about three-fourths of the members are from outside the
Plans. In about 1992 Kaiser Permanente was made a partner in the
TEC program.
(to top)
In 1982 I published an article in the New England Journal in
which I identified the role of guidelines in medicine. After
reading the article, Richard Wilbur MD, the Executive Director of
the Council on Medical Specialty Societies (CMSS) asked me to work
with the CMSS and its members to develop guidelines programs. That
began a long period of work developing the ideas of guidelines and
putting them into practice. I wrote a book for CMSS on how
to design guidelines, intended to be a manual on the subject for
specialty societies. I worked with at least a dozen specific
societies to help them put together programs, most of which still
exist. I taught a four week workshop on how to develop guidelines,
that was attended by 22 specialties as well as representatives of
other organizations (e.g. insurance companies, HMOs, government
agencies).
(to top)
In the mid 1980's I was invited by the Jan
Stjernsward, Director of the cancer unit of the World Health Organization (WHO),
to help developing countries There were more than a dozen trips to
countries such as India, Sri Lanka and Chile. And there was an
impact. For exam[ple the government of India rewrote its five-year
plan for cancer control on the basis of my analysis. (They had
about $1 per person to spend on cancer control.)
Along the way, Jan asked me
to build a model that could be used by developing countries to
design cancer control programs. The requirements were that the
model had to be able to cover all types of cancer, cover all types
of interventions, but fit on a single page and be doable with a
pencil. I thought it was impossible. The idea of how to do it came
to me during a mild drunk at my favorite restaurant in Geneva (Le
Cigne). I still have the napkin. I used a Markov structure, and it worked. The model has
been used in at least ten countries that I am aware of. (to top)
I was also on the steering committee, and
chaired the screening committee for the Year-2000 project.
But the really fun part began when Ed Sondik, who was then the
director of the division that did the analytical work for the US
National Cancer Institute (NCI) saw the WHO model and invited me
to take all the things I had to chop off of it in order to get
down to a page and a pencil. The charge to me was to build a
larger model that could be used to set the US national cancer
control priorities for the year 2000. Like the WHO model it had to
be able to cover
all types of cancer, cover all types of interventions
(but it did not have to be on a single page and I could use a
computer). Called CAN*TROL, the model did end up
being the basis for the year 2000 goals. Later, the NCI hired
professionals to reprogram it (I wrote it in APL) and build a more
user friendly interface. It is still available over the web and, I
am told, is used by communities and states to develop cancer
control programs.
(to top)
A major issue in guidelines and
technology assessment is to synthesize a body of evidence to draw
a conclusion. By the time I came on the problem in around 1984,
there here had been a few efforts to devise ways to combine
evidence, much of it from the social sciences ("effect size"), and
the term "meta-analysis" had been coined. But the existing methods
fell far short of what was needed to do realistic work for
coverage policies and guidelines. The main problem was that they
only worked when the body of evidence was very well behaved (e.g.
all the trials have similar designs, and involved the same
population, treatments, outcomes, and follow-up times, and had no
biases). With some help from Vic Hasselblad and Ross Shachter I
developed a general method for synthesizing evidence from a wide
variety of sources, for a wide variety of designs, and including a
wide variety of biases. Specifically, it can synthesize evidence
that involves
|
Multiple pieces of
evidence |
|
Different experimental
designs (e.g. clinical series, randomized controlled trials,
regression equations, anecdotes, etc.) |
|
Different types of
outcomes (e.g. dichotomous, continuous, count, categorical) |
|
Different measures of
effect (e.g. absolute differences, percent increases and percent
decreases, odds ratios, relative risks, etc.) |
|
Biases to internal
validity (e.g. dilution and contamination, patient selection bias,
errors in measurement of outcomes, errors in ascertainment of the
treatment, etc.) |
|
Biases to comparability
and external validity (e.g. the people or the treatments in the
trial is different than the people or treatments in which you are
interested) |
|
Indirect evidence (e.g.
one body of evidence connects a treatment to an intermediate
outcome, while another body of evidence connects the intermediate
outcome to the health outcome of interest) |
|
Mixed comparisons (e.g.
you have a trial that compares treatment A to B, another body that
compares A to C, and you wan tot know how A compares to C) |
|
Gaps in the evidence,
leaving you with no option but to include subjective judgment or
prior beliefs for some of the pieces) |
|
Uncertainty -- about
every possible piece of the problem |
The Confidence Profile Method can take all that and grind out
joint probability density functions for all of the outcomes of
interest. It' really very powerful. I felt as though I was on fire
when the main ideas were coming out. Vic's role was to ensure that
the method was anchored to classical statistics. Ross integrated
it with Influence diagrams, and developed a general solution to
the optimization problem. I wrote a book on it that was published
as part of a prestigious series and got good publicity. It won an
international prize (from the international Society of Health
Technology Assessment). In a review of the book, Fred Mosteller
(often considered the "Dean" of medical statistics) call it a
"monument". A paper first-authored by Vic won a prize from the
Environmental Protection Agency. Vic and I wrote software so that
non-mathematical people could use it.
Unfortunately, it didn't take off. The fact that the software
was written in DOS just as Windows was coming out hurt a lot. But
I also suspect that the math was just over the heads of the
audience I was developing it for -- doctors who like to do
evidence reviews and technology assessments. Indeed, there are
sections that I, the author, can not read today. Oh well, you
can't win 'em all.
In 1990, after I resigned my chair at Duke, I became a Senior Advisor to Kaiser Permanente Southern
California. There I began to work with practicing clinicians and
administrators on information systems, guidelines, technology assessment, and
applications of mathematical methods to clinical problems. These
problems involved a much higher level of clinical
and administrative detail, and much more realism than the national
and international-level work I had been doing up to that point.
Furthermore, although I had done a considerable amount of work
with specialty societies on guidelines, and with insurers on
technology assessment, the down-to-earth nature of my work with
Kaiser Permanente added a new and very important perspective. Many
of the ideas that worked their way into my series of
essays for JAMA, and virtually all of
the motivation and design criteria for the
Archimedes modeling project came from this work. A few
projects are especially memorable
|
The initial work on the
clinical information system in the Southern California region
|
|
The ionic/nonionic
contrast agent guideline
|
|
The osteoporosis
guideline
|
|
The cholesterol
guideline
|
|
The interregional BRCA
(breast cancer gene) guideline
|
|
Breast cancer
screening, especially women under age 50
|
|
Cervical cancer
screening, especially the role of HPV testing
|
Unlike most of my other projects, where someone
asked me to help with a problem, this project is the result of an
idea I took to George Lundberg, who was the Editor of JAMA through
the 1990's. The idea was to write a series of essays that would
provide a unified theory and practical recommendations for
responding to the changing environment in which medicine is
practiced. The essays addressed such issues as
|
physician uncertainty
|
|
guidelines
|
|
patient preferences
|
|
outcomes
|
|
evidence-based medicine
|
|
the cost problem
|
|
the balance between
quality and cost
|
|
the conflict between the
individual and society
|
|
physician
responsibilities
|
|
rationing
|
|
medical necessity.
|
I was extremely fortunate
to be given this opportunity and to have George Lundberg as an
editor.
I have been fortunate to have been able to work
with the National Committee on Quality Assurance (NCQA). I was on
its Board of Directors and at one time or another served on
several other committees (e.g. strategic planning). But the most
fun and the most far reaching was the work on the Committee on
Performance Measurement (CPM), which is responsible for designing
and maintaining the HEDIS measures. In the early years, I served
as the Chief of the Methodology subcommittee. It was during that
time that the CPM laid out its principles or checklist for the
criteria that measures should meet. The CPM itself has
representation from many different constituencies, such as
physicians, health plans, and the public, as well as people with
methodological backgrounds. The measures themselves almost always
represented a compromise between these different perspectives. But
the level of intelligence, motivation, and integrity was always
extremely high.
This is the best of all. It came out of the work
for Kaiser Permanente, and they have provided the funding for it.
It is done hand-in-hand with Len Schlessinger. It gets a
website all to itself.
|
The Institute of Medicine's (IOM) committee on
guidelines
|
|
The IOM's committee on technology assessment
|
|
Achieving Medicare coverage for screening tests
|
|
The National Osteoporosis Foundation's guideline
on osteoporosis
|
|
High dose chemotherapy and bone marrow transplant
for breast cancer
|
|
White House Fellow working on the 1993 Clinton
healthcare reform plan
|
Biography • Top 10 papers • Top 10 projects • Top disappointments • Markov models
|
|
