Dr. Angell, in a previous session, made the distinction between the scientific principles underlying the methodology and scientific methodologies.  I think this is very important.  We can say with confidence that we understand the underlying principles, but I don't think we can say that there aren't new methods out there that we need to explore.

Unfortunately, the authors rarely do provide us with these innovative methods, and this brings me to my recommendations to the Commission.  There is a need for more academic departments and research positions, at post-doctoral level, to attract the brightest and best into the field.

There is also a need for more research money for practitioner training schools, as they are the ones with the knowledge and insights needed to develop appropriate methodologies.  There is a need to expand the range of acceptable models of quality clinical research, beyond the formulaic, large clinical trial protocols, while retaining integrity and authority, and adhering to those principles previously mentioned.

There is also a particular need for sponsorship from independent funding sources that will encourage innovative and imaginative but rigorous research.

I was speaking to a colleague a couple of days ago.  He is from Vienna.  Just as an example of what I am talking about, there are a lot of interesting groups in Europe who are doing all sorts of experimental science and trials of different methodologies.  This one group was testing out a diagnostic treatment device, rather like a vega device for testing of allergies.

You plug this machine in, and you put in various substances to which people might be allergic, and then apply part of this diagnostic technique to the patients.  Then one of them said, hey, what happens if you unplug it?  So they unplugged it, and it still worked.  Then the next question, of course, was, well, how far away does it have to be and still work, and they found that it could be as far as five feet away and it still worked.

Well, what I am saying here is, you do need people to ask these simple but dramatic questions that raise all sorts of other interesting issues.

DR. GORDON:  Time is up.

MS. WOOTTON:  Yes.  I have left the statistics.  I don't know whether you want me to say those very, very quickly.

When the journal changed from quarterly to bi-monthly at the beginning of '99, more libraries subscribed.  Currently, 60 percent of our subscriptions are medical or biomedical school libraries, and libraries in research institutions.  The other 40 percent of individual subscribers are equally divided between clinicians or practitioners, researchers, and others.  Thank you.

DR. GORDON:  Thank you.

Phil Fontanarosa.          Presenter: Phil B. Fontanarosa, M.D.

DR. FONTANAROSA:  Dr. Gordon, distinguished members of the White House Commission, thank you very much for the opportunity to participate in this meeting and to provide information on the topic of publication of complementary and alternative medicine, CAM, research results and peer-reviewed medical journals.

My remarks are intended to address several of the questions you provided by discussing, briefly, three questions relevant to peer-reviewed general medical journals, and from the perspective of my role as executive deputy editor of JAMA.

First: Why should medical journals consider publishing research on complementary and alternative medicine?

Well, as you all well know, all too well, of course, the use of complementary and alternative medicine and visits to alternative medicine practitioners have increased dramatically, along with marked increases in expenditures for these therapies and increasingly prevalent third party reimbursement for these therapies.

At academic medical centers, there has been increased interest among medical students, residents, and faculty members in gaining knowledge about alternative therapies.  Now, educational opportunities for CAM have been added to the curriculum in many medical schools.

In addition, there have been significant increases in federal funding for national academic centers dedicated to CAM research, along with increased funding from government and from industry to examine CAM therapies.

Yet, despite the increasing interest in CAM, high-quality scientific evidence that clearly establishes the safety and efficacy, or a lack thereof, for many of the widely used CAM interventions is lacking.  Consequently, physicians have traditionally viewed CAM in general, and most practices contained therein, with skepticism and a degree of mistrust, I would say.  Nonetheless, many patients use CAM therapies along with their conventional medical therapies.

I believe that physicians should be the health professionals who are aware of and responsible for coordinating all medical care for their patients.  Accordingly, physicians require reliable information on CAM to understand the complex interplay of CAM therapies and conventional therapies to be prepared to serve as resources for their patients, and to be comfortable answering their patients' questions about CAM, and most important, to provide evidence-based guidance and advice about various CAM therapies.

Ideally, this information should be based on critically evaluated methodologically rigorous and scientifically valid articles published in reputable journals, rather than on anecdotal reports, unfounded opinion, or unproven theories regarding CAM therapies.

To be relevant to readers, medical journals must respond to trends in medical science, react to trends among the public regarding medical therapies, and also must serve the needs of their readers.  In a 1997 survey, physician readers of JAMA ranked alternative medicine as the seventh most important topic of 70 topics we presented them with for the journal to address.  The Editorial Board and editorial staff, after much debate and deliberation, ranked alternative medicine in the top three subjects to address for that year.

Thus, JAMA and the nine Archives journals published by the AMA, The Archives of Internal Medicine, Archives of Pediatrics and Adolescent Medicine, and so on, planned and developed an initiative to objectively assess various CAM therapies.  We published coordinated theme issues, largely devoted to CAM topics, November of 1998.

Through this initiative, we provided more than 80 articles on CAM topics to the peer-reviewed biomedical literature.  I was responsible for planning, coordinating, and editing this issue of JAMA that we published on complementary and alternative medicine, and also helped to coordinate efforts with the Archives journals.

The second issue: How do peer-reviewed journals evaluate research on complementary and alternative medicine?

I believe peer-reviewed journals should be willing to consider papers on virtually any topic relevant to the practice of medicine, medical research, and relevant to their physician readers.  Most research papers on CAM should not be rejected out of hand, simply based on the topic, but should receive an unbiased, objective editorial evaluation based on assessment of the following: the scientific importance of the study; the methodologic rigor with which the research is conducted; the novel contribution to the existing literature; and the relevance of the study results for the journal's readers.

Evaluation of the scientific qualities should involve the same rigorous and critical appraisal used to assess studies reporting research on conventional therapies and the accepted framework of scientific evidence should and must prevail.

Some key aspects in the evaluation of CAM research studies include assessment of whether the paper presents the following: and explicit, focused research question; scientific or biologic plausibility; defined target disease or condition; established and accepted rigorous research methods, appropriate controls, for example; measurable, objectively assessed endpoints, such as by blinded assessment; meaningful patient-centered outcomes; information on adverse effects and safety data; and appropriate discussion of the clinical context, public health importance, and study limitations.

The assessment of CAM research studies frequently involves peer review.  Peer review of these studies involves selection of appropriate peer reviewers who have expertise in the research methods and subject matter of the study.  The goal is to obtain an objective and unbiased evaluation of the research.

Reviewers are identified and selected in several ways, but most commonly from a large database of reviewers we have at JAMA.  This database includes self-designated areas of expertise of the reviewers, along with editors' ratings of the quality of the review.  We also frequently perform literature searches for individuals who have expertise with the research topic under evaluation.

These peer reviewers serve as important consultants.  They provide the editors with information regarding the context and the contribution of the study, in light of existing research, as well as a critical appraisal of the research methods and statistical analyses.

Peer reviewers also provide recommendations about the suitability of the study for publication in the journal, and constructive, objective suggestions to improve the research report.

Comments of the peer reviewers are considered very carefully in editorial decision-making about all research papers, including CAM research.  Physician reviewers are now available for most topics evaluated in CAM research.

In general, decisions to publish CAM studies in a general medical journal, like JAMA, for instance, involve assessment of the quality of the research, the clinical applicability and importance of the research, and the generalizability of the research findings.

An important consideration is the assessment of the merits of the study compared with the information provided in the many other papers submitted for consideration for publication.  As you might expect, in large-circulation, general medical journals, competition for space is extremely keen, particularly keen, with a large number of submitted papers that span the entire gamut of medical research, clinical care, and medical and surgical specialties.

In 2000, the overall acceptance rate for all manuscripts submitted to JAMA was about 10 percent, and for unsolicited manuscripts, the acceptance rate was about 8 percent.  With this competition for space, reports of large-scale, high-quality, multi-center, randomized trials, our highest level of evidence, on clinically important topics, are more likely to receive priority for publication than are single-center studies reporting intermediate or preliminary results about specialized CAM therapies.

In fact, during the past year, of the several studies that we published in JAMA on CAM therapies, most have been randomized trials.

The third question I would like to address, the final, is: What are some of the policy issues regarding publication of CAM research in peer-reviewed journals?

In general, I would support Dr. Campion's policy recommendations, and in addition, would offer several others regarding publication of CAM research in peer-reviewed journals.

Carefully conducted, high-quality, credible research by established biomedical investigators is necessary to evaluate the efficacy and safety of complementary and alternative medicine therapies.  However, until convincing evidence is available that demonstrates safety, efficacy, and effectiveness of these interventions, the uncritical acceptance and widespread application must stop.

With the recent increases in research funding, the establishment of national centers for CAM research, future publication of high-quality studies on CAM therapies should help to establish the evidence base that supports or refutes the safety and efficacy of these many therapies.

Complementary and alternative medicine therapies that are shown to cause, or that are demonstrated to have no beneficial effect should be abandoned.  Decisions regarding the use of and reimbursement for CAM therapies should be based on published evidence and proper cost effectiveness analyses, rather than tradition, anecdotal reports, testimonials, consumer interest, market demand, competition, or political pressures.

Answering fundamental questions about CAM therapies requires critical and objective assessment, using accepted standards for scientific investigation and rigorous evaluation of scientific evidence.  Peer-reviewed medical journals provide one important mechanism for such rigorous evaluation of CAM research, and for providing physicians with reliable information about these diverse and complex therapies.

Dr. Gordon, members of the Commission, thank you very much for the opportunity to present these perspectives on publication of CAM research in peer-reviewed medical journals.  I hope this information is useful in your deliberations on complementary and alternative medicine research policy.  Thank you very much.

DR. GORDON:  Thank you very much.

David Riley.              Presenter: David Riley, M.D.

DR. RILEY:  First, I would like the thank the Commission, Dr. Gordon and the other members of the Commission, for inviting me here.

My name is David Riley.  I am a physician, board-certified in internal medicine and the editor-in-chief of the Journal Alternative Therapies in Health and Medicine.  We began publication in 1995, and were indexed in 1996, and are in the National Library of Medicine.

We view ourselves as a forum for the development of scientific information about the use of alternative therapies and their role in preventing and treating disease, healing illness, and promoting health.  We don't endorse any particular methodology, but we promote the rigorous evaluation of all therapies.

We publish a variety of disciplined inquiry methods, focusing on high-quality scientific research, including randomized, controlled trials, outcome studies, and case reports.

So you might ask the question, why do we need a journal focusing on CAM, and also, is a journal focusing on CAM able to be scientific.  I think a recent letter to "Science" illustrates many of these questions.  It was entitled, "Where Is The Next Einstein?"

It was submitted with a score of signatures from respected academics, including Nobel laureate, Sir Harry Kroto, who is a chemist in England at Sussex, and Dudley Herschbach, a chemist at Harvard, stating that, "All too often the academic research environment favors objectives selected by consensus.  Pioneers and consensus can be very poor bedfellows.  The vogue for expensive, collaborative projects leave little space or money for alternative thinkers.  Thinkers whose research may appear initially irrelevant, but who may come up with stunning work."

I might also add that this letter was rejected to the editors of "Science".  We feel that our journal tries to address this need, and provide a home for some promising and innovative research.

Our journal is by paid subscription only, and the readership consists, primarily, of conventionally licensed health care providers, 60 percent physicians; the balance made up of nurses, other licensed health care providers, naturopaths.  It is interesting that the readership is primarily conventional physicians.

I have some statistics here that I want to go over, briefly, and then I will talk a little bit about the complexity of reviewing articles in a peer-reviewed scientific journal devoted to CAM.

Eighty-seven percent of the original research articles are rejected, 79 percent of brief reports are rejected, 73 percent of hypothesis articles, which is a new section, are rejected -- those statistics could change -- and 64 percent of review articles are rejected.

We have taken an approach that we need a variety of area in the journal to address different types of research.  So we have an Original Article section, which is devoted, primarily, to randomized, controlled trials, although I would say, frankly, that we generally will not get large, randomized, multi-center trials submitted to the journal.  We have, on occasion, and we have one in this issue.  More commonly, we will get smaller studies.

We also have brief reports.  We also have review articles, sometimes solicited.  We also have some other sections, too, that I think are critical.  We have case reports, which are beginning to lay the groundwork for future research that might be done, and I think it is important to have these areas there.

We also have introduced a new section that will start in July on research letters, because, frankly, one of our roles is to educate people who are doing research, and to work with authors so that they can begin to learn how to communicate, because if we are going to communicate with the medical establishment, one of the principles is that you have the communicate in their language, and their language is primarily the peer-reviewed biomedical publication.

So a medical journal that is going to focus on complementary and alternative medicine, by definition, is not designed just for the specialists in a given field.  They have their own forums for communication amongst themselves.  Our goal is to try to communicate the results in complementary and alternative medicine with the biomedical establishment.

I want to talk a little bit about the difficulty of reviewing articles in complementary and alternative medicine, because they can be considerable.  If you look at a conventional medical journal, particularly a specialty journal, there may be a lot of collegiality; everybody knows everybody.  The research areas are fairly well defined.  The unknowns are known.  Frequently, the question becomes one of statistics and methods.

Certainly, in our field, statistics and methods are important.  All articles that are submitted go through a methodological review.  One of our associate editors is a methodologist, and we have several statisticians and methodologists on our editorial board that we turn to regularly, probably too regularly.  That is just the way it is.

We talked a lot about acupuncture research here today.  Let me take that as an example.  If someone submits an article on acupuncture, you need to know, is the author coming from a five-elements school; do they do traditional Chinese medicine; is this a medical acupuncture community; are they doing Korean hand acupuncture.  You name it, there are a of different areas that you need to know.

So not only do you need to find experts in methodology in the given area, but you also have to find an expert in that given therapy.  So you have to pay particular attention.  We have run into issues where, all of a sudden, we realized these two groups that are actually subgroups of a therapy, may not agree with other people about how these things are practiced.

So it can be quite challenging.  Most articles that come in, the original research articles, go through at least revisions, which means there is a review at the end of each revision, because they can be quite complex.  There may be as many as four or five, and I have had even six reviewers on some articles.  Consequently, one of the problems with that is it can delay publication significantly, because each review process, you kind of start anew each time.

So publishing a biomedical journal that is peer-reviewed and indexed, and focusing on complementary medicine is quite challenging, but I also think it is quite necessary, because we need to begin to lay the groundwork for how to communicate with the biomedical establishment.  I also think we need to have a way to begin to publish other kinds of studies.

The conventional medical establishment is much more mature than the complementary and alternative medicine community.  The biomedical model is known.  It is taken for granted.  It is taught in all medical schools.  In complementary and alternative medicine, there are multiple models present.

Just as the randomized, controlled trial is the current best tool we have for investigating a particular type of decision, it is implemented against a background of information, knowledge, and understanding that may not be quite as widespread in complementary and alternative medicine.

I would use the analogy, it is similar to looking at a mural on a wall of building, if you would imagine, with a telescope.  You might get very finely developed information about a very small part of that mural, but you won't understand the mural.

Many of the randomized, controlled trials that are being done, it is difficult to evaluate the results, positive or negative, because you may not actually be looking at an effective evaluation of that treatment.

And so, I think this is a really challenging time and opportunity to begin to develop an evidence profile for complementary and alternative medicine, and to begin to stimulate the next generation of research.  Thank you.

DR. GORDON:  Thank you, David.

Christine Laine.            Presenter: Christine Laine, M.D.

DR. LAINE:  I am Christine Laine and I am the Senior Deputy Editor of Annals of Internal Medicine.  I thank the Commission for the opportunity to present Annals' perspective at this meeting.

A little bit of background about Annals.  Annals is a peer-reviewed biomedical journal that has been published by the American College of Physicians since 1927.  Our circulation is over 100,000, making us the third largest U.S. general medical journal, and the largest medical subspecialty journal, since our focus is on internal medicine.

Our readers are largely internists, but not all.  Some may practice integrative medicine or prescribe CAM therapies themselves, but almost all of them who are clinicians probably take care of patients who seek CAM therapies as well as conventional medical therapies.

Our journal has a high impact factor.  We publish original research reviews, editorials, perspectives, and guidelines.  Aware of the prevalent and increasing use of CAM by people who also seek care from internists, Annals believes strongly that high-quality research and discourse concerning the effectiveness and the safety of commonly used CAM therapies is of great relevance to our readership.

While the number of papers relating to CAM submitted to Annals is increasing, the number remains quite small.  In preparation for this conference, I looked over the information on submitted manuscripts from January 1999 through April of 2001.  Of the, roughly, 6,000 papers submitted to our journal for consideration during that period, only 35 papers related to CAM therapies.

Consequently, we have been developing a series of papers on CAM in close collaboration with two guest editors who are expert in the area.  This series will include somewhere between 15 and 20 papers, and will include a combination of original health services research, critical reviews of the literature, and commentary on a variety of CAM-related issues.

The papers submitted for this series will be subject to Annals usual review process.  Unfortunately, the series will not include original research on specific CAM therapies.  We do anticipate the publication of the series will send out a signal that Annals is interested in the topic and may stimulate investigators in the area to send their original research reports to Annals.

When CAM papers are submitted to Annals, they are subject to our usual rigorous review process.  Unfortunately, many of the papers concerning CAM have important methodological weaknesses, which doesn't make them all that different than many of the papers that concern conventional therapies.

We try to get reviewers that have expertise in alternative medicine, as well as reviewers that have expertise in the particular clinical condition under study.  We have an electronic database of about 10,000 reviewers, but fewer than 100 of the people in that database admit to having expertise in CAM.  So it is a challenge to get reviewers with this expertise.

Let me tell you a little bit about what happened to the 35 papers on CAM submitted to Annals between January '99 and April of 2001.  Two were ultimately published as full articles; one is the letter to the editor; 31 were rejected; and one is in my briefcase to be discussed at our Manuscript Conference tomorrow.

This represents an 8 percent acceptance rate, which is not all that different from Annals' general acceptance rate, which hovers somewhere between 10 and 14 percent, depending on the year.  The acceptance rate for the papers that have been invited to the series is somewhat higher.  We have a special commitment to working with authors through many more rounds of revisions of those papers.  Seventeen have been submitted; nine are accepted; three have been reinvited after substantial revision; three are currently under review; and two have been rejected.

Over recent decades, journals, like the larger medical community, have become increasingly aware of the importance of evidence-based medicine, the growth of large, clinical trials, advances in statistical and analytic approaches, consensus guidelines such as the consolidated standards of reporting trials, and consortia such as the Cochrane Collaboration, are among the many factors contributing to the greater general awareness of the importance of methodological rigor in studies of health care interventions.

In previous times, the medical community gave more weight to anecdotal evidence from case reports, case series, and uncontrolled trials.  In current times, these forms of investigation are considered hypothesis-generating at best.

Unfortunately, much of the CAM-related literature relies on methods that lack the rigor conventional medical journals expect.  This may be in part, and as we have heard today, because CAM practitioners generally do not have the training and education in the experimental process that has become part of the conventional medical education.

Many of the studies of CAM that authors submit to our journal for consideration have important weaknesses such being underpowered, uncontrolled, conducted in non-representative populations, or having inadequate adjustment for potential confounders.

The manuscript that is in my briefcase is one of these.  It is an RCT of a common CAM therapy for a common medical condition.  It is funded by the NIH, HRSA, and industry.  My prognosis is that it is unlikely to make it into publication in Annals, despite having all the trappings that elevate it.  It has federal funding and it is a randomized, controlled trial.

Part of the problems are that they used a preparation that differs substantially from the ones that are in common use in the community.  They studied an atypical study population that doesn't, probably, represent the larger community that uses this therapy; there is a lack of objective outcome measures; and they have marginal statistical power.

Now, the study is reported as negative, but when you look at it closely, it is not truly negative; it is inconclusive.  I think journals take the wrap for being unwilling to publish negative trials.  We have been very aware at Annals, and have one of our statistical editors who has a particular interest in proving that there is no relationship between two things.

So we very carefully look at negative studies.  Most of them are inconclusive, not negative.  Our feeling is, to do a study that is unlikely to give you a conclusive result is, at best, a waste of resources, and, at worst, unethical.

Our suggestions are that we move beyond anecdote and observation to higher levels of evidence; that there is a particular focus on safety; another focus on the interactions between CAM therapies and conventional therapies; and that the CAM therapies be subjected to study with the rigor that the public has come to expect for conventional therapies, realizing that innovative study design and methodologies may be necessary, but that rigor can still exist.  Thank you.

DR. GORDON:  Thank you very much.

Arnold Relman.             Presenter: Arnold Relman, M.D.

DR. RELMAN:  Dr. Gordon and members of the Commission, I cannot speak for the current policies of the New England Journal of Medicine because I retired from the post of editor-in-chief nearly 10 years ago, after serving in that capacity for 14 years.

However, I was familiar with NEJM's policies under its next two editors until the present editor assumed his post, a little less than a year ago.  Given those limitations, I can state confidently that from July 1977, when I first became editor, until July 2000, when the current editor took over, the New England Journal treated manuscripts dealing with CAM in exactly the same manner it treated those concerned with more conventional therapies, and that is the way it should be.

You have just heard Dr. Campion talk about the philosophy of the current editorial leadership, and it appears to be unchanged.  You also heard a very forceful and very articulate statements Drs. Fontanarosa and Laine, speaking for JAMA and Annals, supporting that idea.

Now, I emphasize this because one commonly hears, particularly if you are an editor and you go around to meetings and so on, you commonly hear the complaint that your journal, the New England Journal, or some other leading peer-reviewed journal, is biased against manuscripts dealing with CAM.

Now, all rejected authors tend to have a certain amount of paranoia about their manuscripts and the reasons that they were rejected.  That is part of life.  If you are an experienced editor, as I was for decades, you expect this kind of paranoia and you know how to deal with it, and that is by being absolutely sure you and your staff apply the same standards to all would-be authors and all submissions, including those from your mother, your brother, or from eminent and powerful people in your field who may affect your future in medicine.

If you don't do that, if you apply different standards, you are in deep trouble.  The first advice I have always given to new, young editors who used to come to me and ask me what philosophy do I want to impart to them, is, one, always tell the truth because you can't remember if you lie, and you will be caught sooner or later.  Always tell the truth, and always apply the same standards for the same reason.  You can't live with yourself, and your journal can't survive without that.

So I want to lay to rest the idea that CAM manuscripts, just because they are about alternative medicine modalities, are treated somehow differently.  As far as I know, with the best peer-reviewed journals, and you have heard a few representatives, they are treated the same.

Now, I don't know how many CAM manuscripts we at the New England Journal received during those 23 years that I know something about.  We kept no special file, but there was certainly only a handful of such submissions, and you hear now that there are still only a small number.  In any case, they were handled just like all the other submissions and judged purely on their scientific merits and their suitability for our kind of journal and our readership.

Now, that means that, yes, each editor and each editorial staff has to decide what its readership will be interested in and what will be useful and will determine the tone and the quality of that particular journal.

But given the mounting interest and the growing discussions about its credibility, an editor would be very ill advised to do anything but give CAM submissions at least as much consideration as the others.  I have to tell you that, and I freely admitted my bias, I was looking for good CAM manuscripts to publish.  We got very few, and very few of them met our publishing standards, but I tried very hard to find the best that I could find.  The fact that the journal only published a few, primarily reflects the paucity of really good submissions.

Now, I do not have any statistical data on the relative success rates of CAM papers versus other papers, but you heard some data from Annals.  I just don't know whether it is true of the New England Journal or not, but certainly it was not any conscious editorial policy.

In any case, I think the issue is a fairly straightforward one.  I don't think you need to be a rocket scientist to come to these conclusions.  If CAM research is to have any credibility, it will have to be submitted to the same range of peer-reviewed journals as the more conventional manuscripts dealing with similar illnesses, and it will have to pass the same kind of peer review.  There cannot and there should not be two standards of science.  You have heard that before, nor should there be two standards of peer review.

Unless CAM clinical investigators want to stay outside the mainstream, they must play by the same scientific rules of evidence and publish in the same journals.

So my advice to you is the following:

(1) Emphasize to your constituents and your stakeholders -- that seems to be a popular word.  I don't know what that means, but you know what it means -- your stakeholders, emphasize to them that the time has passed to make arguments based on unsubstantiated claims and undocumented anecdotes; finished, enough of that, enough of the books that claim miraculous cures with no documentation, enough of all the statements in public meetings about marvelous results from a particular case without documentation.

I do not say that anecdotes are useless, and I do not say that anecdotes are not used in ordinary, conventional medical literature; they have been and they should be, but the anecdotes must be documented.  They must be documented.

(2) Encourage the submission of CAM manuscripts to the most rigorously peer-reviewed journals over a wide spectrum, not just to CAM journals but to all journals.  Journals, after all, are not ultimate arbiters of ultimate truth, neither the New England Journal, nor JAMA, nor any other journal of which I am aware, was given by God insight into what ultimate truth is and has the power and has the credibility to say that what we publish is, for sure, true, and what we reject is, for sure, wrong.  Not so.

Journals do the best they can, as Ed Campion said, they do the best they can.  They try to sift out bad evidence from better evidence, and they try to publish the best evidence on the most interesting and the most relevant and important questions for their readership.  That opens the door for lots of journals, but they have to be rigorous.  They have to deal with evidence, quantitative, where possible, objective evidence and documented data.

Finally, my advice to you is to remember that quality, not number of publications, is the important thing.  You don't understand the quality until you read the whole paper, including the fine print, and the methods, and the statistics, and the analysis, and the discussion.  You don't just read the summary, and you don't read somebody else's opinion about what the journal said.

There is no substitute for evidence, and there is no substitute for saying, well, what, actually, did they do; how did they do it; and, what their data are.  When you do that, you often come to different conclusions than those that are generally bandied about.

For example, and I close with this, you heard earlier this morning, that a meta-analysis of homeopathic medicine -- published, I think it was in Lancet.  I read the paper, but I don't remember, was it Lancet, or maybe BMJ -- a meta-analysis showed, beyond any doubt, that homeopathy was effective.

Now, I can't quite remember what was in the abstract, but I know very well what was in the paper, and if you read that paper, it is very clear the data show, and the authors of the paper recognize that the data show, that the best studies, the most rigorous studies, as judged by the analysts, were inconclusive, and the ones that were statistically significant were, by and large, much less rigorous.  A slightly different conclusion from what you heard a little while ago.  Look at the evidence, look at the data, and be guided by the evidence.  Thank you very much.                    Panel Discussion

DR. GORDON:  Thank you.  Thank you, Dr. Relman.

Thank you all.  It is a pleasure to see all of you working so collaboratively with us in addressing these questions.  I feel there has been so much progress made in terms of the dialogue with this field and the major journals.

There was only one thing I was disturbed about, Dr. Relman.  Forty years ago, when I entered Harvard Medical School, I could have sworn they told me God dictated the contents of the New England Journal every week.

[Laughter.]

DR. RELMAN:  Well, you remember, Dr. Gordon, and I was reminded of this just recently in a trip to Jordan, that Moses was denied entrance to the Promised Land, and the New England Journal has not yet gotten there.

[Laughter.]

DR. GORDON:  Thank you for that clarification.

I had one brief question.  As I was listening to a general consensus about the kinds of research that might be published, it occurred to me that one of the places where there might be more attention to CAM or integrative approaches, particularly in the mainstream journals, because I know it is there in the CAM or integrative medicine journals, is in some of the columns that are opinion columns, or the summaries of the literature, or summaries of the clinical experience.

I wonder if you might respond to that, because there has not been very much that I have read in the mainstream journals.

DR. RELMAN:  Is that addressed to me?

DR. GORDON:  Well, you, Dr. Campion, Dr. Laine, Dr. Fontanarosa.

DR. RELMAN:  Well, most of the general medical journals publish a variety of material.  I think, as Dr. Fontanarosa said, we publish not only original such articles, but also reviews of the field, and commentary, and editorial opinion, and so on.  Yes, of course.

I remind you, The New England Journal published the first study by David Eisenberg on the prevalence of alternative medicine use, a very influential study.  We didn't consider it as an original research article in the usual sense.  I think it was a special article.  That is a category we have for thing that have documentation but don't involve the classical kind of research.

DR. GORDON:  No, I know that.  I wasn't asking in an accusatory way.  I was wondering if that might not be a way to open up publication to some of these other issues and deal with them in a thoughtful, critical way.

Phil.

DR. FONTANAROSA:  I would just like to comment on two of the categories of articles you mentioned, one, reviews, and one, commentaries or opinion pieces.  We require a very systematic approach for reviewing articles, to look at the totality of the evidence.

For example, the executive summary of the NCP Cholesterol Guidelines that are published in today's issue of JAMA is the result of a summary and a review of more than 800 references in a 200-page document vetted by 27 experts over about two years.  I think if someone looked at the evidence behind the CAM therapy with that type of rigor, I think a journal would make a mistake not to give it very serious consideration.

On the other extreme, our opinion pieces, or commentaries, I gave you some data for our acceptance rates overall for JAMA.  I will tell you that the acceptance rates for commentaries are even lower than that.

We get a lot of papers that come in to stimulate ideas or thought, or to provoke attention.  Many of those tend to be, quite frankly, not well supported by references, but floating ideas, and we really have to make a determination if we want to spend space on those that would eliminate space from a research paper.

So I would say that the two extremes are rigorous totality of the evidence for reviews, absolutely.  Commentaries or opinion/thought pieces have their place, but they tend to fair less well.

DR. CAMPION:  I would just add that opinion pieces are certainly an appropriate forum for discussion of the issues.

Whenever there is some hot area of controversy in medicine, we tend to hear about from people and get opinion articles.  I am wondering why we are not getting much about alternative medicine and CAM, and I wonder if it is because CAM is actually a lot less controversial than some would say.  It is there; it is accepted by people that use it.

Perhaps people have some perspective on it.  Physicians are well aware of its popularity and may not be as threatened by it as is sometimes said.  Perhaps it is not as controversial.  This does not mean it is accepted as effective, but as a part of our system and of what the consumer wants.

DR. LAINE:  Part of our aim in inviting a series of papers, the papers in the series are much of the type that you describe.  There is a little bit of reviews that are not as critical as we would like, but most of the papers in that series of papers aim to describe the context with which alternative medicine exists in the medical community and the larger society.  So, economic issues, ethical issues, medical, legal issues, that have some relevance to our readership.

So while we are still waiting for the right for the rigorous scientific evidence about specific therapies, we hope that those sorts of articles help conventional physicians understand the context that this whole discipline exists in.

As a sideline, I think there is a perception, like Dr. Relman said, that the conventional medical journals are biased against alternative medicine, papers just reporting on alternative medical therapies.  I don't think we are.  In fact, one of our hopes is that this will show that we are interested, and maybe will stimulate people to send their research to us.  Then, of course, it has to go through the review process.

But one of the reasons we don't publish much, is we don't get much.  The second reason is, of the stuff we get, a lot of it has methodology problems, but that is no different than a lot of the original research that we get on conventional therapies.

DR. GORDON:  Thank you.  My thought was that there may be some articles that are equivalent to discussions of the hypothalamic pituitary adrenal access, for example, summarizing the data that may be interesting, and certainly would be interesting to readers of mainstream medical journals, as well as, perhaps, opinion pieces.

Dean, and Joe, and Effie, and Wayne.

DR. ORNISH:  Well, again, I want to thank the people for such eloquent testimony.  I really appreciate it.

Several people have said that a distinction should not be made between CAM and other medical modalities, either from the standpoint of funding or publication.  I certainly completely share that ideal and your commitment to doing good science.  It is certainly how I spend most of my time, but we have heard testimony from several people who have testified before our Commission that certain journals are more likely to publish an article showing a negative finding from a CAM modality than a positive one.

I am addressing this question to Dr. Campion.  In the three priorities that you outlined in your testimony a few minutes ago, the top one was to give priority to those few complementary and alternative medicine methods that can cause harm.

Do you also have the same commitment to articles that show safety, efficacy, and effectiveness?

DR. CAMPION:  I think, as Dr. Roman outlined, the basic principle is, we will consider any manuscript on it merits.  If the subject is important, if the intervention is important, which means it is something that is in wide use or has potential benefits, we will consider it.  We don't get many of those.  I am not sure if we have gotten any.

We do publish a fair number of negative studies, over 20 percent of the randomized, controlled trials that we publish are negative studies.  Those are of conventional medicine.  We think that negative studies are more important than people realize, both with respect to CAM and with respect to conventional medicine.

DR. GORDON:  Joe.

DR. FINS:  Dr. Fontanarosa uses the phrase "a reputable journal," and as a conventional practitioner, when I see something in the Annals or JAMA or the New England Journal, I feel much more comfortable in feeling that it is true, at least by the standards that I have lived and practiced by.

Dr. Relman, this question is directed to you, but to any of the editors, I know there is a consortium of medical editors that have a working group that have published standards.

Is there any outreach to the CAM journals, or at these other medical journals?  What are the criteria, and how do we know what is a reputable journal?  Because I think that is really the threshold thing.

Yesterday or the day before, we were talking about reimbursement for benefits if things were in a peer-reviewed journal, but the question is, what is a peer-reviewed journal; what is a reputable journal.

How do we establish what the threshold is, and does that organization, that entity of journal editors, have a role in establishing what the cut-off might be?

DR. RELMAN:  Although I was one of the founders of the Vancouver Group, more years ago than I can quite remember, and as of the time I stepped down 10 years ago, there was no attempt to define what was a good journal or what was a bad journal.  The membership in that group was fairly arbitrary, and it didn't have much to do with quality.  It had to do with other relevant editorial things.

So I would ask my colleagues who are now active members of the Vancouver Group whether there is any attempt to define quality, but I would also say that I would be very dubious about any such attempt.  It has got to be very arbitrary.  At the very least, you have to say it ought to be peer-reviewed if it is going to publish original research.

If it says, no, that is not what we are about; we are going to publish commentary, then you don't need, necessarily, peer review, but if you are going to publish original research, it has to be peer-reviewed.  So that would be my only observation.

DR. LAINE:  I have more recent experience than Dr. Relman's with the Vancouver Group, because it met Thursday and Friday of last week in Philadelphia.  So I spent two days with the Vancouver Group.  That isn't its role.  It is a volunteer group of editors from medical journals who came together to create a document called the Uniform Requirements for Manuscripts Submitted to Biomedical Journals to create some standard guidelines that journals can, or choose not to, buy into.

I think, from speaking with people who have been on the Vancouver Group since it started, it sounds like originally it was more focused on the format of the references, and the nuts and bolts of putting together a manuscript.

In its current inclination, in the revision of that document that is in progress, a lot of concern with ethical issues that relate to publication in biomedical journals, but not to give editors some place to refer to when they are trying to sort out difficult issues related to conflicts of interest on the parts of authors, reviewers, and the like, but not in any way setting criteria for what is a good journal and what is a bad journal.

DR. FINS:  It is not unrelated because the other issue is the firewall issue, and advertisements and supplements in some of the CAM journals.

DR. LAINE:  Right.

DR. FINS:  So there may be a role for the Vancouver Group.  For CAM journals, if they were to accept membership in this consortium, it might lift the standards of the ethical issues, the homogeneity of references, and there may be some cross-over effect.

It is just an issue that might be worth --

DR. LAINE:  Right.  That is an interesting point, because one of the things in this document are recommendations about advertising and where it should sit, and where it shouldn't sit, and what types of products are appropriate according to this volunteer group of editors to be in biomedical journals.  That is an interesting point.

MS. WOOTTON:  If I could just make the point that the blue journal, the Journal of Alternative and Complementary Medicine, does not accept advertisements at all.  This was a conscious policy right at the very beginning at its inception because it is such a tricky area for advertisements.  So we are clean.

DR. RELMAN:  Let me interject.  I think that is just great.  If I had had my way when I first became editor of the New England Journal, I would have said to the owners of the journal, live on our modest profits from subscriptions and classified ads.  We were probably the only journal that could have made a profit from just subscriptions and classified ads, but fortunately, not yours.

MS. WOOTTON:  Absolutely.

DR. RELMAN:  Most journals are now caught in the economic vice of advertising income and conflicts of interest.  This is not irrelevant to your concerns.  Conflicts of interest between those who profit from the sale of alternative therapies and alternative devices, and so on, and those who set policies, set government policies, and constitute advisory groups, are rampant, and you ought to be worried about that.

DR. GORDON:  Dr. Campion, please.

DR. CAMPION:  I would just add in response to your question about journal quality, there is no stamp, and there should be none.  Journals can change over time, but journals have to be judged on the process, on what they publish, and on their place in the scholarly community, how readers, the critical audience responds to them.

One indicator of that is the impact factor, how often do other researchers and scholars quote articles from any particular journal.  That is just one rough indicator.

DR. GORDON:  David.

DR. RILEY:  We send all authors the Uniform Requirements for Manuscript Submissions and the Consort Guidelines in terms of publications.  Now, of course, they don't always follow it, and lots of times, that is probably the primary reason for rejection of original research articles.

DR. GORDON:  Effie.

DR. CHOW:  Thank you very much for your input.  I have got many questions, like Wayne usually has, but I will restrict them to one or two.

DR. GORDON:  If you have too many questions, we won't have much lunch.

DR. CHOW:  Regarding Dr. Campion's remark that it may be more accepted than we believe it is, coming from an era in 1970-something when acupuncturists were jailed, and homeopaths were jailed, some of the leaders were jailed, and Christopher Hill from holistic health was jailed, and we were helping to get them out of jail.

I would say it is more accepted now, but it is acceptance on the public level.  Now it seems like we have the challenge of influencing or educating the top level echelon of scientists and policymakers and so forth.  In your reviews of the papers -- 9,000 reviewers, that is really a lot -- you talk about physicians being the managers of CAM processes and so forth.

How much education do you think is enough to educate the physicians and the health professionals who know, really, basically, nothing about energetics or any of the practices?

That is our educational question, too.  This goes into research.  So I am curious, and I think we would like to be enlightened as to, how much education do you think will make a physician and a nurse, a physical therapist, et cetera, qualified to judge the value of a particular practice.

DR. RELMAN:  May I respond to that?  Education has to follow, not precede, the evidence.  You will not influence what the leaders of American medical education teach and what they believe without presenting, without having available persuasive scientific evidence.

That is why it is so important to support research, to publish good research, to teach those who believe in the value of alternative medicine how to do good research.  That is the only way.  There is no other way that you are going to influence the real leaders of American medicine, and there is no other way that you should want to do it.

DR. GORDON:  Dr. Campion, did you want to respond?

DR. CHOW:  We do believe in research.  I just wanted to say that.  That is a big issue here.

DR. CAMPION:  I believe that education of physicians should mainly focus on trying to be better physicians.  There is plenty for physicians to learn already, and there is this huge, poorly defined area of a million different types of CAM, which everybody knows, there is not a lot of objective data there.

Hence, I don't see need for trying to inject the medical curricula with quasi data, or some pretense of data when there isn't any.  I think physicians need to be aware of what their patients want, where their patients go, what other practices are involved, particularly when they come in direct competition.

DR. GORDON:  Phil.

DR. FONTANAROSA:  I would like to respond briefly, if I may.  Your question about how much education is enough is a very difficult question.  Being involved with teaching two courses to the first-year medical students at Northwestern, I can tell you that time in the curriculum is rare for adding on any additional materials.

In fact, if you add something, something else has to get bumped out.  There are so many hours and so many ways that you can do it.  So I would agree with Dr. Relman, that decisions to add things to the curriculum must be based on existing evidence and a good reason for those being introduced, and bumping out something else.

From the standpoint of how much education for physicians in practice, a lot of that is self-directed, of course, by continuing medical education.  I do think that physicians need to be aware of the various CAM therapies, their limitations, their applications, not that they are going to practice them.

I think it would be just as much a mistake for a physician who is not trained in acupuncture to perform it as someone not trained in orthopedic procedures to set a broken wrist -- there needs to be that expertise -- but to be aware of the current information available on these therapies is important so that when patients ask their physicians these questions, they won't just dismiss the question because of their lack of knowledge, and they will be able, more importantly, to see how the CAM therapies might interplay with the conventional therapies, drug/drug interactions and so on.

DR. GORDON:  Wayne.

DR. JONAS:  Thank you for some very, very wonderful comments by everyone.  That was a good discussion and panel.  I hope I can formulate this.  I will put my question first.  How about that?

Do you think anything needs to be done in this area that, we as a federal body trying to look at federal regulations, need to do?

I don't personally see anything that needs to be in this particular area.  It looks like the process is going on like it ought to go on, both with the biases and the lack of evidence.  I think anyone who reads the historical literature and the sociology of science sees that journal publications is the world in which biases are fought out.

We get data and we have editors and reviewers that are biased, and then there is, hopefully, an open and full discussion, and eventually some of the good stuff forms to the top.  There are multiple examples of this.  Parapsychology is fraught with them, chiropractic has documented this recently, Ernst has documented both sides of this problem in publishing, the marked publication bias in favor of positive studies in the complementary medicine journals, very marked.

The reverse of that, the reviewer bias against some complementary therapies among the conventional reviewers.  So I think this exists, and Davidoff [ph], actually, at a previous conference, said this directly.  The question is, is there anything to do about this, because the bottom line is, there isn't very much good research.

So if the goal for all the journals is to get high-quality, rigorous research -- and let that be the arbitrator of this -- if that doesn't exist, is there anything that we have in relation to this, other than what we have already discussed, which is, let's try to get better and more research infrastructure going, and actually outcomes, so that we have a better selection among all journals to pick from?

I am wondering if any of you have an opinion about that.  Is there anything that should be done about these issues, from the publication point of view, anything that we should be doing?

DR. GORDON:  Please, respond.

MS. WOOTTON:  Can I respond to that one?  I think I would go back to the idea of innovative methodology.  This is what the CAM journals can do best, is to nurture new methodologies that are rigorous but bring in new ideas, new approaches

To go back to my recommendations, I think probably the middle one, that there needs to be more research money for practitioner training schools because they are the ones with the knowledge and the insights that can bring forward these appropriate methodologies.

At the moment, there is a dearth of training there.  I think Richard Hammerschlag addressed this one extremely well.  That would be my recommendation.

DR. RELMAN:  Dr. Jonas, I am on your side.  I preach the same sermon that you are preaching.  We need more good evidence.  In the fullness of time, with good evidence and full publication, the reliable will be sorted out from the unreliable; the true from the false.

I would take exception to my colleague, Jackie Wootton's, suggestion that we practitioners ought to be taught how to do research.  That is a special skill.  It requires a different kind of discipline, and neither in so-called conventional medical research, nor in alternative medicine research, I think, can you expect that the average practitioner is going to play an important role.

They can be part of a team, but the team has to be directed and disciplined by people who are professional investigators and who understand the discipline of clinical research.

MS. WOOTTON:  Could I just come back, quickly, to correct any misapprehension.  I am not saying that practitioners should be taught research methodologies.  I am saying that practitioner training schools should be given encouragement to develop new, innovative methodologies, because I think this is where the innovation will come, not by channeling and focusing down the same route that the conventional biomedical research is going in at the moment, whilst at the same time, sustaining the rigor and the basic principles of scientific methodology.

DR. GORDON:  Thank you.

Other responses to this question?  Dr. Fontanarosa?

DR. FONTANAROSA:  Dr. Jonas knows this all too well, from his previous experiences at OAM.  I think that the two places that there might be a way to improve this is being sure that the grant review process is such that there is prioritization of important, answerable research questions.  So clear prioritization of grant funding.

The other seems, probably, easier said than done.  There are, I can't remember how many, national independent centers for CAM research around the country, each sort of focused on its own area of expertise, and I wonder if some better collaboration among these centers might help to answer questions more efficiently.

DR. GORDON:  I wanted to ask you all, too, to follow up on Wayne's question, whether there might not be a place for the editors of some of the journals to give seminars to help some of the investigators develop more appropriate tools for presenting research and papers, a kind of outreach, perhaps, at some of the conferences or some kind of independent meeting.

Would that make sense?

DR. RILEY:  I think if people focus on the Uniform Requirements for Manuscript Submission and the Consort Guidelines.  The Consort Group meets regularly.  There are other guidelines being proposed now, the Moose Guidelines for Outcomes Research.

The information is out there, but it probably needs to be provided more to CAM investigators, or CAM authors when they are writing up data for submission.

DR. GORDON:  I was thinking of something more interactive, actually, a kind of encouragement and a kind of working with on developing papers of higher quality.

DR. LAINE:  I think developing the paper might be too late a step, because once the data are collected, the data are the data.  If the paper has good data and is addressing an important question, as an editor, you can fix up the presentation.  That is a big part of our job, but you can't fix fundamental study design.

So I think there may be a role for the sort of activity you are talking about, but it has to be earlier in the process.

DR. RELMAN:  Dr. Phillips, in describing the program at Harvard, was telling you that one of the things they try to do is train people interested in alternative medicine to learn how to design good studies to find out whether what they are doing works or not.  That should be part of the education of people who want to make a contribution to the field.

DR. GORDON:  Thank you.

Tieraona.

DR. LOW DOG:  It is quick, because everybody is hungry.

DR. LOW DOG:  Thanks for all of your comments.  My question relates to the database and self-described experts, or what they self-describe their expertise in.

There has been wonderful articles in all of these journals, but some mistakes continue to be repeated, especially in the area of botanicals, which I think physicians are interested in because of potential herb/drug interactions, and because they are pharmacologically active substances.

So we get articles that are published that have the wrong species identified.  So we say Siberian ginseng is actually Panix ginseng, which is a completely different genus, or we have an herb/drug interaction article published that talks about hepatotoxicity of echinecea, but having some mistakes around the biochemistry.

So I think that those are errors we wouldn't have in conventional medicine because we are more familiar with the pharmacology.  Have you thought about getting botanists, or trying to expand the database a little bit more, so that they have specialized areas, just to make sure that those errors don't occur?

DR. LAINE:  Yes.  That is something that we constantly do when we go to our reviewer base.  We don't have any process or the resources to go check into people's credentials.  Once they have done a review for us, each reviewer is graded, and that information is included in our database.  Somebody whose done a lousy review for us is unlikely to be chosen to do another review, but we have to rely on what people tell us their area of interest and expertise is.

We do go out and try to find reviewers in CAM therapies, and in other areas, too, where there seems to be a paucity of expertise among our reviewer database, but mistakes will always get by.  I think it is probably just as common in conventional medicine, although in areas where there are lots of reviewers with that background and expertise, it is probably less likely.

Particularly, in alternative therapy, it sometimes seems like we are hitting on the same reviewers over and over again, and then the potential is there for their biases to creep into the reviews, which is another problem, different, but maybe just as great as inaccurate information.

DR. FONTANAROSA:  I would agree with Dr. Laine for the most part, and I would add one thing, that in addition to appropriate reviewers for papers, and this is true of any paper, if there a new mutational analysis for a genetic predisposition, you try to get someone familiar with that methodology, the same thing.

I would also add that editors have a responsibility in reviewing papers, not solely to rely on the reviewers comments, but to do independent work on his or her own, to pool research articles, to try to verify, independently, names of drugs through various textbooks or other resources to find that out.

Once something is published, it is really out there.  There is no pulling it back easily.  So to try to be sure that these T's are crossed, I's are dotted, the right drug is named -- or, the right herb is named, excuse me -- is very important, and I think it is also the editor's responsibility, in addition to the reviewer's information.

DR. GORDON:  Thank you.  Thank you all very much.  This has been a wonderful panel, and we have really enjoyed the discussion and look forward to continuing to work with you all.

We are going to take a break now for lunch.  We will come back at 1:45 for small group meetings.  Small groups will meet until 2:45, and then at 2:50 we will come back here for the large group.

[Lunch recess taken at 12:50 p.m.]                          + + +
           A F T E R N O O N  S E S S I O N                                             [2:55 p.m.]

DR. GORDON:  Which group is going to present first?  Tieraona.        Group IIIA: CAM Research and Publication

DR. LOW DOG:  My panel, we had Tom, and we had Joe, and we had Julia, we had Conchita, we had George.

So our first question was, evaluating the research literature.  So we tried to look at barriers and obstacles that were involved, and we divided it up into two groups, one was evaluation of research literature by practitioners; how could they use it to change their practice habits, and then we also divided it up into reviewers, people that were actually reviewing the literature.

For the practitioners, we feel that everybody would benefit, conventional medical doctors, massage therapists, acupuncturists, chiropractors, if all practitioners had training in school, or through continuing education from people who are already graduated, in how to critically evaluate published research.

That would include conventional practitioners having some familiarity with unique aspects of evaluation of CAM.  It would also allow CAM practitioners to have a better handle on how to critically evaluate the research that they read in other journals.  So we had that.

Also, for practitioners, sort of in that area, we wanted to encourage the continued development and support for training CAM professionals who can teach research methodology and evaluation, because part of the problem is, who is going to teach in these schools.  If we are saying we want the chiropractors or the naturopaths or the acupuncturists to be able to evaluate the research, we have to have people that are capable of teaching them in their schools.

To that end, we encouraged collaboration between the conventional academic institutions and the CAM schools.  So there is a wealth of people who are well-trained in research methodology at universities to be able to help teach in these schools.

Also, we wanted to encourage funds that could help support and sustain library types of materials.  A lot of times, large academic institutions have endowments and funds that are not available to CAM schools, and that there should be some sort of funding available or set aside so that they could have access to journals, MEDLINE, computers, et cetera.

DR. GORDON:  Question.  In this first one of training people to evaluate research, did you have any particular pathway by which you thought that could be done?

DR. LOW DOG:  Well, part of it is already being done through NCCAM, who is helping to teach CAM practitioners research methodology, but we also feel that part of this is going to have to be done in the short term through collaboration with universities and academic centers where you will have Western-trained people, basically, going to CAM schools to teach research methodology.

The ideal, actually, would be to have people that are specialists in their own modality go through and learn research evaluation.  We are hoping that they could use that through NCCAM, such as what we heard today, through NCCAM means.

I hope I answered your question.

DR. GORDON:  Sort of like NCCAM having a training program.

DR. LOW DOG:  With the sole purpose not to train researchers, but to train people who teach practitioners how to evaluate research.  So a little bit different there.

Then again, we wanted to make sure that we earmarked funds to accomplish, making sure that practitioners could have access to this.

Then we separated evaluating research literature, looking at barriers and obstacles, into reviewers, people that just review the evaluation of this literature separate from publication.  We have carved out this niche.

We just, again, wanted to encourage that there be at least somebody who is reviewing it on the team that has familiarity with that CAM-specific modality, so that they would know if that was even appropriate; was it appropriate to test this herb for that purpose so that there is somebody specific.

DR. GORDON:  I'm sort of lost.  What is the context in which you are talking?

DR. LOW DOG:  When you are evaluating research literature, like for doing a systematic review, when you are doing a meta-analysis, when you are gathering data, that those types of reviewers need to also have some sort of expertise, or have some sort of collaboration with somebody who knows about CAM.

We didn't have much on there.  We just sort of thought that came before the published issues.  That is all we had for evaluation of research literature.

Was there anything I forgot, guys?  No?

So the next question was, publication of research results.  Again, here, we tried to look at ways that we could promote and improve the availability of research being published in peer-reviewed journals.

One of the things that we really did want to encourage, because you can't mandate, but we just wanted to recommend that peer-reviewed journals involve at least one person on the review team when they send an article out for review that was specifically familiar, or had expertise in that CAM modality.  They don't have to be experts in the research methodology.  They have other people for that, but just the appropriateness of the way the study was done involving that modality.

Part of what we tried to consider about publication was, one, we don't want complementary and alternative medicine to be seen as second class.  We would like us to try to get more of the articles into peer-reviewed journals, but we thought, perhaps, since we do believe there is some bias at this point, which may be just because we haven't got to the place where we have good enough studies, that maybe NCCAM could do something like, either consider a quarterly journal, less frequently; more frequently, or do what we call a "year's best," where they take the best studies that have been done in CAM for that year, or six months, or however frequently it is done, and publish that and make that available, similar to what ODS does now, so that you know what the 25 bests were for the year.

We thought that that might be a way of making people more aware of the research that has been done because maybe they didn't get into the New England Journal or JAMA.

We also just bandied around other ways of getting that information out there, and wanting to encourage NCI, CDC, Office of Dietary Supplements, HCFA, people who are already taking trials and summarizing or putting them on their sites.  We would like them just to consider the inclusion of CAM trials that were well done, and putting them in their sites, because sometimes we think there is just an exclusion of them for no real reason, maybe just a reminding that good trials should be included there, too.

Anything on publication stuff that we had, that we bandied around?

This took a lot of time.  We had a lot of work on those issues, about where things should be published and who should pay for what.

Questions 3 and 4, we sort of put together.  This was looking at issues around non-profit support, people that are not for profit, and private sector support.

Based on some of the previous testimony that had been given to us, we came up with a couple of ideas that we liked.  One was that there is a lot of private foundation money and not-for-profit money out there, but nobody knows about them.  So, how would a CAM researcher even know they existed?

We thought that NCCAM might be able to just have a site under their researcher site, where they just actually list contact names for foundations, so that if you are an acupuncture wanting to do a trial, there is a place that you could go to that you could get information of other foundation that might be willing to sponsor it, and that they could maintain that as sort of a liaison.

We weren't sure who it would be, this ambiguous, appropriate government body, but we wanted to have an appropriate group to serve as a coordinator for a group of public, private, and governmental agencies who convene to discuss how to better coordinate CAM research.  We think there is a lot of duplication.  We think there is a lot of discoordination, that there is precious little money, and the money is sometimes not being used in a very coordinated fashion.

So if there was a group that could come together that could help coordinate the public money, the private money, the government money, the foundation money, we might be able to get more work done in a better and more quick fashion than in this disjointed fashion it is now.  We didn't know if that was NCCAM or who, so we just left it open.  

Finally, for foundations and not-for-profit, we wanted to recommend that we try to get a group of them to pool monies together, so that they put a CAM pot together, if you will.

Some of them, we heard, they are only willing to put so many dollars in; so that we get them to contribute a group of dollars, and then they could announce a competition, and then could take applications in and get the best few that could compete for these CAM research dollars.  We might be able to get some bigger studies funded, instead of $10,000 here and $42,000 there.  If we could get some of them to come together and pool their research money.

We didn't address Question 5.

DR. GORDON:  Either additions from Tieraona's group, other points that anyone in the group wants to make, or questions for her about this?  Wayne, go ahead.

DR. JONAS:  What was Question 5?

DR. LOW DOG:  It was CAM research and regulatory process.

DR. GORDON:  Other questions for Tieraona?  Because I have one, and I don't know whether this fell into Wayne's group or yours, or somewhere in between.

Oh, you wanted to ask a question?

DR. JONAS:  Yes.  I just wanted to make a comment and add something that has worked in the past and could work in the future, and that is OMAR.  They are the ones that do consensus conferences.  When we were originally looking at the whole list, gee, what ought we do through OMAR.

A lot came up.  We did the mind-body stuff, we did the acupuncture area, but there were a whole bunch of herbal products in which the question was, gee, is there enough research to currently recommend under what conditions that these are safe and effective.

It is exactly in those controversial areas that consensus conferences are held, to have the debate and the discussion, and also bring out the current evidence.  So they, I think, would be an area of focus in terms of some of the areas.

DR. GORDON:  I want to ask Tieraona, both you and Wayne, the question, yesterday when AHRQ and Brian were talking about different ways of producing information.  Was it your task to figure out how all that should go together?  Was it your group rather than yours?  It was not, okay.

DR. LOW DOG:  I am sure it was Wayne's.  I know it was Wayne's.

DR. GORDON:  I don't think we can necessarily do it here, but I would like to hear some more thoughts, like the consensus conferences, which I think is really important, about an overall strategy, because I think what we want to do is make some recommendations, should we go in some of the direction of outsourcing, in a sense, to private groups, like the Cochrane Collaboration.  The production of information, should it go to AHRQ, should it be NLM, or NCCAM.  I would like to hear a little discussion here about that.

DR. JONAS:  There is a real debate within NIH as to whether they should do that.  They have done it in the past, and there is a real debate as to whether they should continue to do it.

For example, they contract with groups to do special databases to produce research on health services research and cancer, and a variety of other things, toxicology.

Some of the areas they are not particularly happy with having to do that, but on the other hand, they do it a lot.  One of the things that might be useful would be to actually contract with a group that does an ongoing summary of current information, past information.  I think that would be a very useful thing, and it wouldn't be very expensive.  We actually looked at the cost of that, and it is not very expensive.

DR. GORDON:  Yes.  I would like to hear other people's thoughts about that, because I think this is a really important issue.  I think this may be one of the ones we want to address in the Interim Report.  It may be complex, but it is reasonably straightforward at the same time.

DR. JONAS:  One of the issues is that, often, people will look at those sites and say, that is the authoritative site, and therefore the individuals that want to market a supplement, if it is listed on that site, then they will say, okay, now we have an authoritative body that has said this is worthwhile, we can market it, and they will use that through the courts and things like that.  So there are complications to doing that.

Again, the government is nervous, rightfully so, lest they be interpreted as recommendations in some way.

DR. GORDON:  But AHRQ does that, in a sense.

DR. JONAS:  AHRQ no longer does that, actually.

DR. GORDON:  But they do these, what are they called, evidence reports.

DR. JONAS:  Yes.  They do do those, exactly.  The Cochrane work does do that.  So they do do that, and they have a whole process for making sure that you don't take this as a recommendation, that they are available for groups that want to then make guidelines out of those, if they feel that they warrant them.  But it is possible to do that.  That becomes very expensive, however.

DR. GORDON:  If AHRQ does it.

DR. JONAS:  Yes, because if they are doing those evidence reports, they are much more elaborate than simply summaries of current reviews, just with a database, or even systematic reviews, than what Cochrane is doing.

DR. GORDON:  Tom, and Joe.

MR. CHAPPELL:  I just wanted to maintain a principle in the choice, the ultimate choice here, that CAM research has access to practitioners and to the public.

DR. GORDON:  Exactly.

MR. CHAPPELL:  So that the more public the domain, the entity that is doing the evaluation and the publication, the better.

Now, if the public entity decides to privatize but still maintains control, that would be fine, too.  I just feel that we need a public entity assessing the evaluation of these trials and taking responsibility for publication to various channels of groups.

DR. GORDON:  Great.

Joe.

DR. FINS:  I very much want to endorse the "year's best" concept that you folks came up with.  I think that is really a way of bridging the gap between the alternative medicine journals and the New England Journal level, and it gives it a kind of validity, and I think that would be very helpful.

The other thing, the practice guidelines, or recommendations for reaching out with the clinicians out there, is through the specialties societies, through the American College of whatever.  They often have board review, documents, or annual updates.  If there could be some kind of linkage with what NCCAM, or whomever is going to do that, with those entities in reaching out so it gets distributed widely, that would also be very helpful.

DR. GORDON:  Okay, great.

Other thoughts?

Tieraona, go ahead, and then George.

DR. LOW DOG:  In the AHRQ, they have the really big, thick technology report, and then they have that brief summary, but it is still not consumer-friendly.  It would be nice if somehow there would be some funding.  That information is public and it is from the government, and it would be nice if there was a lay version of that for the consumer that was just one page, or on a website that would be available as well.

DR. GORDON:  I think it is up to us to fashion exactly the kind of information we want to have out there about the research.  So these points are going to help us make that recommendation.

George.

DR. BERNIER:  We had tremendous exposure today, I think, to the leadership of medical journalism.  I would like to point out, and this sort of came up in our conference today, that very people publish what they are interested in in the New England Journal of Medicine.  It is really hard to get into, and you say, if I get one of those every 10 years, I have got it made.

So it is not a bad sign to take something less than the Annals and the New England Journal, but I think we have been believing it is.

DR. GORDON:  You mean it is not eternal damnation if you are not published there.

[Laughter.]

DR. BERNIER:  That's right.

DR. GORDON:  Okay, thank you.

Other comments?  Joe.

DR. PIZZORNO:  I appreciate that comment, George, and it gets back to the question I asked Brian, and that is, where is the bar.  One of the concerns I have is that, if you look at the New England Journal, et cetera, a very high bar.  If you look at the Cochrane Collaboration, a very high bar.  If you look at the AHRQ, a very, very high bar.

My concern is that, while at some point we want to reach that bar, there is some standard of evidence that is appropriate for us to utilize when treating our patients, and I don't think we know where that bar is.  If we could, in our wisdom, come up with some recommendations, or at least a process, for determining that, I think we could do a lot of good.

DR. JONAS:  May I just comment on that?

DR. GORDON:  Yes, please.  Go ahead.

DR. JONAS:  One of the things, again, we just mentioned, is the possibility of a specialized database contracted out which reports on current research.  It simply reports on research.  NLM does this a lot.  They are not in the role of evaluating it, they simply report it, and report it accurately.

DR. GORDON:  In a sense, what you are saying is, you report on it, and you report to height of the bar as you are reporting on it, you say how good the research is.

DR. JONAS:  No.  They don't even report the height of the bar.  All they do is just describe it in a clear, precise way.  They do summaries of it, and they produce it and put it in one place where if someone wants to get that research, they can go there.  It is ongoing, so it is real-time with the current research.

DR. GORDON:  Joe, go ahead.

DR. PIZZORNO:  Something that we have done, and a project that we are working on, is, we actually developed, by looking at what the Cochrane Group did, a ranking system for the quality of the research.

I am wondering, maybe if we had some kind of central repository which looked at the research that was out there, and basically each study was ranked, just how good is that study, how reliable is that study, that way, we could at least get a sense for where the body of research is shifting or establishing itself.

DR. GORDON:  I think what we have been hearing is that people need some estimate of the quality of the research.  The researchers need it to some degree, but certainly the public does need that.

DR. JONAS:  The way most conventional journals do this, like Annals has JPC Journal Club in which they report abstracts of ongoing research.  Most of the time, they have commentaries, because having a score, tagging a score, is very unreliable, and there is no standard score that everybody accepts.  There are fatal flaws of even high-quality studies, and good studies that are low quality, and so they have a description of that.  Then that gets into a little more complicated --

DR. GORDON:  Joe.

DR. FINS:  There is a lesson from the death-and-dying movement, which had a textbook project that Steve McPhee [ph] from UCSF headed up.  It looked at the death-and-dying care content of the leading textbooks.  It has also been done for nursing as well.

Maybe we can reach out to the kinds of places where conventional practitioners go for authoritative information and try to get some infiltration of the kind of basic themes that we talked about in the December meeting, about what the conventional doc needs to know.

I think if it is there, it gives a tremendous legitimacy.  Of course, it would be a conservative part of the entire spectrum of CAM, but it would be an initial foothold.

DR. GORDON:  Tom.

MR. CHAPPELL:  Dr. Heirich, in his first proposal, says, "Congress should instruct public agencies, that is NIH and AHRQ, to gather data, both on clinical effectiveness of various CAM approaches and on the relative costs of more conventional allopathic, and of CAM treatments for the same condition.

"This information, supplemented by other information found in peer-reviewed journals, should be brought together and made available in an easily accessible website, which is updated constantly as new data becomes available.

"The public agency responsible for the website should be instructed to draw this information to the attention of insurance companies, managed care organizations, HCFA, and the continuing medical educational programs and schools on an ongoing basis."

So I just refer you to that as more specific.

DR. GORDON:  Right.  We are looking at several different kinds of data.  We are looking at research data, we are looking at practice data, we are looking at cost data.  It sounds like many people would like that information made available.

Wayne.

DR. JONAS:  There are plenty of examples of how this is done; HIV-AIDS clearinghouses.

Steve, you put together a Rare Diseases Clearinghouse, the Health Services Research Database.  There are many examples that could probably very easily be modified.

DR. GORDON:  Thank you, Wayne, for that.

Maybe one of the things on this issue, in particular, is that staff could put together some possible, specific recommendations for ways to fulfill this general mandate to produce information for the public and researchers.  Then we can pull that together and take a look at it, and come up with recommendations based on that data, and make that data available for everybody.  Then we can talk more about it.

It sounds like we are all pretty much in the same ballpark, feeling the importance of this, the importance of some kind of discussion, if not necessarily a numerical evaluation, the importance of not only making it available to researchers, but making it easily available to the public.

Joe.

DR. FINS:  One other thing.  I don't know if there are any editors from the CAM journals here, but if there could be sections in the journals specifically addressing methodology and the design of studies in the literature tutorial, deconstructing good studies, explaining what were the elements, as a way of doing a kind of continuing educational effort, I think the readership would probably benefit from that.

DR. GORDON:  That is a separate item.

DR. FINS:  A separate issue.

DR. GORDON:  A separate item, but on the same general topic of information about research.  Thank you.

Any other issue related to these topics, related to journals or philanthropy, that anyone would like to bring up at this point?  Yes, Tieraona.

DR. LOW DOG:  Just the whole issue of the public.  We certainly didn't adequately address it, and I am not sure it has been, really, adequately addressed overall, is, how do you synthesize the research to the public that is truthful.  When you take a study and it says something, what does it really mean, and synthesizing it, because it has to be more than just this study was done and it showed this, because they can't interpret it.

So I do like the AHRQ, and trying to come up with some body that can give truthful information about the research in a lay sort of way.  We really focused more on practitioners, reviewers, and sites, and things like that, but I think that that is one of the biggest jobs that we have, is, how do we get the information to the consumer.

DR. GORDON:  I think, if the staff can bring up some of those models to us, we will see the different ways that it has been done.  One of the things I think that has been clear throughout is that there has not been enough information for the public.  It is not easily available, and it is not answering most of the questions that they have.

DR. LOW DOG:  It is often tied to a product, too, which makes it hard for them.

DR. GORDON:  Right, exactly.

Anything else on this?  Great.

Wayne, do you want to talk about your group? Group IIIB: CAM Research Methods, Training, and Concerns

DR. JONAS:  We were tasked with methods, training, concerns, federal agencies, academic health centers, and design.  I made a mistake when I read these, and I thought it was methods and framing.  So I started with that.

So we had some assumptions that we thought should frame the whole research discussion that we have heard a number of times.  There is a close consensus among some of these.  They are fairly simple.  Research is important, we should be doing research; it is key, it is important.  The research has to be high-quality, it has to be good, and it should include multiple methods.  There isn't, simply, a mono-method approach to understanding complementary and alternative medicine, or any medicine, for that matter.  There has to be multiple methods.

Now, given the fact that there are multiple methods, we felt like, or, at least we outlined, six major areas that we heard discussed both in the prior research discussions that were in the summary your book, as well as over the last few days where people wanted to see emphasis on certain kinds of domains, if you will, and other domains.

That included systematic reviews; consensus conferences; meta-analyses; summaries of current research; randomized, controlled trials; basic science research.  We heard that.  We heard a lot of emphasis then on health services research, especially looking at costs.

Cost effectiveness was something that was emphasized this time; outcomes research and observational data; and then one that was only mentioned briefly, but I put it in there because I thought it is very important for public input, and that is qualitative research, to really get at some of more subtle items that are difficult to capture in complementary medicine, or address issues that the public finds important that, maybe, is not captured in some of these other methods.  So qualitative research is an important area, also.  We just wanted to frame that.

One of the things that we did not address is the whole prioritization process, how do you decide how much you put into health services research and what areas, and should we all do randomized, controlled trials, or how much basic science research and that type of thing.

So this perhaps, then, is an item which is addressing prioritization.  Perhaps, one of the things that needs to be done is to have a prioritization process developed, if you will, that is explicit and clear, and that includes all the major audiences that are involved in complementary medicine to make sure the kind of data that they need is produced in research.

So that was the framing idea, and actually, this is the kind of thing that the Institute of Medicine does.  They have actually already written a number of reports on this, including one we heard from in the first conference, which said that the public should be especially more involved in this prioritization process.

For complementary and alternative medicine, we all felt that that was a very important area, because this is a publicly-driven phenomena.  So, how to do that is a question.

In terms of other mechanisms, we thought it was extremely important that research be done in multi-disciplinary teams.  That includes not only scientific teams, experts in a variety of sciences in these areas, but experts in the actual methods, the complementary and alternative medicine practitioners who actually do this and are trained in this, as well as public participation.

I think that most people have agreed that there needs to be good research methodologists, experts in the clinical area that is being examined, in the complementary medicine area that needs to be examined, but how to get the public really involved on the research team is a key issue, and again, probably you need a special item for complementary and alternative medicine.

We had a discussion of outcome measures, selecting outcome measures as being important in complementary and alternative medicine.  Several of them that were mentioned that appear to be important in this area are: quality of life; patient satisfaction; adverse effects; and then a category that we struggled a little bit with coming up with the terminology, but special outcomes specific to complementary and alternative medicine, such as energy or qi, for example; measurements, or the more subtle aspects that are difficult to capture and may not have a Western conception, even, and therefore not a Western outcome measure; and then also, ways to capture unexpected or serendipitous outcomes that occur from interventions that are focused on healing.  A lot of times, things that happen are unexpected.

So again, that is where qualitative research comes in, to try to capture things that you can't anticipate, don't have a measurement item for.

We felt that one of the main focuses, in terms of setting a research agenda that could go fairly quickly into policy, is the area of safety, that we need to look at safety specifically in the areas of supplements.  We have heard that over and over again, and there are several ways that that might be done.

Again, many people say that DSHEA somehow needs to be either fully implemented or modified, or discussion of DSHEA in order to identify and allow it to get at the safety issues that appear to be inadequate, currently.

We started to get into different ways to do that, and then kind of retreated and said, no, we don't want to get into modifying DSHEA; it is too complicated, but we want to acknowledge that it is important to do and that this should be done in some way.

Safety also includes the monitoring of adverse effects, and also the quality of products.  There are particular ways in which that could be done, and we should empower groups like the FDA, the CDC, the NIEHS, USP and industry, to move into these areas to make sure the quality is improved and that there is monitoring for adverse effects interactions in that way.

I suggested but we didn't really discuss, but we have heard it in the past conference, that the level of evidence, or type of evidence, should, in some way, be stratified by risk so that those that are higher risk require more data, more evidence than those that are lower risk.

This is done, actually, in the government.  We heard from the FDA Devices Division, in which they have an explicit way of doing this.  This should be looked at in a broader way, because certainly some things that are low-risk need different types of evidence than some things that are high-risk.

Also, in safety issues that needs to be addressed, is the indirect effects from incompetent practitioners, although that is slightly different.  That is more in the training issue, and certification and licensing, but it is an important component of safety.  So there should be research looking at safety, and that should be a highlighted item.

There was a discussion of trying to get at those things -- again, this was a little bit difficult, and maybe, Joe, you can help me with this -- getting at the meaning aspects, information about meaning, culturally relevant information, including anthropologists, social science involved in providing information about complementary and alternative medicine research as being very important.

National Endowment for the Humanities, I think, was suggested as a place that, perhaps, this could be a targeted.  So that was a suggestion.

We then looked at stimulation of support in research in federal agencies, academic health centers, and foundations, which was not our charge, but we felt it was very clear that we needed to encourage funds, stimulate more federal involvement in research.  That could be done through multiple methods, RFAs, Special Emphasis Panels, money.

What else did we say, perhaps opening up coordinating offices, either in Health and Human Services, initially.  Then maybe in other agencies that are not within Health and Human Services' purview, perhaps this could be done in a stepwise manner, so you do, first, one in HHS, and then subsequently, you open up as those areas develop to try to specifically encourage research in those areas, similar to what happened at the NIH when the OAM was started and that began to gather a lot of attention, and kind of develop those areas.

One very touchy area is that, first of all, I put up there, "should be blank percent of future budget increases should go to integrative medicine research."  I was hoping they would fill in the blank for me with a number.  They didn't do that, they said, well, there ought to be more, but it should be a significant portion.

There are many ways to do that, but we don't feel like we should specifically say how much that should be.  There are a variety of methods that could be done, line items, appropriations in language, requiring reporting on the amount invested every year in these areas in complementary medicine.

DR. GORDON:  Wayne, could you just define what you mean when you say "integrative medicine"?

DR. JONAS:  Integrative medicine involves the use of complementary and alternative medicine within the conventional health care system as part of our health care delivery in some way.  It doesn't necessarily mean it needs to be in hospital, it just needs to be part of the individual's health care complex.

We listed a number of agencies in which that could occur.  Joe just mentioned to me that we didn't address the whole issue of targeting specific vulnerable populations, areas like end-of-life, but my feeling is that if we played on the strengths of these agencies, that many of them have special populations, like HRSA, that they deal with, and that we perhaps would then get encouragement across the board in terms of different types of research, especially areas that are outside HHS.  DoD, the VA, has a lot of research capacity and is willing.  We heard from them, and they are willing to do research in these areas.

Let them build on the strengths of what they do best.  Of the six general areas that we framed in the beginning, different agencies do different aspects of those better than other agencies.  AHRQ, for example, does the reviews better than anyone else.  They do health services better than anyone else.  NIH likes to focus, and does a lot of good work in randomized, controlled trials, basic science research.  That should, perhaps, be an emphasis for them in other areas, et cetera.

Finally, we also asked, how can there be stimulation of research in other areas besides the federal government, academic health centers.  I think one of the things we thought was especially important that we heard over the last couple days was investment in infrastructure in academic health centers.  These academic health centers include complementary and conventional health centers.

They may require different mechanisms, different approaches to that, but they all need training, they all need equipment, they all need expertise in methodology, they all need to take a team approach, they all need informatics aspects.  The federal government has a number of ways in which they invest specifically in infrastructure to get a critical mass of researchers, and those should be encouraged.

Under foundations, I think you addressed this.  We didn't talk about it too much, but we agreed, relatively easily, we could facilitate coordination among foundation, private and federal, through meetings and things like that, so that they are at least talking to each other, and talking about the kinds of things they are providing.

In the private sector, there were a number of mechanisms that policies could be targeted towards.  Some kind of market protection was mentioned for private groups that invest in research and products and particular practices.  I put down patents.  That is a major issue.  It hasn't been discussed a whole lot here, but again, that is another way of coming up with market protection.

DSHEA, we have already mentioned.  Modification or implementation of DSHEA in a way that encourages research, rather than discourages it; tax incentives, SBIRs.  The orphan drug approach was thought of as being a way of providing some kind of market protection, or as an example of providing some kind of market protection.

Training.  Since I misinterpreted training as framing, we didn't talk about it.  Never mind.

Regional collaboration was felt to be extremely important.  In other words, getting complementary groups and conventional groups to work, such as in the Northwest, was brought up as an example of where that is occurring, and should be encouraged.

We didn't really didn't get into details about how to get state and local governments more involved in the area, but thought it was important, and having a repository of research methods, facilities, resources, et cetera, that have an interest and are doing work in complementary and alternative medicine, an information source for research itself, not the results of research, which is information, but actually, the funding sources and approaches, and types of things they are looking for among these agencies, would be useful.

Oh, yes, we did mention paradigms, and we assumed that for 20 cents on your tax dollar, you could get a couple.

DR. GORDON:  Thank you, Wayne.

Tom, and then Joe.

MR. CHAPPELL:  I am wondering if we could begin a drafting of something for the area of herbs, botanicals, and so forth, as an interim report to postpone this.  We are beginning to get a better understanding of this, and had a lot of testimony.  I think it would be wise to begin to draft a set of recommendations regarding this whole area of concern.

Since the committee, this working team, has identified it as a problem, could we begin to draft some language?

DR. GORDON:  So, what is the problem?

MR. CHAPPELL:  The DSHEA.  So whether DSHEA is sufficient or insufficient.  Do we have sufficient laws.  If not, what are our recommendations going to be?

DR. GORDON:  Dean.

DR. ORNISH:  This is one of the things that came up in the discussion we just had, which is whether we get into that level of detail, or whether we just simply say that we think there are issues within DSHEA that need to be addressed.  I think that if we get too much into that level of detail, it might be polarizing or distracting from what our overall mission may be.

DR. GORDON:  Did you want to address DSHEA?

DR. FINS:  I think I agree with Dean.  I think the notion is, we have identified problems.  Whether it is resolved with an amendment, a repeal, better implementation, I don't think we are in a position to address, but I think it is valuable to simply put it out there that there is a problem, and not to get to the specifics.

DR. GORDON:  My thought is, there are two levels to DSHEA.  One is, there are certain things that have come out consistently in these meetings about DSHEA.  One is that there needs to be good manufacturing practices.  Second is, that the labeling needs to reflect, which is related, what is inside the bottle.  The third is, the FDA has to have more authority, not the same authority that they have, and more money to enforce the provisions of DSHEA.

So I think there are some basic recommendations we can make, that we have heard from every part of the spectrum, about DSHEA as it presently exists, and I think we may or may not want, depending on an analysis of what is there, to take a look at other possibilities.

Tieraona.

DR. LOW DOG:  I just want to remind us that the first two things you said are already in place, though they are not enforced.

DR. GORDON:  That's right.

DR. LOW DOG:  The first thing is, it has to be what is in the bottle; it has to be what it says on the label.  That is by law.  I know it is poorly enforced.

The second is, GMP, which, the FDA is eventually going to publish and release those findings.  So GMP is there.  It is a matter of giving and empowering the FDA the power to be able to enforce, or perhaps looking at rather they should have pre-market kind of authority.  The FTC for label claims, the FDA, if it shown to cause harm, and too, if it is not meeting claims.

DR. GORDON:  That is what I was basing my analysis on.  The point is, that I think it is important for us to take a strong public stand on these provisions that are already in DSHEA.  So if we try it and we really do it, we may find it works rather well, or we may not.  Then there are the whole complex issues that you are talking about of DSHEA.

Joe, and Tom.  Anyone else?

DR. FINS:  I would suggest, because I think there are other issues that we need to talk about here, that maybe we ask the staff to draft what you just said in the Interim Report that will be circulated, and then we can deal with the specifics.  I think there are wide areas of consensus, and I think it would be helpful to have a real text in front of us.

If I could move onto another -- no?

DR. GORDON:  Let's finish up with the DSHEA, and then come back.

DR. JONAS:  I agree with what you said.  I want to add one item that relates to this research area, and that is, it should also be looked at in terms of its ability to facilitate research, not inhibit research, in addition to those others.

DR. GORDON:  Tom.

MR. CHAPPELL:  I think we need to add safety to that list of three that you identified that needs to be looked at in this Interim Report, and we need to look at the question beyond DSHEA and the FDA in terms of the efficacy.

I think Dr. Grollman's proposals, specifically Recommendations 3 and 4 relating to funding efficacy trials at NIH, are very important here because the problem is, it is very hard to find the mechanism of action when it deals with this category.  It is a far greater research burden to approach efficacy than it is to approach safety.

So I think safety and efficacy need to be treated on different levels, different recommendations, and I don't think it is too complex for us to deal with, so I am glad you are willing to begin the drafting of it.

DR. GORDON:  I think it is complex in every way, especially politically.

MR. CHAPPELL:  Well, that is okay, but that is why we are here.

DR. GORDON:  No, I understand.  So what I am saying is that, for the Interim Report, I think we need to lay out where we have consensus, and then for the Final Report, if we start looking at some of the issues, then we can have some time to discuss them and figure out where we stand.

MR. CHAPPELL:  Thank you.

DR. GORDON:  Joe.

DR. FINS:  I just want to make passing reference to the idea about the NEH, the National Endowment for the Humanities.  I think one of the things that we really haven't addressed, that I think we really need to address more fully, is this phenomenon, this social movement which is CAM and what it means for American society, good, bad, and indifferent.

I think that the reason I really favor the NEH is that it is a different kind of scholarship.  Conclusions that are drawn in the medical humanities do not necessarily have to adhere to the scientific method because they are asking different kinds of questions.  The sociology, our cultural values, why this Commission itself was created at this time and at this place.

I think future historians will really be appreciative of the fact that we made a nod to the NEH and the humanities to address these broader issues, as well.  I think a small amount of resources directed towards the humanities will have enduring effects that will outlive any of us in this room.

I just want to make an appeal for that.  I know Linnea feels strongly about that, and some of the other Commissioners.  So I hope that that gets some traction.

DR. GORDON:  Joe, could you say a little bit more about what you mean by NEH?

DR. FINS:  The National Endowment for the Humanities.

DR. GORDON:  No, I know that, but what are you thinking of?

DR. FINS:  Well, they fund a lot, and there are state humanities councils that actually go out and foster the fine arts, literature, history projects, the kinds of things that really address the cultural richness of our society.  I think that the evolution of CAM is part of our cultural inheritance and the legacy that this generation will leave to future American generations.

So I think we just need to understand what it means in a different kind of way.  I am a big proponent of the scientific method for scientific questions, but I think that these are different kinds of social, cultural, anthropological kinds of concerns.

DR. GORDON:  Joe.

DR. PIZZORNO:  Jim, I would like to second what you said about DSHEA.  What we have in place needs to be enforced, and the FDA needs to be funded for it.

However, I also would like to take one step further, and that is, I believe DSHEA does not have adequate safety components to it.  I think before a product is allowed on the market, there should be basic safety documentation, and that is not required right now, and I think that is a serious flaw.

DR. GORDON:  Do you think we have enough consensus to put that in the Interim Report, or do you see that for the Final Report?

MR. CHAPPELL:  You have my consensus.

DR. LOW DOG:  Can I say something?

DR. GORDON:  Sure.

DR. LOW DOG:  There are 2,400 botanicals sold, if you include all the herbs in commerce, 2,400.  They have not all been tested adequately for safety, and that is going to cost an awful lot of money to test them all.  It will limit acupuncturists in their ability to prescribe and use many of the Chinese remedies because they have many in them that have very little data that would be accepted or used here.

I agree, DSHEA absolutely missed t