The researcher must always consider which method is best suited for answering the research question. Different research questions might demand different research methods. An interview will demand a different method than observation. In addition it depends on who the subject is.
Clear concepts and methods
Doing a study of for example pain, the researchers must consider how they are going to measure pain. Or, if the goal of a study is to examine the connection between diet and disease, the researchers must decide which criteria shouuld apply to the concepts of a "healthy" or an "unhealthy" diet:
What does a healthy diet mean? Which groups of food are considered healthy, and in which amounts? Which foods should be attributed to an unhealthy diet?
It is important to have clear definitions of the concepts used in the research. If researchers are operating with unclear concepts and methods of measurement their research will become less reliable.
How can we know if the research is of good quality?
What separates good from bad research is often how precise and consistent the process has been. The researchers will strive to get their results published in reputable journals when the research project has ended.
Before a research article can be published in a scientific journal, it is critically reviewed by a group of other researchers with knowledge from the specific field. This is called a peer-review. This increases the probability that the research is of good quality; meaning that the results are reliable - but there are still no guarantees. Also reputable scientific journals may accidentally publish studies with methodological weaknesses.
Some of the known weaknesses in the research of complementary and alternative medicine (CAM) are:
Some studies examines how a treatment component (the prick from a needle, the pill, the grip and pressure applied, and so forth) affects a specific disorder. Such studies often conclude that the therapy has no effect more than placebo; which is often interpreted as that patients don't benefit from seeing a therapist for the provision of it.
Most CAM therapies weigh several other factors in addition to that one specific treatment component. CAM providers often talk about the total treatment package, which may include suggestions for lifestyle changes, changes in personal circumstances and help with processing existential and emotional problems, with the goal of improving the patient's health. CAM providers also means that the total effect of their treatment adds up to more than the sum of each individual component.
This is why it is important to be aware of the difference between studies examining one single treatment component, and studies examining the effect of a more complex treatment situation. This distinction is not always done explicitly in scientific articles.
In some studies, all the subjects with the same diagnosis get exactly the same treatment (needle pricks in the same spot, same type of pill, etc.). These studies often conclude that the treatment has no effect besides placebo, and that the patients don't benefit from seeing a therapist to receive this treatment.
Many CAM therapies gives weight to the treatment being consciously adapted to the specific patients' clinical picture. This opens up for the clinical medicinal diagnosis not being the only thing that determines which treatment that is chosen, or how the treatment is carried out. But also other factors such as lifestyle, personal circumstances , personality, health history, outer bodily signs etc. should be take into account.
If the treatment being researched isn't sufficiently in compliance with how the treatment actually takes place in reality, it can be claimed that it is not clinically relevant - "That is not how it is provided in real life".
A research method is a specific scientific approach. Here, we explained some of the most commonly found scientific approaches below, with examples connected to CAM.:
Basic and applied research
Research can be parted in two main categories: basic and applied research.
Basic research is experimental or theoretical activity carried out to gain new knowledge about how something works, is constructed or connected. Basic research analyzes traits, structures and connections, aiming to formulate and test hypotheses, theories or scientific laws.
One might have theory of how the mechanisms of contraction in the heart works in humans affected by infarction, relative to humans with healthy hearts. To find out if this theory holds, one can experiment with rats and mice.
The basic research does not have as a goal to put forward research with a specific use, but is supposed to provide knowledge which can be crucial for researches within applied science. Within the health field, the basic research seeks knowledge about the human biological structures, and the cellular and molecular processes of disease.
Basic research within the field of health and disease may answer the following questions:
- "What happens in the pancreas of people with diabetes?"
- "What function does the pituitary gland have?"
- "What happens to the activity in the brain of people getting acupuncture?"
- "Could healing affect the immune response or metabolism in cancer cells?"
Applied research aims to put forward knowledge that can be used for specific practical purposes. Applied research builds on, among other things, knowledge from basic research: A better understanding of what is going on in the pancreas of people with diabetes, may contribute to new ideas for research that can provide better medicines with fewer adverse effects.
Applied research in the field of health and disease may give answers to the folloing questions:
- "Can medication X reduce the number of epileptic seizures?"
- "Can the intake of selenium prevent prostate cancer?"
- "Can acupuncture reduce post-operative nausea in children?"
- "Does thought field therapy (TFT) have an effect on social anxiety?"
- "Can healing contribute to cancer patients who are recieving chemotherapy, becoming less susceptible to infections?"
Systematic reviews
A systematic review is a review of all the research, for example what the science says about the efficacy of a specific treatment.
Systematic reviews are carried out by researchers gathering all studies regarding treatment they will research, and take with them only those of good enough quality. The researchers then examine what these studies collectively says about the effects of the treatment. This way one gets a quality assured overview of what we know about the efficacy of the treatment on a specific health problem. NAFKAM reports the efficacy of therapies, herbs, foods, diets and supplements based on the conclusions from systematic reviews.
Systematic reviews has several names in Norwegian. Mostly used are the terms kunnskapsoppsummering, oversiktsstudie, metastudie and oppsummert forskning.
In some cases the researchers can only find one study that fulfills the inclusion critera. In these cases the systematic review does not give us results on the highest level (1a), but on the level of the type of individual study included.
Oxford Center for Evidence-based Medicine has a overview of the levels of evidence .
Clinical efficacy study
A clinical efficacy study is an experiment where one examines whether a specific therapy/medicine has an effect on humans with a specific disorder. For such a study to provide a definite answer, there must be many participants in the study who are randomized to one test group and one control group. This type of studies are called randomized controlled trials (RCT).
Typical research questions in clinical effect studies may be:
- "Does acupuncture have effect on morning sickness?"
- "Will natto supplements reduce the risk of fractures?"
Pilot study
Large-scale research projects are often very costly and time consuming. Before one starts big projects one usually cunducts a pilot study. This a test with the intent of examining whether the study design works and whether it is possible to get enough participants.
A pilot study involves only a small group of participants (also applies to animals). Through the pilot study researchers can identify any deficiencies and weaknesses with the planned research project, and get an indication of whether if it has a purpose to initiate a large-scale study.
Research on mechanisms of action
A study of mechanisms of action does not examine whether a treatment works, but how it works through psychological or bodily mechanisms. The starting point might be that other research has shown that a specific treatment has an effect on a specific disorder, but without being able to explain how it works. This is a method used to get knowledge about how of why a treatment works.
An example of a typical research question might be:
- "Which biological and/or physiological mecanisms might explain the positive effect on nausea by acupressure treatment?"
Research on user experiences
In research on user experiences one might be interested in how patients themselves mean a treatment works. This is about patients' subjective opinions, and not what objective methods can tell us about the treatments effect. This type of studies gives in-depth knowledge about patients' individual experiences og the effect og a treatment, both on their quality of life and the symptoms.
User surveys can be combined with clinical efficacy studies to get knowledge avout the relationship between objective, measurable efficacy and experienced efficacy.
An example of a typical research question in user surveys within CAM may be:
- "How do cancer patients experience the meaning of daily meditation on their perception of quality of life?"
Quantitative og qualitative research
In quantitative research a phenomenon is measured and counted, in relation to another phenomenon. One example is examining to what degree there is a connection between lifestyle and high blood pressure. In such a study the researchers must have to measure the blood pressure of a large number of people who also answer a number of questions about ther lifestyle, from diet and exercise to working conditions, use of tobacco and alcohol.
Through statistical analyses the researchers can describe the connection (association) between lifestyle and blood pressure in the test subjects. If the researchers wish to examine whether exercise lowers blood pressure they must do that in separate studies with some duration over time, or in an experiment (RCT).
An example of a research question demanding a quantitative method may be:
- "Does exercise have an effect on the blood pressure of men over 50 years of age?"
In qualitative research the aim is often to gain knowledge that is hard to measure or count. Qualitative reserach methods is used when we want to get in-depth knowledge about a phenomenon. If we are searching for a connection, as above, between lifestyle and blood pressure, a qualitative interview study might give an elaborate description of the different parts of the lifestyle of the test subjects with high blood pressure. These elaborate descriptions may for example contribute to more precise questions and definitions of concepts in a possible quantitative part of the study study.
Qualitative research method are suitable when researchers are aiming at understuanding human relations and interaction.
Examples of research questions where qualitative research methods are suitable are:
- How does serious illness affect young people's self-confidence?
- How do patients communicate with different professions within the health care system?
Qualitative and quantitative research sheds light on different sides to a phenomenon, like the prevalence of lifestyle related high blood pressure on one side, and how people describe and substantiate their lifestyle on the other side. In an increasing number of studies quantitative and qualitative method as combined, to get knowledge both about the prevalence of a phenomenon, as well as a theory about the cause(s) of the phenomenon.
Research in vitro
During in vitro studies an isolated part of a whole is studied. Examples of this might be cells, cell parts, organs and tissue studied outside the human body. In the development of new medicines the researchers often start with in vitro research: They study, for example, how cancer cells in a tube reacts when a specific medicine is added.
It is important to know that even though the research results are positive in vitro, there is still a long way from there to knowing how the medicine affects cancer cells in the human body. One does not know how it will affect other cells and processes in the body, or which dosage is the best. Research in vitro must always be followed up by in vivo research, where the medicine is tried on the whole organism. Ususually in vitro studies are followed up by in vivo studies with animals, before in vivo studies on humans are conductet
Research on animals
Some parts of the research with the field of health and disease will always have to be done on animals, before one moves forward and does the research on humans. For example when exploring medicines. The reason behond this is the strict rules of law regarding which experiments one expose humans to.
Testing new medicines or surgical techniques always carry a risk for adverse effects. Animal testing is therefore used in the early stages of the research where possible mechanisms of action and toxity can be examined. When a medicine or treatment method shows positive results and few adverse efects in animal tests, the next step can be to test this on humans.
Results from animal tests are not necessarily transferable to humans. Even though animals are very similar to humans there is still such big differences betwen species that one can never use any treatment on humans, based solely on animal testing.
Research on effects of the system vs effects of the component
Researching the system effect is about examining whether the treatment with all components has an effect. Then one can examine whether seeing a homeopath has an effect on haayfever, in relation to not seeing a homeopath.
If one wants to examine whether one specific homeopathic remedy has an effect on hayfever, one are examining the effect of the component in the treatment. In the case of homeopathy for hayfever a study on the effect of the component would be done by giving one group of people with hayfever a specific homeopathic remedy, and another group (control group) gets placebo medicine, and then compare the results.
Resrearch on the system effect might answer if there is something else and/or more that affects the results that the specific treatment component. It may be about the relationship between the therapist and the test subject, or about lifestyle advice given to the test subject as part of the treatment.
Randomizert controlled study (RCT)
An RCT is an experiment which can be used both for examining the system effect and effect of the component in a treatment. This type of studies can be used to examine the effect of a medicine, a therapy (acupuncture) or a treatment strategy (for example the follow-up of patients using telemedicine).
In an RCT the test subjects are randomized to an intervention group or to a control group. The research question decides which treatment the different groups get. An example: The people in the intervention group gets active medicine, acupuncture or follow-up with telemedicine, while the people in the control group will get placebo, no treatment or regular medical treatment. Placebo-controlled studies are those where test subjects get the placebo medicine.
The groups that are compared must be as like as possible in regard to factors that may affect the results of the experiment. These factors can be age, gender, lifestyle, other diseases or diet. When the study is concluded the results are gathered from both groups, and compared to see whether the treatment in the intervention group had a greater effect than what could be observed in the control group.
If the aim of the study is examining the effect of a medicine, the study can be double blind. This means that neithe the test subject or the therapist knows who gets the active medicine or who gets placebo. In some studies however, it is difficult to do an experiment without the therapist knowing who gets the active medicine, Then the study is blinded, not double blinded.
In these cases it is important that the analyzer is blinded and does not have knowledge of who the test subjects are or which treatment they got.
Examples of research questions in randomized controlled studies in the field of health and disease might be:
- "Can the supplying of vitamin D to women treated for breast cancer enhance their survival?"
- "Can acupuncture therapy reduce hot flashes with menopausal women?"
- "Can orthopedic patients be followed up as well by telemedicine as with a regular consultation?"
- "Which package of treatment works best for people with a stiff neck? Seeing an acupuncturist, a homeopath or a doctor?"
Case study
In a case study, the focus on research of one or few subjects or phenomenons. The purpose ot this type of study can be to get in-depth knowledge about a phenomenon with the intent of developing a theory or testing a hypethesis on a small group, before incvesting in a large-scale study on a represenative selcetion.
In the first case the interest might be in developing a theory about possible psychosocial causes of the development of fibormyalgia. Then it will be purposeful to do in-depth inteviews with a few selected people with the diagnosis fibromyalgia, to map their life and health situation in the years before the diagnosis.
In the other case one might have a hypothesis about regular visualization exercises over time, might change patterns of thought to a degree that patients ways of thinking and acting is changed. To check if there might be something to this hypothesis one can start by testing it on a few people. A case study might, in other words, also be a pilot study.
Epidemiological studies
Epidemiology is the science studying the prevalence of diseases in populations and which factors that are affecting the prevalence. Epideiologi research is always done on populations. By a population one means groups of people with something in common. Sometimes these are people living in the same geographical area, other times it might be people who smoke, a specific profession og people who work out alot.
Epidemiological studies are often divided in two main groups: Descriptive and analytical epidemiology.
Descriptive epidemilogy examines and describes the prevalence of morbid conditions in a population. Descriptiv epidemiology provides an important basis for research on causalities, what is calles analytical epidemiology. The Cancer Registry of Norway is one example of data/information that can be used for descriptive epidemiology. It can provide important knowledge which can be used in analytical epidemiology to identify possible risk factors for the development of cancer.
Analytical epidemiology examines possible causalities. For example examining what could be possible reasons why Norway is one of the countries in the world with the highest prevalence of multiple sclerosis (MS) an osteoporosis. Analytical epidemilogical research questions usually occurs on the background of the results from descriptive epidemiological studies.
There are two main groups of analytical epidemiological study designs: Observational and experimental epidemiological studies.
Observational epidemiological studies:
A case-control study is an observational epidemiologic study where one compares a group of people witha a disease or symptom, to a group of healthy people. The healthy people are the control group. The intent of this type of study is to find the factors that may have made the sick people sick. This is called a retrospective study because they start with the people who are already sick (and healthy people being the control group for comparison) and then they gather information about the sick people, what they have experienced before they became ill. If all the sick people did something of the same, which the control group did not do, then one can formulate a hypothesis about this being a possible cause of disease.
One of the biggest sources of error in case-control studies is that it is very hard to make the selection of healthy control subjects who are representative for all healthy people. The group "healthy people in Norway" (eg.) is not a uniform group. Another significant source of error is the fact that information about possible causal factors are gathered after the sick people have become sick. People's memory is not always great, and it might be that the sick and the healthy have varying reasons for "twisting their brain" for information from way back in time.
Cohort studies
A cohort study is an observational epidemiologic study testing a hypothesis, for example from a case-control study. In a cohort study one follows a group of people over time, most of them healthy. The group must be subject to health examinations several times during the research period which may last for many years. In addition, they must answer a line of questions about their health, diseases in their family, lifestyle, life situation and so on. The Tromsø study is an example of a large cohort study. They are interested in looking at who develops which diseases, and if they have anything in common in the time before the breakout of the disease.
Another example: Researchers may have a hypothesis that women working the night shift has an increased risk of breast cancer. They will then follow a relatively large group of women over a long period of time, to see if they who work nigh shifts develops breast cancer to a greater degree than their day-working fellow sisters.
The biggest challenge for researchers in this type of studies is to spot other possible factors the group might have in common, and which might the actual risk factor.
In ecological studies one examines if the prevalence of a phenomenon (eg. a disease, a symptom of a type of behavior) in a population is connected to the prevalence of something assumed to be the cause of the phenomenon (eg. a type of behavior or environmental influence).
Examples of possible ecological studies:
A study comparing mortality from breast cancer in one country to the sales of birth control over a period of time.
A stuy comparing the prevalence of oseoporosis to the consumption of dairy products in a population for a specific time period
Ecological studies are so-called hypothesis forming studies. This means that if one finds h high prevalence of osteoporosis in a population with high consumption of dairy products, it can form a basis for several new hypotheses and new research projects. Then one can examine why milk, containig high amounts of calcium, apparently does not seem to have a preventive effect on osteoporosis, or examine why the population in question can't make use of calcium from dairy products.
Experimental epidemiological studies:
In this type of studies the effect of influence on a population or group is examined. Unlike observational studies, the researchers controls what the test subjects are exposed to. One can for example run information campaigns to promote a healthy diet and more exercise in the population. To check if it has had an effect on the health of the population, one can examine one ore more representative and randomly selected groups in the population, before, and a while afte the campaigns. It is also possible to choose a group of the population, part it in two different and randomly selected groups. One group sees the campaign, while the other works as a control group.
