Glen A. Fox / Journal of Toxicology and Environmental Health, 33:359-373, 1991
Glen A. Fox
Wildlife Toxicology and Surveys Branch
Canadian Wildlife Service, Environment Canada
Ottawa, OntarioEnvironmental scientists and managers must determine whether a relationship between an environmental factor and an observed effect is causal and respond accordingly Epidemiologists have, over the past 150 yr, developed a systematic approach to evaluating these relationships. Their criteria for objectively evaluating the relationship between a suspect cause and a chronic disease are (1) probability, (2) time order, (3) strength of association, (4) specificity, (5) consistency on replication, (6) predictive performance, and (7) coherence. These criteria can be used, with little modification, to evaluate associations in relation to diseases in fish and wildlife suspected to be caused by exposure to chemical pollutants. Some populations of fish and wildlife are members of the same guilds as subpopulations of humans. Investigations of chemically induced disease in these sentinel populations of fish and wildlife may identify the potential risks posed to these human subpopulations. Evidence evaluated using the epidemiologic criteria may assist environmental managers to determine whether a substantive case can be made to initiate preventative or remedial action. By applying the null hypothesis, scientists are forced to consider how much information must be ignored to conclude that a causal relationship does not exist.
I thank John M. Last, Professor of Epidemiology and Community Medicine, University of Ottawa, for his stimulating introduction to the science of epidemiology and for encouraging me to apply it to my wildlife studies. M. Gilbertson, T. M. Yuill, and J. M. Last made many useful comments on an earlier draft of this manuscript.
Requests for reprints should be sent to Glen A. Fox, Canadian Wildlife Service, National Wildlife Research Center, Environment Canada, Hull, Quebec, Canada K1A OH3.
Our Laurentian Great Lakes are the world's largest freshwater ecosystem; they contain 18% of the world's fresh surface water and 80% of North America's supply. Approximately 30 million people, one-sixth of the combined population of Canada and the United States, live within the Great Lakes Basin and approximately 25 million get their drinking water from the Great Lakes and the St. Lawrence River. In addition, the Great Lakes provide water for domestic uses, sanitation, industrial processes, agricultural operations, hydroelectric power generation, navigation, recreation, and habitat for aquatic wildlife. The influx of biological and chemical wastes from municipal, industrial, and agricultural sources has adversely affected the quality of the Great Lakes. There are about 150 major abandoned hazardous waste sites that have been identified in the Great Lakes Basin, and the chemicals from many find their way into the lakes. There is a growing public concern over the occurrence of toxic chemicals in air, soil, water, and food chains in the Great Lakes Basin. Forty-two areas of concern have been identified where chemical contamination has resulted in the loss of beneficial uses of the area; remediation of such sites is a complex and costly process. At the present time, about 30,000 compounds of commercial and industrial significance are used in the Great Lakes Basin and at least a thousand new compounds are introduced each year. It is little wonder that about 500 toxic compounds have already been identified in the waters of the Great Lakes. Many new contaminants are being discovered as the detection limits for hazardous substances are lowered by improvements in our analytical capability.
In 1987, commercial fishermen landed 89 million pounds of fish from the waters of the Great Lakes, with a regional economic impact in excess of $260 million. Sports fishermen spent in excess of 60 million angler days annually, fishing in Great Lakes waters, resulting in a regional economic benefit of about $2 billion. However, frequent sport fish consumption advisories and lost marketing opportunities for the commercial catch have resulted from toxic chemical contamination of Great Lakes fish. Evidence is accumulating that suggests that chemicals in Great Lakes food chains are toxic to wildlife, inducing reproductive toxicity and other physiological problems in birds and tumors in fish. The list of species affected comprises three fish, one reptile, eight birds, and two mammals. Evidence is also accumulating that these chemicals may pose a hazard to humans, a possibility that points to an urgent need for further investigation.
The economic and global ecological significance of the Great Lakes ecosystem, the size of the human population at risk, observations of adverse effects in fish and wildlife confirming the presence of toxic substances in Great Lakes food chains, and our limited understanding of their behavior and effects continue to generate widespread concerns about the ability of industry and governments to ensure the safety and health of the inhabitants of the Great Lakes ecosystem. The subtlety of many of the effects seen in experimental studies has meant a high degree of uncertainty and skepticism about the significance of the results and confusion about regulatory approaches that should be adopted. The Great Lakes and St. Lawrence River are bordered by eight states and two provinces, which, along with the two federal governments, assume some responsibility for managing the Great Lakes ecosystem. A wide variety of environmental laws and regulations exists within these jurisdictions. Because society cannot control all pollution sources completely, rational choices will have to be made between them and about the degree of control that is most desirable. Although the social and monetary costs of regulation and remedial action may be great, the potential societal and ecological costs of inaction are far greater.
What are decision makers to do when a problem is serious and urgent, the cause of the problem uncertain, and the data incomplete? Somehow we need to draw together all the disparate threads of evidence and make them into a coherent whole, so that scientifically and socially defensible regulatory decisions can be made. When faced with similar problems during the 1848-1854 cholera epidemic in London, John Snow successfully tested hypotheses to establish how the disease was spread and to identify common events that linked patients with cholera and distinguished them from healthy controls. He hypothesized that cholera was transmitted by water contaminated with feces from infected humans. (Microbial agents were not yet recognized-it was not until 1883 that Koch identified the cholera vibrio). In testing his hypothesis, Snow demonstrated that persons who consumed water, heavily polluted with sewage, drawn from the Thames River in central London had a rate of cholera that was significantly greater than that observed in consumers of less contaminated water drawn upstream. In one outbreak in London, he established that most victims drank water from a single point source, the Broad Street pump. He removed the handle from the pump and halted the epidemic. That intervention experimentally confirmed his hypothesis. John Snow's achievement was based on his logical organization of observations, his recognition of a natural experiment, and his quantitative approach in analyzing the occurrence of a disease in a human population. Within 2 yr of the submission of his findings, legislation was passed requiring that all of the water companies in London filter their water. Snow is considered the father of modern epidemiology, and his methods, the essence of which is the gathering of population-based observations that can compare disease rates between groups with different exposures, are still applied today in the search for causes and means of preventing diseases where no etiological agent has been identified. The usefulness of epidemiology as "a technique for taking a first pass at a new problem" makes it applicable to a wide range of interesting phenomena (Fraser, 1987). Recently, epidemiological methods have helped to explain diverse phenomena including child murders, deaths due to heroin, clusters of deaths associated with improper medical practices, and the risk of death or injury from such environmental hazards as hot weather, volcanic eruptions, and tornados (Glass, 1986). Why not apply epidemiological methods to clarify our understanding of many of the apparent associations observed between chemical contamination of Great Lakes food chains and adverse effects in fish, wildlife, and humans? The Henle-Koch postulates have been used for 100 yr in the evaluation of causal relation ships involving infectious agents, and have served as a model upon which modern concepts of causality are based.
The following definitions describe both the "process" and "scope" of epidemiology:
A sequence of reasoning concerned with biological inferences derived from observations of disease occurrence and related phenomena in human population groups. (Lilienfeld and Lilienfeld, 1980)
The study of the distribution and determinants of disease and health related status in populations and the application of results to the prevention and control of health problems. (Last, 1983)
Epizootiology is the nonhuman animal equivalent.
The definition of disease has been universalized by the modern pathobiologist to include any failure of normal homeostatic processes at any level of biological organization.
The objectives of epidemiological studies of the effects of environmental agents on health include:
To assess risk to the public's and ecosystem's health from exposure to environmental hazards.
To provide decision makers and health workers with the information needed for the establishment of health criteria and programs for the control of pollution and other environmental hazards.
To assist in evaluating efficacy of preventive and remedial measures.
To improve scientific knowledge.
Thus, epidemiology is expected to provide the bulk of the answers that the citizen, workman, employer, government, and scientist need concerning the relationship between various aspects of the environment and health.
Epidemiologic methods can be applied to the study of any phenomenon within a population. Bro-Rasmussen and Lokke (1984). have coined the term ecoepidemiology for the "causistic discipline describing ecological disturbances and damages in relation to their specific causes."
ecoepidemiology is the study of the ecological effects that are prevalent in certain localities or among certain population groups, communities, and ecosystems and their potential causes.
According to Bro-Rasmussen and Lokke, ecoepidemiological studies concentrate on the description of effects, identification of causes, and determination of links and pathways between these. Man is considered part of the environment in ecoepidemiological studies, and damages would include diseases in individuals and species, as well as populations, disturbances in communities, and disruptions of ecological systems.
In a study based on group or population characteristics, the observer compares effects in different populations or groups in the hope that he or she can relate the observed differences to differences in the local environment, life style, etc., of these populations. Such relations or ecological correlations provide clues to causal or etiological hypotheses, which may then be tested in individuals (Lilienfeld and Lilienfeld, 1980).
It is generally accepted that statistical associations derived from well-controlled experimental studies represent causal relationships. In epidemiology, however, most studies are observational, and an experiment to identify a cause-effect relationship may be difficult or impossible to carry out. Policy decisions affecting public health and preventive medicine may be made on the basis of observational evidence alone. It is important, therefore, to have some basis for deciding whether a statistical association derived from an observational study represents a cause-effect relationship (Friedman, 1980).
The logician's definition of "cause;" that a factor must be both necessary and sufficient before it can be considered causal, implies that there must be a one-to-one relationship between a factor and its effects. We know that this is not necessarily the case even in infectious diseases. Lilienfeld and Lilienfeld (1980) suggested that, in medicine and public health, it would appear reasonable to adopt a pragmatic concept of causality:
A causal relationship would be recognized to exist whenever evidence indicates that the factors form part of the complex of circumstances that increases the probability of the occurrence of the disease and that a diminution of one or more of these factors decreases the frequency of that disease.
This concept of causality is obviously different from that applied in law or philosophy. However, in prevention, it is necessary to identify an exposure without necessarily identifying the ultimate cause of the disease. Epidemiology frequently provides a basis for action despite ignorance of mechanism. It was necessary to know that polluted water was a major vector of cholera; lack of knowledge that a specific bacterium is the causative agent did not prevent authorities from introducing legislation mandating that all water companies in London filter their water, thus greatly controlling the disease. Similarly, cigarette smoke has been identified as the contaminated vehicle that is associated with increased rates of lung and other cancers, and heart and respiratory disease. It was not necessary to identify precisely which component in the smoke is the prime offender before instituting preventive measures.
The criteria published in 1964 by the Surgeon General's Advisory Committee on Smoking and Health (U.S. Department of Health, Education, and Welfare, 1964) for objectively evaluating the relationship between a suspect cause and a chronic disease have been generally accepted by the epidemiologic community. Their five criteria, (1) consistency of the association, (2) strength of the association, (3) specificity of the association, (4) temporal relationship of the association, and (5) coherence of the association, are identical to those published by the American epidemiologist Susser (1986a). Similarly, Susser's five criteria incorporate all nine of the aspects of an association that the eminent British epidemiologist, Sir Arthur Bradford Hill, felt should be considered in causal judgments by those practicing occupational medicine (Hill, 1965; also see Table 1). Susser (1986b) introduced two additional criteria, (6) probability and (7) predictive performance. Evans (1976) has presented 10 criteria for causation that provide a unified concept for considerations of causation for both acute and chronic disease.
The epidemiologists' criteria do not provide the kind of proof demanded by the experimentalist but form a process and framework upon which one can build a balanced judgment. They provide no guidance in weighing one criterion against any other or for judging the strength of the support provided by the evidence; we can only argue that, using these criteria, our evidence supports or detracts from a causal judgment. Of these criteria, only four, (1) strength of the association, (2) consistency of the association, (3) predictive performance, and (4) statistical coherence in the form of a monotonic dose-response relationship, strongly affirm causality. Similarly, only (1) incompatibility on the basis of time order or (2) factual plausibility, and (3) lack of consistency upon replication, detract from causality sufficiently to reject a causal hypothesis with confidence. The remaining criteria are indeterminate (Table 2). Application of these criteria is of greater assistance in negating causal hypotheses than in confirming them. Causal inference is an informal and subjective rather than a rigorous judgment. Philosophers have taught us that complete logical certainty is not available in science, hence our results will often be inconclusive, and the best we can expect to do is to reach the most reasonable explanation based on the evidence at hand. Ziman (1978) reminded us that "the aim of research is to publish a scientific result of adequate plausibility, not complete proof."
Susser (1986b) was the first to suggest that probability be considered as one of the criteria for causation. Statistical significance may help us decide how much attention to give to a particular result. However, as Susser pointed out, "lack of significance gives quantitative but not logical grounds for rejecting an epidemiological hypothesis." Statistical power must always be considered. However, the demonstration of a statistical relationship between observations of a disease and biological or social characteristics is but the first step in the epidemiologic analysis of its etiology and/or natural history. The second step is to ascertain the meaning of the relationship. Unfortunately, the word "significant" in statistically significant is often misinterpreted as representing the biological significance of the association. According to Hill (1965),
No formal tests of significance can answer these questions. Such tests can, and should, remind us of the effects that the play of chance can create, and they will instruct us in the likely magnitude of those effects. Beyond that they contribute nothing to the "proof" of our hypothesis .... Yet too often I suspect we waste a great deal of time, we grasp the shadow and lose the substance, we weaken our capacity to interpret data and take reasonable decisions whatever the value of P And far too often we deduce "no difference" from "no significant difference."
Interpretation of such an association must be conducted in a systematic manner.
Time-order refers to the necessity that the cause precede the effect in time. This is an essential criterion that stems from the definition of cause. However, it must be remembered that in the case of chronic diseases, the timing and nature of the initial events are often obscure, and long latent periods may exist between cause and effect. Determinants that are personal attributes or predisposing factors may change with time and will compromise knowledge of time order.
By strength of association we refer to
the degree to which the supposed cause and outcome coincide in their distribution, or
the size of effect produced by the presumptive cause, or
the relative risk.
In epidemiology, the most common measure of the strength of the association is relative risk: the ratio on one side of incidence of a disorder in the population exposed to the suspect causal factor, and on the other side, the incidence of the disorder in a comparable population not exposed to the suspect factor. The larger the relative risk for any hypothetical cause, the more likely it is to be causal. Weak relationships are not uncommon in environmental epidemiology. They are susceptible to confounding, and their weakness may reflect a poor measure of exposure or outcome. However, they may have tremendous public health significance.
On a more practical note, the strength of association is relevant to the genesis of the causal hypothesis. It seems unlikely that a suspicion of an effect could be raised unless the association was remarkable enough to be noticed (Hatch and Stein, 1986). The initiating event in almost all epidemiological investigations has been the spontaneous and voluntary reporting of an unusual event. Frequently, the initial observations that ultimately lead to an understanding of the relation between a chemical and an illness derive from a small number of observations made by one or more physicians practicing medicine.
Specificity refers to the precision of the association between X and Y. Does X lead only to Y (specificity of effect), or does only X lead to Y (specificity of cause)? Specificity in the causes of a given effect enhances the plausibility of a causal claim, even when there is a low relative risk. However, it is by no means a requirement.
Specificity in the effects of a given causal factor does little to strengthen a causal claim. It is illogical to expect causes of a given effect to be without other effects. Everyday experience teaches us repeatedly that single events may have many effects. Current biological knowledge suggests that
Many diseases have multiple causes.
A single environmental factor may be a vehicle for several different substances.
A single pure substance in the environment may have a number of different biological effects with different underlying mechanisms.
A single biochemical event may result in a diverse array of biological responses.
Has the association been repeatedly observed by different persons, in different places, circumstances, and times? A hypothesis gains credibility if it is supported across studies in different times, places, and populations, by different investigators, with different research designs.
The epidemiologist and ecotoxicologist deal with individuals in changing societies in an ever-changing world where the conditions of study are constantly changing. Opportunities for exact replication by means of which the physical scientist demonstrates consistency and rules out chance are not available to the epidemiologist-his alternative to exact replication is the consistency of a result in a variety of repeated studies. In ecoepidemiology, the occurrence of an association in more than one species and species population is very strong evidence for causation.
However, there are many situations where repetition is impossible. In the case of the exposure to chemical wastes of residents of the Love Canal area, hundreds of chemicals were dumped, the majority of them unidentified. When a study of birth weight records showed lower birth weights in offspring of Love Canal residents (Vianna and Polan, 1984), there was no possibility of repeating a study with those particular, undefined, exposures.
We must also keep in mind, however, that lack of consistency does not rule out a causal association, since some effects are produced by their causes only under unusual (and as yet not understood) circumstances.
Predictive performance relies on the testing of a deduction. The criterion requires that a hypothesis drawn from an observed association predicts a previously unknown fact or consequence, and must in turn be shown to lead to that consequence. Predictive performance is a strongly affirmative criterion, particularly when it produces new knowledge.
The cause-effect interpretation of our data should not seriously conflict with the generally known facts of the natural history and biology of the disease. Coherence is comforting; incoherence by itself is often not destructive of a hypothesis but emphasizes gaps in scientific understanding.
Although a statistically significant association must be present before any relationship can be said to exist, only biologically plausible associations can result in "biological significance" (Lilienfeld and Lilienfeld, 1980). However, judgments on this basis are bound by the imperfect knowledge existing at any time. An association that does not appear biologically credible today may prove to be so tomorrow; indeed, the observation of a seemingly implausible association may initiate the extension of our knowledge. As Sherlock Holmes advised Dr. Watson, "when you have eliminated the impossible, whatever remains, however improbable, must be the truth."
There was no biological knowledge to support or, for that matter, refute the observation in 1776 by the English physician, Percival Pott, of the excess of scrotal cancer (so-called "soot wart") in chimney sweeps, which he believed was directly due to the infiltration of soot into the skin of the scrotum. Based on Pott's observations, the Danish Parliament required chimney sweeps to bathe daily, and thus directly reduced exposure and the incidence of scrotal cancer. This was 150 yr before experimental work on chemical carcinogenesis was begun, work that has shown numerous associations between the PAHs in soot, coal tar, and petroleum products and carcinogenesis.
Biological common sense contributes greatly to the elucidation of the pathways and mechanisms by which a cause may take effect. The search for biological plausibility commonly draws on the findings and experiments from other species. While useful in constructing mechanistic theory, such evidence can also bear strongly on causal inference. For example, teratogens found to affect humans have also been found to be teratogenic in at least one other animal species, and show some manifestation of developmental toxicity in most species that have been tested. However, as a general rule, extrapolations across species require a knowledge of species-specific physiology.
Any response proportional to dose is suggestive of a causal relationship. However, causal relationships need not be linear or monotonic; in some, there is a marked threshold, others are sigmoid, and yet others are parabolic. If the association is one that can reveal a biological gradient or dose response, we should look carefully for such evidence. If the response is measured as a function of comparative incidence, this condition will ordinarily be met. However, there is often difficulty in securing a satisfactory quantitative measure of the environmental variable or exposure to it, which will allow us to explore this dose response. Relative intensity and duration of exposure are frequently used surrogates of dose. It must be noted that some causal associations, such as the association between a mother's exposure to diethylstilbesterol and the occurrence of adenocarcinoma of the vagina in her teenage daughter, show no apparent trend of effect with dose. Hence, although the presence of a regular dose-response relationship is highly supportive of the hypothesis, its absence has little bearing on whether an association is causal or not.
Experimental evidence is often telling but seldom available for free-living populations prior to remedial action. However, the results of interventions or remedial action, in terms of altered frequency or intensity of the associated events, can provide the strongest support for the causal hypothesis.
In concluding this process, we must ask: Does the evidence suggest that the relationship is real, probably causal, and biologically or ecologically significant enough to warrant the attention of those of us engaged in public health or ecoepidemiology, and do the relative risks pose enough of a problem to initiate preventive or medial action? In making such a decision, we must ask ourselves, "what evidence must I ignore to conclude that a causal relationship does not exist?"
Hill reminds us that
All scientific work is incomplete-whether it be observational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. That does not confer upon us a freedom to ignore the knowledge we already have, or to postpone the action it appears to demand at a given time. (Hill, 1965)
We are continuously brought back to the fundamental question-what alternative explanation will fit a set of observations, what other differences between our contrasted groups could equally, or better, account for the observed incidences. That is the crux of the matter-and no X2 test or other application of the Greek alphabet will answer it. It demands an experience and acumen, in what to collect, how to seek in the data for essentials, how to interpret. And that . . . demands an apprenticeship in the subject-matter. [italics mine] (Hill, 1962)
Today, unlike public health workers examining possible relationships involving the etiology of infectious disease, ecoepidemiologists frequently find themselves asking, "How is it that our society has denied awareness of environmental degradation, often using the language and models of science to support this denial?" The "acid rain debate" is a perfect example. In their recent review "Forest Decline and Acidic Deposition;" which appeared in the journal Ecology, Pitelka and Raynal (1989) suggested that "it is essential to identify the specific pollutants and mechanisms that are responsible;" implying that a tight causal proof is necessary before instituting controls that could protect forests from destruction by pollution.
To this, the eminent American ecologist George Woodwell (1989), in the same issue, countered:
If such proofs are required, there is little that scientists can do to prevent the destruction of forests and other resources. Tight causal proofs are elusive in ecology as in most science. While we all share the desire to be able to define the myriad of strands of cause and effect in the destruction of forests by acid rain and air pollution, science can never be large enough to do so; it is a futile wish, at least for most issues for the moment. Reason and the scientist's commitment to objectivity require acknowledgement of uncertainty; the acknowledgement is often used to block clear advice.
Pitelka and Raynal would like to stop the impoverishment of the world's forests by pollution, but said they don't know how. Woodwell countered:
They know much more than they admit and they have more strength than they recognize. Scientists and other scholars know much about forests, the details of the causes and effects of pollution, and the costs to society of losing forests over such areas .... Scientists always feel more comfortable with data as opposed to experience, but in making the judgments as to how to manage resources, it is experience that counts. The complicated details of the chemistry of life that keep forests vigorous are worthy of detailed pursuit by our best minds .... They tell us how nature works; they confirm that there is a difference of several orders of magnitude between what we know about nature and what we know about managing our wastes .... Such answers are part of the stuff of scholarship, but not all of it. Much of the stuff is experience, the basis of projections and the basis on which we must proceed with management in a world growing rapidly smaller. Data may be preferable, but experience is what we have to work with .
. . . And the experience is rich, rich enough to provide ample basis for effective action now . . . . The response to the challenges of pollution is cleaning up the mess; containing the effects of human activities to preserve the physical, chemical and biotic integrity of the earth. We know enough to do that; we know why and how and how much, and it is long past time that we emphasized the inadequacies of current efforts .... We err in not clarifying the need and in accepting permissive limits based on narrowly focused expenses or risks to human health without consideration of the much more stringent requirements for avoiding biotic impoverishment .
. . . The spirit of compromise is often commendable. There are places where compromise is not appropriate, or even possible. One is our scholarship, which we work hard to keep objective, uncontaminated by worldly considerations. Another is our stewardship of the earth's biota. [italics mine]
As Beland (1988) put it in his thought-provoking article on the plight of the St. Lawrence Belugas,
Searching for definite proofs in a traditional sense is illusory and should not bog us down into inaction when the survival of an important element of our environment is at stake.
The very wise science of statistics teaches that you can make two types of errors: in type 1, you reject evidence for lack of sufficient proof, even though the case is good; in type 2, you accept the evidence, although it later turns out that the case was wrong.
The present state of our planet results from society having consistently preferred to guard itself against the first type of error. I suggest that type 2 errors would, in the long run, have proved less costly in social and economic terms. [italics mine]
I wholeheartedly agree! Should we not in fact be willing to risk the Type 2 error in our decision making-be willing to take the risk that the findings we accept as true turn out not to be true-rather than learn that we were not protective enough of human health and planet Earth, the only place in the universe known to sustain life?
Beland, P 1988. Witness for the prosecution. Nature Can. 17(4):29-36.
Bro-Rasmussen, F., and Lokke, H. 1984. Ecoepidemiology-A casuistic discipline describing ecological disturbances and damages in relation to their specific causes; Exemplified by chlorinated phenols and chlorphenoxy acids. Reg. Toxicol. Pharmacol. 4:391-399.
Evans, A. S. 1976. Causation and disease: The Henle-Koch postulates revisited. Yale J. Biol. Med. 49:175-195.
Fraser, D. W. 1987. Epidemiology as a liberal art. N. Engl. J. Med. 316:309-314.
Friedman, G. D. 1980. Primer of Epidemiology, 2nd ed. New York: McGraw-Hill.
Glass, R. I. 1986. New prospects for epidemiologic investigations. Science 234:951-955.
Hatch, M. C., and Z. A. Stein. 1986. Agent Orange and risks to reproduction: The limits of epidemiology. Teratogen. Carcinogen. Mutagen. 6:185-202.
Hill, A. B. 1962. The statistician in medicine. J. Inst. Actuaries 88:178-191.
Hill, A. B. 1965. The environment and disease: Association or causation? Proc. R. Soc. Med. 58:295300.
Last, J. M., ed. 1983. A Dictionary of Epidemiology New York: Oxford University Press.
Lilienfeld, A. M., and Lilienfeld, A. B. 1980. Foundations of Epidemiology, 2nd ed. New York: Oxford University Press.
Pitelka, L. F., and Raynal, D. J. 1989. Forest decline and acidic deposition. Ecology 70:2-10.
Susser, M. 1986a. Rules of inference in epidemiology. Reg. Toxicol. Pharmacol. 6:116-128.
Susser, M. 1986b. The logic of Sir Carl Popper and the practice of epidemiology. Am. J. Epidemiol. 124:711-718.
U.S. Department of Health, Education, and Welfare. 1964. Smoking and Health. Report of the Advisory Committee to the Surgeon General. Washington, D.C.: Public Health Service Publication 1103.
Woodwell, G. M. 1989. On causes of biotic impoverishment. Ecology 70:14-15.
Vianna, N. J., and Polan, A. K. 1984. Incidence of low birth weight among Love Canal residents. Science 226:12171219.
Ziman, J. 1978. Reliable Knowledge: An Exploration of the Grounds for Belief in Science. Cambridge: Cambridge University Press.
Received June 30, 1990
Accepted March 28, 1991
|
If you have come to this page from an outside location click here to get back to mindfully.org |
Theoretical
Biological