Andy Stirling, Ph.D., Sue Mayer, Ph.D.
There are strong scientific reasons for holding the broader scope of precautionary approaches to be more consistent with the scientific foundations of rational choice and probability theory than are conventional narrow risk assessment techniques. The imperatives both of science and precaution can be seen to pull in the same direction. The regulatory appraisal of risk should become more systematic and broader in scope. In particular, a set of criteria can be developed concerning the need for greater humility, completeness, transparency, and participation in regulatory appraisal, with specific attention to the comparison of different options (including mixtures of options), the consideration of benefits and justifications, and the systematic "mapping" of the ways in which different framing assumptions lead to different pictures of performance. A case study of a pilot exercise applying a mufti-criteria mapping method to the regulatory appraisal of a genetically modified crop is reported. The results are more complete than orthodox risk assessment, in that they embody consideration of an unlimited array of issues and include consideration of a wide range of different strategic alternatives to the use of GM technologies. It is concluded that conventional regulatory appraisal might be adapted to better address the imperatives of both science and precaution. Key words precautionary principle; risk assessment; probability theory; regulatory appraisal; genetically modified crops; mufti-criteria mapping.
Received from
SPRU-Science and Technology Policy Research, University of Sussex, Brighton,
East Sussex, United Kingdom (AS); and GeneWatch UK, Tideswell, Buxton,
Derbyshire, United Kingdom (SM).
Address correspondence and
reprint requests to: Andy Stirling, PhD, SPRU, Mantell Building, University of
Sussex, Brighton, BNI 9RF, United Kingdom.
INT J OCCUP ENVIRON HEALTH 2000;6:296-311
The precautionary principle is an increasingly prominent theme in the debate over technologic risk. Many questions are raised over the implications for policy making. In particular, concerns have been expressed over the relationship between precautionary and more traditional science-based approaches to decision making such as cost-benefit and risk analyses. Fears are sometimes raised that-unlike risk assessment-a precautionary approach is too ambiguous and impractical to serve as a basis for real decision making, that it is somehow antagonistic to science, and even that it threatens to stifle technologic innovation and economic growth.
The first part of this article takes a close look at some of the key practical and theoretical issues bearing on the relationship between `science' and `precaution' in the management of technologic risk. It is found that-far from being in tension-these two concepts are entirely consistent and even mutually reinforcing. The real distinction is found to lie between narrow risk-based concepts of regulatory appraisal and broader precautionary approaches. A series of eight criteria are developed against which appraisal techniques can be evaluated in terms both of their scientific and their precautionary validity.
The second part of the article reports on a recent pilot exercise that attempted to apply these criteria to the specific case of the regulatory appraisal of a genetically modified (GM) crop. The main features of the new mufti-criteria mapping method are outlined. Key findings are reported, with particular attention to the extent to which the method conforms to a set of scientific and precautionary quality criteria.
SCOPE AND COMPLEXITY OF ENVIRONMENTAL AND HEALTH RISKS
Risk is a complex concept. Even under the most narrowly defined of quantitative approaches, it is recognized that risk is a function of at least two variables-the likelihood of an impact and its magnitude. However, it is only very rarely the case that a series of technology, policy, or investment options is seen to present only one form of hazard. Normally, the characterization of risks associated with any individual option requires the consideration of a wide variety of disparate risks. In the energy sector, for example, risks can take forms including greenhouse gas emissions, radioactive wastes, heavy metals, persistent organic pollutants, soil erosion, thermal discharges, ambient noise, ecologic disturbance, or aesthetic intrusion in the landscape. All of these risks manifest in different ways, with different physical, biologic, social, cultural, and economic connotations.
The conventional analytic response to this breadth and diversity of issues in regulatory appraisal is to identify a single major yardstick of performance and seek to measure all the various aspects of risk using this as a metric. The chosen unit of measurement in conventional risk assessment is usually human mortality or morbidity rates. In some areas, the techniques of cost-benefit analysis are employed in seeking to subject a wider range of impacts to measurement under a common monetary metric and so allow comparison with the associated benefits. It is hoped that in this way the multiplicity of magnitudes typically confronted in the regulation of environmental and health risk may usefully be reduced to a single key factor, thus apparently simplifying the process of appraisal. This process of reduction is an essential element in what is sometimes described as a science-based approach to the regulatory appraisal of risk.
Of course, one crucial consequence of this artificial narrowing and conflation of the full diversity of technologic risks is effectively to exclude from consideration many classes of effects. For instance, it is clear that only a minority of the types of energy risks mentioned above is meaningfully addressed by a mortality, morbidity, or even a monetary metric. Moreover, even with respect to the single issue of human health, risk is an inherently multidimensional concept. For instance, are exposures voluntary or controllable? Do they manifest as disease, injuries, or deaths? How familiar are the risks? How immediately are they realized and how reversible once identified? To what extent are they concentrated in large events or dispersed in small routine incidents? How are they distributed across space, time, and society? Mortality, and even morbidity, indices fail to capture these important contextual features?
Beyond this, further scope for divergent approaches to regulatory appraisal lies in the characteristics of the assessment process itself. Should appraisal take account of social, economic, cultural, and ethical issues, as well as environmental and health factors? With respect to the more narrowly defined physical factors, to what extent should appraisal seek to address the potential additive, cumulative, synergistic, and indirect effects associated with particular environmental and health risks? With how wide an array of potential alternatives should each individual technologic or policy option be compared in appraisal? Should attention be confined simply to the operation of the options concerned, or should it extend to the manufacture, decommissioning, and disposal of plant, as well as to the various inputs (such as energy and materials) and associated risks at each stage? To what extent should the relative benefits of different options be taken into account in appraisal so that they can be offset against the associated risks?
In an ideal world, the appropriate response to factors such as these is easy to determine. All else being equal, the regulatory appraisal of risk should be as complete and as comprehensive as possible. However, aspirations to completeness concerning the different classes and dimensions of risk and benefit and comprehensiveness concerning the different types of options provide only a rather loose operational guidance in the practical regulation of risk. Moreover, even were appraisal to be fully complete and comprehensive, in some hypothetical sense, then there would still remain the problem of how the different aspects of risk should be framed and prioritized in analysis. For instance, what assumptions should be made about adherence to best practice in the various activities under appraisal? What relative priorities should be attached to different effects such as toxicity, carcinogenicity, allergenicity, occupational safety, biodiversity, and ecologic integrity? What weight should properly be placed on impacts on different groups, such as workers, children, pregnant and breastfeeding mothers, future generations, disadvantaged communities, foreigners, those who do not benefit from the technology in question, or even animals and plants as beings in their own right? Even if they were practically feasible, objectives such as completeness and comprehensiveness do not assist in addressing issues of framing and prioritization of this kind. No one set of assumptions or priorities may be claimed to be uniquely rational, complete, or comprehensive.
It is here that we come to a classic and well-explored dilemma in the field of social choice theory, but one that is frequently forgotten in risk assessment and regulatory appraisal. The disciplines of risk assessment, economics and decision analysis have developed no single definitive way of addressing the problems of comparing apples and oranges. Even the most optimistic of proponents of rational choice acknowledge that there is no effective way to compare the intensities of preferences displayed by different individuals or social groups.3 Indeed, even where social choices are addressed simply in relative terms, the economist Kenneth Arrow went a long way towards earning his Nobel Prize by demonstrating formally that it is impossible definitively to combine relative preference orderings in a plural society.4
Put simply, the point is that it takes all sorts to make a world. Different cultural communities, political constituencies, or economic interests typically attach different degrees of importance to the different aspects of environmental risk and look at them differently. Within the bounds defined by the domain of plural social discourse, no one set of values or framings can definitively be ruled more rational or well informed than can any other. Even were there to be complete certainty in the quantification of all the various classes and dimensions of risk, it is entirely reasonable that fundamentally different conclusions over environmental risk might be drawn under different-but equally legitimate-perspectives. It is a matter of the science of risk assessment itself, then, that there can be no analytic fix for the scope, complexity, and intrinsic subjectivity of environmental and health risks. The notion that there can be a single unambiguous "science based" prescription in the regulatory appraisal of risk is not only naive and misleading; it is a fundamental contradiction in terms.
THE DEPTHS OF INCERTITUDE
This problem may seem serious enough. Unfortunately, the difficulties encountered in the regulatory appraisal of risk are even more intractable than this. Thus far we have considered only the issues associated with the characterization of the magnitude aspects of risk. What of the likelihoods? Here we come upon some profound limitations to the applicability and robustness of probabilistic approaches that are as seriously neglected in regulatory appraisal as are the difficulties discussed above concerning the comparison of magnitudes.
Figure 1-The formal definitions of risk, uncertainty, ambiguity, and
ignorance.
In economics and decision analysis, the well-established formal definition of risk is that it is a condition under which it is possible both to define a comprehensive set of all possible outcomes and to resolve a discrete set of probabilities (or a density function) across this array of outcomes. This is illustrated in the top left corner of the diagram in Figure 1. This is the domain under which the various probabilistic -techniques of risk assessment are applicable, permitting (in theory) the full characterization and ordering of the different options under appraisal. There are a host of details relating to this picture (such as those hinging on the distinction between frequentist and Bayesian understandings of probability), but none of these alters the formal scientific definition of the concept of risk %7
The strict sense of the term uncertainty, by contrast, applies to a condition under which there is confidence in the completeness of the defined set of outcomes, but where there is acknowledged to exist no valid theoretical or empirical basis confidently to assign probabilities to these outcomes. This is found in the lower left corner of Figure 1. Here, the analytic armory is less well developed, with the various sorts of scenario analysis being the best that can usually be managed.8 While the different options under appraisal may still be broadly characterized, they cannot be ranked even in relative terms without some knowledge of the relative likelihoods of the different outcomes.
Both risk and uncertainty, in the strict senses of these terms, require that the different possible outcomes be clearly characterisable and subject to measurement. The discussion in the previous section has already made it clear that this is often not the case-the complexity and scope of the different forms of environmental risk and the different ways of framing and prioritizing these can all too easily render ambiguous the definitive characterization of outcomes. This may be so even where there is relatively high confidence in understandings of the likelihood that at least some form of impact will take place (top right corner of Figure 1). An illustrative example here might be the prospects for regional climatic, ecologic and socioeconomic impacts arising from the human-enhanced greenhouse effect.
Where these problems are combined with the difficulties in applying the concept of probability, we face a condition that is formally defined as ignorance (bottom right corner of Figure 1). 9-13 This applies in circumstances where there not only exists no basis for the assigning of probabilities (as under uncertainty), but where the definition of a complete set of outcomes is also problematic. In short, recognition of the condition of ignorance is an acknowledgement of the possibility of surprises. Under such circumstances, not only is it impossible definitively to rank the different options, but even their full characterization is difficult. Under a state of ignorance (in this strict sense), it is always possible that there are effects (outcomes) that have been entirely excluded from consideration.
Figure 1 provides a schematic summary of the relationships between these formal definitions for the concepts of risk, uncertainty ambiguity, and ignorance. It is quite normal, even in specialist discussion, for the full breadth and depth of these issues to be rolled into the simple concept of `risk' (and sometimes `uncertainty'), thus seriously understating the difficulties involved. In order to avoid confusion between the strict definitions of the terms risk and uncertainty as used here and the looser colloquial usages, the term `incertitude' is used to cover all four subordinate conditions. Either way, it is not difficult to see that it is the formal concepts of ignorance, ambiguity, and uncertainty-rather than mere risk-that best describe the salient features of regulatory decision making in areas such as energy technologies, toxic chemicals, and genetically modified organisms. Indeed, many of the most high-profile technologically-induced "risks" of recent years-such as stratospheric ozone depletion, endocrine-disrupting chemicals, and BSE, for instance are all cases where the problem lay not so much in the determination of likelihoods, but in the anticipation of the very possibilities. They were surprises!
The crucial point is that intractable uncertainties, ambiguities, and ignorance are routinely treated in the regulatory appraisal of technology simply by using the probabilistic techniques of risk assessment. This treatment of uncertainty and ignorance as if they were mere risk effectively amounts to what the economist Hayek dubbed (in his Nobel acceptance speech) "pretence at knowledge."14 Far from displaying a respect for science in regulatory appraisal, the effect of such scientistic oversimplification is actually to ignore and undermine the scientific principles on which risk assessment itself purports to be based. Given the manifest inapplicability-in their own terms-of probabilistic techniques under uncertainty and ignorance, this is a serious and remarkable error. The self-contradictions in aspirations to a "science based" approach reliant solely on quantitative risk assessment, already noted in the last section, are thus further underscored and reinforced.
PRACTICAL CONSEQUENCES FOR RISK ASSESSMENT
The problems discussed so far-the multidimensionality of environmental and health risks and the conditions of uncertainty, ambiguity, and ignorance-may all seem a little abstract and theoretical. It is perhaps also partly for this reason that they remain relatively neglected in the business of regulatory appraisal. Unfortunately, however, they have some important practical consequences that, though often concealed, hold profound implications for the interpretation of orthodox risk-assessment results in all fields, extending from the regulation of energy options through chemicals and industrial hazards to genetic modification technologies.
In all these areas, the typical response to these difficulties in regulatory appraisal is to reduce and simplify focusing on those aspects that are either the most tractable or the most reasonable under certain dominant perspectives. In this way, individual studies can construct a picture of environmental risks, which appears to be quite unambiguous and precise. The scale of the discrepancies becomes evident only on occasions when attention is extended to a series of different appraisal studies, all applying subtly different-but equally "reasonable" and "legitimate"-framing assumptions concerning the different dimensions of appraisal discussed here. When this takes place, it becomes dear that the apparent relative riskiness of different options can vary quite radically, depending on the framings and priorities attached to the hidden variables during the process of appraisal.
Figure 2 illustrates this by showing the results obtained in 32 large-scale risk assessments of eight different energy technologies conducted in industrialized countries over the past two decades. Here, environmental and health effects are characterized using the techniques of cost-benefit analysis as monetary external costs expressed in standardized form per unit of electricity production.2 This case is taken as an example because both the techniques employed, and this particular field of application, might arguably be seen as being among the most mature and intensively explored areas of application of comparative risk assessment. The picture is not specific to these techniques or this field. A similar pattern may be found in a variety of other regulatory fields, including transport, toxic chemicals, and food safety. The same pattern is also evident in the underlying physical and mortality indices on which these monetary results are based. A number of salient features can be seen.
First, individual studies present their results with great precision, often as a single value rather than a range and sometimes expressed with as many as four significant figures (one part in ten thousand). Yet, the variability in the results obtained in the literature as a whole for any one option is radically larger. For instance, the uppermost values of the highest range assessing the risks associated with coal power amount to the equivalent of some 20 dollars per kilowatt-hour of electricity production. The lowest values of the bottom range in Figure 2 are less than four hundredths of a cent per kilowatt-hour. The difference is more than four orders of magnitude-a factor of more than fifty thousand! Detailed analysis of the reasons for these discrepancies show that they do not arise as a result of any single factor. It is not a simple matter of some studies' being more accurate or reasonable than others in any definitive sense. Instead, the variability is the cumulative consequence of the adoption of divergent assumptions and priorities concerning the whole range of the different dimensions of appraisal identified in the preceding sections.2
The second crucial feature that is illustrated in Figure 2 concerns the ambiguities in the ordering of the different options under appraisal. The lowest values obtained for the worst-ranking option (coal) are lower than the highest values obtained for the apparently best-ranking option (wind). Since the effect of the particular assumptions adopted in individual studies is to produce results at the high end of the overall range for some options but lower in the distributions for others, the overall picture yielded by the literature as a whole would accommodate virtually any conceivable ranking order for these eight options! By the judicious choice of framing assumptions, then, radically different conclusions can be justified for regulation.
This evident disjuncture between precision and accuracy in supposedly science based risk assessment paints a rather negative picture. One of the first and most basic tasks in the management of risk is to construct some robust overall notion of the relative merits of the different options under consideration from the point of view of society as a whole. This then serves as a basis for regulatory intervention, market-based measures or investment initiatives. Where this cannot be achieved in any absolute (or even relatively robust) sense, then the value of appraisal lies in systematic exploration of the relationships between different assumptions in analysis and the associated pictures of the relative importance of different options. Where aspirations to the science based appraisal of risk lead to the assertion of the intrinsic authority of narrow risk-assessment procedures, then these crucial exogenous factors typically remain unacknowledged and unexplored. In this event, the problem is not simply one of a lack of rigor concerning the theoretical contradictions noted in the previous sections. The difficulties are also very concrete and pragmatic. For, without a robust appreciation of the assumptions under which appraisal yields differing pictures of performance, serious questions must be raised over whether the associated results no matter how confidently and precisely expressed-are of any practical policy use at all.
Figure 2-Varlablllty In technologic risk assessments (an example from energy
technologies): stated external environmental cost (bars represent range over a
variety of studies). 1995 c/kWh.
"SCIENCE" AND "PRECAUTION" IN THE REGULATORY APPRAISAL OF ENVIRONMENTAL RISK
It is with increasing realization of these practical and theoretical limitations to the value of orthodox risk assessment in regulatory appraisal that interest in complementary and alternative approaches is growing. In particular, the precautionary principle is becoming an ever-more-prominent feature of the regulatory debate on environmental risks and of national and international legislation.15-17 A precautionary approach acknowledges the difficulties in risk assessment by granting greater benefit of the doubt to the environment and to public health than to the activities that may be held to threaten these things. A host of different practical instruments and measures are variously proposed in different contexts as embodiments of a precautionary approach or as means to implement a precautionary principle. For present purposes, attention will concentrate on the way in which a precautionary approach offers a direct response to the practical and theoretical problems in regulatory appraisal that have been discussed so far.
One key theme in the debate on these matters surrounds the frequent assertion (and sometimes assumption) that-whatever form it takes-a precautionary approach to the management of environmental risk is somehow in tension with (or even antithetical to) the generally uncontroversial aspiration that regulatory decision making should be based on sound science. Of course, this does not address the extent to which orthodox "scientific" approaches such as comparative risk assessment may themselves be claimed to yield sound results. The thrust of the discussion thus far has been to raise serious doubts over this. Nevertheless, the important question remains as to what exactly the relationship between so-called science-based and precautionary approaches to the regulation of environmental risk is?
A necessary starting point for this analysis is a clear characterization of exactly what is meant by "science" and "precaution" in the context of decision making relative to environmental risk. Drawing on a wide literature, Figure 3 displays some idealized attributes of scientific approaches to regulatory appraisal.18 A scientific approach to the management of risk should, ideally and at minimum, be transparent in its argumentation and substantiation, systematic in its analytic methods, skeptical in its treatment of knowledge claims, subject to peer review, independent from special interests, professionally accountable, and continually open to learning in the face of new knowledge. These aspirations may not always be realized, but they represent fundamental, and relatively uncontroversial, principles guiding any "science-based" approach to regulatory appraisal.
Likewise, it is possible broadly to characterize the essential features of a precautionary approach to the management of risk. A precautionary approach involves the application of principles that prevention is better than cure, that the polluter should pay, that options offering simultaneously better economic and environmental performance should always be preferred (no regrets), that options should be appraised at the level of production systems taken as a whole and that attention should be extended to the intrinsic value of non-human life in its own right (a 'biocentric ethic'). In effect, this means a certain humility about scientific knowledge and an acknowledgement of the complexity and variability of the real world It implies recognition of the vulnerability of the natural environment and living organisms and the prioritizing of the rights of those who stand to be adversely affected. It requires scrutiny of claims to benefits and Justifications as well as risks and costs, with full account given to the available alternatives. Finally, a precautionary approach involves the adoption of long-term, holistic, and inclusive perspectives in regulatory appraisal. 19
In many ways, these attributes of a precautionary approach concern different aspects of the breadth of the regulatory appraisal process. A broad regimen is one that takes account of a wide range of different types of impacts, including qualitative as well as quantitative issues; including indirect as well as direct effects; accommodating a diverse array of different points of view (including, importantly, those of potential victims); and anticipating a wide range of possibilities in the face of uncertainty and ignorance. It extends consideration to the benefits and justifications associated with the introduction of the technology in question and examines a variety of alternative ways in which the benefits of a regulated technology might be realized at lower levels of risk. Taken together, these features constitute a more precautionary approach because they increase the number and intensity of the constraints that any technologic option must satisfy in order to be approved by the regulatory process, thus making it more difficult for certain innovations to pass through the regulatory "filter." At the same time, however, such measures might equally serve to encourage other technologic innovations that might otherwise remain neglected.
Figure 3--Model of the relationships between risk, science, and precaution.
|
NARROW
REGIMEN |
'UNSCIENTIFIC
APPRAISAL' |
BROAD
REGIMEN |
|
|
||
|
'SCIENTIFIC
APPRAISAL' |
What is interesting about this characterization of precaution in terms of the breadth of the associated regulatory regimen, is that it reveals a consistent-and in many respects complementary-relationship between precaution and science in the management of technologic risk. Figure 3 distinguishes between different approaches to risk management based on the degree to which each embodies the respective characteristics of `scientific appraisal' and `breadth of framing' identified here.
Although the broad/narrow and the scientific/unscientific dichotomies drawn here are highly stylized and simplified, the general picture revealed in Figure 3 is at least richer and more realistic than the prevailing one-dimensional dichotomy between science and precaution. The combination of these two dichotomies generates a fourfold array of idealized permutations. The adoption of a "narrow" regimen without reference to scientific understandings or disciplines in appraisal might be described as a permissive position. Taken to an extreme, this would amount to an entirely uncritical anything-goes approach to the regulation of technology of the kind associated with caricature `cornucopian' visions of technologic progress. Likewise, a broad-based regimen might be similarly unscientific. The resulting restrictive position might be associated with a caricature `apocalyptic' vision of technology. In the extreme, it would lead to a situation of paralysis under which no new technologic innovation that offends in the slightest respect would ever be approved for deployment. The crucial point is that neither the permissive (cornucopian) nor the restrictive (apocalyptic) position as defined here would be subject to challenge or reversal by the disciplines of scientific discourse associated with the vertical axis.
It is clear that neither the established procedures of risk regulation (based on relatively narrowly framed risk assessment methods) nor the emerging precautionary approach (based on broader perspectives and considerations) actually resemble these stylized permissive or restrictive caricatures. Existing risk-assessment-based regulation includes a host of effective checks and balances. It certainly does not necessarily provide for the uncritical approval of any new technology that may be developed. Likewise, even the most progressive formulations of a precautionary principle are circumscribed in their scope, admit an incremental series of instruments, and allow for regulatory approval under a host of favorable conditions. Both approaches are compatible-at least in principle-with the requirements of systematic method, skepticism, transparency, accountability, quality control by peer review, professional independence, and an emphasis on learning, which are held here for the purposes of this discussion to be among the key aspirations of a science-based approach.
It is at this point that it is useful to return to the earlier discussion of the profound importance of the conditions of uncertainty, ignorance and multidimensionality in risk assessment. It is shown in earlier sections of this article that questions over the scope of appraisal, the plurality of different value positions and framing assumptions, the diversity of different anticipated possibilities, and the degree of confidence placed in the available knowledge are all matters that are central to the scientific status of the appraisal process. As was shown, it flows directly from the theoretical foundations of risk assessment, cost-benefit analysis (and, indeed, all `rational choice' approaches to decision making on risk) that probabilistic approaches are inapplicable under strict uncertainty and ignorance. It also follows equally directly from these fundamental theoretical principles that different priorities, framing assumptions, and value systems cannot be definitively aggregated across different groups. For both these reasons, it is clear that there can be no analytic fix for the definitive ranking of different technology or policy options in the social appraisal of risk. All that can be done to maximize scientific rigor in appraisal is to ensure that the process is as broadly framed as possible in terms of the value systems and framing assumptions that are included and the options and possibilities that are addressed. Seen in this way, then, key elements of the breadth of the regulatory regimen themselves become issues of sound science in the management of environmental risk, as well as institutional features of the wider regulatory regimen. Precaution, in this sense, is not just entirely consistent with science-it is a necessary prerequisite for as truly scientific approach to the regulatory appraisal of risk.18
IMPLEMENTING A PRECAUTIONARY APPROACH TO APPRAISAL
In the discussion so far, environmental and health risks have been treated at a high level of generalization. Of course, it is obvious that at a greater level of detail, different technologic and policy options will vary radically in the magnitude and character of the risks that they present. Likewise, the ranges of different possible regulatory interventions do not fall neatly into permissive or restrictive categories, but lie along a series of continuums, and range between being more and less precautionary in their effects. With respect to both precautionary and other approaches to the regulation of risk, then, different measures will be appropriate in different contexts. This general picture is the subject of a vast literature.15-18, 20 Here, however, the purpose is to focus specifically on some key implications for regulatory appraisal. It is possible to construct a series of eight evaluative criteria against which the regulatory appraisal of risk can be assessed in terms of both its scientific rigor and its precautionary qualities.18
•
•
•
•
•
•
•
•
These are the considerations that have informed the development of the multi-criteria mapping technique employed in the present case study.21,22 The motivation behind this approach is to seek to combine the openness and qualitative flexibility of participatory deliberation with the clarity and focus of quantitative assessment. The specific case study with which this method has been piloted concerns the hotly contested debate over the use of genetically modified (GN) crops in UK agriculture.
THE UK DEBATE OVER THE REGULATION OF GENETICALLY MODIFIED CROPS
The agricultural use of GM technologies is held in some quarters to promise great benefits. On the other hand, there is general agreement that there exists at least the potential for serious, irreversible harm. In the United Kingdom, formal regulatory appraisal of GM crops has centered on the question of whether or not they are safe for the environment and for human consumption. However, there is considerable scientific incertitude over the form and magnitudes of the possible effects and, as yet (by contrast with chemical or nuclear risks), little accumulated practical experience to draw upon. This has led to the evolution of a set of controls that are intended to be precautionary in nature-where it is accepted that action to avoid harm may be taken in the absence of scientific proof-with the conduct of risk assessment being required before experimental or commercial use of a particular genetically modified organism is allowed.
Despite this somewhat precautionary approach to risk regulation enshrined in the European Commission's Deliberate Release Directive (90/220/EC) and the Novel Foods Regulation, the regulatory appraisal process has failed to gain confidence, either of nongovernmental organizations (NGOs), private industry,23 or the general public.24-25 This lack of confidence arises, among other reasons: because the scope of the regulatory appraisal is still held in many quarters to be too narrow; because there is a general lack of trust in official reassurances of safety (particularly in the wake of BSE); and because justifications and benefits are not explicitly included in the evaluation process. Industry and regulators have expressed frustration, believing that the precautionary approach is being invoked in too burdensome a fashion, with unrealistic demands being made concerning absolute proof of safety.
It has also been almost impossible to gain agreement between European Member States over whether particular commercial releases of GM crops are environmentally safe, despite a supposedly common approach to their risk assessment 26,27 Disputes routinely emerge over the appropriate scope of regulatory appraisal. Even where there is agreement over the possibility that effects will occur, notions of what constitute adverse effects remain strongly contested.
These sorts of problems with the current regulatory appraisal of GM crops are typical of those that beset the use of conventional risk assessment and cost-benefit analyses in other areas. Taken together, these characteristics of the U.K. debate over the application of GM technologies in agriculture make it a challenging candidate for a case study concerning the application of the general principles of scientific and precautionary appraisal enunciated above.
THE MULTI-CRITERIA MAPPING PILOT STUDY
The details of the multi-criteria mapping method and this particular application in the GM field in the United Kingdom are described at length elsewhere.22,27 Only those features bearing on the examination of the quality criteria given above are outlined here. The pilot study took place between April 1998 and September 1999. It was funded by the transnational food firm Unilever, but was entirely independent in its conception, design, implementation, and reporting. The two authors come from an academic and NGO background, and the project was overseen by a steering group comprising a wide range of environmental, consumer, farming, and food industry representatives. The specific topic chosen for the study was the production of oilseed rape in the United Kingdom. Six basic options were identified in advance for the purposes of comparison (Table 1).
TABLE 1 The Definitions of the "Basic Options" Appraised by All Participants in the Pilot Study
|
Option |
Definition |
|
Organic agriculture |
All farming and food production conducted under present-day organic standards |
|
Integrated pest management |
All farming and food production conducted using systems designed to limit but not exclude chemical inputs and with greater emphasis on biologic control systems than conventional systems |
|
Conventional agriculture |
All farming and food production conducted under present-day Intensive systems |
|
GM* oilseed rape with segregation and present systems of labeling |
Labeling based on the presence of foreign DNA or protein In the final product |
|
GM* oilseed rape with postrelease monitoring |
Monitoring for effects (mainly environmental) conducted on an ongoing balls attar commercialization |
|
GM* oilseed rape with voluntary controls on areas of cultivation |
Areas of growing of GM oilseed rape restricted on a voluntary basis to avoid unwanted effects such as gene-flow and cross fertilization of non-GM crops |
|
Up to six additional options to be specified by participant |
Any option of participant's choice, including combinations of the above if desired |
| *Genetically modified. | |
In consultation with the steering group, 12 individual high-profile protagonists in the GM food debate were selected as participants. These individuals came from a variety of backgrounds, including academics and government regulators, environmental, consumer, and religious organizations, and representatives from the farming, food, and biotechnology industries. Of course, it is impossible to claim any statistically representative status for such a small sample. However, care was taken that, between them, the group of participants covered a full envelope of specialist and sociopolitical perspectives ranging, for instance, from strongly opposed to strongly in favor of the use of GM crops. Given the character of the U.K. GM debate at the time of the study (1998-99), it was necessary to give undertakings of anonymity in order to secure participants' involvement. However, the viewpoints of the participants are reproduced individually, each being identified by a code letter and an affiliation with one of four general groupings of perspectives: academic, NGO, industry or `government' (Table 2).
TABLE 2 The Participants In the Pilot Study
|
Area |
Code* |
|
Agriculture and food Industry |
B, L, H, K |
|
Academic scientists |
C, J |
|
Government safety advisors |
E, F |
|
Religious and public Interest groups |
A,D,G,I |
|
*Each letter represents an Individual participant. |
During a two-to-three-hour individual interview, each participant undertook a four-stage process: 1) the identification of additional options; 2) the defining of appraisal criteria under which the options should be assessed; 3) the scoring of the performance of each option under each criterion; and 4) the weighting of each criterion in terms of its relative importance. A straightforward linear additive multi-criteria procedure (Figure 4) was employed using a simple spreadsheet model mounted on a portable computer in order to display to the participant the results of the scoring and weighting process as it developed. The process then iterated cyclically until the participant was satisfied that the results accurately reflected his or her own personal perspective and professional judgment relative to the issues at hand.
One crucial feature of the scoring process was the deliberate eliciting of a range of performance scores for each option under each criterion. Rather than providing a single best guess, this allowed consideration of a range of optimistic and pessimistic assumptions under each perspective. The discussion of the associated issues was then carefully documented and provided an interesting reflection of the importance of technical uncertainties in the final picture of performance derived under each perspective.
Figure 4-The multi-criteria mapping process.
In the period following the interviews, the participants were contacted and asked to consider the role of deliberate diversification among options as a means to implement two crucial features of the precautionary approach discussed above: the desire to hedge against intractable uncertainties and ignorance and to accommodate a variety of sociopolitical viewpoints. For this purpose, a simple index of diversity was borrowed from mathematical ecology and information theory in order to model different tradeoffs between the value attached to the performance of individual options and the value attached to diversity in the mix of options. The details of this method are described in more detail elsewhere.21,22,29 In short, participants can choose to assign a zero, low, medium or high weighting to diversity. Where it takes a zero value, then only the best-performing option is included in the favored mix. Where higher weightings are placed on diversity, progressively greater contributions are included from lower-performing options-drawn on in proportion to their relative performances under the perspective in question. In this way, the study was able to elicit a crude notion of the importance of ignorance and pluralism under the different perspectives.
As analysis proceeded, the participants were asked to review the results of a thorough sensitivity analysis conducted on their own sets of weightings. Along with an invitation that participants retain the spreadsheet model and freely experiment with it at their leisure, this provided a further means to verify that the final results represented a robust reflection of the different viewpoints. Finally, all participants were invited to a concluding workshop, at WHICH the results were again confirmed and used as a basis for wider-ranging discussion.
TABLE 3 Additional Options Defined by the Participants
Labelling and/or other controls
GM crops with segregation, full labeling and post-release monitoring and legally binding growing contracts A* GM crops with segregation, current labeling and post-release monitoring F GM crops with segregation, full labeling and post-release monitoring H GM crops with segregation and labeling according to means of production and source of gene, plus postrelease monitoring G GM crops with segregation, comprehensive labeling based on process and generic restrictions on some classes, e.g., In center of origin I GM crops within controlled sectors (compulsory control) A GM crops with legally binding threshold for gene transfer to non-GM stream A
Agricultural system
GM crops, IPM system G GM crops, IPM system J No GM crops-conventional and organic as now K GM crops in conventional and organic systems K GM crops, organic agricultural system, plus segregation, labeling and other regulations as required J
Assessment criteria
GM crops with quality B GM crops with assessment of Indirect agricultural Impact and assessment of need I
Other
GM crops only in USA A No GM commodity crops A Complete public control over choice C*Letters represent Individual participants (see Table 2).
FINDINGS
The procedure summarized above generated a rich body of empirical material, reflecting a very wide range of issues bearing on the regulatory appraisal of GM technologies in agriculture. This is reported in more detail elsewhere,22,28 but the key points can be summarized here.
TABLE 4 Broad Groupings of Criteria Defined by the Participants
Environment
Biodiversity, e.g., field boundary ecology, other environmental risks Chemical use, e.g., reduction In use of existing herbicide sprays, benefits of contact herbicides versus soil-acting residuals, longer-term pollution of air and water Genetic pollution, e.g., gene flow to other crops and native flora Wildlife effects, e.g., Impact of enhanced weed-control efficiency on wildlife, other practices affecting wildlife value of agricultural systems Unexpected effects, e.g., potential for effects not foreseen under this scheme Visual, e.g., amenity Impacts Aesthetics, e.g., feelings about environment
Health
Allergenicity, e.g., from food consumption Toxicity, e.g., human or animal health Nutrition, e.g., to consumers Unexpected effects, e.g., unexpected interactions between Ingredients, stability of genetic Insert Ability to manage , e.g., traceabilllty and ease of recall
Agriculture
Weed control, e.g., invasive volunteers and weedy relatives Food supply stability, e.g., sustainability, tendency to monocultures, global food security Agricultural practice, e.g.., farmers' rights, choice and quality of life, land requirements
Economy
Consumer benefit, e.g., retail price Producers' benefit, e.g., shorter-term costs, yield or longer-term value added Benefit to processor, e.g., profitability Socioeconomic Impact , e.g., welfare of small farmers, substitutions for developing countries
Society
Individual Impacts, e.g., consumer choice, transparency, accessibility, participation, pluralism Institutional Impacts, e.g., concentration of power, Institutional trust, regulatory complexity Social needs, e.g., new opportunities, opportunity costs, misuse of science, employment, quality of life
Ethics
Fundamental principles , e.g., animal welfare, taking care of nature Knowledge base, e.g., hubris about scientific knowledge
Options
In addition to the six basic options, a further 18 alternative agricultural strategies were identified and evaluated by participants. These included many different labeling and control regimens and the application of a series of different regulatory assessment procedures and criteria. They also involved a variety of different agricultural strategies, including a focus on hypothetical strategies using GM technologies under an organic farming regimen. Interestingly these tended to perform relatively well under the perspectives of participants from both sides of the GM debate (Table 3).
Criteria
A total of 117 appraisal criteria were defined by different participants. Although some of these are apparently closely related, all embody important differences of emphasis or framing. It is interesting that the criteria typically reflect not only issues of consumer choice but also issues of citizenship and wider questions of participation and agency. Collectively they cover a very wide range of considerations, including environmental, agricultural, health, economic, social, and ethical issues. Most of these issues are very remote from the factors included in orthodox risk assessment. Indeed, for no participant was it true that the full range of his or her concerns is fully addressed by existing regulatory appraisal procedures in the United Kingdom (Table 4).
Performance Scores
The pattern of the performance scores assigned by the participants under the various groups of criteria shows that the differences between perspectives are not simply due to variations in the weightings placed on different criteria. Instead, the picture yielded in this exercise is very similar to that illustrated above with respect to risk assessment of energy technologies (Figure 3). The participants adopted a variety of different framing assumptions; resulting in significant differences in the scores assigned by different participants under the same criteria. This has implications for conventional multi-criteria analysis, in which scoring is often conducted by a separate body of experts, with an assumption that different value judgments can be captured simply in the weightings. Despite these differences, however, the patterns evident in the scoring do allow certain general conclusions. For instance, under all perspectives the organic option tended to perform well under environmental criteria. Also, the evident divergence between the patterns of performance under health and environmental criteria challenges conventional assumptions that these aspects of performance will necessarily be well correlated.
Weightings
Notwithstanding the complications raised above with respect to scoring, the patterns evident in the weightings do illuminate some interesting, if rather unsurprising, features. Perspectives drawn from the biotechnology industry and food supply chain are conspicuous in their relative under-emphasis of the social and/or environmental and safety considerations that are prominent under all other perspectives. The perspectives adopted by government advisers have the distinctive characteristic of being at the same time relatively narrow in scope while emphasizing environmental and safety considerations. The perspectives expressed by the non-industry participants share a markedly lower emphasis on economic or agricultural considerations.
Rankings
Perhaps the most important output of the multi-criteria mapping procedure is the patterns in the final rankings obtained under the various participants' perspectives. Indeed, it is this picture that constitutes the "map" referred to in the name of the technique, reflecting the way that final results vary with starting assumptions. Figure 5 displays the results obtained in the present exercise for the ten participants from whom it was possible to elicit all the necessary quantitative inputs (the remaining two expressed difficulties either with the scoring or with the weighting process). Each chart represents the rankings elicited from a single participant. The sequence of basic options is the same in each case, running left to right from organic farming through integrated pest management and conventional intensive agriculture to GM crops with (respectively) segregation and labeling, monitoring, and voluntary controls. The height of each vertical bar reflects the overall performance of the option under that perspective-the higher the bar the better the performance. The scale is in arbitrary subjective linear units of performance. Each option is represented by a pair of bars, with the left bar expressing the ranking of that option under pessimistic assumptions and the right bar expressing the ranking under optimistic assumptions. The pessimistic and optimistic scales have been normalized in order more clearly to show the relative orderings.
A number of features can be distinguished in this picture. First, it is clear that there exist profound differences between the practical implications of the perspectives adopted by different participants. In an orthodox regulatory appraisal procedure (such as risk assessment) such variability would typically be concealed by the emphasis on a single aggregated position. In the present analysis, on the other hand, the idiosyncrasies of each individual position are clearly displayed. Second, it is clear that the differences between rankings obtained under optimistic and pessimistic scores are generally rather small compared with the differences between perspectives. This suggests that it is not generally the technical dimensions of uncertainty that are the key issue, but rather more intangible qualitative aspects concerning the divergent interests, values, and framing assumptions adopted by different participants. Third, despite the variability in the evaluations of different participants, there emerges a series of consistencies. With only a few exceptions, the organic option tended to perform relatively well. Conventional intensive agriculture tended to perform uniformly badly. The voluntary-controls regimen for GM crops was regarded rather poorly by industry, academic, and NGO participants alike, being viewed favorably only by the government advisers.
Diversity
The final issue concerns the merits of diversification. Figure 6 shows the diverse mixes of agricultural strategies that were favored by the seven participants who responded to this part of the analysis. The partitioning of the pie charts represents some arbitrary measure of importance, such as "share of output" or "share of land in production." For present purposes, the details do not affect the broad-brush conclusions. What is interesting is that six of these participants-drawn from all sides of the debate-favored the deliberate pursuit of some degree of diversity. The remaining participant-a senior UK government safety adviser-displays a uniquely high degree of confidence in the validity and robustness of his own perspective. Together with the idiosyncrasies evident in the rankings displayed in Figure 5, this is, in itself, an informative result. Taken as a whole, this part of the multi-criteria mapping analysis might be taken to raise questions over the extent to which R&D and regulatory policy making should be geared towards active encouragement of a variety of techniques rather than assuming or emphasizing a single particular trajectory. Given the broad support expressed from all sides of the debate for the principle of diversity, questions might also be raised over how to treat options that display characteristics that militate against diversity.
In summary, this pilot multi-criteria mapping exercise permits a series of quite subtle and specific observations to be made concerning the character of the debate and the positions of the different protagonists. However, it also provides a basis for some general normative conclusions concerning the options under appraisal. These findings are all the more robust for being set firmly in the context of systematic exploration and acknowledgement of the underlying variability and dissent. In particular, in the present case, the picture is quite unequivocal with respect to the relatively negative performances of conventional intensive farming and voluntary controls on the use of GM crops. If equal attention is paid to each perspective, then it is clear that the organic-farming and integrated-pest-management options tended to perform significantly better overall.
Figure 5 -- Final rankings for the basic options under the perspectives of individual participants
|
C |
J |
E |
F |
|
A |
D |
I |
B |
|
K |
L |
 |
KEY
TO BASIC OPTIONS
1
organic |
| Each bar
chart shows the final rankings for the six basic options
obtained by each individual (identified by their code
letter). The bars show the relative performance of the
options according to an arbitrary linear scale. The
pessimistic and optimistic bars display the uncertainty
range in scoring that was expressed by each participant. The
six core options are described in Table 4.
|
|||
Figure 6 -- Diverse mixes of options favored by individual participants.
|
L |
B |
D |
E |
KEY TO
OPTIONS |
|
F |
I |
K |
Additional
Options |
|
|
Participants
were asked their opinion concerning diversity among options.
The results for each participant are shown as a pie chart
displaying the proportional importance of each option in
their favored mix of options. The six basic options 1-6 are
color coded. The additional options defined by each
participant are briefly described above. Full descriptions
are given in Table 4
|
||||
CONCLUSIONS
This article shows significant theoretical, methodologic, and practical grounds for concern over the utility of orthodox narrow risk-assessment techniques in the regulatory appraisal of risk. The frequently cited dichotomy between science-based and precautionary approaches to regulatory appraisal is shown to be unfounded and misleading. Indeed, there exist strong scientific reasons for holding the broader scope of precautionary approaches to be more consistent with the scientific foundations of rational choice and probability theory than are conventional narrow risk assessment techniques. The imperatives both of science and precaution can therefore be seen to pull in the same direction. The regulatory appraisal of risk should become more systematic and broader in scope. In particular, a set of criteria can be developed concerning the need for greater humility, completeness, transparency, and participation in regulatory appraisal, with specific attention to the comparison of different options (including mixtures of options), the consideration of benefits and justifications, and the systematic "mapping" of the ways in which different framing assumptions lead to different pictures of performance.
In order to explore how these criteria might be practically operational, the second part of the article examines as a case study a pilot exercise applying a novel multi-criteria mapping method to the regulatory appraisal of a genetically modified (GM) crop. The results of this exercise are more complete than orthodox risk assessment, in that they embody consideration of an unlimited array of issues. They also include consideration of a wide range of different strategic alternatives to the use of GM technologies. The same attention is given to the justifications and benefits as to the impacts and costs associated with the various options. The appraisal is inherently participatory, in that the results fully reflect the perspectives of a wide range of different socioeconomic and cultural interests. The relative simplicity and auditability of the weighting and scoring approach retain a high degree of transparency. A central feature is the focus on the systematic mapping of the way in which specific results relate to specific assumptions in appraisal. This, together with the explicit attention given to more diverse mixes of options, embodies a greater degree of humility in the face of ignorance and pluralism than is normally seen in risk assessment.
In this way, it may be concluded that there seems no reason in principle why conventional regulatory appraisal might not be adapted to better address the imperatives of both science and precaution. The present pilot exercise was not prohibitive in scale, being conducted by two researchers over a period of 18 months. Although necessarily provisional, the consistencies displayed in the results show that it is possible-despite the breadth of the process-to draw some quite robust conclusions over the relative performances of options such as organic farming, conventional intensive agriculture, and voluntary control regimens for the use of GM crops. Although the techniques of risk assessment continue to offer essential tools in addressing specific aspects, there seems to be no compelling theoretical, methodologic or practical reason for persisting in basing the entire business of regulatory appraisal on these inherently constrained and limited methods.
References
European Commission. Externe: Externalities of Energy. Vol 1-6, EUR 16520 - 16525 EN, Brussels, Belgium, 1995.
Stirling A. Limits to the value of external costs. Energy Policy. 1997;25:5.
Bezembinder T. Social choice theory and practice. In: Vlek C, Cvetkovitch G. Social Decision Methodology for Technological Projects. Dordrecht, The Netherlands: Kluwer, 1989.
Arrow K Social Choice and Individual Values. New Haven, CT: Yale University Press, 1963.
Knight F. Risk, Uncertainty and Profit. Boston, MA: Houghton Mifflin, 1921.
Luce R, Raiffa H. Games and Decisions. New York: Wiley, 1957.
Morgan M, Henrion M, Small M. Uncertainty. A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. Cambridge, UK Cambridge University Press, 1990.
Funtowicz S, Raveaz J. Uncertainty and Quality in Science for Policy. Amsterdam, The Netherlands: Kluwer, 1990
Loasby B. Choice, Complexity and Ignorance: An Inquiry into Economic Theory and the Practice of Decision Making. Cambridge, UK Cambridge University Press, 1976.
Smithson M. Ignorance and Uncertainty: Emerging Paradigms. New York: Springer, 1989.
Wynne B. Uncertainty and environmental learning: reconceiving science and policy in the preventive paradigm. Global Environmental Change. 1992; 111-27.
Faber M, Proops J. Evolution, Time, Production and the Environment. Berlin Germany: Springer, 1994.
Stirling A. Risk at a Turning Point?, J Risk Res. 1998;1:97-110.
von Hayek F. New Studies in Philosophy, Politics, Economics and the History of Ideas. Chicago, IL: University of Chicago Press, 1978.
O'Riordan T, Cameron J. Interpreting the Precautionary Principle. London, UK Earthscan, 1993.
Harding R, Fisher E. Perspectives on the Precautionary Principle. Anandale, CA: Federation Press, 1999.
Raffensberger C, Tickner J. Protecting Public Health and the Environment: Implementing the Precautionary Principle. New York: Island Press, 1999.
Stirling A. On Science and Precaution in the Management of Technological Risk Vol 1-a synthesis report of case studies. Institute for Prospective Technology Studies. Sevilla, Spain, EUR 19056 EN, 1999.
Stirling A. On Precautionary and Science Based Approaches to Risk Assessment and Environmental Appraisal. Vol 2. Institute for Prospective Technology Studies. Sevilla, Spain, 1999.
Reno O, Klinke A. Risk Evaluation and Risk Management for Institutional and Regulatory Policy., Vol 2. Sevilla, Spain: Institute for Prospective Technology Studies, 1999.
Stirling A. Multicriteria mapping: mitigating the problems of environmental valuation? In Foster j led). Valuing Nature: Economics, Ethics and Environment. London, UK: Routledge, 1997.
Stirling A, Mayer S. Rethinking Risk: A Pilot Multi-criteria Mapping of a Genetically Modifed Crop in Agricultural Systems in the UK. Sussex, U.S.: SPRU, 1999.
Mayer Sm Hill J, Grove-White R, Wynne B. Uncertainty, precaution and decision making: the release of genetically modified organisms into the environment. Global Environmental Change Programme Briefings Number 8. Global Environmental Change Programme: University of Sussex, Brighton, East Sussex, UK 1996.
Biotechnology and the European Public Concerted Action Group. Europe ambivalent on biotechnology. Nature. 1997;387:845-7.
Grove-White R, Macnaghton P, Mayer S, Wynne B. Uncertain World. Genetically Modified Organisms, Food and Public Attitudes in Britain. Centre for the Study of Environmental Change, Lancaster University, United Kingdom, 1997: Lancaster, 64 pp.
Von Schomberg R. An Appraisal of the Working in Practice of Directive 90/220/EEC on the Deliberate Release of Genetically Modified Organisms. STOA, European Parliament, Luxembourg, Belgium, 1998.
Wynne B, Mayer S. The Release of Genetically Modified Organisms to the Environment. In: EU 98. European Environment Agency, Copenhagen, Denmark, 1999 (in press)
Stirling A. Mayer S. Forthcoming, 2000.
Stirling A. Diversity and ignorance in electricity supply investment: addressing the solution rather than the problem. Energy Policy. 1994; 22, 3.
|
If you have come to this page from an outside location click here to get back to mindfully.org |
