# DUAL-USE LIFE SCIENCE RESEARCH AND BIOSECURITY IN THE 21ST CENTURY: SOCIAL, TECHNICAL, POLICY, AND ETHICAL CHALLENGES

EDITED BY: Jonathan E. Suk, Kathleen M. Vogel and Amanda Jane Ozin PUBLISHED IN: Frontiers in Public Health

#### *Frontiers Copyright Statement*

*© Copyright 2007-2015 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

*All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88919-568-8 DOI 10.3389/978-2-88919-568-8

## About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

## Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

## Dedication to quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

## What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

## **DUAL-USE LIFE SCIENCE RESEARCH AND BIOSECURITY IN THE 21ST CENTURY: SOCIAL, TECHNICAL, POLICY, AND ETHICAL CHALLENGES**

#### Topic Editors:

**Jonathan E. Suk,** European Centre for Disease Prevention and Control, Sweden **Kathleen M. Vogel,** North Carolina State University, USA **Amanda Jane Ozin,** European Centre for Disease Control and Prevention, Sweden

Image design: Tobias Hofsäss, http://www. tobiashofsaess.com/

In September 2011, scientists announced new experimental findings that would not only threaten the conduct and publication of influenza research, but would have significant policy and intelligence implications. The findings presented a modified variant of the H5N1 avian influenza virus (hereafter referred to as the H5N1 virus) that was transmissible via aerosol between ferrets. These results suggested a worrisome possibility: the existence of a new airborne and highly lethal H5N1 virus that could cause a deadly global pandemic. In response, a series of international discussions on the nature of dual-use life science arose. These discussions addressed the complex social, technical, political, security, and ethical issues related to dual-use research. This

Research Topic will be devoted to contributions that explore this matrix of issues from a variety of case study and international perspectives.

**Citation:** Jonathan E. Suk, Kathleen M. Vogel and Amanda J. Ozin, eds. (2015). Dual-use life science research and biosecurity in the 21st century: social, technical, policy, and ethical challenges. Lausanne: Frontiers Media. doi: 10.3389/978-2-88919-568-8

# Table of Contents


Kirsten X. Jacobsen, Kirsten Mattison, Marianne Heisz and Sandra Fry

## *80 Novel Dutch self-assessment Biosecurity Toolkit to identify biorisk gaps and to enhance biorisk awareness*

Petra C. C. Sijnesael, Linda M. van den Berg, Diederik A. Bleijs, Paul Odinot, Carin de Hoog, Mieke W. J. C. Jansen, Evelien Kampert, Saskia A. Rutjes, Martien Broekhuijsen and Sander Banus

*85 Antipodal biosecurity? Oversight of dual use research in the United States and Australia*

Frank L. Smith III and Adam Kamradt-Scott

*88 H5N1: a cautionary tale*

Susan A. Ehrlich


EDITORIAL published: 02 February 2015 doi: 10.3389/fpubh.2015.00013

## Biosecurity and dual-use research: gaining function – but at what cost?

## **Kathleen M. Vogel <sup>1</sup>\*, Amanda J. Ozin<sup>2</sup> and Jonathan E. Suk <sup>2</sup>**

<sup>1</sup> North Carolina State University, Raleigh, NC, USA

<sup>2</sup> European Centre for Disease Control and Prevention, Stockholm, Sweden

\*Correspondence: kmvogel@ncsu.edu

**Edited by:**

Jos W. M. Van Der Meer, Radboud University Nijmegen Medical Centre, Netherlands

**Reviewed by:** Christopher G. Atchison, University of Iowa, USA

#### **Keywords: biosecurity, dual-use research, influenza, editorial, bioterrorism**

In September 2011, scientists announced new experimental findings that would not only threaten the conduct and publication of influenza research, but would have significant policy and intelligence implications. The findings presented a modified variant of the H5N1 avian influenza virus (hereafter referred to as the H5N1 virus) that was transmissible via aerosol between ferrets (1, 2). These results suggested a worrisome possibility: the existence of a new airborne and highly lethal H5N1 virus that could cause a deadly global pandemic. In response, a series of international discussions on the nature of dual-use life science arose (3). More proposed "gain-of-function (GOF)" research on the flu, and other respiratory viruses such as severe acquired respiratory syndrome (SARS) and middle east coronavirus (MERS-CoV), has led to this work being labeled as having "potential pandemic potential (PPP)."

Scientists and other interested parties are increasingly asked to more clearly state the risks and benefits of this kind of research and whether new regulations and oversight mechanisms are needed. More recently, controversies such as reported accidents and lax controls over dangerous pathogens in high profile research labs have once again raised the issue of accounting and safeguards of dangerous pathogens, with new calls for greater transparency of the oversight of these materials (4, 5). The emerging field of synthetic biology is also raising concerns about its current and future impact on human health and the environment, and its potential for bioterrorism by do-it-yourself biologists. With the Ebola outbreaks happening as we began to work this editorial, we have encountered additional (but fairly speculative!) discussion about the threat of bioterrorism during naturally occurring outbreaks and how this risk could be dealt with by the health security agenda.

Regardless of where one finds oneself on the topic, it seems clear that advances in the life sciences are creating new ethical, safety, regulatory, and security challenges. To what extent such research should be conducted, published, and governed? Who should have a say in these outcomes? What viable alternatives exist? Since 2001, there have a variety of national and global initiatives to increase biosecurity, while not unduly inhibiting responsible scientific innovation. Various countries are continuing to develop or revamp their biosecurity regimes. The traditional "bottom up" approach of scientist self-governance for biosecurity is increasingly in question, but controversial changes to the National Science

Advisory Board for Biosecurity in the United States indicate that top-down approaches are also not a panacea.

This special issue was devoted to contributions that explored this matrix of issues from a variety of case study and international perspectives. This issue was a challenge to manage because of the rapidly evolving nature of developments in the field even from when we first issued the call for papers in December 2013. Emblematic of the topic itself, this made it difficult to draw a line on what to include in the issue as new submissions were entered and new controversies arose. The debate and discussion over the dual-use implications of emerging infectious diseases and the life sciences continues and will continue in the foreseeable future. We thank the authors of this special issue for an excellent set of papers to starting framing and prioritizing the national and international dialog on these timely issues.

The articles in this issue ultimately clustered around five central themes: (1) dual-use as a unique kind of policy problem; (2) involving diverse stakeholders in dual-use discussions; (3) instituting a culture of responsibility among scientists; (4) producing more evidence-based risk-benefit analyses of dual-use research; and (5) developing greater oversight, control, and standardization of dual-use procedures.

For the first theme, Rappert (6) noted that dual-use has been a largely"non-problem"– a curious phenomenon. Rappert (6) notes that although much concern has been cited with the misuse of the life sciences since September 11, there have been very few research identified as "of concern." Moreover, Jefferson and colleagues (7) find that there has been a lot of mythmaking around synthetic biology that has been used to mobilize support, resources, and action for focusing policy attention on this field. Koblentz (8), however, finds that dual-use is an inherently "wicked problem" that makes it resistant to long-lasting solutions. In contrast, Murdock and Koepsell (9) argue that dual-use research is a classical principal-agent problem and that this kind of asymmetry between governments and scientists, creates the tensions that we see in regulation.

Connecting to the second theme, Suk et al. (10) argue that the public health sector could be brought in more to dual-use discussion to help guide policy decisions and promote actions along all phases of the research cycle. To date, the authors find that public health perspectives have been an underutilized resource in dual-use/biosecurity discussions. Kosal (11) argues more dramatically that improving public health could serve as a powerful active deterrent to those who might wish to launch a bioterrorist attack.

Laboratory biosafety and biosecurity are closely intertwined concepts, both dependent on compliance with appropriate regulations, laws, and oversight mechanisms. In both, there is no such thing as "zero risk." As concerns biosafety, it is important to note that safety consists not simply of the design of laboratories, but also crucially in the trained people that work there, the implementation of regulations, and the use of robust risk-based approaches to mitigate adverse events. In the third theme cutting cross the papers, Jacobsen et al. (12) and Sijnesael et al. (13) argue that we need to internally cultivate responsibility within specific organizations that handle dangerous pathogens, as well as the larger life science community. These authors also argue that we need a more diverse set of stakeholders in discussions about dual-use issues and in the development and implementation of new oversight and assessment measures. Further, Jacobsen et al. (12) as well as Klotz and Sylvester (14) both argue that we need more quantitative and qualitative risk-benefit analyses for assessing research with dual-use potential. In the area of governance, Smith and Scott (15), as well as Lev and Samimian-Darash (16), Ehrlich (17), and Jacobsen et al. (12) all advocate for the need for new oversight and governance structures for research and funding of dual-use science.

In sum, what we see from these papers and continuing media coverage is that the debate of dual-use is growing, gaining more public, expert, and policy attention. But as the papers in this issue suggest, these debates need to happen at higher policy levels. Moreover, there is the need for more inculcation of scientific responsibility and norms at the local, national, and global level. Countries need to continue to work on improving their biosecurity efforts and develop some key indicators to not only show that they are committed to biosecurity and biosafety, but they are implementing, monitoring, and assessing key aspects at the local and national level. This needs to include not only academic and government research institutions, but also those in the private sector. Within this context, the underlying objective should be, ultimately, to improve and not threaten public health, but exactly how to do this is remains an outstanding and elusive question. Finally, we need more review and accounting both nationally and globally about what biosecurity measures are in place, what gaps still exist, and how to remedy these shortcomings.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 08 December 2014; accepted: 09 January 2015; published online: 02 February 2015.*

*Citation: Vogel KM, Ozin AJ and Suk JE (2015) Biosecurity and dual-use research: gaining function – but at what cost? Front. Public Health 3:13. doi: 10.3389/fpubh.2015.00013*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2015 Vogel, Ozin and Suk. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Why has not there been more research of concern?

## **Brian Rappert \***

Department of Sociology, Philosophy, and Anthropology, University of Exeter, Exeter, UK

#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Tomoko Y. Steen, Georgetown University School of Medicine, USA Judith Reppy, Cornell University, USA

#### **\*Correspondence:**

Brian Rappert, Department of Sociology, Philosophy, and Anthropology, University of Exeter, Exeter EX4 4RJ, UK e-mail: b.rappert@ex.ac.uk

Amid the renewed concern in the last several years about the potential for life science research to facilitate the spread of disease, a central plank of the policy response has been to enact processes for assessing the risks and benefits of "research of concern."The recent controversy regarding a proposed redaction of work on the modification of a H5N1 avian influenza virus is perhaps the most prominent such instance. And yet, a noteworthy feature of this case is its exceptionalness. In the last 10 years, life science publishers, funders, and labs have rarely identified any research as "of concern," let alone proposed censors. This article takes this experience with risk assessment as an invitation for reflection. Reasons for the low number of instances of concern are related to how the biosecurity dimensions of the life sciences are identified, how they are described, how the assessments of benefits and risks are undertaken, how value considerations do and do not enter into assessments, as well as the lack of information on the outcomes of reviews. This argument builds on such considerations to examine the limitations and implications of the risk–benefit experiment of concern framing, the politics of expertise as well as the prospects for alternative responses.

**Keywords: rationality, risk–benefit assessment, dual-use research of concern, precaution, biological weapons convention**

## **INTRODUCTION**

Throughout recorded history, attempts have been made by some to stop others from acquiring means of inflicting harm. From sixth century BC, efforts to check the spread of the formula for Greek fire to twentieth-century efforts to restrict designs for atomic and nuclear weapons, groups, and nations have exerted themselves to limit the potential for the diffusion of destructive capabilities – sometimes with specific users in mind, sometimes simply to anyone else. Attempts at control have extended far beyond weaponry itself. In different ways, natural resources, animals, information, and individuals have been subject to restriction, sanction, and suppression. Such attempts have been conceived in response to the hopes, events, fears, and preoccupations of their times.

Particularly since 9/11 and the subsequent anthrax postal attacks in the US, research in the life sciences has become an object of apprehension in relation to who might use it for what purposes. The question of how to prevent the life sciences from becoming the death sciences has been posed and answered in ways that raise questions for longstanding preoccupations and practices. Attention has extended beyond the access to pathogenic agents to also include scrutiny of what can be called "information products." For instance,a central plank of recent biosecurity-related responses has been to develop processes for assessing the outputs of experiments. Much of this attention has been couched in terms of the imperative to weigh risks and benefits of openness. For instance, since 2003 a number of civilian science journals have established procedures for reviewing individual submissions in relation to whether "the potential harm of publication outweighs the potential societal benefits" (1).

This and similar activities undertaken by funders, university departments, and others have prompted wide-ranging discussion, typically framed in terms of where the balance should be struck between scientific freedom and national security. Much debate, sometimes heated, has taken place about the appropriateness of restricting what research gets done and how it is communicated.

Interestingly, though it is widely acknowledged that almost any knowledge and techniques in the life sciences can be used for destructive purposes, in practice it has been rare that risk assessments have identified anything as "of concern"; meaning that it poses clear possibilities for harm. It has been much rarer still that the harms of research have been deemed to outweigh its benefits.

This article takes this experience as an invitation to question how and why this is the case. The argument is divided into six sections. The next section recounts the recent history of attention to the security implications of the life sciences, with particular reference to the identification and assessment of "research of concern" and related designations. As will be argued, despite the limited identification of concerns and frequent expression that weighing the future benefits and risks associated with individual instances of research is not feasible, the enacting of assessment procedures remains a central strain of current international biosecurity efforts. The third section then asks how it is that the measures enacted to spot concerns rarely do so.

The fourth section elaborates on the pervasive but tensionridden notions of "rationality" that underpin the assessment of experiments of concern. The fifth section offers alternative ways of conceiving of concerns associated with the destruction implications of the life sciences. These speak to issues about the politics of expertise. In particular, it will be argued that rethinking the terms of the present debate enables new possibilities for understanding the relation between science and society as well as the place of precaution in biosecurity.

#### **A RECENT HISTORY OF CONCERN**

Regard for the link between the production of knowledge and the capabilities for inflicting disease has a long history. A recurring theme of much of the previous century and a half of modern biology has been the manner in which the latest understanding of disease fed into state and other biological weapons programs (2). This section elaborates how such regard has led to the recent notion that research might be "of concern."

To begin with, it can be noted that proposals for controlling intangible knowledge and information did not figure prominently within Western life science policy discussions in past decades. For instance, in the years prior to 9/11, many analyses considered the new destructive possibilities enabled by developments in biology and related fields (3–5). Proposals for what needed to be done centered on strengthening physical controls on the transfer of pathogen agents and who has access to them. In this vein, in the immediate aftermath of 9/11 and the US anthrax letter attacks, initial legislative measures (such as the 2001 US *PATRIOT Act* and the later *Public Health Security and Bioterrorism Preparedness and Response Act of 2002*) enhanced requirements on the registration, movement, storage, and use of deemed dangerous bioagents as well as who could legitimately access them (6). Similar controls were introduced in a number of other countries.

Of note then, post-9/11, there have been suggestions that the outcomes of fundamental research might need to be scrutinized and restrictions imposed because of their security implications. As an example, in late 2001 the former head of research at SmithKline Beecham, George Poste, in the role of chair of a US Department of Defense task force on bioterrorism called on biology to "lose its innocence" regarding its security sensitivities (7). For him that meant enacting procedures for vetting, classifying, or otherwise restricting what research gets done and published. Similarly, at that time Epstein examined the possible contribution of civilian science for enabling destructive capabilities. He offered the category of "contentious research" to denote "fundamental biological or biomedical investigations that produce organisms or knowledge that could have immediate weapons implications, and therefore, raise questions concerning whether and how that research ought to be conducted and disseminated" (6).

A prime example of the type of research that raised questions for both Poste and Epstein was the early 2001 publication detailing how Australian scientists inserted the interleukin-4 gene (IL-4) into the mousepox virus as part of efforts to devise a contraceptive for rodent populations (8). This manipulation resulted in a modified mousepox with significant mortality rates for nonimmunized, immunized, and genetically resistant mice. The worry was that the publication of these results could provide a technique for enhancing the lethality of other pox viruses, including smallpox. Like others at the time, both Poste and Epstein also voiced apprehension that if scientists did not initiate a discussion about what controls might be needed for security sensitive knowledge, then they risked others imposing draconian measures on them.

At least in the US, efforts were made during 2001–2003 to set in place a potential basis for restricting research findings because of how they might aid bioterrorism. The *Homeland Security Act of 2002* included the requirement that US government agencies "identify and safeguard homeland security information that is sensitive but unclassified" (9); a provision that was feared would be applied to basic science. One discussion about the potential for restricting publications identified likely problems and stipulated that any system of publication review should have the "support of the international scientific community, which must perceive that the security benefits of restricting open publication outweigh the possible costs to science" (10).

At the time, there was little evidence of such widespread support. As previously mentioned, in early 2003 an informal group of 32 largely American based journal editors agreed voluntary guidelines for reviewing, modifying, and if necessary rejecting research articles where "the potential harm of publication outweighs the potential societal benefits." (1) Yet this enactment went hand in hand with expressions of apprehension – not least voiced by those signed up to the guidelines – that security motivated restrictions or oversight measures might unduly jeopardize the advancement of science (11–14). A common refrain expressed both by those with roles in national security agencies and in life science professional organizations was that security might well be compromised overall if the said free exchange of information underpinning research was hindered (15–18).

What would become arguably the most prominent statement about the potential for the techniques, methods, and knowledge generated through life science research to aid destructive purposes was given in late 2003 by a US National Academies report titled "*Biotechnology Research in an Age of Terrorism*" (19). It recommended extending existing (largely self-governance) mechanisms already in place in the life sciences. In relation to the themes of this article, one recommendation called for the initiation of a system of pre-project review for so-called "experiments of concern." Seven such categories were specified in the report; this included research that would:


It was argued that work that fell in these categories should be reviewed by existing biosafety and recombinant DNA review procedures for its security implications. Echoing a theme prevalent elsewhere, the report recommended this while also noting the importance of not jeopardizing the norm of open communication in science.

Through the sorts of initiatives mentioned in the previous paragraphs emerged a sense of the potential security implications of the life science research outcomes and the need for oversight measures. Those notions largely emanated from the US and they were directed at discrete instances of research situated at the nexus of terrorism and biology. In other countries at the time, the"experiments of concern" framing would be varyingly taken up, rejected, or ignored (20).

In the years after 2001, just how much of a threat was really posed by research was subject to varying assessments informed by alternative criteria about what harms mattered as well as what lessons should be drawn from past history about the likelihood and severity of bioattacks (21). Despite such differences, calls for identifying and assessing sensitive knowledge at the time generally shared a number of features including: the stated need not endanger the benefits of science that are derived from its openness; the encouragement to scientists to act before controls was placed on them from elsewhere; and the object of scrutiny being the future risks and benefits associated with individual experiments.

In relation to the last point, regard was directed at a limited number of such instances. Besides the previously mentioned IL-4 mousepox research, other prominent experiments were the 2002 publications detailing the successful artificial chemical synthesis of poliovirus (22) and the comparison of a type of smallpox and its vaccine that suggested a means of increasing the vaccine's lethality (23).

#### **EXPERIENCE WITH ASSESSMENTS**

The attempts to identify and assess sensitive knowledge noted above sought to establish key points at which to make determinations about whether specific instances of research should go ahead or be communicated; this based on their anticipated potential future harms and benefits. In the years that followed the initial articulations of the "experiments of concern," this manner of framing the security implications of the life sciences would become more widespread within international policy discussion. For instance, after the publication of *Biotechnology Research in an Age of Terrorism,* a number of similar calls were made to put in place "harm–benefit" or "risk–benefit"-related reviews of research, such as the World Health Organization's (WHO) *Life Science Research: Opportunities and Risks for Public Health* and the American Medical Association's *Guidelines to Prevent Malevolent Use of Biomedical Research*. The British-based Biotechnology and Biological Sciences Research Council, Medical Research Council, and Wellcome Trust did adopt review procedures for grant applications that posed a potential for misuse in 2005 (24). Despite such developments, little public articulation was given to how such assessments could be or were being conducted in practice.

Following directly from one of the recommendations of *Biotechnology Research in an Age of Terrorism,* in early 2004 in the National Science Advisory Board for Biosecurity (NSABB) was formed to provide advice on oversight strategies, guidelines, and education regarding the handling of federally supported "dualuse" research. Included within its remit was the devising of criteria for identifying and evaluating the risks and benefits. In 2007 as part of the document *Proposed Framework for the Oversight of Dual-Use Life Sciences Research*, it offered a split between two kinds of science: "dual-use research" was used "to refer in general to legitimate life sciences research that has the potential to yield information that could be misused to threaten public health and safety and other aspects of national security such as agriculture,

plants, animals, the environment, and material" (25). Since nearly all science could be used in this manner, NSABB offered another category of "dual-use research of concern" (DURC). This denoted "research that, based on current understanding, can be reasonably anticipated to provide knowledge, products, or technologies that could be directly misapplied to pose a threat to public health and safety, agricultural crops and other plants, animals, the environment, or material" (23).

Within the framework envisioned by NSABB, should Principal Investigators determine that they are conducting DURC research it would then be subjected to institutional risk review to assess: "the likelihood that the information might be misused; the potential impacts of misuse [and] [s]trategies for mitigating the risks that information from the research could be misused" (26) In this way, a general framework for the risk assessment of individual research instances was elaborated.

While the activities of NSABB and others in relation to the scrutiny of research results have generated public, policy, and ethical discussion about the dangers they pose for science (27–29), one notable feature of the reviews is how few publications, grant applications, or project proposals have been identified as posing concern. Take the time period following the initial articulations of the category of "experiment of concern." In a sample of 16,000 manuscripts submitted to the journals of the American Society for Microbiology after they adopted the 2003 journal publication guidance, only 3 were subjected to additional biosecurity peer review. By the end of 2006, the Wellcome Trust reported having identified three proposals as requiring additional security scrutiny with none judged to pose an overall concern on balance (26). Also, a US National Research Council report titled *Seeking Security: Pathogens, Open Access, and Genome Databases* argued against the prospect of being able to identify genomic data with significant security worries (17). Even in the case of the 2005 publications related to the sequencing of the 1918 Spanish Flu virus (30) and its subsequent artificial reconstruction (31), the benefits were deemed to outweigh possible risks by the journals involved. It was such experience up until 2007 that lead NSABB to anticipate "few" cases would fit into the DURC category and therefore that the initial assessment of experiments by Principal Investigators should not be time consuming (32).

This overall pattern of finding little of concern has continued through until today (33). Between 2009 and early 2014, the Wellcome Trust has flagged only two applications to its funding committee for scrutiny in relation to their misuse potential, with both not being funded on the basis of their scientific merit rather than due to security concerns (David Carr, personal communication, 12 February 2014). Of the 74,000 biological submissions to the Nature Publishing Group between 2005 and 2008, only 28 were identified as having a dual-use potential, with none rejected for this reason (34). The Danish Centre for Biosecurity and Biopreparedness has licensed projects in the Denmark that produce new technologies of a directly weapons potential and has not identified any cases of DURC publications (John-Erik, personal communication, 29 January 2014).

Such an overall situation is remarkable within the context of the multi-billion dollar increase in biodefense research funding in the US after 2001, much of it supporting civilian research (35). This massive expansive directed funding toward the type of work that would likely be of concern, and yet few such instances have subsequently been identified in practice. The US National Institute of Allergy and Infectious Diseases of the National Institutes of Health (NIH) distributed much of the funding for biodefense research. Its director reportedly indicated that in recent decades, the NIH has never had an instance in which funded research was retroactively judged as having been funded or published improperly (36, 37). Instead of large number of diverse instances of research being flagged on a regular basis, since 2003 a limited list of several experiments have come to be repeatedly cited (38), the latest at the time of writing being the reverse genetics creation and then mutation of a virus resembling the 1918 Spanish Flu virus (39).

Such experience makes it important to note that while the *Proposed Framework for the Oversight of Dual-Use Life Sciences* and other initiatives outlined processes of assessment, they did not specify in practice how potential future benefits and harms could be assessed and weighed. At the time, perception of this gap led to calls for the development of new risk assessment tools, often couched in terms of the need for objective quantification of the likelihood and impacts of bioattacks (40, 41). Within the work of NSABB itself, belief in the prospect of rigorous and value neutral calculations have been made alongside recognitions that the evaluation of dual-use potential of research inevitably would be subjective (42).

While in practice, few experiments were being identified as posing significant security concerns until and after the launch of the 2007 *Proposed Framework for the Oversight of Dual-Use Life Sciences*, this has come alongside contentions that practicing scientists have been largely unaware of the malign applications of their research. The World Medical Association, the US National Academies, the British Royal Society, the International Committee of the Red Cross, the Wellcome Trust, the InterAcademy Panel, NSABB, the International Council for Science as well as others have argued that practitioners needed greater education about the potential dangers associated with their work (43). In theory at least, the need for such enhanced understanding left open the possibility that a different pattern of review outcomes might emerge once individuals possessed the requisite awareness.

Calls for greater education have not been restricted to scientists though. Another accompanying current of dual-use discussions has been the repeatedly expressed anxiety about public understanding. For instance, over the course of its deliberations the NSABB Communications Working Group expanded attention from the time of its creation on the security threats stemming *from* research to include the threats *to* research posed by public misconceptions (44).

#### **THE EXCEPTIONAL CASE OF H5N1**

Between June 2007 and late 2011, NSABB's *Proposed Framework for the Oversight of Dual-Use Life Sciences* faced an uncertain future waiting for an official response by successive US administrations. The attention to dual use transformed significantly in late 2011 when a set of experiments on the H5N1 influenza virus became high profile. At that time, two groups lead by Ron Fouchier at Erasmus Medical Center and Yoshihiro Kawaoka at the University of Wisconsin, Madison submitted manuscripts to *Science* and *Nature* respectfully related to the mammalian transmissibility of a strain of H5N1; specifically indicating how a genetically mutated form of the H5N1 influenza virus could become transmissibly airborne between ferrets (45, 46). Up until that time, H5N1 was only known to be transmittable through direct physical contact. Although exactly what had been demonstrated would become a matter of controversy, this work identified a possible casual link between genetic munitions and airborne transmission between mammals more generally.

National Science Advisory Board for Biosecurity reviewed the publications and concluded they should go ahead, but minus certain details so as to reduce their malign potential (47). In the wide-ranging debate that followed, a year long moratorium was initiated by a group of 40 flu researchers (48). Both these moves reignited debates about the security implications of the life sciences – typically framed in terms of whether the freedom of science should be jeopardized in the name of security. The WHO convened an international meeting in February 2012 that heard additional non-public information about the experiments (49). That meeting concluded that full versions of the articles should be published once issues associated with public messaging had been addressed. In response to the controversy, in March 2012 the US Department of Health and Human Services issued a revised policy for DURC life science research (50).

While this experience with H5N1 has come to dominant recent discussions associated with the governance of experiments of concern and spurred renewed attention to implementing review procedures (51–53), what is perhaps most notable is its *exceptionality*. It is exceptional both in relation to the recommendation to withhold details for security reasons and the extent of policy and public discussion that took place.

With regard to the former, the recommendation of restricting details was to be subsequently overturned. In late March 2012, NSABB was reconvened and reversed its decision in voting overall in favor of publishing revised forms of both disputed papers. In justifying this shift, the Board cited the availability of new information that reduced worries about the ability of the research to immediately enable malign capabilities and that increased its public health benefits (54).

The case of H5N1 is similar to other discussions about experiments of concern though in its fraught relation with risk–benefit assessment. In reversing its initial decision, for instance, NSABB contended that "The Board's discussions were informed by the analytical frameworks that it previously developed for considering the risks and benefits associated with the communication of DURC." (54) That framework was the 2007 *Proposed Framework for the Oversight of Dual-Use Life Sciences Research: Strategies for Minimizing the Potential Misuse of Research Information.* Yet, as previously mentioned, this framework did not specify how potential future benefits and harms could be assessed and weighed in practice. Instead, it laid out organizational processes for handling DURC instances.

As another strain of the troubled status of risk–benefit assessment, apprehensions about the way NSABB conducted the assessment of benefits and risks was given in a critical response letter to the NIH leaked to the press. With regard to one of the controversial papers (subject to a 12–6 split decision in favor of publishing at the March 2013 NSABB meeting), a Board member lamented:

I believe there was a bias toward finding a solution that was a lot less about a robust science- and policy-based risk–benefit analysis and more about how to get us out of this difficult situation. I also believe that this same approach in the future will mean all of us, including life science researchers, journal editors and government policy makers, will just continue to "kick the can down the road" without coming to grips with the very difficult task of managing DURC and the dissemination of potentially harmful information to those who might intentionally or unintentionally use that information in a way that risks public safety (55).

Some commentators would go further, drawing the conclusion that weighing benefits and risks in relation to DURC issues was not feasible (56). Yet elsewhere, belief continued to be placed on the need for "careful consideration of the scope and magnitude of the potential risks and benefits associated with the research proposal, evaluation of whether the risks outweigh the benefits, and strategies for mitigating potential risks" (57) – as stated in the early 2013, NIH guides for US Department of Health and Human Services' framework for funding decisions on individual proposals involving highly pathogenic avian influenza H5N1 viruses.

International attention to devising processes for identifying and evaluating research along these lines continue. The need for DURC-type oversight frameworks has been made elsewhere, including by some governments as part of the Biological Weapons Convention (58).

#### **WHY IS THERE NEARLY NOTHING?**

For more than a decade, attention has been cast to the potential destructive application of knowledge generated from life science research and what, if any, governance measures need to be in place to advert their realization. While varying in their specifics, the attention to what can generically be called "research of concern" indicates a movement beyond traditional biosecurity preoccupations about materials, equipment, and personnel.

The previous section though drew attention to some curiosities: despite the importance often attached to assessing concerns, in practice few such instances have been identified. Moreover, since 2003 it would appear that (in the end) in no case of civilian formal reviews have the risks been deemed to outweigh benefits. On the back of this track record, important questions can be asked, such as: "how is it that so little concern has been identified?,""how is belief in the value of assessment processes maintained despite their apparent lack of implications?," and "what alternative ways of understanding are possible?"

This section principally addresses the first of these questions. It does so by examining the identification and weighing benefits and harms in order to suggest why cases have not been identified.

#### **WHAT ARE THE OBJECTS OF CONCERN?**

Consider first the basic framing given to what is of concern. Whatever their other differences, the varied attempts to establish research of concern have generally shared the bounding of evaluations around specific instances of research. Both within assessment procedures and educational material (59), this means attention gets cast at individual (or in some cases more than one closely related) research applications, experiment proposals, and submitted manuscripts. Such instances are envisioned as the holders of potentially sensitive knowledge.

With such a focus, signaling out one piece of knowledge as of concern requires being able to separate out its contribution to the general stock of knowledge from all others. As scientific and technical developments are typically cumulative accomplishes, this is often difficult. Against past attempts to contend that a particular set of findings raised concern, counter claims have been made that previous work was suggestive of or already indicated grounds for concerns (60–62). The less a distinctive break from what was previously known, the more difficult it becomes to justify any security apprehension.

In contrast, rarely in policy discussions to date have assessments been offered at lines or programs of work (63). Taking these as the object for scrutiny though arguably opens up a space for wider set of questions and possibilities. For instance, the publications in 2005 pertaining to the sequencing of the 1918 Spanish Flu virus and its artificial reconstruction were only the end culmination of a long line of funded and published research (64). As a result, it was possible to scrutinize the activities associated with the 2005 publications well before the results were sent to in *Science* and *Nature*. Instead of asking "should this particular experiment go ahead or be published?" alternative broader questions could include "what lines of research should be funded in the first place?" The latter is important to acknowledge because in situations of limited funding, choices are inevitably made about, which research to support and which to not (65). As such, when a WHO report on its 2013 DURC meeting stated:

Scientific research is conducted in virtually all countries and is critical to strengthening global response to all health threats and hazards, including those posed by naturally occurring and by accidentally or intentionally released biological agents. The only way to eliminate the potential for misuse of DURC is to not perform research. Such an extreme solution, however, is neither feasible nor advisable (66).

It arguably did not make a room for acknowledging that choices are routinely made to back some lines of research over others (65). For all the roads taken, there are many not pursued.

The limiting of attention to individual experiments or publications is also consequential for the identification of concerns because it generally directs attention toward the latest, and thereby often most technically sophisticated, expensive and thereby exclusive research. Because of this sophistication, doubts can be raised about how feasible that it is that other groups can reproduce the work (67). The resulting situation is one much more difficult to assess than if consideration were directed at what capabilities are becoming widely accessible.

#### **HOW ARE CONCERNS IDENTIFIED?**

Working within the common conceptualization of individual instances of research being the potential holders of concern, further questions can be asked of the assessment procedures and practices enacted to date.

As previously noted, a variety of organizations have underscored the importance of practicing scientists being cognizant of the destructive potential of their activities. Without this awareness, assessment procedures reliant on Principal Investigators to identify concerns could not function as envisioned. Against this need though, many empirical studies have indicated such an awareness is possessed by relatively few practitioners (68). Thus, the relative infrequency of the identification might be attributed to a lack of awareness. This consideration along with the conflict of interest associated with researchers judging their own work led the Center for International and Security Studies at Maryland to forward an oversight system that requires independent peer review to include those with scientific and security expertise (69).

The contingencies associated with how research is and is not identified as posing concern can be highlighted through examining the regard given to the potential of research both before and after periods of prominent attention. For instance, in the case of the early 2001 IL-4 mousepox publication, the Australian scientists involved have argued that work undertaken prior to 2001 by others and in follow-on work they performed *after* 2001 indicated how to enhance the lethality of viruses (70). Yet, professional and public regard for those developments has been muted.

Other grounds can be offered for suggesting formal reviews might be limited in how they determine concern. The comparison between formal reviews and informal practice is one such basis. In a 2007 survey undertaken by the US National Research Council and the American Association for the Advancement of Science (AAAS), AAAS members with an interest in the life sciences were asked about their familiarity and experiences associated with dual use. Nearly one in six indicated they had made some sort of change to their research – for instance, whether it was undertaken, with whom, and how it was communicated – because of worries that the knowledge, tools, or techniques might be used in bioterrorism. The low response rate (16% completed the survey in full) means the findings were not statistical representative. However, they signal a level of regard not being registered through the formal assessment procedures enacted by published, funders or organizations (71). The criteria individuals employ in making self-determinations about the potential of their work would be a likely important topic for understanding rates of identification.

A relatively prominent recent case of *researcher-initiated* restrictions was the publication in 2013 of a new type of botulinum neurotoxin designated as BoNT/H (72, 73). With no effective treatment for this form of botulism, the researchers decided to withhold the sequence data on BoNT/H from their write-up of the research until an antitoxin is developed. In this case, the authors first consulted with various US federal government agencies about the advisability of publishing these and then secured agreement from the journal to publish without the sequence data or their submission to the International Nucleotide Sequence Databases (74, 75).

#### **HOW ARE RISKS AND BENEFITS DETERMINED?**

Even when concerns are recognized, determining the risks and benefits has proven highly taxing and would likely be so into the future.

One challenge is that assessments of risks and benefits vary considerable. For instance, based on lab observation research and interviews, Bezuidenhout has argued distinct ways of making sense of risks and benefits exist between scientists in sub-Sahara Africa and those prevalent in Western dual-use discussions to date (76). Within the former, dual-use risks were regarded as hypothetical, biosecurity harms were frequently defined in relation to gross lab deficiencies in local waste disposal, and the benefits of research were associated with its ability to address disease in the immediate term.

Another often identified challenge is the inability of the many of those associated with the life sciences to assess the potential for malign applications. In classic risk assessment models, the expected value of risk is taken as a function of the likely probability of an event times its consequences. In relation to formal reviews for research of concern, given how the objects of concern are typically defined, what is demanded then is a way of assessing the possibility that unspecified users would draw on individual sets of findings toward the development of an unfixed range of destructive capability in a time frame that is not specified. Then assessors need to determine the expected consequences of such an action against likely available countermeasures. A fully developed notion of threat would also require regard for the intent of potential users.

As many have contended, practicing scientists are often not knowledgeable about the capabilities or intent of those that might employ their work for hostile ends (6, 70). The same has been argued for those that typically make up biosafety committees in universities and elsewhere (56). In this regard, it should be underscored that what is required for assessing dual use is twofold: one, information about matters such as motivations and capabilities and two, a competency through methods, concepts, and theories to assess experiments (77).

The extent to which either dimensions can be grasped at all in the case of dual-use life science research appears an open question. Just how much information is available and could be made widely accessible about the motivations and capabilities of wouldbe users is unclear; especially given the relative dearth of bioattacks in recent years that might provide a (however tentative) baseline for future extrapolation (78, 79), the clandestine status of any existing state or sub-state bioweapon programs, and the focus in reviews given to cutting edge capabilities enabled by the latest science.

In addition, though, despite the aforementioned importance often attributed to devising methods for determining the security risks associated with research of concern, little by the way of detail have been given about how this could take place (69, 80). The absence of methods for determining risks is a particularly salient point in relation trying to make sense of concerns outside of traditional agents used within biowarfare programs.

As such, much of the consideration of research of concern could be characterized as taking place in conditions of "ignorance" – that is in conditions characterized by limitations in both information and methods for assessment (77).

Yet, a further sense of the difficulties of determining risks is evidenced in how security related implications should be interpreted. To start, as has been repeatedly argued in relation to the DURC designation developed in the US, "characterization of research as DURC should not be viewed pejoratively" (81), meaning it need not necessarily be stopped, censored, or otherwise restricted because it is determined to be "of concern." But questions of interpretation go beyond this point of a non-negative evaluation. The identification of concern has heighted the *positive* value attached to research because of what it suggests for assessing threats and countermeasures (82, 83). A notable feature of many of the experiments of concerns of the last decade is how the initial work led to follow-on activities undertaken worldwide and justified on both scientific and biodefensive grounds. The identification of the need for such follow-on work has led some to express anxiety about the risks to society from *restricting* dual-use information (84).

Whereas the downside potential of research is widely regarded as difficult to assess and often subject to radically diverging evaluations, the contention that benefits can be expected to accrue (however, much in the future, however, indirectly) is a starting point for many commentaries (80). In short, research is categorically taken as "an essential public good" (85). While the certainty or even likelihood of research leading to health improvements has been queried elsewhere, such doubts are rarely voiced within dualuse discussions (86). The case of H5N1 was a notable exception in the manner in which detailed questions were raised about its utility (87).

#### **HOW ARE RISKS AND BENEFITS WEIGHED?**

In classical risk assessment models, once risks and benefits are identified, these should be weighed against each other so that a net assessment can be reached. In the case of research of concern, for instance, this is expressed in the manner some publishers have committed themselves to assessing whether "the potential harm of publication outweighs the potential societal benefits" (1). Given the "ignorance" that often characterizes determinations of dualuse risks though, undertaking such a weighing has and will likely be bedeviled by problems.

In theory at least, such a situation could lead to a range of possible outcomes. For instance, post-9/11 in the US [and elsewhere (88)], fears about low probability but high-consequence terrorist attacks justified a range of domestic anti-terrorism measures and military actions (77). Parallel uncertainties and unknowns in relation to research of concern could have resulted in sweeping restrictions. This, however, has not taken place.

What explains this difference between the types of responses made in previous years? One set of considerations would seem to be the basic presumptions informing weighing. For instance, as mentioned above the default position has been that risks with research of concern need to be substantiated, whereas the benefits from research are typically assumed (41). Another prominent set of presumptions is that life science research – in the absence of security related controls – is characterized by the free and open flow of information, that such a situation is vital for the scientific progress, and that therefore any attempt to move away from this default needs to be justified (80). A related corollary is that once knowledge has been generated, it is not possible to undo it or restrict its flow (89). With such widespread presumptions, controls are difficult to justify.

Both lines of thinking are arguably questionable though. Social studies of the practice of science have indicated how the exchange of information in research is frequently subject to negotiation and limitation in practice – not least because of commercialization goals (90). In addition, to subscribe to the view that knowledge once generated is simply "out" and uncontainable relies on a reduction of knowledge to abstract and explicit propositional statements. In contrast, it is possible to highlight the practical skills, understandings, and competencies necessary to reproduce and utilize specific research. These ways of knowing are crucial to many aspects of the production of biological and nuclear weapons and, as such, some scope exists to affect (and even reverse over time) the proliferation of capabilities (91).

As Buchanan and Kelley argue though, the very attempt to pitch risks and benefits against each other and ask how they can be "traded off" is consequential. Such an approach often discounts what does not fit under the heading of "open science"or"security." As they argue, within the typical dual-use framing:

. . .it is the interests of only two parties that are likely to be strongly represented: scientists who fear constraints on the pursuit of knowledge, and government officials whose worst nightmare is a bioterrorist attack that could have been prevented. Therefore, one of the dangers of an overly simplistic framing of the ethics of biodefense is that it largely ignores or arbitrarily discounts values that have been central to the research ethics debate since its inception: the protection of research subjects, both human and non-human [i.e., animal] (92)

With this silencing, weighing is likely to be skewed.

This formulation of the limitations of dominant framings today itself though arguably makes questionable presumptions. As with much of the discussion about biosecurity generally and research of concern specifically, Buchanan and Kelley treat the issues at stake as subject to contention by two competing communities with distinct interests: those on the side of "science" and those on the side of "security" (93). It is the latter "security community" that is treated as seeking restrictions on what research gets done and how it is communicated. Appeals to such a community have been routinely evoked in dual-use discussions, though without defining its membership.

In practice, it is difficult to identify a coherent security community in relation to the specific topic of "research of concern," let alone one that has worked in a concerted effort to imposing restrictions. This is the case both outside of the US (where, in general, dual-use concerns have been more muted and biosecurity expertise within national security communities is more limited) as well as in the US. Indeed, some of those raising the most significant worries about threats to science have been those that would likely be identified as part of "the security community" (89). In the absence of a coherent group consistently forwarding securityinspired restrictions, the track record of the last 10 years is not surprising.

#### **HOW HAS EXPERIENCE BEEN EVALUATED?**

In models for managing risk, much emphasis is often placed on scrutinizing experience and modifying assessments in response. As with the aforementioned components, here too points can be suggested about why there has been little research of concern.

One pertinent point is the lack of systematic data on how often experiments and publications of concern have been identified and the decisions reached as part of formal reviews. While some figures have been made available at meetings or in publications, and some analysts have complied information (33), the resulting picture of practice has been fragmentary and partial. Such a situation stifles learning from experience.

In this respect, an interesting feature of the discussion about this topic is how experience to date is often not taken as relevant to informing policy recommendations. For instance, in an otherwise wide-ranging and empirically rich analysis of the dual-use policies of biomedical journals, Resnik and colleagues lamented on the low rate of journals with such policies in place (94). To correct for this, they called for journals to develop such policies. Yet, this analysis did not seek to determine the implications (if any) of the reviews undertaken and thereby their practical relevance (95). Instead, the utility of reviews was assumed. In general, a lack of evidence about the results of reviews undertaken characterizes other prominent statements on this topic (63).

At least in relation to US federally funded research, the absence of information may change. In March 2012, the Federal government issued a policy titled "United States Government Policy for Oversight of Life Sciences Dual-Use Research of Concern." It calls for a "regular review of United States Government funded or conducted research with certain high-consequence pathogens and toxins for its potential to be DURC in order to: (a) mitigate risks where appropriate; and (b) collect information needed to inform the development of an updated policy, as needed, for the oversight of DURC." (96). Figures compiled by the NIH in early 2012 indicated 381 extramural and 404 intramural projects using high-consequence pathogens or toxins. Ten of the extramural projects and none of the intramural projects were designated as DURC (97). At the time of writing, however, it is unclear what information agencies in the US will release on the outcomes of reviews.

#### **ASSESSMENT AND RATIONALITY**

Taken together, the previous sections suggested recent discussions about research of concern have been tension-ridden. On the one hand, much of the attention to this topic has been initiated in response to individual experiments, yet that object of scrutiny also delimits the scope for consideration. While a handful of instances of contentious research have served as prompts for wide-ranging calls to rethink the oversight of the life sciences, few other such examples have been identified and it has been exceedingly rare that risks have been deemed to outweigh benefits. Vocal, resolute, and repeated apprehension has been expressed about how securityinitiated reviews threaten the scientific enterprise, and yet to date formal reviews have had seemingly little bearing on what activity gets done or how it is communicated.

Despite the divergent ways of making sense of whether and what kind of concern should be associated with the informational products of research, much of the discussion shares a common object for scrutiny and a common language for thinking about assessing concern: namely, a focus on weighing the future benefits and risks of individual elements of research. An often recurring assertion has been that the extent of concern can be rendered known, and thereby manageable, through rationalistic "risk–benefit" assessment procedures.

At times, highly ambitious goals have been ascribed to assessments. A 2009 Royal Society workshop report titled *New Approaches to Biological Risk Assessment*, for instance, suggested dual-use risk assessments need "to link epidemiological modeling of disease, economic modeling, and qualitative social science modeling of human behavior" (98). Moreover, it added, "public perceptions and media reactions play an important role in driving policymakers" decisions on biological risks, particularly in the context of risk management and communication. Therefore, any risk assessment methodology needs to encompass assessment of human behavior and motivations, and any model needs to incorporate feedback loops to address the public's reaction to government risk management policies" (98). Achieving such aspirations for comprehensive rigor was said to require national and international harmonization through multidisciplinary analysis, a point echoed elsewhere (99).

The stating of such ambitions have sometimes gone hand in hand with recognition that doing so in practice would be frustrated by the demands of determining the risks associated with biological attacks. At times, these difficulties have been presented as surpassable through re-doubling efforts. For instance, in response to the recognition of uncertainty, the Royal Society's *New Approaches to Biological Risk Assessment* advocated that "given the different nature of the risks across the spectrum and varying availability of data against which to derive or test mathematical models, a common approach should incorporate a range of specific assessments at points on the spectrum coupled with an overarching model to unify the resultant risk assessments" (100).

On other occasions, a more fraught relation between expectations and demands has been presented. In 2013, an international meeting of prominent government officials, practicing scientists, law enforcement officials, life science representatives, and others met at Wilton Park for a meeting titled "Dual-Use Biology: How to Balance Open Science with Security." The outcome report of that meeting displays a desire, necessity, and possibility of definitive measures of risk and benefits as well as the challenges of producing them. With regard to the former, it was argued that:

Appropriate risk assessment should be part of the first phase of the research. Much work needs to be done to identify appropriate risk assessment factors relevant to DURC, taking into account the wide range of possible security concerns. In the future, a broader approach to risk could assess physical safety; economic security costs; diplomatic security; social and political stability; fear and anger and risk of research leading to the diminishing trust in government. It should also look at probability and take into account possible actors motives as well as intelligence on terrorist actors. Current DURC risk assessments have been largely "risk–benefit" analyses, and there is a need for much more comprehensive and quantitative risk assessments that specifically evaluate what could go wrong with certain research. The assessment should not be left solely to researchers and we need to incorporate all bodies and have a debate including governments which are responsible for crisis management and therefore need to consider responses (63).

And yet, while it was stated that "quantitative assessment sounds attractive because it feels evidence-based and hence more dependable and less open to counter-argument" (9), the Wilton Park report also noted that"the chances are that firm statistical data will be hard to come by, and that the sort of risks inherent in dual-use biological research cannot be quantified easily (which is not to say that they cannot be quantified at all)"(9). It was further contended that there is no"common understanding on how to conduct sound risk/benefit analysis; this is an issue between different states but also between different communities (scientific, security, etc.)" (7).

Though varying in their portrayal of the likelihood of achieving it, aspirations for comprehensive risk assessment methods have been made for years – this despite the lack of progress in that time toward specifying how risk–benefit analysis of research of concern could take place in practice. On this last point, the Wilton Park report contended that between "2005 and 2011 the NSABB established a risk/benefit methodology"; (3) a statement, which appears to conflate the process for the handling of risks and benefits with a methodology for determining risks and benefits.

Academic analysis of the prospects for risk–benefit assessment shares many of the same dynamics in treating research of concern as (more or less) susceptible to rational (often quantitative) analysis, but in practice being able to offer a limited articulation of how such assessment could be conducted (101, 102).

The need and prospect for elaborated formal risk–benefit assessment as a basis for decision making is not universally shared. Interviews undertaken by the author with one national biodefense establishment, for instance, indicated a preference for processes of dialog and professional judgment to identify concerns in contrast to the type of comprehension quantitative analysis sought elsewhere. The latter was judged as not necessary and not feasible.

Thus, the points above would suggest the continuing value placed with assessment processes has been promissory – the *future promise* of comprehensive assessments have been widely forwarded without explicit consideration of the ongoing inability to articulate how determinations of risk assessment could be made along the lines advocated (103). Such calls have shored up at least the prospect of the rational management of the dual-use concerns and thereby worked against arguments for rethinking the basic rationalistic framing of debates.

In contrast, this article has also offered reasons for questioning the prospect for achieving the types of comprehensive assessments envisioned. Arguably the situation is not simply one of uncertainty about the details of certain parameters associated with the type and extent of misuse risk nor is it the case that is only difficult to describe the likely outcomes of the malign application of research. Rather in many cases, both probabilities and outcomes are characterized by many unknowns and subject to different interpretations in such a way as to confound the devising of methods of assessment. If this appraisal is correct then it is necessary to foster other ways of understanding in order not to prematurely close down thinking. It is also necessary not to lend a false confidence to what is being grasped by existing review processes. For instance, the listing of funder and publisher review procedures has been forwarded at times as grounds for assurance about the level of scrutiny today (104). Whether that implication is warranted seems open to question given the argument above that the details about how assessments are being made makes it highly unlikely that expected risks would ever outweigh anticipated benefits.

#### **ALTERNATIVE POSSIBILITIES**

In recent decades, considerable effort has gone into asking how risks associated with science and technology can be handled more generally. A recurring theme from such investigation has been the need to recognize the fact that risk–benefit assessments are often of limited applicability in making decisions. When the outcomes and probabilities can be straightforwardly and consensually characterized, such methods can play a significant role in risk management. In the absence of such conditions though, reducing decision making to conventional risk–benefit analysis should not be seen as rational or reassuring (105).

In relation to the specific topic of this article, how then might we move away from the narrow question of whether this or that particular instance of science will likely result in more risks than benefits? One manner in which this has been done is by asking about the place of "precaution" in making sense of issues. The remainder of this paper considers what space can be opened up through taking inspiration from this topic.

While diverse in their formulations (see below), efforts to inject precaution into science and technology policy have usually shared the premise that definitive evidence of negative consequences need not be demonstrated to justify deliberation or even action (106). Instead, attempts have been made to ask what uncertainties, unknowns, and ignorances imply for who has to prove what to whom and for what purpose.

Precaution has become an overarching principle in national and international regulations such as the Cartagena Protocol on Biosafety, the Rio Conference on Environment and Development, and the Montreal Protocol on Substances that Deplete the Ozone Layer. And yet, despite the widespread reference to "the precautionary principle," especially in environmental policy, the practical relevance of these types of orientations is disputed (107, 108).

Within biosecurity life science discussions, precautionary orientations to risk have been dismissed at times. As argued, for instance:

Using an alternative method such as the precautionary approach to try to overcome these problems would be quite inappropriate for governing dual use technologies. Although the precautionary approach casts a wide net, precautionary regulations over every potential technology that could be misused would be not only prove to be infeasible in the case of dual use research and technologies but may have a dramatic social costs through stigmatizing the legitimate applications of these technologies (109).

Despite what is implied in such an evaluation, precautionary ways of orientating to risk are diverse. Peterson spoke of this diversity in considering how these approaches differed in their answers to the questions:

• What level (threshold) of threat or potential for harm is sufficient to trigger application of the principle?


As implied by these questions, formulations of precaution still depend on the identification of risk, but they need not invest risk–benefit assessments with the definitiveness that is implied in dual-use discussions today.

Other attempts to map the range of precautionary orientations have set out taxonomies (111, 112). Luján and Todt, for instance, distinguish versions of precautionary principles according to how they handle scientific uncertainty about consequences, make judgments in relation to disputed harmful consequences, and view the controllability of technology (113). With these criteria, Luján and Todt offer three different interpretations.


Against this taxonomy, it is possible to suggest how dual-use discussions to date are already (albeit mainly implicitly) infused with precautionary-type reasoning. For instance, as argued previously, discussions about how to assess research of concern often start with presumptions – such as that dual-use risks need to be substantiated, whereas, in general benefits from research can be assumed – that shape assessments of what needs doing.

Alternative starting presumptions have been voiced elsewhere. In relation to the H5N1 controversy, two new former members of NSABB, Michael Osterholm and David Relman, contended that the risks at stake were so grave (catastrophic human pandemic) and benefits unclear, that "the precautionary principle" should be evoked to err on the side of not doing harm – meaning that the work led by Fouchier needed to be censored (114).

Another precautionary paralleled facet of responses has been the opting of categorical approaches requiring specific logics of decision making rather than case-by-case assessments. *A Framework for Guiding U.S. Department of Health and Human Services Funding Decisions about Research Proposals with the Potential for Generating Highly Pathogenic Avian Influenza H5N1 Viruses that are Transmissible among Mammals by Respiratory Droplets*, for instance, stipulates that there is a category of research that is different from others (115). Within the US Department of Health and Human Services, funding proposals that fall into this category must undergo review scrutiny wherein the work must meet certain criteria (such as that there is no feasible alternative method to address the same scientific question in a manner that poses less risk and that the information generated is anticipated to be broadly shared in order to advance global health).

#### **FROM DECISIONS TO PROCESSES**

Up until this point in this article, precaution largely has been conceived as a factor in decision making. Precaution as a decision rule that prescribes action, however, is only one (and perhaps a highly) limited conceptualization of the notion. In practice, precautionary orientations to risk enacted to date have rarely provided definitive operational rules for making decisions or even stipulated clear cut criteria. Instead of being a rule for decision making, precaution can be thought about for what is implied for the *process* of deliberating risks. Consider a number of dimensions to this.

#### **Examining foundations**

With the acknowledgment given to uncertainties, ambiguities, and ignorances, attention should be directed at the starting points that shape understanding. These should be made explicit and a topic for reflection. In other words, the values underpinning interpretations to risks must be acknowledged and scrutinized. These may, for instance, have significant implications for how the burden of proof is distributed (105). In this sense, making scope for precaution itself does not imply that specific concerns take priority (for instance, preserving scientific development, environmental sustainability, avoiding a catastrophic pandemic, etc.), merely that the (likely varied and multiple) commitments for making sense of uncertainties, ambiguities, and ignorances be the subject of examination (116).

#### **Shifting discussion terms**

In fostering certain kinds of deliberation, precautionary-inspired deliberations can lend credibility and legitimacy to some arguments. In relation to how references to the precautionary principle entered into deliberations about conservation in fishing, for instance, it has been argued that the effects have been significant:

first by enhancing the credibility of certain types of arguments and diminishing that of others; second, by providing a framework within which conservationist arguments can be presented; and third, by pointing to interests and values other than those of states as legitimate objectives which the conservation regime should pursue (117).

Elsewhere precaution has diminished the credibility of narrow, notionally "scientific" forms of determining risks (108).

The need to reconsider the relevancy expertise in the process of making sense of the malign applications of science was given in an examination by Vogel of how US intelligence analysts assessed the H5N1 experiments (118). Her conclusions were three-fold:

First, U.S. intelligence analysts do not have adequate social and material resources to identify and evaluate the tacit knowledge, or know-how, that underpins dual-use experiments such as those in the H5N1 case. Second, they lack dedicated structures and methods to sort through the politics that characterize the use of technical expertise in such controversial biosecurity issues. Third, they require new types, structures, and assessments of expert knowledge to enable them to make more informed and balanced judgments of biosecurity threats (48, 80).

As part of enacting these recommendations, she contended that intelligence analysts need to be able to draw on a wider range of experts, including those in the social sciences.

#### **Promotion of alternative methodologies**

In maintaining the applicability of traditional forms of risk assessment are limited due to uncertainties, ambiguities, and ignorances, those adopting precaution orientations have sought alternative methods for making sense of risk. These have been either replaced or complement conventional assessments (105). Examples include scenario analysis, interval analysis, Q-method, horizon scanning, and societal impact assessment (119). Whereas conventional risks assessment might be done with the aim of weighing risks and benefits so as to make decisions, methods based on the recognition of incertitude aim to understand the limits of what is known, aid professional judgments, identify starting assumptions, reframe debates, and promote dialog and interaction. Making use of such methods can result in the participation of a different range of individuals than conventional risk assessment. Along these lines, as part of the analysis of H5N1, Vogel suggested how intelligence analysis could benefit from new forms of engagement that tested its limitations (118).

One area where these dimensions of precaution come together is public engagement. Within precautionary orientations, the overall attention to the limits of scientific certitude in determining risks and their acceptability opens a space for a wide range of contributions; including by those in publics. As argued, though:

"broadening out" of the social appraisal of technology that precaution may also be seen to entail a more generally *comprehensive* approach to decision making. A key consideration here concerns the many ways in which precaution is inherently interlinked with participatory approaches. This is not only as an aspiration to enhanced democracy. Nor is it just about fostering greater public trust or education. Far from second-guessing technical expertise with irrational public anxieties, precautionary participation is a matter of improved analytical rigor (emphasis in original) (105)

It would be difficult to over-estimate how much of the dual-use discussion to date has cast the public as a threat to science due to the potential for "misunderstanding" and sensationalism. As detailed elsewhere, within the Communication Working Group of NSABB,"the public" has come to occupy a central (if not the most prominent) place due to fears of public misunderstanding and sensationalism (120). In response to fears about the public, advisory documents such as NSABB"s *Proposed Framework for the Oversight of Dual-Use Life Sciences Research: Strategies for Minimizing the Potential Misuse of Research Information* provide many points about the need to message the publication of dual-use research so as to highlight the safeguards on research and its benefits.

Elsewhere in science policy over the last two decades, attempts have been made to recast the public away from being a problem for the acceptance of science and technology. Instead, efforts have been made to promote the engagement of the varied and numerous publics within a dialog (121). Public participation has been sought, for instance, as a means to highlight the importance of social values, to challenge technocratic framings, to identify alternative paths for the development of technology, and to promote what is coined as "responsible innovation" (122). While realizing such aspirations in practice is highly demanding, a more positive and arguably more productive role for the public is envisioned within them that typifies dual-use discussions to date (123).

## **CONCLUSION**

This article has examined the origins, emergence, resurrection, and implications of the category "research of concern." Throughout, attention has been given to a curiosity: the rarity that anything is identified as "of concern." The previous argument would suggest that the outcomes of review procedures enacted to date are the result of contingent practices that are consequential in the manner they structure a sense of what is going on and why, as well as what needs doing and by whom. In theory, this situation leaves open the possibility that a different pattern of review outcomes might take place if alternative conditions are in place.

More critically, as part of making the case for contingency, the preceding argument has questioned the continuing prominence given to conventional rationalistic "risk–benefit" assessment in managing the dual-use dimension of the life sciences. The notion of "weighing risks and benefits" may have substantial symbolic purchase for some, but arguably has limitations as a way of framing responses to research of concern. Without an acknowledgment of these, it is possible that a misplaced confidence is invested in reviews as currently conceived and that alternative policy possibilities are not sought out. Like other complex social and scientific issues, arguably it would not be wholly unfair with respect to the topic of this article to contend that "not only is the solution unknown, but the problem itself is initially not well defined, and the values that ought to drive its investigation and the valid methods to do so are unknown, unclear, or in dispute, as are the set of applicable theoretical models, the solution set, and the criteria for successful resolution" (124).

In reply, this article has outlined one set of different possibilities associated with "precaution." Though varied in their formulations, precautionary orientations generally begin with the aim of acknowledging conditions of uncertainty, ignorance, and ambiguity in order to ask how issues can be sensibly approached nevertheless. As argued, adopting such a starting basis could open spaces for alternative ways of thinking and responding to a set of issues that are bound to uncomfortably accompany the life sciences into the future.

#### **ACKNOWLEDGMENTS**

The author would like to thank Brian Balmer, Louise Bezuidenhout, Carol Stone, Kathleen Vogel, Judith Reppy, and Tomoko Y. Steen for their comments on earlier drafts of this article. The production of this article was supported by an ESRC, Dstl, and AHRC project: "The Formation and Non-formation of Security Concerns" (ES/K011308/1) and the work undertaken as part of this project was approved by the College of Social Science and International Studies Ethics Committee at the University of Exeter. By the end of 2014, data from the project will be accessible via http://www.data-archive.ac.uk/.

#### **REFERENCES**


to a reassortant H5 HA/H1N1 virus in ferrets. *Nature* (2012) **21**(June):420–8. doi:10.1038/nature10831


*Proposals with the Potential for Generating Highly Pathogenic Avian Influenza H5N1 Viruses that are Transmissible among Mammals by Respiratory Droplets*. Washington, DC: Department of Health and Human Services (2013).


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 08 May 2014; accepted: 24 June 2014; published online: 22 July 2014. Citation: Rappert B (2014) Why has not there been more research of concern? Front. Public Health 2:74. doi: 10.3389/fpubh.2014.00074*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Rappert . This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Synthetic biology and biosecurity: challenging the "myths"

## **Catherine Jefferson, Filippa Lentzos and Claire Marris\***

Department of Social Science, Health and Medicine, King's College London, London, UK

#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Luis Campos, University of New Mexico, USA

#### **\*Correspondence:**

Claire Marris, Department of Social Science, Health and Medicine, King's College London, Strand, London WC2R 2LS, UK e-mail: claire.marris@kcl.ac.uk

Synthetic biology, a field that aims to "make biology easier to engineer," is routinely described as leading to an increase in the "dual-use" threat, i.e., the potential for the same scientific research to be "used" for peaceful purposes or "misused" for warfare or terrorism. Fears have been expressed that the "de-skilling" of biology, combined with online access to the genomic DNA sequences of pathogenic organisms and the reduction in price for DNA synthesis, will make biology increasingly accessible to people operating outside well-equipped professional research laboratories, including people with malevolent intentions. The emergence of do-it-yourself (DIY) biology communities and of the student iGEM competition has come to epitomize this supposed trend toward greater ease of access and the associated potential threat from rogue actors. In this article, we identify five "myths" that permeate discussions about synthetic biology and biosecurity, and argue that they embody misleading assumptions about both synthetic biology and bioterrorism. We demonstrate how these myths are challenged by more realistic understandings of the scientific research currently being conducted in both professional and DIY laboratories, and by an analysis of historical cases of bioterrorism. We show that the importance of tacit knowledge is commonly overlooked in the dominant narrative: the focus is on access to biological materials and digital information, rather than on human practices and institutional dimensions. As a result, public discourse on synthetic biology and biosecurity tends to portray speculative scenarios about the future as realities in the present or the near future, when this is not warranted. We suggest that these "myths" play an important role in defining synthetic biology as a "promissory" field of research and as an "emerging technology" in need of governance.

**Keywords: synthetic biology, biosecurity, bioterrorism, biological weapons, DIY biology, iGEM, policy discourse, non-proliferation**

#### **INTRODUCTION**

"Synthetic biology strives to make the engineering of biology easier and more predictable." [(1), p. 6]

A dominant narrative has emerged in policy arenas, in which advances in the biosciences are seen to make biology easier and more accessible, and this is presumed to increase the so-called "dual-use" threat, i.e., the potential for the same scientific research to be "used" for peaceful purposes or "misused" for warfare or terrorism. Developments in synthetic biology, a field that emerged at the start of the twenty-first century with the stated aim of "making biology easier to engineer" (1, 2), have further fueled these concerns. Fears have been expressed that synthetic biology will lead to further "de-skilling" and that, combined with open online access to the genomic DNA sequences of pathogenic organisms and the reduction in price for DNA synthesis, this will make biology increasingly accessible to people operating outside well-equipped professional research laboratories, including people with malevolent intentions. The emergence of do-it-yourself (DIY) biology communities and the student iGEM competition has come to epitomize this supposed trend toward greater ease of access and the associated potential threat from rogue actors.

In this article, we analyze this dominant narrative and identify five"myths"that permeate discussions about synthetic biology and biosecurity. We describe each of these myths and provide illustrative examples of how they are deployed in policy arenas. We then demonstrate how each of these myths is challenged by more realistic understandings of the scientific research currently being conducted in both professional and DIY laboratories, and by an analysis of historical cases of bioterrorism. In particular, we show that the importance of tacit knowledge is commonly overlooked in the dominant narrative: the focus is on access to biological materials and digital information, rather than on human practices and institutional dimensions. Sonia Ben Ouagrham-Gormley and Kathleen Vogel have argued, on the basis on their in-depth analysis of the US and Soviet biowarfare programs, that there are important intangible barriers to the proliferation of biological weapons (3–5). These authors show how tacit knowledge has been marginalized in assessments of the dual-use threat of biotechnologies in the twenty-first century.

Tacit knowledge is crucial to conduct advanced bioscience research, and is by definition difficult to share. This is encapsulated by Polanyi's remark that "*we can know more than we can tell*" [(6), p. 4, emphasis in original]. As a result, researchers who work within institutionalized laboratories acquire tacit knowledge through experience, by working in teams and participating in professional scientific networks. But acquiring tacit knowledge is much more difficult for people who operate outside of such institutions, such as DIY biologists and bioterrorists. Broadly, tacit knowledge refers to skills and techniques that cannot be readily codified but, rather, are acquired through a process of "learning by doing" or "learning by example," and often take considerable time and effort to gain. According to Harry Collins, a distinction can be made between "weak,""somatic limit," and "collective" tacit knowledge (7, 8). Revill and Jefferson (9) have drawn on Collins' classification to explore the importance of tacit knowledge in the practice of synthetic biology and the conduct of bioweapons programs. They explain that "[w]eak tacit knowledge is that which could, under certain circumstances, be rendered explicit but either through inability, unwillingness, or practicality remains unwritten and implicit" [(9), p. 3]. Individual, or somatic, tacit knowledge "refers to things that our bodies can do, which we cannot articulate, transfer and replicate as knowledge without the recipient learning by doing" (*ibid*., p. 4–5). These are the skills, mechanical techniques, and idiosyncratic know-how obtained by individuals through trial-and-error problem-solving or through a master-apprentice style relationship. Collective, or communal, tacit knowledge "is the combined knowledge that is developed through interaction between experts with different disciplinary backgrounds working together" (*ibid*., p. 6). This can be conceptualized "as the bringing together of different disciplinary experts that are greater than the sum of their parts" (*ibid*., p. 6–7), or, following Vogel (10), can be understood as "communally synthesized tacit knowledge" that comes from the ongoing interactions between different types of expertise. Revill and Jefferson (9) provide examples of tacit knowledge from each of these categories from the history of biological weapons programs and the practice of advanced biological sciences, including synthetic biology.

Ben Ouagrham-Gormley and Vogel, and Revill and Jefferson, argue that a better understanding of tacit knowledge could improve the assessments of the dual-use threat posed by modern biotechnologies. Yet, tacit knowledge continues to be overlooked in policy arenas. In this paper, we examine the way in which the biosecurity threat posed by synthetic biology has been framed within the dominant narrative that permeates scientific and policy arenas. We identify five recurring "myths" that emerge from this analysis:


The use of the term "myths" is not intended here to imply falsity. We are not simplistically opposing "myth" and "reality," and we are not arguing that there is no threat. Rather, our aim is to convey the pervasiveness of misleading assumptions about both synthetic biology and bioterrorism that frequently underlie discussions about the dual-use threat of synthetic biology, and to draw out some of the subtleties that frequently disappear from these discussions. Moreover, we do acknowledge that these myths have power and perform real functions such as mobilizing support, resources, and action. Thus, the dominant narrative identified in this paper helps to bring into being a particular hoped-for future, and attributes roles and influence to different actors. It influences the way in which the problem is defined, and thus the kinds of solutions that are proposed. These "myths" are real enough to influence policy in significant ways and that why it is important to examine them more carefully.

## **MATERIALS AND METHODS**

The research presented here draws on participant observation in scientific and policy arenas, and on a review of a wide range of written materials.

All three authors have been participant-observers in either synthetic biology arenas, or biosecurity arenas, or both, for a number of years. Filippa Lentzos has been regularly attending and actively participating in a wide range of events on biosecurity, biological arms control and non-proliferation for over a decade. Catherine Jefferson has been involved in discussions on bioweapons, biosecurity and arms control for a decade. Claire Marris has been attending and participating in a wide range of scientific and policy events on synthetic biology for 5 years. Filippa Lentzos has been engaged in the field of synthetic biology for the last 7 years. The synthetic biology events include scientific meetings ranging from large-scale international conferences such as those in the SBx.0 series to laboratory meetings at the Centre for Synthetic Biology and Innovation (CSynBI) that all three authors are members of, or informal conversations with CSynBI and other collaborators in the field of synthetic biology. Our involvement also includes participation in expert committees and working groups, and public debates organized by scientific organizations.<sup>1</sup>

The key insights reported in this paper emerged from this immersion in the worlds of synthetic biology and biosecurity, which provided the authors with regular opportunities to interact with synthetic biologists, government officials, security analysts, technical experts, diplomats, public health officials, law enforcement agents, DIY biologists, and others who have assembled around the "problem" of synthetic biology "misuse." These interactions took place in"natural"settings (as opposed to, for example, an interview setting), in places and during events that these actors – and the authors – were participating in through the course of their work.

It is through this fieldwork that we became aware of the prevalence of particular ways of framing the issues at stake, and were able to analyze how actors mobilized particular arguments. This was complemented by a review of written materials, which has been utilized mostly to confirm the hypotheses developed through our fieldwork, and to select citations to illustrate our results. This was necessary because many of the meetings that we participated

<sup>1</sup>A list of some of our higher profile engagements can be found here: http://www.kcl.ac.uk/sspp/departments/sshm/research/Research-Labs/CSynBI@ KCL-Impact.aspx.

in were not public and/or were not recorded, so it is not technically possible to provide verbatim quotations from those events. Moreover, in most cases, providing such quotations would not be compatible with research ethics. The documents reviewed are mostly from the "gray" literature: reports produced by scientific and biosecurity institutions. But they also include relevant academic articles, websites, blogs, and print media. The key criteria for selection of documents and citations were that they should be produced or written by key institutions or influential individuals in the fields of synthetic biology and/or biosecurity, for example: Drew Endy, Rob Carlson, George Church as leaders in the field of synthetic biology; Jonathan Tucker, Tara O'Toole, and Laurie Garrett as US experts in the field of biosecurity; Markus Schmidt as a key European commentator on "ethical, legal, and social issues" related to synthetic biology; US government officials and politicians; and institutions such as the Biological Weapons Convention (BWC), the European Commission, the US National Academy of Science, the US National Research Council, the US National Advisory Board for Biosecurity (NSABB), the J. Craig Venter Institute (JCVI) and the UK Royal Academy of Engineering. Moreover, the illustrative citations are taken mostly from documents and from (individual or institutional) authors that are themselves routinely cited by actors in discussions about synthetic biology and biosecurity.

Ethnographic data from participant observation and the literature review was complemented by a 1-day workshop convened by the authors at King's College London on 28th February 2014 (11). This workshop brought together a group of 23 scientists, policy experts, science journalists, and social scientists (mostly from the UK) with specialist expertise in either synthetic biology or biosecurity (or both). A draft of the present paper was circulated in advance of the workshop and participants were asked to comment on it. The comments received and the discussions that occurred during the workshop provided additional information and confirmation of our hypotheses.

#### **RESULTS**

**MYTH 1**

#### **Synthetic biology is de-skilling biology and making it easier for terrorists to exploit advances in the biosciences**

Founding leaders in synthetic biology have argued that developments in the field would lead to a situation where biology would not only become "easier to engineer," but that it would become easier for *anyone* to engineer biology. For example, during his early campaigns to garner political and financial support for the field, Drew Endy stressed that synthetic biology would lead to "the probable inability to control the distribution of technologies needed to manipulate biological systems" (12). Rob Carlson, in an article published in a biosecurity journal, emphasized that it would lead to the "inevitable" "proliferation of skills" [(13), p. 7]. Endy and Carlson both pointed to potential dual-use threats, whereby the powerful technology that they were promoting could be misapplied for harmful purposes. George Church also raised these issues in his "Synthetic Biohazard Non-proliferation Proposal" (14). The JCVI funded a report on "Options for Governance" that also focused almost exclusively on such risks (15).

The idea that synthetic biology could make it easier for nonspecialists, including those working outside of institutions, to exploit this powerful technology for both benevolent and malevolent purposes, has to a large extent become a hallmark of the field. For example, in an article entitled "Diffusion of synthetic biology: a challenge to biosafety" Markus Schmidt, who was the leader of the first European Commission-funded project on the "Ethical, Legal, and Social Issues" of the field (SYNBIOSAFE) and who has become a prominent commentator on the risks involved, has argued, in a paper that has been cited 52 times in Google Scholar (accessed 10th July 2014), that:

With this"de-skilling"agenda, synthetic biology might finally unleash the full potential of biotechnology and spark a wave of innovation, as more and more people have the necessary skills to engineer biology [(16), p. 1].

This portrayal of synthetic biology focuses on the powerful positive impact that could be "unleashed" by "de-skilling," and inevitably leads to concerns that such power could fall into the hands of people with malevolent intentions. As a result, policy experts have routinely expressed concerns that synthetic biology could be used by terrorists to produce biological weapons. For example, political scientists from the Massachusetts Institute of Technology (Gautam Mukunda<sup>2</sup> and Kenneth Oye), who were both at the time working for the US Synthetic Biology Research Center (Synberc), published an article on synthetic biology and biosecurity in 2009, in which they stated:

Synthetic biology includes, as a principal part of its agenda, a sustained, well-funded assault on the necessity of tacit knowledge in bioengineering and thus on one of the most important current barriers to the production of biological weapons [(17), p. 14].

The European Group on Ethics in Science and New Technologies to the European Commission also emphasized this in their 2009 Opinion on synthetic biology:

Ethical issues arise particularly from dangers of using synthetic lethal and virulent pathogens for terrorist attacks, biowar, or maleficent uses ("garage terrorism", "bio-hacking"), particularly if knowledge and skills on how to produce such pathogens are freely available [(18), p. 43].

## **Challenges to Myth 1**

These concerns are based on the assumption that synthetic biology already has made it, or shortly will make it, easy *for anybody* to "engineer biology." The underlying vision is one where well-characterized biological "parts" can be easily obtained from open-source online registries and then easily assembled, by people with no specialist training and working outside professional scientific institutions, into genetic "circuits,""devices," and "systems" that will reliably perform desired functions in live organisms (1, 2). However, this does not even reflect current realities in academic or commercial science laboratories, where researchers are still struggling with every stage of this process (19, 20).

<sup>2</sup>Gautam Mukunda is now at the Harvard Business School.

Moreover, synthetic biologists who participated in our recent workshop (11) argued that although historical experience with other forms of (non-biological) engineering demonstrate that dependence on the craft skills of a small number of highly trained individuals is reduced for some parts of the production process, usually by standardization and mechanization, this does not mean that skills become irrelevant or that all aspects of the work become easier. Specialized expertise, teamwork, large infrastructures, complicated machinery, advanced technology, trouble-shooting, and organizational factors continue to be required when a design and engineering approach develops. Thus, even though the engineering approach of synthetic biology aims to make processes more systematic and more reproducible, this will not make it easier for anybody to engineer biology. Indeed, some aspects of the work may become more complex, and new skills may be required.

A useful analogy to aeronautical engineering was used at the workshop to illustrate this. Planes are built from a large number of well-characterized parts in a systematic way, but this does not mean that any member of the general public can build a plane, make it fly, and use it for commercial transportation. Thus, it is too simplistic to suggest that if synthetic biology becomes an engineering discipline it will necessarily become easier for anybody to engineer biological systems, including dangerous ones. More care needs to be taken in the interpretation of statements about how synthetic biology will lead to "de-skilling" and "make the engineering of biology easier."

Furthermore, the experiences of iGEM teams tend to demonstrate the challenges of successfully performing synthetic biology experiments, and demonstrate the ongoing need for guided instruction and collective expertise. iGEM is the annual International Genetically Engineered Machine competition, which brings together undergraduate students from across a range of disciplines to work collaboratively to design and build biological systems and operate them in living cells. The iGEM competition is linked to the parts-based approach to synthetic biology through its contributions to the Registry of Standard Biological Parts, and provides a proof-of-principle for the synthetic biology agenda (21).

iGEM teams typically receive considerable guidance from senior faculty members and, while iGEM is a collaborative exercise, biologically trained students still tend to be the ones who have the central roles in daily laboratory activity. Balmer and Bulpin (22) describe the collaborative experiences of one undergraduate iGEM team:

Over the course of the project, as time pressures became more significant, it became natural, when assigning the activities of the day, for them to conduct the procedures in which they had each become experts, as otherwise it would require them teaching someone else. [. . .] As one of them explained:"From the start I had the idea that I would take a main role in modelling but also get some experience in the lab. However, I quickly gave up on lab work after the first few weeks because *the time frame for the project we had was not enough to learn the basics needed for the lab and apply them*." Owing to these contextual, material specificities of the laboratory and modelling work, *the sub-teams were quickly separated by knowledge, time and space* [(22), p. 14, emphasis added].

Cockerton (23) reported similar findings from her ethnography of two iGEM teams:

both teams found that many protocols were not streamlined as descriptions of synthetic biology often present. There was a great deal of tedious work, which involved small volumes of clear liquid and lots of waiting time. Many cycles of failed experiments had to be repeated (p. 306).

## [. . .]

the reality of everyday design-experiment-fail-redesign (and so on. . .) cycles serves as a sobering reminder that the foundations of synthetic biology were not then (when I was in the field in 2009), and are not yet (2011), stable. Many experiments don't work out as planned because many BioBricks from the Registry don't function reliably. Presently, engineering that is accomplished with BioBricks in one lab and described in a standard fashion, certainly does not guarantee that the same result is reproducible in another lab (p. 307–308).

These in-depth analyses of synthetic biology in action illustrate the importance of collective expertise in synthetic biology research and the challenge posed by tacit knowledge, especially for wet lab work. Members of iGEM teams have or acquire distinct specialist sets of knowledge and skills, which are then applied to the collective project. Training by experienced professional researchers, and specialist skill sets acquired through trial and error, are still highly relevant to the success of synthetic biology projects.

The challenge of acquiring the specialist skill sets to perform laboratory work is also demonstrated by the experiences of some members of the DIY biology (DIYbio) community. DIYbio.org describes itself as "an organization dedicated to making biology an accessible pursuit for citizen scientists, amateur biologists, and biological engineers who value openness and safety"<sup>3</sup> . The organization comprises over 2000 members globally, although the actual number of members regularly conducting biological experimentation is much smaller. Some DIY biologists work in home laboratories assembled from everyday household tools and second-hand laboratory equipment purchased online. However, the majority conduct their experiments in community labs or "hackerspaces" (24).

DIY biologists typically comprise a wide range of participants of varying levels of expertise, ranging from complete novices with no prior background in biology, to trained scientists who conduct DIY experiments in their own time. The experiences of amateur DIY biologists demonstrate how a lack of indoctrination in the practices of biology can present significant challenges. As Revill and Jefferson (9) note:

For example, the London Biohacker group [. . .] have noted the challenge of overcoming "pipetting errors" when trying to optimise techniques for DNA extraction and PCR process. MadLab, a bio group based at the Manchester Digital Laboratory, experienced similar difficulties during their "PCR

<sup>3</sup>http://diybio.org/about-2/, accessed 14/07/2014

challenge," in which they pitched their home-made Arduinobased PCR machine against the open-source OpenPCR kit and the commercial PCR at Manchester Metropolitan University: . . ."the hardest part of the process was getting our samples into the gel using a micropipette. It turns out there is a bit of an art to pipetting . . . The more experienced pipettors claimed that it took them weeks to get the proper technique." (p. 6)

Scientists typically build up these skills over the course of their training, but they present notable challenges for amateurs. Thus, while representations of Myth 1 imply that the material, informational aspects of synthetic biology will make it easier for anybody to exploit this technology to do harm, further examination of the social dimensions of scientific practice reveal the continued significance of local, specialized knowledge, and the importance of enculturation in laboratory practices.

At the workshop recently convened by the authors, an interesting tension was revealed. On the one hand, if tacit knowledge remains important in synthetic biology, then this implies that it will not be easily accessible to outsiders and this reduces concerns about the dual-use threat. On the other hand, if synthetic biology is an engineering discipline *and if*, as stated by Mukunda et al. in the citation above, this represents "an assault on the necessity of tacit knowledge" (17), then this implies that it will become more accessible to outsiders and this increases the dual-use threat. Thus, biosecurity concerns are heightened when more extreme depictions of synthetic biology's ability to engineer biology are emphasized. We characterize this as the "synthetic biology/engineering conundrum" (11).

#### **MYTH 2**

#### **Synthetic biology has led to the growth of a DIY biology community, which could offer dual-use knowledge, tools, and equipment for bioterrorists seeking to do harm**

Developments in synthetic biology are seen to be closely associated with the growth of the DIYbio community, and concerns are expressed that this could offer knowledge, tools, and equipment to bioterrorists seeking to do harm. This was a key thrust in Carlson's 2003 article, which started with the phrase: "The advent of the home molecular laboratory is not far off." Schmidt also stressed this notion in his 2008 article, saying, for example: "[Imagine] a world where practically anybody with an average IQ would have the ability to create novel organisms in their home garage" [(16), p. 2]. This anticipated rise of a form of biology that could be performed by amateurs in their home garage or kitchen (25), sometimes referred to as "biohacking," was understandably picked up by biosecurity experts. Jonathan Tucker, a well-recognized expert on chemical and biological weapons, wrote several articles on this topic, and in the most widely cited of these (cited 96 times according to Google Scholar, accessed 07/07/2014), he said:

The reagents and tools used in synthetic biology will eventually be converted into commercial kits, making it easier for biohackers to acquire them. Moreover, as synthetic biology training becomes increasingly available to students at the college and possibly high-school levels, a "hacker culture" may emerge, increasing the risk of reckless or malevolent experimentation [(26), p. 42].

Such concerns became prevalent at the NSABB, an organization established in 2005 to provide advice to the US government on biosecurity issues:

As synthetic biology techniques become easier and less expensive and the applications become more widely relevant, the range of practitioners expands to include scientists from a variety of disciplines; students at all levels, including high school; and amateur scientists and hobbyists who may lack any formal affiliations with universities or research institutions. The diversity of practitioners will also include individuals of different ages and varied social and educational backgrounds who may not have been sensitized to the ethical social and legal norms of the traditional life science research communities [(27), p. 11].

By 2014, this idea had become so widely accepted among experts in the field of Chemical, Biological, Radiological, and Nuclear (CBRN) weapons that an article entitled "DIY Bioterrorism Part II: the proliferation of bioterrorism through synthetic biology" was posted on the CBRNePortal.com. This article stated that:

The threat may be changing with the continued advancement of synthetic biology applications. Coupled with the ease of information sharing and a rapidly growing do-ityourself-biology (DIYbio) movement,the chances of not only more attacks but potentially more deadly ones will inevitably increase (28).

#### **Challenges to Myth 2**

The link between synthetic biology and DIYbio, and the level of sophistication of the experiments typically being performed in DIYbio community labs,is overstated (24,29). Members of DIYbio communities who are involved in more sophisticated experiments tend to be trained biologists, not amateurs and, as noted in the previous section, the experiences of amateur members of the DIYbio community demonstrate the challenges posed by tacit knowledge to successfully conduct even rudimentary biological experiments.

Furthermore, members of the DIYbio community tend to be proactive in addressing and engaging with safety and security concerns and many community labs have strict rules about access (24). For example, BioCurious, a community lab in silicon Valley, requires all members working in the wet lab to undertake a safety orientation, regardless of formal education or previous laboratory experience. BioCurious also has a safety committee that reviews requests to work with organisms not already on an approved list, and can approve, modify, or reject experimental design<sup>4</sup> .

DIYbio.org has also been active in promoting responsibility within the community. For example, in partnership with the Synthetic Biology Project at the Wilson Center, DIYbio.org has developed a Draft Code of Ethics that includes a focus on transparency, safety, and peaceful purpose<sup>5</sup> . In January 2013,

<sup>4</sup>http://biocurious.org/faq/, accessed 14/07/2014

<sup>5</sup>http://diybio.org/codes, accessed 14/07/2014

DIYbio.org also launched an "Ask a Biosafety Officer" web portal<sup>6</sup> in which anyone with a question can submit their query to a panel of volunteer biosafety experts. DIYbio Europe has established a set of Community Lab Safety Guidelines, with an emphasis on communication, openness, lab organization, and user and environmental safety (30). The US Federal Bureau of Investigation (FBI) weapons of mass destruction outreach program has also launched a series of efforts to promote outreach and oversight of the DIYbio community (31).

#### **MYTH 3**

#### **DNA synthesis has become cheaper and can be out-sourced, and this will make it easier for terrorists to create biological threat agents**

DNA synthesis is one of the key enabling technologies of synthetic biology. There are now a number of commercial companies that provide DNA synthesis services, so the process can be outsourced: a client can order a DNA sequence online and receive the synthesized DNA material by post within days or weeks. The price charged by these companies has greatly reduced over the last 20 years, and is now around 0.3 US\$ per base pair, which puts it within reach of a broad range of actors. This has led to routine statements suggesting that it is now cheap and easy to obtain a synthesized version of any desired DNA sequence. This popularized image of DNA synthesis is well represented by the Wikipedia entry (accessed 02/07/2014) for "artificial gene synthesis," which states that:"it is possible to make a completely synthetic double-stranded DNA molecule with no apparent limits on either nucleotide sequence or size."

Rob Carlson first published his now famous "Carlson curves," illustrating the increasing productivity and reducing cost of DNA synthesis, in an article in the journal *Biosecurity and Bioterrorism*, which focused on how to combat the "potential for mischief or mistake" associated with advances in biological technologies (13). This illustrates how synthetic biology was, early on, promoted alongside discussions of a related biosecurity threat.

The key concern raised has been that bioterrorists could create dangerous viruses or other pathogens "from scratch," meaning without access to the biological material from nature, from a strain repository, or from a laboratory. Instead, they would start with DNA or RNA genomic sequences for pathogenic viruses and bacterial pathogens that are increasingly freely available online. Such fears were heightened in 2002 by an experiment in which poliovirus was synthesized without the use of any natural virus or viral components (32). The research team, led by Eckard Wimmer, obtained published poliovirus RNA genome sequence information and converted this into DNA sequence data, which they then ordered from a commercial DNA synthesis company and assembled into a viral genome. The DNA was converted back into RNA and the RNA was used to produce a functional virus. Publication of this research in a scientific journal article immediately raised concerns that terrorists could use it as a recipe to synthesize dangerous viruses without needing access to biological material. These fears were further fueled when a journalist from *The Guardian* reported that he had been able to order online a synthesized DNA fragment from the smallpox virus genome and have it delivered to a residential address. According to this journalist, this showed "the ease with which terrorist organizations could obtain the basic ingredients of biological weapons" (33).

As Garfinkel et al. [(15), pp. 5–6) point out, although these experiments built upon previous work on DNA synthesis, "Wimmer's work demonstrated for the first time in a post-September 11 world the feasibility of synthesizing a complete microorganism, in this case, a human pathogen – using only published DNA sequence information and mail-ordered raw materials." Such concerns were further crystallized when, the following year, researchers at the JCVI similarly synthesized the bacteriophage phiX174 (a virus that infects bacteria) (34), and when researchers at the US Centers for Disease Control and Prevention "reconstructed" the Spanish flu virus (35), thought to have killed around 50 million people during the 1918 pandemic (36). This demonstrated that even viruses that could not otherwise be easily obtained in nature or from laboratory collections could be recreated (by well-resourced university researchers).

Together, the reconstruction of poliovirus and Spanish influenza virus have come to epitomize the threat narrative that DNA synthesis has become faster and cheaper, and that this will make it easier for terrorists to create biological threat agents. This is illustrated by statements from biosecurity experts such as Jonathan Tucker and Raymond Zilinskas<sup>7</sup> :

One potential misuse of synthetic biology would be to recreate known pathogens (such as the Ebola virus) in the laboratory as a means of circumventing the legal and physical controls on access to "select agents" that pose a bioterrorism risk. Indeed, the feasibility of assembling an entire, infectious viral genome from a set of synthetic oligonucleotides has already been demonstrated for poliovirus and the Spanish influenza virus [(26), p. 37].

Another article published in 2007 by Stephen Maurer and Laurie Zoloth stated that<sup>8</sup> :

Synthetic biologists have already shown how terrorists could obtain life forms that now exist only in carefully guarded facilities, such as polio and 1918 influenza samples [(37), p. 16].

In an early article highlighting this concern, security analysts from the Johns Hopkins Center for Civilian Biodefense Strategies wrote:

An editorial in a prestigious scientific journal reporting on the successful decoding and manipulation of the genetic sequence of the influenza A virus noted that "one can only

<sup>6</sup>http://ask.diybio.org, accessed 14/07/2014

<sup>7</sup> Jonathan B. Tucker was at this time a senior fellow at the Center for Nonproliferation Studies (CNS) of the Monterey Institute of International Studies, where he specialized in biological and chemical weapons issues. Raymond Zilinskas was and still is the director of the Chemical and Biological Weapons Nonproliferation Program at CNS.

<sup>8</sup> Stephen Maurer was then and still is at the University of California-Berkeley's Goldman School of Public Policy and Director of the Goldman School Project on Information Technology and Homeland Security. Laurie Zoloth was and still is Professor of Bioethics at Northwestern University.

speculate as to how quickly our knowledge. . ..will progress, now that every nucleotide of the viral genome can be mutated and engineered back into the genome, in nearly endless combinations with other mutations." [. . .] Using such technologies, which have been utilized to investigate Ebola, pandemic flu, influenza, hanta viruses, lassa, rabies, and Marburg viruses,there is no need for a bioweaponeer to isolate the virus from an infected patient, acquire it from a germ bank, or culture it from nature. All the required starting materials, such as cell lines and DNA synthesizers, are widely available and used for many beneficent purposes. And the sequences for a growing variety of viruses that infect humans, animals and plants, including Ebola, pandemic influenza, and smallpox, are published in the open literature [(38), p. 30].

Tara O'Toole, Director of the Johns Hopkins Center for Civilian Biodefense Strategies and co-author of the article, was also the principal author of "Operation Dark Winter" (in 2001-2002) and "Atlantic Storm" (2005), the disaster response exercises that simulated covert outbreaks of smallpox in the United States. She went on to become Under Secretary of the Science and Technology Directorate of the Department of Homeland Security and, on the 10-year anniversary of the "anthrax letters," reiterated her Johns Hopkins group's earlier concerns with synthetic biology in testimony to the Senate Committee on Homeland Security and Governmental Affairs:

More than a decade ago, the Defense Science Board affirmed that, "there are no technical barriers to a large-scale bioattack." We are living in the midst of a biotechnology revolution where the knowledge and tools needed to acquire and disseminate a biological weapon are increasingly accessible. It is possible today to manipulate pathogens' characteristics (e.g., virulence, antibiotic resistance), and even to synthesize viruses from scratch. These procedures will inexorably become simpler and more available across the globe as technology continues to mature (39).

Concerns about terrorist use of DNA synthesis to create biological weapons spread internationally, and synthetic biology has become a regular feature of the science and technology reviews of the international treaty banning biological weapons: the BWC. In one of these reviews for BWC members, the Chinese delegation noted that:

With the spread of synthetic biology, some small scale research groups and even some individuals are now able to make the deadly Ebola and smallpox viruses and even some viruses against which all drugs are ineffective, thus making it much harder to counter bioterrorism. Furthermore, it has become much easier to obtain sensitive information. Using publicly available DNA sequences, terrorists can quickly synthesize pathogenic microbes that had previously been eradicated. [(40), p. 4].

During a 2012 Meeting of Experts of the BWC, the US delegation noted that:

These technologies [enabling technologies, including highthroughput systems for sequencing, synthesizing and analyzing DNA; bioinformatics and computational tools; and systems biology] could potentially be used for purposes contrary to the Convention, including making pathogens or toxins easier and less expensive to manufacture *de novo*, and further into the future, enabling development of biological weapons agents designed to evade countermeasures or target certain human populations [(41), p. 1-2].

Similar concerns have also been highlighted by individual bioweapons experts. Recent examples include Laurie Garrett's<sup>9</sup> article in the November/December 2013 issue of Foreign Affairs (42), which was widely disseminated and became the subject of a "Foreign Affairs Focus" video interview with the author published online on 15th January 201410. In this article Garrett asserts that:

All the key barriers to the artificial synthesis of viruses and bacteria have been overcome, at least on a proof-of-principle basis (42).

Another example is the article written by Adam Bernier and Patrick Rose for the CBRNePortal, which states:

Non-state actors who wish to employ biological agents for ill intent are sure to be aware of how tangible bio-weapons are becoming as applications of synthetic biology become more affordable and the probability of success increases with each scientific breakthrough (28).

Synthetic biologists have not sought to deny these risks, and have led several initiatives to consider how these potential biosecurity risks could best be addressed. These initiatives re-enforced the association between synthetic biology, DNA synthesis, and biosecurity threats. For example in his "Synthetic Biohazard Non-proliferation Proposal," George Church stated:

While the likelihood of misuse of oligos to gain access to nearly extinct human viruses (e.g. polio) or novel pathogens (like IL-4-poxvirus) is small, the consequences loom larger than chemical and nuclear weapons, since biohazards are inexpensive, can spread rapidly world-wide and evolve on their own (14).

Similarly, the JCVI report mentioned above concluded that:

today, any synthesis of viruses, even very small or relatively simple viruses, remains relatively difficult. In the near future, however, the risk of nefarious use will rise because of the increasing speed and capability of the technology and its widening accessibility. [. . .] Ten years from now,it may be easier to synthesize almost any pathogenic virus than to obtain it through other means [(15), p. 12–13].

And a group of synthetic biologists (including Drew Endy and George Church) published, together with leading DNA synthesis companies and four FBI staff, a commentary in *Nature Biotechnology* on "DNA synthesis and biological security," which stated that:

<sup>9</sup>Laurie Garrett is a science writer with a special interest in emerging infectious diseases, global health and biosecurity. She works at the Council on Foreign Relations Council, a think-tank that publishes the journal *Foreign Affairs*.

<sup>10</sup>http://www.foreignaffairs.com/discussions/audio-video/foreign-affairs-focuslaurie-garrett-on-synthetic-biology

Like any powerful technology, DNA synthesis has the potential to be purposefully misapplied. Misuse of DNA-synthesis technology could give rise to both known and unforeseeable threats to our biological safety and security [(43), p. 627].

#### **Challenges to Myth 3**

When speaking about DNA synthesis, it is useful to distinguish between (a) the synthesis of oligonucleotides, commonly referred to as "oligos," which are typically less than 100 nucleotides in length; (b) "gene synthesis," a term used to refer to the *de novo* synthesis of "gene-length" DNA sequences, typically 200–3,000 base pairs (bp); and (c) the assembly of *de novo* synthesized gene-length fragments into genetic circuits and whole genomes.

There are a number of ways in which DNA synthesis could be used to create a synthetic viral genome [(44), p. 134]. An entire viral genome could be ordered online from a commercial gene synthesis company. Short, single stranded oligonucleotides could also be ordered from different gene synthesis companies and "stitched" together to create a complete viral genome. Alternatively, oligonucleotides could be synthesized using a purchased or custom-built DNA synthesizer, and these fragments could then be assembled into a complete viral genome. Several challenges should be taken into account when assessing the potential for this technology to be misused.

Ordering short oligos and then assembling them into a genome was the method used in the polio and Spanish flu experiments, but this required specialist expertise, experience, and equipment, which were all available in the academic laboratories involved but would not be easily accessible to an amateur working from home. Obtaining the oligos (as was done by *The Guardian* journalist for the smallpox virus) is only the first step in a complicated process. This is the first challenge to Myth 3.

The second challenge to Myth 3 is that,contrary to what is stated in Wikipedia, and what is often implied in the policy discourse described above, even specialized DNA synthesis companies cannot easily synthesize *de novo* any desired DNA sequence. Several commercial companies provide routine gene synthesis services for sequences under 3,000 bp, but length is a crucial factor, the process is error prone, and some sequences are recalcitrant to chemical synthesis (those that are"complex,"have high GC content, or result in the expression of particular proteins when cloned). Thus, in a recent review of large-scale *de novo* DNA synthesis, Kosuri and Church conclude that:

Today, reconstructions of complete viral and bacterial genomes are testaments of how far our synthetic capabilities have come. Despite the improvements, our ability to read DNA is better than our ability to write it [(45), p. 499].

The polio and phi174 viruses both have relatively small genomes, but these are still 7,400 and 5,400 bp, respectively. Thus, several *de novo* synthesized DNA fragments would have to be assembled in order to produce a full genome and (even if this was not already regulated by voluntary guidelines adopted by DNA synthesis companies) it would not be possible to simply order the full-length genome sequence of a small virus online.

The third challenge is that for sequences longer than 5–10 kb, *assembly* of DNA fragments becomes the crucial step, not *de novo*

DNA synthesis. This was the major technological feat in the work conducted at the JCVI that produced the "synthetic" bacterial genome, and the "Gibson Assembly method" developed for that project is now widely used. The description of that work, however, demonstrates how the assembly of smaller fragments into larger ones and eventually into a functioning genome required substantial levels of expertise and resources, including those needed to conduct trouble-shooting experiments to identify and correct errors when assembled DNA constructs did not perform as expected (46).

The fourth challenge to Myth 3 relates to cost. The price of gene synthesis has declined greatly over the last 20 years, and the policy discourse that underlies biosecurity fears often implies that it will naturally become even cheaper over time, and thus widely affordable. The decline in price has, however, more or less stagnated around 0.3 US\$ per base pair since 2008; and Carlson (47), Kosuri and Church (45), and Shetty (48) each discuss reasons why investment in this area may not be sufficient or well directed enough to generate further significant advances.

The fifth and fundamental challenge to Myth 3 is that constructing a genome size DNA fragment is not the same as creating a functional genome. In particular, ensuring the desired expression of viral proteins is a complex challenge, which has been well documented in Vogel's (5) account of the 2002 poliovirus synthesis experiment. Drawing on interviews with the researchers involved in the experiment,Vogel found that making HeLa cell-free extracts was a crucial step in translating the synthetic genome into infectious virus particles; and it was also one of the most difficult parts of the experiment. Successful preparation of the HeLa cell-free extracts depended on craft-like techniques that require specialized and localized know-how. Yet, as Vogel notes, despite the difficulties encountered in this step of the process, published protocols of the experiment give no indication of this contingency:

As this case study illustrates, successful replication of the published 2002 poliovirus experiment hinges not only on the availability of the genetic sequence of the virus, commercial pieces of DNA, or the posting of the publication on the internet but also on the ability to master the mundane yet idiosyncratic biological techniques and adhere to specific laboratory disciplines [(5), p. 86].

Published accounts of science imply that experiments are readily replicable and transferrable from one lab to the next, but Vogel's analysis demonstrates the significance of tacit knowledge in scientific practice and how this would limit the "proliferation" of skills anticipated in the dominant narrative on synthetic biology. Recognizing the importance of such tacit knowledge would enable more refined analyses of the potential biosecurity threat posed by advances in DNA synthesis technologies.

Additional challenges to Myth 3 include the fact that while DNA or RNA sequence data are available for many pathogenic viruses, genomes published in publicly available databases can contain errors or may be derived from attenuated laboratory strains (49). Producing viral particles in a laboratory is, moreover, not the same as creating and deploying an effective biological weapon. Challenges to the processes of scaling up, storage, and developing a suitable dissemination method are discussed under Myth 5.

#### **MYTH 4**

## **Synthetic biology could be used to design radically new pathogens**

In addition to recreating dangerous viruses, concerns have also been expressed that synthetic biology could be used to enhance the virulence or increase the transmissibility of known pathogens in order to create novel threat agents.

The 2001 mousepox experiment is the most widely cited examples of the dual-use potential of life science research and has come to epitomize the potential to create more virulent viruses. In this experiment, researchers inserted the gene for interleukin-4 into the mousepox virus (50). They aimed to produce an altered virus that would induce infertility in mice and serve as an infectious contraceptive for pest control. However, the altered virus was found to be lethal to mice. Moreover, and most surprisingly, it was lethal to mice that were naturally resistant to mousepox as well as to mice that had been recently immunized against ordinary mousepox. The publication of these findings led to concerns that they could provide instructions to terrorists to produce novel biological weapons.

An early, formative report that shaped concerns about radically new pathogens was *Biotechnology Research in an Age of Terrorism* from the US National Research Council. It noted:

The effects of naturally occurring pathogens are limited by the evolutionary advantage gained by not eliminating their hosts. Among the many implications of the anticipated progress in biotechnology is the presumption that it may be feasible to create novel biological agents that are far more predictable and dangerous than any of the naturally occurring pathogens that have been developed as biological weapons in the past. It may be difficult to engineer a more successful pathogen than those already present in nature that have been perfected by evolution for their niche in life. However, application of the new genetic technologies makes the creation of "designer diseases" and pathogens with increased military utility more likely [(51), p. 25].

These concerns have been echoed in a number of other high profile reports. For example, the very first European Commission report dedicated to synthetic biology, published in 2007, stated that:

The possibility of designing a new virus or bacterium "*à la carte*" could be used by bioterrorists to create new resistant pathogenic strains or organisms, perhaps even engineered to attack genetically specific sub-populations [(52), p. 18].

A 2012 report from United Nations Interregional Crime and Justice Research Institute (UNICRI) raised similar concerns:

Experts felt that as an enabling tool, synthetic biology [. . .] would in the long term likely facilitate the work of those attempting to acquire and use biological weapons. More dangerous and controllable pathogens could be engineered that lead to novel possibilities in designing bioweapons. Advances in modeling could enable improvements in weapons design. Metabolic engineering might confer new qualities and attributes upon agents and offer options for new types of weapons. [. . .] This could have the negative effect of making bioweapons cheaper and easier to acquire, making their use eventually more likely; more reliable and controllable, making them more desirable; and more effective, increasing their potential impact [(53), p. 34].

These concerns are also evident in the statements made by the Chinese and US delegations in the BWC reports identified under Myth 3.

Influential experts have also highlighted concerns about"superpathogens," for instance Marc Collett, a virologist who was commissioned by the JCVI to provide advice for their work on the risks and benefits of synthetic genomics, concluded that:

While nature has provided would-be terrorists an ample supply and selection of quite virulent viruses, there is concern that genetic technologies will be used to modify these already pathogenic agents and create "super-pathogens," viruses that are more lethal and disruptive than naturally occurring pathogens, and that are designed to evade vaccines or to be resistant to drugs [(54), p. 95].

Maurer and Zoloth, in the article mentioned above, similarly stated that:

Synthetic biology's efforts to reprogram life have raised concerns in some quarters that the technology could one day be used to make radically new weapons, such as pathogens that could be narrowly targeted towards populations with known genetic susceptibilities [(37), p. 16].

Laurie Garrett, in her 2013 article for *Foreign Affairs*, raised her concerns as follows:

a simple, ubiquitous microbe such as *E. coli*, a bacterium that resides in the guts of every human being, can now be transformed into a killer germ capable of wreaking far more havoc than anything on [the US National Select Agent] registry (42).

The 2011–2012 controversy over publication of H5N1 "bird-flu" research also centered on concerns that the published research would provide "blueprints" to terrorists to create highly virulent viruses with increased transmissibility. H5N1 does not spread easily from human to human, but it kills between 30 and 80% of people infected (55). In this experiment, researchers in the Netherlands and the US independently developed a novel strain of the H5N1 avian influenza virus that could spread more easily to humans and other mammals. They passed H5N1 among ferrets and found that a mutated H5N1 virus that was air transmissible could emerge, and that this variant was still highly virulent. When two papers relating similar experimental results were submitted for publication to *Science* and *Nature*, concerns were raised about the dual-use risk and the NSABB recommended against full publication of the study. After additional consultations at the World Health Organization, the NSABB reversed its position and recommended publication of revised versions of the papers (56).

#### **Challenges to Myth 4**

The mousepox and H5N1 experiments are frequently cited to demonstrate how dangerous new pathogens could be designed. However, assessments of this threat tend to overlook the fact that, in both these experiments, the researchers did not actually *design* the pathogens. With respect to H5N1, researchers had indeed been trying to design an air-transmissible virus variant for some time, without success. The ferret experiment was set up as an alternative approach, to see whether "natural" mutations could generate an air-transmissible variant. The researchers had no influence on the specific mutations induced. In the IL4 mousepox experiment, the results were unanticipated by the researchers. In other words, they were not planned for.

Moreover, some of the key lessons that came out of the extensive Soviet program to weaponize biological agents were about the trade-offs between improving characteristics that are "desired," in the context of a bioweapons program, such as virulence, and diminishing other equally "desired" characteristics, like transmissibility or stability. One project, for example, aimed to develop strains of *F. tularensis* (which causes tularemia) that were resistant to current vaccines and to multiple antibiotics. Genes coding for antibiotic resistance were successfully transferred into *F. tularensis*, but the new strain lost its virulence. Domaradsky, who led the research, wrote:

Everyone who has ever dealt with the genetics of bacteria knows how complicated it is to produce a new strain, indeed, to create a new species! [quoted in (57), p. 186)].

The Soviets did, however, eventually succeed in developing a strain of *F. tularensis* that was resistant to multiple antibiotics and retained its pathogenic characteristics. They also worked on four additional bacterial strains – *B. anthracis* (which causes anthrax), *B. mallei* (glanders)*, B. pseudomallei* (melioidosis), and *Y. pestis* (plague) – with the goal of making each of them resistant to 10 antibiotics, but this proved too technically difficult. As Leitenberg and Zilinskas note in their account of the process:

The most difficult problems had to do with pleiotropic effects and a lack of stability in engineered strains. Antibioticresistant cells had a distressing habit of losing virulence or exhibiting lesser yields (or both) when propagated in culture. As for stability [. . .] when the construct for resistance to one antibiotic was introduced into the host cell, an earlier emplaced construct was often lost. This sort of problem required additional rounds of research,which were both labor intensive and time consuming [(57), p. 188].

Pleiotropic effects (where a single gene affects more than one characteristic) and genetic instability are common in microorganisms, and while it is too simple to say that increased transmissibility will always be associated with reduced virulence, this is often the case for strains produced in laboratories. In the case of viruses, this is in part because the production of virus molecules necessitates passage through a series of host organisms, and that during this scaling-up process the virus is not subject to any evolutionary pressure to maintain virulence, and thus – although this cannot be taken as a definitive rule – the virus tends to accumulate mutations that generate an attenuated strain. Similarly, bacteria cultured in laboratories will tend to lose virulence.

#### **MYTH 5**

#### **Terrorists want to pursue biological weapons for high consequence, mass casualty attacks**

Underlying the first four myths are certain assumptions about who the terrorists might actually be, what their intentions are, what capabilities they might pursue, and the level of skills and resources available to them. Despite a lack of analysis of the potential adversaries involved in the misuse of life science research, the bioterrorism threat has generally been portrayed in policy circles as an imminent concern, and emphasis is placed on high consequence, mass casualty attacks, performed with "weapons of mass destruction" (WMD).

For example, in one of the President George W. Bush's earliest statements following 9/11 and the "anthrax letter" attacks that drew the American people's attention to the biological weapons threat, he said:

Since September 11, America and others have been confronted by the evils these [biological] weapons can inflict. This threat is real and extremely dangerous. Rogue states and terrorists possess these weapons and are willing to use them (58).

Later, he set up a WMD Commission and tasked it with examining the threat posed by the nexus of international terrorism and the proliferation of weapons of mass destruction. In its report, this Commission asserted:

Unless the world community acts decisively and with great urgency, it is more likely than not that a weapon of mass destruction will be used in a terrorist attack somewhere in the world by the end of 2013. The Commission further believes that terrorists are more likely to be able to obtain and use a biological weapon than a nuclear weapon. The Commission believes that the U.S. government needs to move more aggressively to limit the proliferation of biological weapons and reduce the prospect of a bioterror attack [(59), p. xv].

Bioterrorism became one of the Bush Administration's key security concerns over its two terms in office. One estimate of civilian biodefense expenditure across the federal government since 2001 is that more than \$70 billion have been spent (60). Despite this, on the 10-year anniversary of 9/11 and the "anthrax letter" attacks, the former US senators who chaired the WMD Commission, Bob Graham and Jim Talent, released a "report card" on America's bioresponse capabilities that concluded the US was still unprepared to respond to large-scale biological attacks. It also warned:

Naturally occurring disease remains a serious biological threat; however, a thinking enemy armed with these same pathogens — or with multi-drug-resistant or synthetically engineered pathogens — could produce catastrophic consequences. A small team of individuals with graduate training in several key disciplines, using equipment readily available for purchase on the Internet could produce the type of bioweapons created by nation-states in the 1960s. Even more troubling, the rapid advances in biotechnology, such as synthetic biology, will allow non-state actors to produce increasingly powerful bioweapons in the future [(61), p. 11].

We see here how the myths we previously discussed, about deskilling and increased access, and about the ease of designing new dangerous pathogens, underlie concerns about terrorists' potential ability to launch a mass attack, and how these are connected, by actors, with the advent of synthetic biology.

The senators were not alone in their assessments. For instance, the US Senate Majority Leader Bill Frist made a similar warning in an earlier speech outlining the global threat of infectious disease and bioterrorism, and the need to better prepare the US and the world to respond to epidemics and outbreaks:

No intelligence agency, no matter how astute, and no military, no matter how powerful and dedicated, can assure that a few technicians of middling skill using a few thousand dollars worth of readily available equipment in a small and apparently innocuous setting cannot mount a first-order biological attack . . . Never have we had to fight such a battle, to protect so many people against so many threats that are so silent and so lethal (62).

Similar messages were reinforced at the highest level. Addressing BWC members at their five-yearly meeting in 2011, Secretary of State Hillary Clinton said:

The advances in science and technology make it [. . .] easier for states and non-state actors to develop biological weapons. A crude, but effective, terrorist weapon can be made by using a small sample of any number of widely available pathogens, inexpensive equipment, and college-level chemistry and biology (63).

She also acknowledged, however, that not everyone in the international community shared the US assessment:

I know there are some in the international community who have their doubts about the odds of a mass biological attack or major outbreak. They point out that we have not seen either so far, and conclude the risk must be low. But that is not the conclusion of the United States, because there are warning signs, and they are too serious to ignore (63).

The belief that the focus should be on mass attacks was bluntly stated by an FBI agent at a symposium on synthetic biology this year (1st May), when she warned: "These technologies do not just pose a risk to individual buildings or cities, but if cleverly deployed, can reduce our population by significant percentages" (64).

## **Challenges to Myth 5**

There are two dimensions to Myth 5. The first is about the intention of would-be terrorists, and the assumption is that terrorists would seek to produce mass casualty weapons and pursue capabilities on the scale of twentieth century state-level bioweapons programs. While most leading biological disarmament and nonproliferation experts believe that the risk of a small-scale bioterrorism attack is very real and very present, they consider the risk of sophisticated large-scale bioterrorism attacks to be very small (65). This is backed up by historical evidence. The three confirmed attempts to use biological agents against humans in terrorist attacks in the past were small-scale, low casualty events aimed at causing panic, and disruption rather than excessive death tolls: (i) the Rajneesh cult's use of *Salmonella* on salad bars in local restaurants to sicken potential voters and make them stay away from the polls during Oregon elections in 1984; (ii) the 1990–95 attempted use of botulinum toxin and anthrax by the Japanese Aum Shinrikyo cult; (iii) and the "anthrax letters" sent to media outlets and members of US Congress in 2001 resulting in at least 22 cases of anthrax, five of which were fatal (66, 67).

The second dimension to Myth 5 is the implicit assumption that producing a pathogenic organism equates producing a weapon of mass destruction. It does not. Considerable knowledge and resources are necessary for the processes of scaling up, storage, and developing a suitable dissemination method. These processes present significant technical and logistical barriers. Drawing from her in-depth study of the Iraqi, Soviet, and US bioweapons programs (3, 4), Ben Ouagrham-Gormley explains:

Scaling up fragile microorganisms that are sensitive to environmental conditions and susceptible to change — and viruses are more sensitive than bacteria — has been one of the stiffest challenges for past bioweapons programs to overcome, even with appropriate expertise at hand. Scaling-up requires a gradual approach, moving from laboratory sample, to a larger laboratory quantity, to pilot-scale production, and then to even larger-scale production. During each stage, the production parameters need to be tested and often modified to maintain the lethal qualities of the agent; the entire scaling-up process can take several years (68).

The dissemination of biological agents also poses difficult technical challenges. Whereas persistent chemical agents such as sulfur mustard and VX nerve gas are readily absorbed through the intact skin, no bacteria and viruses can enter the body via that route unless the skin has already been broken. Biological agents must either be ingested or inhaled to cause infection. To expose large numbers of people through the gastrointestinal tract, possible means of delivery are contamination of food and drinking water, yet neither of these scenarios would be easy to accomplish. Large urban reservoirs are usually unguarded, but unless terrorists added massive quantities of biological agent, the dilution effect would be so great that no healthy person drinking the water would receive an infectious dose (66). Moreover, modern sanitary techniques such as chlorination and filtration are designed to kill pathogens from natural sources and would probably be equally effective against a deliberately released agent. Bacterial contamination of the food supply is also unlikely to inflict mass casualties. Cooking, boiling, pasteurization, and other routine safety precautions are generally sufficient to kill pathogenic bacteria.

The most likely way to inflict mass casualties with a biological agent is by disseminating it as a respirable aerosol: an invisible cloud of infectious droplets or particles so tiny that they remain suspended in the air for long periods and can be inhaled by large numbers of people. A high-concentration aerosol of *B. anthracis* or some other pathogen, released into the air in a densely populated urban area, could potentially infect thousands of victims simultaneously. After an incubation period of a few days, depending on the type of agent and the inhaled dose, the exposed population would experience an outbreak of an incapacitating or fatal illness. Although aerosol delivery is potentially the most lethal way of delivering a biological attack, it involves major technical hurdles that most terrorists would be unlikely to overcome. To infect through the lungs, infectious particles must be microscopic in size – between 1 and 5µm in diameter. Terrorists would therefore have to develop or acquire a sophisticated delivery system capable of generating an aerosol cloud with the necessary particle size range and a high enough agent concentration to cover a broad area. Overall, an important trade-off exists between ease of production and effectiveness of dissemination. The easiest way to produce microbial agents is in a liquid form, yet when such a "slurry" is sprayed into the air, it forms heavy droplets that fall to the ground so that only a small percentage of the agent is aerosolized. In contrast, if the bacteria are first dried to a solid cake and then milled into a fine powder, they become far easier to aerosolize, yet the drying and milling process is technically difficult.

The Aum Shinrikyo cult struggled with dissemination (67, 69, 70). In one of its anthrax dissemination attempts, it sprayed unknown, but probably very large, quantities of a liquid aerosol (most likely crude culture, unprocessed in any way) of *B. anthracis* from the roof of the Aum's headquarters building in Tokyo. For the dissemination, the Aum set up two sprayers on the roof of the eight-story building,each within a large round cooling tower. Pipes were extended from the cooling towers to tanks below, which were filled with a liquid suspension of *B. anthracis*. The device worked poorly, producing large droplets rather than the very fine aerosol needed for effective transmission of anthrax. It also appears the spore concentration was very low (at least five orders of magnitude below that necessary for a highly infectious wet aerosol).

In another dissemination attempt, targeting the area around the Kanagawa prefectural office and the Imperial Palace, the Aum equipped vehicles with spraying devices, but according to prosecutors' statements, the nozzle of the sprayer clogged and the operation failed. Despite its 200 m<sup>2</sup> laboratory containing, amongst other equipment, a glove box, incubator, centrifuge, drier, DNA/RNA synthesizer, electron microscope, two fermenters each having about a 2,000 litre capacity, and an extensive scientific library, and despite its repeated attempts at dissemination, the Aum was unsuccessful in causing any disease, and in retrospect it is clear that the cult did not even make the first substantive step toward an effective bioweapon.

If, despite the odds, aerosolization was achieved, the effective delivery of biological agents in the open air is highly dependent on atmospheric and wind conditions, creating additional uncertainties. Only under highly stable atmospheric conditions would the aerosol cloud remain close to the ground where it can be inhaled, rather than being rapidly dispersed. Moreover, most microorganisms are sensitive to ultraviolet radiation and cannot survive more than 30 min in bright sunlight, limiting their use to night-time attacks. One major exception is anthrax, which can be induced to form spores with tough outer coats that enable them to survive for several hours in sunlight. Terrorists could, of course, stage a biological attack inside an enclosed space such as a building, a subway station, a shopping mall, or a sports arena. Such an attack, if it involved a respiratory aerosol, might infect thousands of people, but even here the technical hurdles would by no means be trivial.

Finally, even if a biological weapon had been disseminated successfully, the outcome of an attack would be affected by factors like the health of the people who are exposed to the agent, and the speed and manner with which public health authorities and medical professionals detected and were able to respond to the resulting outbreak. A prompt response with effective medical

countermeasures, such as antibodies and vaccination, can significantly blunt the impact of an attack. Simple, proven ways to curtail epidemics, such as wearing face masks, hand washing, and avoiding hospitals where transmission rates might soar, can also prove effective in stemming the spread of a disease. Indeed, this aspect of a bioterrorism attack is often underplayed in scenarios like Tara O'Toole's "Dark Winter" and "Atlantic Storm," where the rates of contagion used are often significantly higher than those in historical cases of natural outbreaks (71).

## **DISCUSSION**

We have identified a number of assumptions that underlie policy discourse on the biosecurity threat posed by synthetic biology. We characterize these assumptions as "myths" that pervade discussion on this issue and have identified important challenges to those myths. In particular, we argue that the myths overlook significant difficulties faced when seeking to design and/or produce a pathogen because they focus mostly on material features, thus missing important socio-technical factors, such as tacit knowledge. We have also shown that this dominant narrative underestimates a crucial step needed to mount a terrorist attack, especially a mass attack: the need to produce weapons, not just pathogens. Thus, we conclude that the five myths that recur in the dominant narrative embody misleading assumptions about both synthetic biology and bioterrorism.

The purpose of identifying and challenging these "myths" is not to dismiss the threat of a bioweapons attack. Of course, it is prudent to take measures to prepare against the possibility of a biological weapons attack and concerted action across a policy continuum that extends from prevention through preparedness to consequence management is necessary. However, as we have demonstrated, any bioterrorism attack will most likely be one using a pathogen strain with less than optimal characteristics disseminated through crude delivery methods under imperfect conditions, and the potential casualties of such an attack are likely to be much lower than the mass casualty scenarios frequently portrayed. This is not to say that speculative thinking should be discounted as it can, in some policy contexts, be helpful to represent possible, though not necessarily probable, future scientific developments, in order to encourage thinking on long-term security challenges. However, problems arise when these speculative scenarios for the future are distorted and portrayed as scientific reality in the present, which, as this paper demonstrates, has occurred in policy narratives related to synthetic biology and biosecurity.

We have shown that much of the debate in policy forums about the biosecurity threat of synthetic biology is based on naïve and simplistic interpretations of synthetic biology's ability to "make biology easier to engineer," and in particular on the misleading assumption that the skills and knowledge necessary to perform synthetic biology will necessarily become accessible to people with no specialist expertise working outside professional scientific institutions, including hostile actors who would seek to misuse the technology to develop biological weapons.

In order to understand why such myths develop and persist, it is important to consider the role that they play in the social dynamics of synthetic biology. Drawing on the literature in the sociology of expectations (72), we suggest that particular portrayals of synthetic

biology are mobilized by various actors – deliberately or not – to strengthen their own perspectives and interests, and to help bring into being their own "hoped-for" future. The myths act as "prospecting retrospects": prospects that are deployed in the realtime now, in order to construct particular futures (72). Discourses about the future are *performative*, meaning that they "perform" functions (they "do work") and are also *relational*, meaning that they bind together and enroll actors and other resources into networks (73). Thus, discourse is "wishful enactment" not just "wishful thinking" (74).

With respect to synthetic biology, different communities of actors stress particular issues in particular contexts. This frames the debate in particular ways and plays an important role in constructing and maintaining resources and support for each of these communities. For example, scientists such as Rob Carlson, George Church, or Drew Endy, who are heavily engaged in the promotion of synthetic biology, need to portray an optimistic vision of the potential of the engineering approach to biology as part of their endeavors to develop support for a new field of research which they believe has great significance and potential. Actors in the security field (including some policy makers, social scientists, and natural scientists) play a different role and often exaggerate the "dual-use threat" in order to attract attention and resources to their own work. Researchers from our own field of science and technology studies (STS) are not immune from such processes: we will generally seek to emphasize the complexity of real world situations and the importance of social dimensions of science, in order to justify the need for our expertise. However, at least until now, STS framings have had less influence on the dominant narrative than the discourse mobilized by actors from the fields of synthetic biology and biosecurity. Thus, the myths we have discussed in this paper have played an important role in defining synthetic biology as a "promissory" field of research and as an "emerging science and technology" in need of scenario forecasting, regulation and governance. Our aim is not to denigrate the behavior of those who deploy these narratives. Rather, we suggest that when discourse is understood as something that seeks to change the social world, we can move beyond the battle that we have regularly encountered in discussions about synthetic biology, that focuses on whose prognosis is most accurate and whether or not "it is just hype" (19, 20).

We believe that a better understanding and acknowledgment of the social dynamics at play would help to develop more productive discussions in which the different communities involved could move beyond simply promoting their own interests and perspectives. This is important because in some cases the discourse deployed can have unintended consequences that are detrimental to the interests of the actors themselves, and to the nature of public debate. Thus, overstating the"promise"of synthetic biology applications manifestly leads to parallel overstatements about the "perils" of the field: the promissory discourse of synthetic biology is bolstered by the "promised peril" of misuse by malevolent actors. The fact that these myths (or at least the first 4) serve to bolster the positive promises of synthetic biology helps to explain why these myths continue to persist, despite the fact that they do not accurately reflect current or foreseeable realities for the practice of synthetic biology. This is somewhat incongruous since the hoped-for futures of the actors who promote the benevolent development of synthetic biology do not, of course, include large-scale fatal bioterrorist attacks.

If we are to disentangle synthetic biology and biosecurity concerns, and to have a more refined assessment of both the biosecurity threat and the anticipated benefits, we believe that it is necessary to have more nuanced discussions about the extent to which synthetic biology is, or ever will be, an engineering discipline, and whether, in practice, this would reduce the importance of tacit knowledge, specialist expertise of different kinds, collective work, large infrastructures, and organizational factors. Such discussions would need to identify those aspects of the work that would become easier – in the sense that they can, for example, be automated and reliably performed by a robot – and those which are likely to remain difficult, in the sense that they still require craft skills to be successfully achieved. This would need to take into account not only the material and informational aspects but also other important socio-technical dimensions that will shape the development of the field.

## **AUTHOR CONTRIBUTIONS**

The three authors are listed in alphabetical order and have all contributed equally to this paper, including the conception, analysis and data collection of the research, drafting the text, and revising it critically for intellectual content.

#### **ACKNOWLEDGMENTS**

This research was supported by two awards from the UK Engineering and Physical Sciences Research Council (EP/G036004/1 and EP/J02175X/1). Filippa Lentzos was supported by a UK Economic and Social Research Council award (RES-070-27-0003). The authors would like to thank all the participants at the workshop held at King's College in February 2014, which has informed this research. We would also like to thank Jane Calvert, Piers Millett, Kathleen Vogel, and Luis Campos for providing useful comments on earlier drafts of this paper.

#### **REFERENCES**


**Conflict of Interest Statement:** This research was conducted at the Centre for Synthetic Biology and Innovation (CSynBI), which is a partnership between synthetic biology researchers at Imperial College London and social scientists at the Department of Social Science, Health and Medicine at King's College London. CSynBI seeks to influence the trajectory of synthetic biology and contribute to the development of an appropriate governance regime.

*Received: 11 May 2014; accepted: 23 July 2014; published online: 21 August 2014. Citation: Jefferson C, Lentzos F and Marris C (2014) Synthetic biology and biosecurity: challenging the "myths". Front. Public Health 2:115. doi: 10.3389/fpubh.2014.00115 This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Jefferson, Lentzos and Marris. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# PUBLIC HEALTH

## Dual-use research as a wicked problem

## **Gregory D. Koblentz\***

School of Policy, Government, and International Affairs, George Mason University, Fairfax, VA, USA \*Correspondence: gkoblent@gmu.edu

**Edited by:** Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Kenneth A. Oye, Massachusetts Institute of Technology, USA

**Keywords: dual-use research of concern, biosecurity, wicked problems, H5N1, bioterrorism**

The challenge of dual-use research in the life sciences emerged vividly in 2011 as scientists and policy-makers debated what to do about article manuscripts that described how to modify the H5N1 avian influenza virus so that it could spread between mammals (1, 2). Since H5N1 emerged in Southeast Asia in 2003, it has sickened 667 people and caused 393 human deaths, as well as the deaths of millions of domestic and wild birds (3). The virus has not, however, demonstrated the ability to engage in sustained humanto-human transmission. If a new strain of H5N1 emerged with that capability, and it retained a high level of virulence, it could cause a global pandemic. The experiments by Yoshihiro Kawaoka from the University of Wisconsin-Madison and Ron Fouchier from Erasmus Medical Center in the Netherlands not only demonstrated that mammalian transmission of the virus was possible but also provided information on how to construct such a virus.

These H5N1 experiments are only the latest demonstration of the dual-use dilemma at the heart of the biotechnology revolution: research conducted for peaceful purposes has the potential to be misused for malicious purposes. The H5N1 controversy highlighted the widely divergent views on the benefits and risks of dualuse research held by different stakeholders, including scientists, publishers, biosecurity experts, the national security community, and public health officials. On the one side, proponents of the research focused on the public health benefits of knowing that H5N1 can be transmitted between mammals and which specific mutations can confer this ability on the virus. Opponents of the research highlighted the risks of a laboratory accident and the potential for a

nefarious actor such as a terrorist group or rogue scientist to replicate the research and deliberately release the virus.

The concept of wicked problems provides a new lens for understanding the public policy challenges posed by dualuse research. This concept was first introduced in the 1970s to describe the challenges posed by poverty, urban development, and other social issues (4). Wicked in this context does not mean evil or cool, but instead refers to the intrinsic properties of an issue that make it resistant to long-lasting solutions.

Wicked problems are characterized by multiple, overlapping subsets of problems and high levels of social complexity driven by the number and diversity of players involved in problem-solving. The parties who have a vested interest in how (or whether) the problem is solved are likely to come from different organizations and disciplines with different values and objectives so they will define the problem and acceptable solutions differently. The complex interactions between interconnected issues and the diversity of stakeholder preferences impede the wide acceptance of a definitive statement of the problem. As a result, wicked problems tend to defy traditional linear methods of problemsolving, which rely on a clear specification of the problem to drive the data collection and analysis process. Furthermore, the environment in which stakeholders are trying to solve a wicked problem is dynamic. The constraints on the solution, such as availability of resources and political ramifications, change over time, and stakeholders enter and exit the problemsolving process, change their preferences, or otherwise change the rules by which they address the problem. Since there is

no definitive statement of the problem, there can be no definitive solution. As a result, the problem-solving process ends only when stakeholders run out of time, money, or energy, not when the perfect solution emerges. In addition, solutions to wicked problems are at high risk of having unanticipated effects. Wicked problems are never permanently solved since solutions have implications for other policy domains, which can generate feedback loops or have unintended consequences. The potential for this type of ripple effect increases the scope of stakeholders affected directly or indirectly by policy-making and creates the need for a wider array of information from a broader range of sources to identify the universe of potential solutions and their costs and benefits. In sum, wicked problems are "ambiguous, fluid, complex, political, and frustrating as hell." [(5): p. 2].

Based on these characteristics, dualuse research has all of the signs of being a wicked problem. As two congressional researchers (6) wrote about the H5N1 controversy:

The current issues under debate cut across traditional policy areas, involving simultaneous consideration of security, scientific, health, export, and international policy. Because of the complexity of these issues, analysis according to one set of policy priorities may adversely affect other policy priorities (p. 24).

While wicked problems defy easy and long-lasting solutions, there are several strategies that can be used to manage them. The choice of strategy is dictated by two factors: how concentrated or dispersed power is among stakeholders and how strongly stakeholders struggle for power



amongst themselves (5). Based on these criteria, four coping strategies for wicked problems can be identified: authoritative, hegemonic, competitive, and collaborative (see **Table 1**).

Stakeholders following an authoritative strategy cede the authority to define and solve the problem to a small group of experts. Reducing the number of stakeholders involved in decision-making simplifies and speeds up the process. The use of experts also increases the perceived objectivity, and therefore, legitimacy of the outcome. A drawback to this strategy is that even experts can be wrong or have too narrow of a view (5).

The George W. Bush Administration initially employed an authoritative strategy to address dual-use research. In 2002, the National Academy of Sciences was commissioned to provide recommendationsfor how to balance the costs and benefits posed by dual-use research. As a result of this study, the Bush administration created the National Science Advisory Board for Biosecurity (NSABB) to advise the government on dual-use research oversight. Between 2005 and 2012, NSABB was at the forefront of dual-use research oversight, education, and outreach activities.

When one party is so powerful that it is able to impose its preferred problem definition and solution on other stakeholders, it can employ a hegemonic strategy. While other stakeholders may disagree with the way a problem is defined or solved, the hegemonic strategy simply excludes them from the decision-making process. The main advantage of this strategy is its speed and simplicity: problem-solving by decree. The major disadvantage of this approach is that it is more likely to try to "tame" a wicked problem than actually solve it.

In late 2004, spurred by fears that recent breakthroughs in gene synthesis technology could be used to create dangerous pathogens from scratch, the U.S. Congress made it illegal to synthesize the variola virus,which was defined as"a virus that can cause human smallpox or any derivative of the variola major virus that contains more than 85% of the gene sequence" of variola (7). Scientists objected that the seemingly precise language of the new law could potentially cripple research on smallpox vaccine and other orthopoxviruses since all of these viruses are closely related. This hegemonic strategy was a heavy-handed attempt to"tame" the problem posed by the growing sophistication of synthetic biology by simply outlawing a specific use of the technology.

When power is dispersed and contested, stakeholders view problem-solving as a zero-sum game. Stakeholders pursue a competitive strategy to consolidate their own power in order to define the problem in their preferred way and impose their preferred solution. This strategy can result in more innovative policies due to the struggle by stakeholders to persuade others of their preferred definition and solution. Another advantage of this strategy is that it impedes the centralization of power and creates opportunities for reform when the balance of power among stakeholders shifts. A disadvantage of this strategy is that it is likely to end in stalemate as different stakeholders maneuver to implement their preferred approach and block others from doing likewise (5). The competitive strategy is the default setting for resolving wicked problems in the American political system.

The 2011 controversy over the H5N1 experiments marked a shift from an authoritative to competitive strategy for dealing with dual-use research. The debate over these experiments quickly moved beyond the NSABB and the small community of biosecurity specialists to include the World Health Organization, politicians, scientific publishers, and the scientific community, especially influenza researchers. In an explicit acceptance that the scientific authority of the influenza community was no longer sufficient to shield it from oversight, Dr. Anthony Fauci, director of the National Institutes for Allergies and Infectious Disease (NIAID), told international influenza experts, "The flu

scientific community can no longer be the only players in the discussion of whether the experiments should be done." (8). At the same time, the NSABB's charter was revised to remove its authority to review dual-use experiments and it has not met since late 2012 (9).

Collaborative strategies are best suited for situations where power is dispersed but not contested. Under these conditions, stakeholders can move beyond the zerosum mentality and work together for"win– win" outcomes. This strategy seeks to alter the structure of payoffs to encourage cooperation through repeated iterations to build up trust or create linkages between unrelated issues to expand the potential gains achievable through cooperation. Collaboration can enable stakeholders to achieve results they would not have been capable of reaching on their own and to do so more efficiently. Increasing the number of stakeholders and seeking solutions that are acceptable to as many parties as possible increases transaction costs and delays decision-making.An additional hurdle to collaboration is that each stakeholder brings practice-based "local knowledge" to the table, which is hard to share and difficult for other stakeholders with different identities to internalize (10). Despite these disadvantages, a collaborative strategy has the potential to yield longer lasting policies that are more widely accepted by the relevant stakeholders (5).

Unfortunately, people often have to fail into collaboration. According to Roberts (5),

People have to learn what does not work before they are willing to absorb what they perceive to be the extra 'costs' associated with collaboration. This learning is especially important for people who come from cultures that place a high premium on taking charge, making decisions, being competitive, and using authorities and experts to settle whatever disputes arise (p. 12).

Although the authoritative strategy for addressing the wicked problem posed by dual-use research has now run its course, it is unclear what will replace it. The scientific community views the competitive and hegemonic strategies with a mixture of fear and contempt: contempt for the

push-and-pull of politics that privileges sound bites over the complexities of science and fear of draconian solutions imposed by scientifically ignorant politicians and bureaucrats.

Successful collaboration on dual-use research is more likely to emerge if stakeholders engage in intensive dialog as a means of building a shared understanding about the problem and a shared commitment to solving it. Dialog is not an instrument for decision-making or a negotiating tactic to lead to agreement, but an integral part of the process of creating a shared vision among a diverse group of stakeholders. Getting the right answer is not as important as having stakeholders accept whatever solution emerges (11).

Collaboration can also be facilitated by the emergence of a "collaborative capacity builder" whose role is to ensure the integration of knowledge among stakeholders as part of a long-term strategy to foster a collaborative environment for continuously addressing the dilemmas posed by dual-use research. An individual or organization is empowered to play the role of collaborative capacity builder due to its legal authority, expertise valued by other stakeholders, reputation as an honest broker, or some combination of these values (10).

Recognizing that a problem is wicked is the first step to coping with the problem. Viewing dual-use research as a wicked problem highlights the need for stakeholders to engage in dialog with one another and to adopt collaborative strategies for managing risks in this area. Admitting that the experts do not have all of the answers and giving up the zero-sum view that dominates policy-making in a pluralistic society will be difficult, but the potential benefits of seeking collaborative solutions is well worth the discomfort caused by this mode of problem-solving.

#### **ACKNOWLEDGMENTS**

The author would like to thank the participants in the "Dual Use Dilemma in the Life Sciences" round table at the 2010 International Studies Association Annual Convention. The author would also like to thank Marie Chevrier for her valuable comments.

#### **REFERENCES**


Washington, DC: Congressional Research Service (2013).


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 09 May 2014; accepted: 21 July 2014; published online: 04 August 2014.*

*Citation: Koblentz GD (2014) Dual-use research as a wicked problem. Front. Public Health 2:113. doi: 10.3389/fpubh.2014.00113*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Koblentz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

#### **Keelie Lyn Elektra Murdock <sup>1</sup>\* † and David Koepsell <sup>2</sup>**

<sup>1</sup> Rathenau Institute, The Hague, Netherlands

<sup>2</sup> Department of Philosophy, Delft University of Technology, Delft, Netherlands

\*Correspondence: keeliemurdock@hotmail.com

†Present address: FueledMinds, St. Catharines, ON, Canada

#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Gregory Koblentz, George Mason University, USA David Malet, The University of Melbourne, Australia

**Keywords: principal–agent problems, H5N1, dual-use, science-policy**

### **INTRODUCTION**

When the news broke that Ron Fouchier and his research team at Erasmus Medical Center (MC) in the Netherlands had genetically modified the highly pathogenic avian influenza (HPAI) H5N1 virus and that it had acquired the ability to transmit between mammals, it was a story of scientific discovery and progress and an exciting new development in the international effort to prevent the next pandemic. However, public anxieties and national security concerns would soon become a point of contention between virologists and biosecurity experts in the media and in highly politicized discussions about science-policy. In considering the controversy and the conflicts between scientists and policy-makers, we propose that regarding the situation as a principal– agent problem can yield useful analytical results.

Principal–agent problems occur when two parties that are driven by competing self-interest negotiate the terms of a relationship or contract and act together toward a mutually defined end but an informational asymmetry provides one party (typically the agent) with certain advantages, thus creating tensions. Principal– agent theory provides a template of relational action and the conditional effects of actions in contract situations defined by a functional differentiation, such as between scientists and the government (on behalf of its citizens). Exploring the sciencepolicy nexus from this perspective may further efforts to develop effective policies that address dual-use concerns in the life sciences by offering insights into methods of dispute resolution and the effective design of institutional mechanisms that balance the interests of the parties involved thereby level the playing field.

### **A PRINCIPAL–AGENT PROBLEM?**

Drawing from traditional tools of economic analysis and the theory of rational actors, principal–agent models provide a useful heuristic to explore the behavior of actors and intuitions engaged in contractual relations when there is informational asymmetry and incompletely overlapping or opposing goals. Based on the assumption that the principal requires an agent with a specific skill-set to perform specific actions or functions and both actors enter an agreement to further their interests, the relationship is based on a division of labor. The problem is one of delegation. With imperfect or incomplete information about the interests or the abilities of the applicant, it is possible that the principal will select the wrong agent to pursue its goals and endure the opportunity cost (i.e., adverse selection). If the principal and the agent enter into an agreement, the agent is offered an opportunity to gain from specialization and the informational advantage that it provides, including the conditional authority to act on behalf of the principal, the concomitant financial and professional rewards and some degree of autonomy. The principal must in turn relinquish valued resources and ensure that they receive an adequate return on the investment, in terms of productive labor and output realization.

The central difficulty for principals during the post-contract stage of the relationship, as articulated by Arrow (1), is that "by definition the agent has been selected for his specialized knowledge and the principal can never hope to completely check the agent's performance" (2). The principal must thus bear the risk that the agent will act definitely or in ways that will have consequences for which it will be liable (i.e., moral hazard). The principal also has an incentive to minimize the risk of the transaction by reducing uncertainty and negative externalities (i.e., agency costs). This can be achieved via the strategic introduction of information revelation and generation mechanisms or by offering the agent incentives based on self-interest (such as pay by performance or profit sharing schemes) to create better alignment and ensure better communication and cooperation from the start (3, 4).

#### **RULES OF THE GAME**

As the principal in the relationship between the scientists and the US government, the National Institutes of Health (NIH) retained the Erasmus MC Department of Virology to conduct research in support of the US Department of Health and Human Services (HHS) Pandemic Influenza Plan (5). The terms of the contract between the NIH and the Centers of Excellence for Influenza Research and Surveillance (CEIRS) are defined in the solicitation document and include the provisions for all grant recipients, including foreign institutions, to comply with NIH policies and relevant US regulations (6).

The Erasmus MC Department of Virology was selected to perform research within the CEIRS network because the research proposal defined a problem of mutual interest, in scientific terms as well as in terms that were consistent with the objectives of pandemic preparedness<sup>1</sup> . In addition, Ron Fouchier is specialized in the pathogenicity of respiratory viruses and has an established publication record in this particular domain of research expertise, signaling to the scientific community and funding agencies alike his competency in virology. However, the decision to conduct research on the transmissibility of avian influenza was not a decision that followed directly from NIH funding but had been under consideration at Erasmus MC since the initial detection of the virus in 1997 (7). It was thus against a backdrop of scientific uncertainty and questions about an emerging virus that that Fouchier defined his research questions and the methods of experimentation (8). Given some industry affiliations (including patents), an interest in future commercial applications can also be presumed<sup>2</sup> .

#### **CONTROVERSY**

At an influenza conference, Dr. Fouchier announced that a "stupid" experiment succeeded in creating an airborne strain of the virus and the result was "very bad news" (9). The media had a field day with the story about what was in his terms, "probably one of the most dangerous viruses you can make" (10). It was soon revealed that Fouchier had submitted a paper to *Science* and intended to openly publish the intricate details. The headlines that ensued expressed strong reactions and objections to the research and the publication, including references from reputable biosecurity experts (11).

Upon review by the US National Scientific Advisory Board for Biosecurity (NSABB), the HHS was advised to request redacted versions of two manuscripts, including the paper co-authored by Ron Fouchier and a similar paper submitted to *Nature* by Yoshihiro Kawaoka from the University of Wisconsin-Madison. According to an NIH Press Statement, the board's recommendations called for key details to be removed to prevent the replication of the experiments "by those who would seek to do harm" (12).

The Chair of the NSABB further clarified that the recommendation was based on the perception that the potential negative consequences of publishing the manuscripts outweighed the benefits. The intention, however, was neither to restrict the dissemination of information to persons with a legitimate need to know nor for the US to dominate what was essentially a global issue. Rather, the US government was also considering a mechanism to enable secured access and would pursue "broad" discussions with "global leaders" on matters of policy, science, and public health (13). This point gave credence to concerns that the limitations would interfere with scientific progress and public health preparedness, particularly the recently established and hard wrought Pandemic Influenza Preparedness (PIP) Framework of the World Health Organization (WHO).

The scientists connected to the contentious studies and the Editor-in Chief of both journals conceded to the request for a redaction but on the condition that further progress would be made on matters of policy. The former imitated a "pause" on H5N1 gain-of-function experiments to buy time for scientists to communicate with the public and policy-makers, for governments to consider policy solutions and for the scientific community to assemble and discuss the issues in an international forum (14). The latter indicated that it is next steps would rest on the US government capacity to share the omitted details (15).

At a "technical consultation" hosted by the WHO in Geneva, it was decided by

consensus that the research was essential, that the papers should be published without restrictions, and that limiting access to the research results was missing a practical vision (16) 3 . The NSABB was thereafter requested on behalf of the US government to review two new manuscripts. Clarifications provided by the authors and "non-public data" discussed at the WHO meeting were named as key factors influencing the NSABB decision to revisit the matter (17).

The NSABB concluded by a majority rule that the publications should be "communicated in full" (18). The consideration of new epidemiological data and classified security information relevant to the riskbenefit calculation and the release of a new United States Government Policy for Oversight of Life Sciences Dual-Use Research of Concern (DURC) informed the discussion and influenced the decision. The moratorium, however, was upheld until the following year as NIH funded scientist awaited pending changes in the funding policy for transmissibility studies (19).

#### **DISCUSSION**

Within constraints of science-policy, the roles of the government and of science are institutionally mediated (20). Governments are appointed to serve the interests of the public and retain scientific information to these ends. Scientists are delegated with the authority to conduct specialized research in pursuit of particular goals and missions. The government is empowered to dictate how the agent should act and define the limits of autonomy. Scientists, however, have an informational advantage because they are on the front lines of knowledge development and have other motivations influencing their decisions. The relationship manifests as a dynamic series of moves taken by the principal and the agent to protect their respective interests, beginning with the negotiation of the contract<sup>4</sup> .

<sup>1</sup>The identification of mutations influencing influenza transmission can facilitate the development of medical countermeasures such diagnostics, vaccines, and drug therapies.

<sup>2</sup>At the time the paper was submitted, Fouchier was a part-time employee, the Chief Scientific Officer (CSO) and a shareholder in ViroClinics Biosciences, a lab that conducts virology research in support of clinical trials, diagnostics, and medical treatments. Conflicts of interest in this case, however, were avoided by letting the shares to the Stichting Administratiekantoor Erasmus Personeelsparticipaties (a financial institution established to hold shares for Erasmus University staff).

<sup>3</sup>According to the report, "the group recognized the difficulty of rapidly creating and regulating such a mechanism in light of the complexity of international and national legislation," and thus decided that it was not viable.

<sup>4</sup> In scholarly discussions about the dynamics of science-policy, this process is referred to as "boundary-work" and serves to stabilize the boundary between science and politics.

The contract between the NIH and Ron Fouchier aligned their interests on pandemic preparedness. However, different preferences about how the research results should be communicated combined with insufficient incentives from the government can be perceived as creating a moral hazard. Fouchier's colorful description of his research and outspoken views on openness in science landed him in the political hot seat but the risk was not borne by him. Rather, it is in the interest of scientists to increase their visibility within the research community and to publish in prestigious journals because science communication facilitates research progress and because it can improve their career prospects.

Had the NIH thoroughly considered the objectives of the research methods and the implications in relation to existing dual-use concerns or had Fouchier been compelled to be more explicit about the potential outcomes, the controversy and at least some of the agency costs could have been mitigated.

The agency costs include the research delayed by the extension of the moratorium, the drafting of comprehensive changes in government policies and the implementation and performance thereof. The consequences of the monitoring and new bureaucratic rules, however, will also be felt by the scientists engaged in this type of research.

While not a typical adverse selection problem, the international dimension to this problem raises questions about the relationship between principals and foreign agents and whether discrepancies between national dual-use policies and legal requirements will impact future funding decisions.

The Netherlands for instance requested an export license for Fouchier's manuscript, which delayed the publication and complicated the redaction option (21). In addition, the new US government policies introduced selection mechanisms that exclude certain research projects, including those that cannot be openly communicated or conducted in civilian (non-classified) research laboratories. The review and oversight procedures may also provide a disincentive for scientists to pursue particular research projects or seek certain funding opportunities. This may

impact the international marketplace for research grants. Ron Fouchier for instance has claimed that he "would not be silenced by the Americans anymore," and has continued his studies using funding from the European Union (22).

We propose that exploring the complex entanglement of decisions taken by the many principals and agents in this case, including those on the international level such as the WHO, can provide further insight into the tensions between scientists and policy-makers and indicate what can be done to help all parties involved to achieve their goals. The analytical framework can also help alleviate conflicts and prevent similar problems. For instance, if principals and agents have nonaligned interests, and a stand-off may prevail instead of progress. In addition, when incentives are strong, there is essentially no need for monitoring, which can be an ineffective mechanism if it excessive constraint are placed on the agent (23). These issues have worked to the detriment of the relationship between the principal and the agent in the H5N1 case and of the general public, which may have lost confidence in the institutions involved. This may well have been avoided if a more meaningful negotiation process was pursued or boundary spanning mechanisms were developed to bridge the asymmetry of information, which can improve trust and transparency and increase the possibility of interest mediation (20).

## **ACKNOWLEDGMENTS**

The authors would like to thank Kathleen Vogel for notifying us of this Research Topic and facilitating our participation and also the reviewers for their comments.

## **REFERENCES**


by the federal trade commission.*J Polit Econ* (1983) **91**(5):765–800. doi:10.1086/261181

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 May 2014; accepted: 20 July 2014; published online: 05 August 2014.*

*Citation: Murdock KLE and Koepsell D (2014) Principals, agents, and the intersection between scientists and policy-makers: reflections on the* *H5N1 controversy. Front. Public Health 2:109. doi: 10.3389/fpubh.2014.00109*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Murdock and Koepsell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## **Brett Edwards\***

Department of Politics, Languages and International Studies, University of Bath, Bath, UK \*Correspondence: bwie20@bath.ac.uk

#### **Edited by:**

Jonathan E. Suk, European Centre for Disease Prevention and Control, Sweden

**Reviewed by:**

Samuel A. Weiss Evans, University of California Berkeley, USA

**Keywords: synthetic biology, biosafety, biosecurity, ethics, dual-use, public health, policy**

#### **INTRODUCTION**

In early May, the German Ethics Council produced an in-depth report on the oversight of dual-use research of concern (DURC) (1). The report follows in the wake of recent international emergency reviews of avian influenza research and builds on discussions, which have been taking place internationally for over a decade (2).

In addition to calling for greater awareness raising and education in the scientific community, the report also calls for the establishment of a new legal framework to address DURC within Germany. This framework would provide a legal definition of DURC and would require researchers to report to a newly established central DURC committee before embarking on certain lines of research. Such a legal framework would also generate new responsibilities for those outside the research team who impact upon the research process; from funding right through to publication. For example, this would include new legal responsibilities for Laboratory Biosecurity Officers.

Such an approach would be in stark contrast to the patch-work of largely voluntary measures, which are in place in the rest of the world. The German Ethics Council has also taken the view that Germany should encourage the adoption of similar review models at EU level and internationally.

#### **DUAL-USE AND PRECAUTION**

The summary report of the full 300 page document produced by the German Ethics Council, which is yet to be published in English, notes that:

ethical analysis leads to the conclusion that scientific responsibility in the area of DURC is mainly to be governed by the *precautionary principle* (emphasis added) [Ref. (3), p. 3].

In essence, the precautionary principle places the burden of proof upon the scientific community to demonstrate that research of DURC should be carried out, and is being carried out in a responsible way. The precautionary principle tends to be brought into play in the context of complex risks, which are political challenges characterized by complexity, uncertainly, and ambiguity [Ref. (4), p. 235]. Dual-use issues are complex as they do not involve simple causal chains of events with easily quantifiable consequences, but rather a large set of intervening variables with unknown or even unknowable consequences. This is true not only in relation to thinking about the harms of research, but also its potential benefits. Dual-use issues also involve uncertainty, as there is insufficient data or information to convincingly produce risk verses benefit assessments of single experiments or lines of research. Dual-use issues are also ambiguous, as they typically involve conflicts over ethical and professional values.

Developing a legal and ethical framework to address these issues requires intricate webs of collaboration between institutions, in the context of policy strategy and design, as well as in the context implementation. This creates a challenging environment in which to develop and sustain policy initiatives, directed at problems, which receive only periodic interest from publics and governments.

The reassertion of the role of the precautionary principle in the context of dualuse research provides suitable moment to reflect on broader security concerns related to emerging techno-sciences such as synthetic biology, which extend beyond the single experiments commonly described as constituting DURC. These concerns relate to trends, which could undermine existing models of oversight (such as material and technology containment strategies). This includes concerns about the proliferation of foundation technologies,which could be utilized to modify or synthesize pathogens. These concerns also relate to broader trends in the underlying structures and funding of innovation (5). This includes concerns about de-skilling and proliferation dynamics in life-science research, which could potentially undermine existent and advocated approaches,which place emphasis on local level ethics review, as well as laboratory safety and security (6).

Many of these broader concerns are best thought of as *anxieties* rather than risks, in that discussions about them are largely speculative. However, nonproliferation experts have been keen to reassert, not only that new security challenges are inevitable, but also that existing national and international systems of oversight are poorly prepared (7). It is in this context that the field of synthetic biology has become somewhat of a test-bed for novel security initiatives. In the following section, there is an introduction to how dual-use concerns have emerged in relation to the broader field of synthetic biology, as well as some of the political realities facing those developing policy in this area.

### **EMERGENCE OF DUAL-USE CONCERNS ABOUT SYNTHETIC BIOLOGY IN A US AND EUROPEAN CONTEXT**

There have been discussions of security concerns related to the practices and technologies of synthetic biology, as far back as the community and institutions of the field can be identified. This is perhaps unsurprising, considering that the field emerged in the post 9/11 political environment. However, what is surprising is the high levels of attention this field has received as compared to other contemporary fields of innovation (such as nano-biotechnology).

A key reason for this is that engagement with misuse concerns has been a stipulation of research funding in both the US and the UK. The requirement to address dual-use concerns was incorporated into the National Science Foundation funding criteria for synthetic biology, when the first major publicly funded research center was established [Ref. (8), p. 15]. This led to the establishment of the first major ethical, legal, and social issue (ELSI) thrust with an explicit mandate to consider bioweapon issues. As a result, such concerns also took hold in a European context, as the field was being institutionalized by the research funding bodies. During these early stages a broad range of misuse concerns were under discussion,including those related to the threat of bioterrorism and biowarfare (9–11).

There are two key factors, which are important to thinking about dual-use as an ELSI issue. The first is that dual-use concerns have been a novel addition to more traditional ELSI concerns associated with new and emerging science and technology (such as safety). This means that the issue often competes with more established issues on the ELSI agenda. Security concerns have been more dominant in a US context, but less pronounced in a European ELSI context (12). Added to this, misuse concerns have emerged at a time in which the very concept and practice of ELSI governance is being made subject to transformation.

Both funders and society are increasingly demanding "up-stream" engagement by ELSI thrusts with the innovation process [Ref. (8), p. 15; Ref. (13)]. Up-stream engagement with the innovation process involves engineering safety and security into technologies and research practices, rather than just responding to the challenges raised by the products of innovation. Up-stream engagement is also typically understood to involve pro-active

engagement with key regulators and stakeholders to pre-emptively address potential ethical and legal concerns. Increasingly, such engagement is understood as a part of national government policy to address forward looking concerns about new and emerging science and technology (14, 15).

However, there are several characteristics of dual-use politics, which de-limits the scope and feasibility of such endeavors, which have been reflected in the recent history of dual-use synthetic biology governance in a UK and US context.

### **FROM BROAD ANXIETIES TO NARROW ACTIONS**

The first issue is that despite some of the regulatory back-lash myths, which linger in the US and Europe, governments have not tended to exhibit appetites to legislate *specifically* in relation to dual-use concerns related to synthetic biology. Particularly, with respect to those concerns, which could not be addressed through incremental amendments to law covering laboratory security and safety. Such a situation is symptomatic of a more general trend in dual-use governance in national contexts, in which there is an absence of clear institutional responsibility to develop such policy programs. It is worth noting, however, that even in the absence of "top-down" approaches, regulatory bodies can still play a fundamental role in the fate of so-called "bottom-up" initiatives; by providing financial, political as well as technical support. Such collaboration is also essential if up-stream engagement with the field is actually to result in the development, adoption, and sharing of best-practices nationally.

The second issue is that despite the emphasis on up-stream engagement and best practice sharing at institutions such as SynBERC there has not been substantial investment into systematic and nationwide examinations of the way in which dualuse issues are currently dealt with in different institutional contexts. This is even the case in relation to the field of synthetic biology. Such engagement is necessary if policy discussions about cuttingedge fields are going to be tied to concrete risk identification and management activities in the institutions in which research is taking place. It would seem that without such data gathering, much discussion, particularly in ELSI forum will be condemned to remain an exercise in "speculative ethics" (16).

A final issue is that while considering how to improve security practices at local level is important, there is still a requirement for institutional capacities to identify and respond to much more fundamental trends in S&T, which go beyond the scope of the local level review. In relation to synthetic biology in the US for example, the emphasis on the centrality of local level review, has led to an artificial narrowing of dual-use discussions. For example, the so-called "Sloan Report" (17) still represents one of the most substantial and influential technical reviews of security concerns related to synthetic biology. Yet this report largely externalized those concerns, which could not be identified and managed at local level. Other major reports produced in the US on synthetic biology have also tended to adopt this framing (14, 18).

In particular, the prospect of state level misuse of advances in the life-sciences in the development of weapons has been largely absent from US discussions of synthetic biology as a security concern. In a European context, the issue has only received substantial attention in more recent years. A report on an expert meeting hosted by the United Nations Interregional Crime and Justice Research Institute (supported by the European Commission), describes the various ways in which developments in synthetic biology could re-ignite military interest in biological weapons and potentially undermine existing oversight regimes at national and international level (19).

Such concerns are particularly pressing when one considers the existing challenges, which face the international regime tasked with preventing the development and use of biological weapons. This includes the absence of a system to verify state compliance, which is unlikely to change. While there is slightly more hope for improving the science and technology review system within the regime, improvements continue to be frustrated by a range of bureaucratic and diplomatic issues (20, 21).

#### **CONCLUSION**

To sum up, the point of this article was not to argue that synthetic biology poses an imminent security threat, but instead to argue that while our capacity to imagine misuse scenarios is boundless, our institutional capacities to engage with the less whimsical of these concerns remains quite limited, and developments in policy in this area have been hard won. For some this will not be a cause for alarm, but for others, particularly those with less faith in the resilience of the current norm against biological weapons, this issue continues to be a source of unease.

#### **ACKNOWLEDGMENTS**

I am grateful to Prof. Kathryn Nixdorff for drawing my attention to the work of the German Ethics Committee at a recent meeting on Regional Innovation Biosecurity hosted by the Biochemical Security 2030 Project, at the University of Bath. I am also grateful to the reviewers of this paper and the editors of this Research Topic.

#### **REFERENCES**


*Biotechnol* (2007) **3**(4):333–65. doi:10.1089/ind. 2007.3.333


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 11May 2014; accepted: 26 June 2014; published online: 09 July 2014.*

*Citation: Edwards B (2014) Taking stock of security concerns related to synthetic biology in an age of responsible innovation. Front. Public Health 2:79. doi: 10.3389/fpubh.2014.00079*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Edwards. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

#### **Jonathan E. Suk, Cornelius Bartels, Eeva Broberg, Marc J. Struelens and Amanda J. Ozin\***

European Centre for Disease Prevention and Control, Stockholm, Sweden \*Correspondence: amanda.ozin@ecdc.europa.eu

#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Todd Kuiken, Woodrow Wilson Center, USA Kathleen Vogel, Cornell University, USA

**Keywords: laboratory biosafety, Public Health, infectious diseases, dual-use research of concern, biosecurity, laboratory biorisk management, risk assessments**

#### **INTRODUCTION**

The rapid pace of discovery in the lifesciences can have profound implications for public health, and the focus of much deliberation in recent years has been on how best to ensure that they are positive and not negative. A key focus of debate has been on dual-use research of concern (DURC), which has been defined as life-science research that "could be directly misapplied to pose a significant threat with broad potential consequences to public health and safety, agricultural crops, and other plants, animals, the environment, materiel, or national security (1)." Debates such as the one that surrounded gain-of-function (GOF) research on avian influenza have led to many existential questions about contemporary life-science research, including whether or not such research should even be conducted in the first place, what viable alternative experimental approaches exist, if or how the findings should be made public, and how – or whether – such research can be governed (2–6).

Responding to these questions at a policy level necessarily involve a broader sphere of actors than life scientists alone as they have potential ramifications in different sectors and for society-at-large. Often, it is the security and research communities that have been at the frontline of such debates and driving policy. The public health community tends to enter the fray at later stages, such as after the completion of "concerning" research, at which

point it is asked to either facilitate discussion or comment on the potential public health risks and benefits of research (7, 8). By that stage, public health organizations risk being viewed of as a partisan supporter of dual-use research (9).

In this paper, we demonstrate how the public health sector could more substantially contribute to the debate, guide policy decisions, and promote actions along all phases of the research life-cycle. Before doing so, we articulate the key aspects of the dual-use debate as they are relevant to public health.

#### **"DO NO HARM": PUBLIC HEALTH AND DUAL-USE RESEARCH**

Medical research is intended to promote the health of humans at an individual and a population level along a set of ethical principles laid down in the Declaration of Helsinki<sup>1</sup> and overarching professional ethos in health care to "above all, do no harm" – *primum non-nocere.* What complicates matters with regards to the dualuse research in the area of health is that harm could potentially be done by both promoting and preventing research. On the one hand, research fuels innovation in new medicines, vaccines, and diagnostics, which are fundamental components of public health intervention strategies. The fine balance required for tightening up the regulation of life-science research may, in some instances, lead to some undesirable consequences, such as greater barriers and costs for doing research on listed agents<sup>2</sup> (10). On the other hand, promoting DURC increases the possibility of malevolent use, through providing the information or material that would help state or non-state actors to develop agents for a biological attack. After pathogens are modified or created, responsibilities are created for their physical containment; the issue is thus one of both laboratory biosafety and biosecurity. Even more problematically, once DURC experiments are published, it is nearly impossible to control access to information. In this sense discussions about the risks and benefits of DURC nearly always occur too late – ideally, they should occur before and not after the research has been conducted.

The wide spectrum of potentially problematic issues related to dual-use research necessitates multi-stakeholder engagement including life-science researchers, research funders, regulators, the scientific media, ethicists,social scientists,and security communities. In addition, given the potential societal implications of the topic, greater transparency, and public engagement surrounding dual-use debates is required (11).

There are numerous highly debated issues that have surrounded dual-use research. At the broadest level has been the question of what sorts of life-science research should be conducted, and what constitutes DURC (12, 13). Regulatory measures, such as security clearances for researchers and export controls (10, 14) as well as other risk mitigation measures (1) have been contemplated alongside

<sup>1</sup>http://www.wma.net/en/30publications/10policies/b3/, accessed June 7, 2014.

<sup>2</sup>http://www.australiagroup.net/en/human\_animal\_pathogens.html, accessed May 15, 2014

self-governance measures such as raised awareness and codes of conduct and continued professional development for life scientists (15–18). Furthermore, much attention has focused on whether or how results should be published (6, 19) and how practical measures, such as enhanced laboratory biorisk management processes along (global) standards, can be achieved (7, 20).

From the perspective of public health, these are all important discussions, each aimed at identifying and mitigating potential harm. However, in order to ensure better integration of public health aspects into future policies and actions, a few other key issues must be addressed. One relates to the way in which risk assessments of specific dual-use issues could be strengthened. Another builds upon previous arguments for managing dual-use research by taking into account risk-based approaches and biorisk (biosafety and biosecurity) management in all areas of implementation of the research cycle (7, 21). Addressing all of these public health relevant aspects could offer a next step in structuring the dualuse debate into options to mitigate harm through joint action between the research, security, and policy making communities.

#### **REDUCING INFORMATION ASYMMETRIES IN RISK ASSESSMENTS**

A striking feature of the dual-use debate is that the benefits to public health are often invoked, regardless of the extent to which actors in the public health sector have been consulted. This is particularly ironic given that one core area of public health expertise is the undertaking of risk assessments. For example, one of the rationales for moving beyond the moratorium on publishing A(H5N1) transmission studies was they were"essential for pandemic preparedness" (4). This statement was made after an expert consultation organized by the World Health Organisation (WHO) in February 2012 (7). However, it is important to note that the WHO consultation occurred after the research had been completed and this consultation has been criticized for the absence of a broad range of expertise<sup>3</sup> . It should furthermore be noted that although

the obtained study results are of scientific value, some have pointed out that incorporating detailed sequencing studies on the viral samples from infected patients is, for many countries, likely not the most feasible or effective way of strengthening global pandemic preparedness in the short term (19, 22). Previous research has noted that there are, globally, key gaps in viral sample collection and analysis (23). In addition, although the longer-term benefits may well be substantial,translating the findings from advanced life-science research into tangible and usable products and knowledge can take years, if not decades.

It is essential to keep in mind that advanced research is not always predictable. Discoveries can be serendipitous and researchers themselves may not always be able to predict what sort of outcomes certain experiments might lead to. The point here is not to argue against advanced life-science research but to caution against the creation of overly ambitious expectations, particularly, if these will be the basis for risk-benefit discussions about dual-use research. Scholars of science and innovation, for example, have long observed that the creation of sometimes overly optimistic expectations about the future benefits of research helps to secure funding and lower regulatory hurdles (24, 25). Similarly, it has been pointed out that large funding bodies bring particular interests and institutional cultures to the dual-use debate, one which tends to emphasize the positive benefits of such research (9). In the specific example of GOF research on influenza viruses, it is notable that at least a few prominent scientists believed that the benefits had been overstated (22, 26, 27). One way of mitigating the development of over-expectations for research is to ensure balanced debate and doing so could be achieved by comprehensively including a broader range of perspectives, including public health, in both pre- and post-experimental discussions (11, 26).

There is another important information asymmetry on the other side of the spectrum in debates about dual-use research. This relates to the intents and capabilities of would-be bioterrorists and how this affects risk assessments (28). Presumably, the principle reason for worrying about publishing experimental protocols or genomic information relates to the concern that rogue scientists could replicate the results with malevolent aims. Given the rapid advances in life-science research,the advent of synthetic genomics and the numerous social issues that it raises (29, 30), and the declining costs of doing research, it would seem quite reasonable indeed that a wider range of actors might be able to replicate advanced research. Yet this assumption may appear to overstate the ease with which advanced research can be undertaken, which relies not only on information and materials but on experimental know-how and tacit knowledge (31). The recent history of actual bioterrorist events as well as the findings from a risk analysis appears to support this claim (32). This is a key point. Overstating – or understating – the capabilities and intents of bioterrorists can affect the perception of the "riskiness" of research. Here, the public health sector needs to raise and reiterate the importance of this question. The aim should be to ensure risk assessments integrate the best available information from a variety of sectors, meaning that life scientists, regulators, ethicists, public health actors, *and* the security and intelligence communities will need to become more adept at and comfortable with exchanging information and ideas. This has not always been the case.

## **ADDRESSING ALL PHASES OF THE RESEARCH CYCLE**

The WHO advocated managing dual-use risks by taking account of all stages of the research cycle, which is an approach that we would like to briefly elaborate upon here (7). Public health activities already encompass many of these phases, and thus existing expertise could be harnessed to ensure a broad and comprehensive public health engagement with dual-use research (**Table 1**). Following the discussion above, in the "pre-research" phase, the public health sector could contribute to discussions about the possible risks and benefits of research through consultation with research funding bodies, scientists, and institutional review boards. In addition, good laboratory practice means following

<sup>3</sup>http://www.psandman.com/col/WHO-H5N1.htm, accessed May 14, 2014



ethical and legal guidelines for conducting research on infectious diseases, and for ensuring that robust biorisk management systems are in place. Some European public health institutes have developed tools to facilitate this, such as the Dutch Biosecurity Self-Scan Toolkit<sup>4</sup> or the German development of codes of conduct for potential dual-use research<sup>5</sup> .

During research, biorisk and biosecurity managers should communicate with researchers about emerging findings, and should regularly monitor the adherence to laboratory biosafety and biosecurity standards for research deemed potentially "risky." It is essential to remember that all laboratory research carries a risk. The public health consequences of laboratory accidents are quite serious, particularly if the possibility exists that laboratoryinfected workers expose the general public, as nearly happened with SARS (33). Early communication with public health agencies about findings likely to generate particular attention could also occur during this phase.

Post-research, a clear discussion on the potential public health benefits should be incorporated into risk analyses concerning the implications and publication of findings. Existing protocols for the physical containment of experimental materials should be reviewed and, if necessary, strengthened should DURC be approved.

The recent misplacement of samples of SARS, anthrax and smallpox demonstrates the continued importance of maintaining high biosafety and biosecurity standards for storing physical specimens6,7 .

During all phases of the research cycle, the public health sector works to strengthen its core functions such as surveillance, preparedness, prevention, response, risk communication, and training. In regions where vast amounts of funding dedicated to potentially "risky" life-science research, the public health community should argue for a greater role in participating in research funding prioritization, and risk-benefit assessments. It could also make the case for strengthened and sustainable investments

<sup>4</sup>http://www.biosecuritytoolkit.com/mainMenu.html;jsessionid=AAE30461C0ABD5A9A0D7BC33E91DBBD5, accessed May 20, 2014.

<sup>5</sup>http://www.rki.de/EN/Content/Institute/Dual\_Use/code\_of\_conduct.html;jsessionid=A2FBBD63B5FE63F62EC4B84C0603C295.2\_cid298?nn=4005636, accessed May 20, 2014.

<sup>6</sup>http://www.pasteur.fr/fr/institut-pasteur/presse/documents-presse/communique-presse-l-institut-pasteur, accessed May 20, 2014.

<sup>7</sup>http://www.cdc.gov/media/releases/2014/p0711-lab-safety.html, accessed August 4, 2014

in biosafety, biosecurity, and core public health services: should a dangerous pathogen be released, whether intentionally or unintentionally, then strengthened general defenses against infectious diseases will be essential. Finally, but essentially, should the public health sector seek to develop the role of "honest broker" in dualuse discussions, then it will need to work seriously at fostering engagement with key stakeholders, such as security communities and ethicists in addition to life scientists so as to ensure a comprehensive "web of protection" (4). Hosting international meetings focused on bringing together scientists, the security community, public health workers, regulators and possibly even the public could be an initial way of engaging important sectors of society in the debate.

#### **CONCLUSION**

Advances in life-science research are staggering with the advent of synthetic biology, the latest in a long line of technological breakthroughs. There is little to suggest that this pace of change will slow in the future and the increasingly global nature of science means that all countries have a stake in ensuring that research is conducted and disseminated responsibly. Thus far, debates about dual-use research have tended to invoke public health rationales but there is much room for improvement for ensuring that public health perspectives are fully integrated into discussions. This will be challenging, and mechanisms and fora for doing so will need to be created. Yet there is much to be gained from doing so, for the ethics behind the debate closely matches the public health ethos: *primum non-nocere*.

#### **ACKNOWLEDGMENTS**

The author team acknowledges the inputs from the Chief Scientist Johan Giesecke and other colleagues from the ECDC Disease Programmes, Core Sections, and Directors office for their collaboration and support for this work.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 12 June 2014; accepted: 23 July 2014; published online: 12 September 2014.*

*Citation: Suk JE, Bartels C, Broberg E, Struelens MJ and Ozin AJ (2014) Dual-use research debates and public health: better integration would do no harm. Front. Public Health 2:114. doi: 10.3389/fpubh.2014.00114 This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Suk, Bartels, Broberg , Struelens and Ozin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## A new role for public health in bioterrorism deterrence

## **Margaret E. Kosal \***

Georgia Institute of Technology, Sam Nunn School of International Affairs, Atlanta, GA, USA \*Correspondence: margaret.kosal@inta.gatech.edu

## **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Jonathan E. Suk, European Centre for Disease Prevention and Control, Sweden Amanda J. Ozin, European Centre for Disease Prevention and Control, Sweden

**Keywords: bioterrorism, biosecurity, bioweapons, national security, public health, deterrence, polio, Ebola**

When this commentary was submitted in April 2014, only a handful of scholars and policy-makers in the defense and security communities were following the Ebola outbreak in West Africa, which was over 4 months old at that time. Now that thousands of people have died, cases have spread to the US and Europe, and thousands of US uniformed military are being deployed on humanitarian assistance/disaster relief missions, attention and interest are significantly heightened. The events of the last few months demonstrate the criticality for interdisciplinary thinking, which is more challenging due to different historical contexts, knowledge bases, interests, lexicon, and perspectives.

This commentary will explore the creation of new relationships between deterrence, infectious disease, and public health to reduce the threat of biological terrorism and increase international security. Examining the global spread of re-emerging infectious disease, such as the re-emergence of polio from northern Nigeria, offers a novel case study for thinking about how to deter potential bioterrorists who seek to use infectious disease. Polio outbreaks have more directly affected the developing world compared to the US or other nations with robust public health sectors. This example suggests that a bioterrorist attack would also be more devastating for developing countries in low-resource settings compared to the western world. Credibly, communicating this may offer a new approach to deterring bioterrorism by foreign actors. Although a robust public health sector has long been noted to reduce the vulnerability to a bioterrorism attack, actively promoting the strength of US public health can also serve as a powerful deterrent in its own right. The issue of terrorist groups utilizing biological weapons against other states is a mounting concern, yet little deterrence research in the field of political science addresses methods of dealing with the threat of bioterrorism. Thus, creating new conversations among the life sciences, public health, and political science can lead to new perspectives on deterring bioterrorism.

The issue of bioterrorism deterrence, if addressed, has been often added or subsumed under the auspices of deterrence strategies associated with nuclear weapons. In the second half of the twentieth century, nuclear deterrence dominated geopolitics and national security strategies. At its height, the threat of mutually assured destruction (MAD) existed in which both superpowers possessed arsenals with second-strike capabilities, i.e., the ability to respond to a first nuclear strike on land via use of nearly undetectable submarine-launched ballistic missiles with nuclear warheads.

These historical approaches, however, undermine and oversimplify the distinct challenges of deterring bioterrorism. One such method attempted is focusing on pathogen security, or securing and denying access to the materials necessary to develop biological weapons (i.e., deterrence by denial). Based on the nuclear non-proliferation model, pathogen security strives to control the materials, equipment, and personnel involved with production and use of biological agents. With nuclear weapons, controlling fissile materials proved successful because of key characteristics of the critical materials: fissile material is man-made and can be tracked. Those same characteristics that

make nuclear weapons easier to track are those that make biological weapons material difficult to monitor. These characteristics include the presence of biological agents in nature, lower production costs, increased diversity of materials that could be used in bioweapons attacks, and multiple legitimate uses for biological materials. These differing features have not always been fully considered by policy-makers (1). Rather than focusing solely on securing biological materials and laboratories from misuse, other recommendations and strategies that the US has pursued include prevention measures such as biosurveillance, global laboratory and research cooperation, research and development of diagnostics and countermeasures,international stockpiles of effective medical countermeasures, and increased response and mitigation capabilities (2–6). These approaches aim to reduce consequences of an attack, afford earlier detection, and reduce vulnerability; they do not address the challenge of deterring use and reducing motivation directly, however.

To date, discussions about public health and deterrence have focused on measures such as regular vaccinations; access to timely medical care to treat infected, isolate suspected infected, and mitigate the spread of disease; confidence in the professional nature of health providers, etc. These are largely passive, defensive deterrence measures, in that they demonstrate credible capacity by a state to respond and mitigate the consequences of an attack (post-exposure) or reduce vulnerability to an attack by making it ineffective (preexposure) (7–9). Both approaches mentioned thus far, pathogen security and a defensive approach to terrorism, which ultimately aim to decrease vulnerability by fortifying civilian populations, are examples of deterrence by denial adapted from the realm of nuclear deterrence.

In contrast to these passive approaches, active deterrence strategies have not been explored. Active deterrence is actions and policies preventing a specific opponent from doing something they may wish to do. Traditionally, robust active deterrence has involved the application of expressive force to change the policy or character of the target government or group (10). Forces and policies are used to send a political message. In contrast to passive strategies, active deterrence is more dynamic and may incorporate escalating threats in response to an adversary. What this would look like at the nexus of international security and public health is largely an unexplored area of study or policy. Therefore, there are limited models for thinking about deterrence that have been developed exclusively for bioterrorism. As a consequence, the role of a robust public health system for twentyfirst century active deterrence remains to be explored. There has not been a substantive consideration of robust public health system as a strategic asset in a more active deterrence role.

The threat of inflicting punishing retaliation against some aggressor, not the ability to prevent some hostile act from occurring, is the core of traditional deterrence theory. Within new deterrence approaches in political science, however, there are several types of definable strategies that may be applied to bioterrorism by foreign actors (11). *Indirect deterrence* focuses on third party players and their roles in terrorist attacks. Third parties are most typically state sponsors or supporting financiers. This concept is based on the recognition that while a terrorist may be willing to die for his cause, it is less likely that explicit and tacit supporters are willing to pay a similar retribution. Appealing to or directing bioterrorism deterrence efforts toward tacit supporters is an untapped area. *Collective actor deterrence* utilizes the power and influence of institutions like the United Nations, NATO, or other broad coalitions to deter terrorist actions, highlighting the legitimacy of the organization and the international community rather than the interests of a single state. For bioterrorism, the WHO and African Union's disease

eradication efforts are examples. *Internalized deterrence* plays off the psyche of a terrorist, combining abstract concepts of criminology and social constructivism to subconsciously deter a terrorist through social taboos and norms (12, 13). This might involve leveraging fear of disease spreading to oneself or one's own community. *Tailored deterrence* attempts to individualize each situation to reach the best possible solution,leveraging cultural, political, social, and other specific knowledge. These newer deterrence strategies offer opportunities for dealing with bioterrorism threats by foreign actors, which could be combined with public health information and resources.

In thinking about public health infrastructure as an active or passive part of new deterrence strategies, it is useful to think about the role of missile defense. As the presence of a ballistic missile defense system is supposed to be an existential deterrent itself, so could be a strong public health system. Missile defense is both a passive deterrent and, if used, an active deterrent, as it stops somethingfrom occurring. A strong public health infrastructure is likely to be the key in reducing the vulnerability to bioterrorism attack, as well as having a potential role in deterring a foreign terrorist group from even considering such an attack. If a biological weapon launched by a terrorist group will have little or no effect on the target country because of a known robust public health sector, then a foreign terrorist may be discouraged from launching a biological weapons attack in the first place. If foreign terrorists are also aware of the weak public health infrastructure with their own borders, and the increased risks to them and their publics in the event of an accident in developing biological weapons and/or spread of an infectious disease that they might launch, this may also deter them from pursuing this work. In addition, even the accidental release of a dangerous pathogen or the spread of an infectious disease via attack will most likely cause disproportional negative effects to nations with limited public health infrastructures and affect tacit and explicit supporters in those states.

The role of a robust public healthcare system for its deterrence capacity can be explored through empirically driven case study methods against predominant theories of deterrence in political science (14, 15) and in comparison to other works considering the possibility of deterring bioterrorism (16–20). For example, the reemergence of polio offers a potentially useful example to think about the effects of a potential bioterrorist attack on the developed and the developing world. Polio is both a contagious infectious disease and transmissible from human-to-human (like smallpox and plague). The poliovirus is highly transmissible with a basic reproductive rate or secondary transmission rate (R0) exceeding most suspected biological agents, e.g., standard estimates of R0 for polio range from 5 to 7 (21, 22), whereas R0 for suspected bioterrorist agents like smallpox (1.8–3.2) (23–25); pneumonic plague (0.8–3.0) (26, 27); and even Ebola (1.34– 2.0) (28, 29) are lower. It is not a likely biological terrorism agent, however, due to the low-mortality associated with infection. It is, however, a useful model for thinking about the spread of infectious disease and the importance of a robust public health infrastructure as a deterrence strategy.

At the beginning of 2003, the complete eradication of polio appeared to be within the grasp of the World Health Association and its many partners. In 1998, the World Health Organization estimated there were over 365,000 new cases of polio; by early 2003, the rate of infection had declined to <1,000 new cases worldwide due to a vigilant vaccination effort (30). That trend was interrupted, however, when Nigerian citizens refused to be vaccinated after hearing unfounded allegations of contaminated vaccines that would lead to sterility or cause HIV/AIDs. Before 2003, polio had largely been confined to only a handful of countries; Nigeria, India, Pakistan, and Afghanistan accounted for 93% of the world's cases (31). What started with the refusal of local clerics to allow vaccination led to the reestablishment or importation of the poliovirus to 14 countries that were previously disease-free.

Transport of the contagious virus was not limited to neighboring African states. The poliovirus moved through Sudan to Ethiopia crossing the Red Sea to Lebanon andYemen. The latter was been particularly severely affected, witnessing more than 500 new cases in the first half of 2005. The poliovirus spread as far as Indonesia, where it afflicted more than 150 people in a single year in 2 provinces, predominantly children (32). Prior to this outbreak, Indonesia had been polio free for nine years. Genetic fingerprinting confirmed that the strain imported to Indonesia came from northern Nigeria through Sudan, most closely resembling an isolate recovered in Saudi Arabia in December 2004. A pilgrim returning from Mecca or a returning foreign worker is suspected to have brought the virus to the island of Java, across an ocean and thousands of miles from its source. The polio virus continues to persist in a limited number of states in the developing world, specifically in Nigeria, Afghanistan, and Pakistan, where a ban on vaccination by Islamist leaders in Waziristan remains in place. Since 2013, polio (linked genetically to the strain in Pakistan) has spread from Syria to Iraq (33).

Countries that have witnessed the reemergence of poliovirus outbreaks have some crucial links: social and political challenges that have impeded the development and implementation of appropriate public health infrastructures and measures. Not unexpectedly, there is an inverse relationship between government health expenditure in health and number of polio cases.

Looking at the spread of polio can provide us with a lens to think about the impacts of bioterrorism in states with developed public health infrastructures and those who do not. A bioterrorist attack, especially one with a contagious agent like smallpox or pneumonic plague, will likely impact the developing parts of the world substantially more than the US. One only has to look as far as polio's re-emergence (or more recently the outbreak of Ebola virus disease in West Africa) to see the very real repercussions of a contagious virus and how the most dire causes and effects of infection and spread stem from poor public health infrastructures (34).

Creating a new deterrence strategy for bioterrorism is needed. Credibly, communicating the differential capacities to respond and the comparative likely outcomes will require diplomacy, coordination with civil affairs, specialized knowledge of individual states, and regions of the developing world. These are fundamentally interdisciplinary efforts that should leverage small teams from

diplomatic, development, public health, and defense communities. One single parochial voice will be inadequate. Further improving the US domestic public health infrastructure would be beneficial and cost effective regardless of whether an outbreak is intentional or natural. The devastating Ebola outbreaks serve as a call for urgent investment in public health infrastructures worldwide, to provide both responsive and proactive actions to deter bioterrorism and to deal with natural disease outbreaks. Public health remains a powerful and often underutilized asset for bioweapons defense through vulnerability reduction; leveraging public health may also enable new approaches to deterring bioterrorism threats. International security scholars would benefit from better understanding of and leveraging the knowledge of the public health community.

#### **ACKNOWLEDGMENTS**

Thanks to Katherine Lange and Ana Terron for research assistance and to the reviewers for helpful suggestions.

#### **REFERENCES**


31. Minor PD. Polio eradication, cessation of vaccination and re-emergence of disease. *Nat Rev Microbiol* (2004) **2**:473–82. doi:10.1038/nrmicro906

32. Pallansch MA, Sandhu HS. The eradication of polio – progress and challenges. *N Engl J Med* (2006) **355**:2508–11. doi:10.1056/NEJMp068200


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 29 April 2014; accepted: 27 November 2014; published online: 10 December 2014.*

*Citation: Kosal ME (2014) A new role for public health in bioterrorism deterrence. Front. Public Health 2:278. doi: 10.3389/fpubh.2014.00278*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Kosal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## The irrationality of GOF avian influenza virus research

## **SimonWain-Hobson1,2\***

<sup>1</sup> Molecular Retrovirology Unit, Department of Virology, Institut Pasteur, Paris, France <sup>2</sup> Foundation for Vaccine Research, Washington, DC, USA

## **Edited by:**

Amanda Jane Ozin, European Centre for Disease Control and Prevention, Sweden

#### **Reviewed by:**

Li Xu, Cornell University, USA Gigi Gronvall, UPMC Center for Health Security, USA Hervé Raoul, INSERM, France

#### **\*Correspondence:**

Simon Wain-Hobson, Molecular Retrovirology Unit, Department of Virology, Institut Pasteur, 28 rue du Dr. Roux, Paris 75015, France e-mail: simon.wain-hobson@ pasteur.fr

The last two and a half years have witnessed a curious debate in virology characterized by a remarkable lack of discussion. It goes by the misleading epithet "gain of function" (GOF) influenza virus research, or simply GOF. As will be seen, there is nothing good to be gained.The controversial experiments confer aerosol transmission on avian influenza virus strains that can infect humans, but which are not naturally transmitted between humans. Some of the newer strains are clearly highly pathogenic for man. It will be shown here that the benefits of the work are erroneous and overstated while the risk of an accident is finite, if small. The consequence of any accident would be anywhere from a handful of infections to a catastrophic pandemic. There has been a single open international meeting in this period, which is surprising given that openness and discussion are essential to good science. Despite US and EU government funding, no risk–benefit analysis has been published, which again is surprising. This research can be duplicated readily in many labs and requires little high tech. It falls under the definition of DURC without the slightest shadow of a doubt and constitutes the most important challenge facing contemporary biology.

**Keywords: avian influenza, airborne transmission, human adaptation, DURC, unfalsifiable**

Science excels in making things that work: vaccines, smart phones, and airplanes. This is the implicit promise made to society and one underpinned by basic science. "If you invest in the biomedical research enterprise, ultimately it will deliver products that will impact global health and alleviate suffering." Biomedicine has delivered on this promise with spectacular success – average life expectancy is now out to 80 years and beyond in some countries.

Making things that work relies on solid data that resist the tests of time. With a background in HIV evolution and genetics, I became drawn to the latest hot topic in virology, which is to predict the future of rapidly evolving viruses such as avian influenza A H5N1 or H7N9 (1–4). I was bothered because the claims about delivering vaccines and drugs based on lab experiments did not square with my understanding of rapidly mutating viruses (5).

The issue is how these avian viruses will evolve and whether we can anticipate their trajectories by performing accelerated or forced evolution experiments in the lab. If yes, the reasoning goes that we can make vaccines out of these strains, develop drugs, and stockpile them, so heading off a future pandemic, assuming that a similar, but this time naturally arising strain, emerged. The vaccine goal is the most touted of benefits, no doubt because of their phenomenal cost–benefit ratio.

In the last 100 years, influenza pandemics occurred in 1918, 1957, 1968, 1977, and 2009, meaning that a pandemic can strike every 10–40 years. Influenza A viruses are distinguished by one of 16 hemagglutinins (H) and one of 9 neuraminidase (N) proteins on the surface of the virus, essentially marking them out antigenically. There are 110 strains of avian influenza viruses carrying one of 16 hemagglutinins (H1–H16) and one of 9 neuraminidase (N1–N9) proteins. The reservoir of influenza viruses in ducks, shorebirds, birds, and chickens is by far the largest among animals,

although it is also found among pigs, dogs, and horses, to mention a few species.

As viral crystal-ball-gazing is a new topic with a scant literature, let us look at the track record for predicting influenza pandemics. Pandemic viruses in man are referred to as H1N1 (1918, 1977, and 2009), H2N2 (1957), and H3N2 (1968). For the Spanish flu virus (H1N1 1918), all eight segments of the genome came from bird strains. The 1957 H2N2 and 1968 H3N2 viruses represented mixes of more avian viruses along with parts of the 1918 virus. 1977 H1N1 represented an accidental reintroduction of an old vaccine strain pre-1957, probably from a Russian research lab (6).

Only by the twenty-first century did we have the wherewithal in evolutionary and genetic terms to have even the slightest chance of predicting a pandemic. All bets were on an avian virus spilling over in SE Asia. The 2009 H1N1 pandemic was a surprise on at least three counts: (1) it emerged from swine, (2) it was in NW Mexico, and (3) it represented the first time that a pandemic was initiated by a strain belonging to a virus that was already circulating – descendants of 1977 H1N1 were around in 2009 (7). In short, flu virologists were far from the mark (8–10).

Other avian influenza viruses cross over to humans causing mild to severe disease with case fatality rates that can sometimes approach 60%. Despite this, these viruses are hardly ever transmitted between humans because they lack the mutations enabling them to grow well in the upper respiratory tract. As such, they represent dead-end infections. In the last 17 years, the number of unequivocally documented H5N1 cases in humans is of the order of 650 or so. For H7N9 human infections, the number is 448 and rising. For both examples, the viruses spilled over directly from mixtures of duck, chicken, and bird strains. H7N9 was totally off the radar when it first struck in China in 2013 (11). Indeed, there had not been a single prior report of H7N9 in humans, which was very worrying and again illustrates how little we still know about flu viruses. However, when turning to serological surveys, which are woefully few, one finds that viruses from at least 11 hemagglutinin groups can be detected in farm workers in China (12). Presumably, these represent mild to asymptomatic infections that essentially go unrecorded. It shows that very probably large numbers of avian influenza viruses silently spill over to humans without any fuss.

In an attempt to recapitulate the evolutionary process and to anticipate the future, two groups performed forced evolution experiments on avian H5N1 influenza strains from 2004 and 2005. They used a ferret transmission model. The ferret is the animal of choice in influenza research for a number of reasons, one of which is that the animal sneezes, much as humans do. When housed in adjacent cages with an airflow carrying aerosols from the infected animal across to the receiver ferret, it is possible to ascertain whether a virus is capable of efficient airborne transmission. By repeating the process four to five times, they rapidly selected for such viruses (1, 2). Actually, the number of ferrets in each experiment was so small as to invite criticism on statistical grounds alone (13). While the ferret model has its limits, as pandemic human viruses are transmissible between ferrets by the airborne route, it could be assumed that these viruses will be so.

A more recent study on an H7N1 strain started with a virus that was lethal in ferrets with neurological complications (4). With minimal effort, a strain transmitted by the airborne route was obtained without loss of pathogenicity. In the earlier studies on H5N1 avian influenza, few ferrets showed respiratory distress. Now we are dealing with strains capable of a lethal respiratoryacquired infection. Obviously the"proof"experiment, inoculation of human volunteers with one of these lab-generated viruses, cannot be ethically performed. Yet, this creates a very unsatisfactory situation because science is about resolving conjecture, not making it. Assessing the risk to humans is equally stymied by these unfalsifiable findings, to use a Popperian term. Needless to say, precautionary logic and a savant interpretation of Murphy's Law suggests that these viruses should be considered as highly dangerous for man. In short, the risk level has been enhanced by this work.

At a 2014 meeting on infectious diseases, Dr. Kawaoka reported experiments whereby he forced the evolution of the pandemic H1N1 2009 virus so that it could escape from natural human antibody responses. The experiment was not complicated: it involved simply mixing virus with sera from individuals who had been naturally infected and selecting out the virus that was not neutralized. A total of 15 sites on the virus hemagglutinin protein were identified. Concentrating on five of these sites, he was able to produce strains that completely escaped human antibodies and extant vaccine coverage. This experiment is different from prior avian influenza virus gain of function (GOF) experiments in that we know the virus is readily transmissible among humans – after all these lab-made strains are derived from the pandemic H1N1 2009 virus! As the strains escape vaccine control, they constitute, unambiguously, a HUGE risk to man.

Increased risk *per se* should not be frowned on if there are substantial benefits to be had. So what are the purported benefits of influenza A GOF research?

As the proponents talk about stockpiling preventive vaccines (4), we will examine this hypothesis. Given the annual change in the antigenic composition of a virus, the tried and tested working rule toward making a vaccine is to select from circulating strains those that are most likely to cover the world's population in the next few months. The CDC has just reiterated this logic by their choice of viruses for the season 2014–15 (14). A single antigenic mismatch can substantially reduce vaccine efficiency. As mentioned above, there are 110 genetically confirmed combinations of avian hemagglutinin and neuraminidases presently circulating in ducks, shorebirds, and birds.

Perhaps a subset of these 110 avian viruses might pose a threat to man, yet we simply do not know this. The recent isolation of H10N8 from two patients, one of who had underlying immunosuppression and subsequently died (15), sparked three commentaries from influenza virologists along the lines of "H10N8, the next pandemic?" (16–18). Such dramatic extrapolation from two case reports does not help for sound science. But if we take them at face value, then we need a vaccine to this strain.

For complete coverage in the US, the cost of 314 million doses of a commercially available influenza vaccine to the public sector is presently between \$6 and 15 per dose<sup>1</sup> , or between \$1.9 and 4.7 billion. To be fully prepared for a pandemic would require preventive and stockpiled vaccines for all 110 strains. This ramps up the cost to something of the order of \$209–517 billion. As the shelf life of an inactivated vaccine is ~12 months at 2–8°C, these would be annual costs<sup>2</sup> . Even if the number of vaccines were reduced to one per hemagglutinin (there are more distinct H than N proteins), a minimum number would be 16 stockpiled vaccines, or \$30–75 billion annual costs.

These back-of-an-envelope calculations show that stockpiling vaccines is effectively science fiction, even if, with economies of scale, costs could be slashed by a factor of 10. Of course, these numbers ignore investments in staff and production facilities, to mention just a couple of issues. By comparison, radical investment in developing a near-universal flu vaccine, or vaccines that induced broad immune responses might be much cheaper.

Regarding the development of anti-viral drugs, the response is binary. If the virus is sensitive, society will go with what already exists. If not, the development of novel drugs normally involves a >10-year-cycle to get to market with a winnowing down of a large number of candidate molecules down to a very small number. Again the costs are huge and without a clear virus strain in the crosshairs, backed up by a scientific consensus, industry will not rise to the challenge.

What is the chance of experimentally settling on a combination of mutations that allows, say, avian H12N5 to become transmissible between ferrets that could also be thrown up by nature?

<sup>1</sup>http://www.cdc.gov/vaccines/programs/vfc/awardees/vaccine-management/ price-list/#adflu

<sup>2</sup>http://www.who.int/immunization\_standards/vaccine\_quality/pq\_239\_ influenza\_seasonal\_10dose\_sanofi\_pasteur/en/

I, for one, have not the slightest idea. You could perhaps document a restricted number of mutations but these would still be a reasonably large number allowing numerous permutations. The notion of hitting on a single solution is highly erroneous and constitutes a flawed appreciation of evolution. For example, over the course of evolution, the eye evolved independently something like 40 times. Taking an example from my world of retroviruses, there are at least six different ways to express the reverse transcriptase gene.

And so it goes with influenza. While the initial papers on H5N1 showed that the hemagglutinin gene had acquired important mutations to allow it to bind to human receptor molecules in the upper respiratory tract, a subsequent paper showed that the same mutations did not confer the same phenotype on other H5N1 viruses currently circulating (19). This is not surprising to virologists, or scientists with knowledge of protein structure. The overriding question is how many solutions are out there? It may not be possible to answer this accurately, although with techniques such as saturation mutagenesis, it may be possible to asymptote toward defining a fraction for an individual flu protein. With regular exchange of avian influenza genes, it will be an extraordinarily difficult challenge.

To resume, virological crystal-ball-gazing is even harder than the real thing. Virology can only deliver a limited number of answers in this area, very few of which may be useful for the development of effective pandemic vaccines, anti-viral drugs, or enhanced pandemic preparedness. By contrast, rapid surveillance and communication of findings seems to be *de rigueur* and have been shown to work in the real world. Existing networks are picking up isolated cases of H6N1 and H10N8 avian influenza meaning that they are doing a very good job (15, 20). In all likelihood, these will be dead-end infections that will not set off a pandemic, which are rare events given past information.

These forced evolution influenza virus experiments are most unlikely to deliver much practical information, nothing that a Health Minister could mobilize around. Meanwhile the risks are finite and small, but of catastrophic proportions if ever there was a breakdown of biosafety or biosecurity.

Advocates of this GOF research are off the mark for three other reasons. First, scientists are notoriously optimistic about their work and systematically underestimate risk (21). When pushed, they can hardly find a web page or reference citing the data about lab accidents. Second, they feel that once funded they should be free to publish – it has become a struggle for many in what is a crazily competitive race where publishing in big journals is a question of survival. "A paper in Nature is worth every risk" is how a colleague who fled Budapest in 1956 summed it up. Certainly, scientists should be as unfettered as possible, but enhancing the danger level of a virus impacts public safety and society as a whole. Society is the ultimate arbiter, a fact revealed by the scientists themselves in their grant proposals and papers where they explain how dangerous their virus is. They never miss an occasion that includes mortality statistics in a basic science manuscript, or to point out that a vaccine and drugs to their virus are lacking. *So why is it they are so refractory to discussion and openness?* Where are the three or four flu congresses where this topic has been openly debated?

This inability to discuss goes further. Surprisingly, no government agency, learned body, or independent organization has commissioned a comprehensive risk–benefit analysis on GOF influenza research, despite more than 2 years of controversy. Almost none of my colleagues are aware of the 2007 InterAcademy Panel (IAP) statement that"Scientists [too] have an obligation to do no harm."<sup>3</sup> . Indeed, let us discuss what is and what is not DURC, but let us not hide or forget the IAP statement.

The third point concerns dissemination of information. These flu virologists are not trying to hurt their fellow beings, nor deliberately trying to set off an influenza pandemic. Yet, they are part of the paper race to publish. Once published, these studies can be reproduced at far less cost. Knowledge that it can be done is enough. And if reproduced in other labs with lower containment facilities or scrutiny, these strains could well proliferate, the corollary being an increased risk of an accidental release. Most scientists would see this as not part of their brief; "I did my bit, I'm not responsible for others." In short, the larger picture, that of DURC, is simply not on their radar, the terms not part of their vocabulary. This absence of reflection was apparent in the debate about redacting parts of the original manuscripts on H5N1 GOF research. The papers were uploaded via the Internet, which means that they were on the cloud. Any computer security jock could find the data. Correct? I asked a colleague to track down a manuscript from my lab that had uploaded to a major journal. He had only the title page and agreed not to hack the Institut Pasteur server, which would have been too easy. He retrieved a complete pdf of the manuscript compiled by the journal web site in <2 min! This shows just how computer naïve the original discussion was.

The DURC issue, particularly in our Internet age, needs far more debate. Even though the risk of an error or a lab accident may be small, the consequences could be catastrophic. For this work to proceed, there needs to be a clear consensus based on an open discussion that the benefits outweigh the risks. Judging by the controversy, a consensus is clearly lacking (22, 23).

By resorting to semantics, or hiding behind the cloak of freedom to investigate, or whipping up fears about increased regulation, yet continuing GOF work, these researchers are showing themselves to be remarkably cavalier, disdainful of public opinion, and totally averse to discussion, which is a contradiction in terms for scientists. Their imperviousness will ultimately boomerang on the flu community that has shown an esprit de corps typical of a medieval guild. Ultimately, society will have the last word. My fear is that before the issue is settled there will be an accident or an incident resulting in a terrible backlash on biomedicine. Thereafter, the new dynamics will harsher. I do not even want to contemplate the nightmare of a man-made pandemic.

#### **REFERENCES**


<sup>3</sup>http://www.interacademies.net/10878/13912.aspx


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 08 May 2014; paper pending published: 09 June 2014; accepted: 25 June 2014; published online: 16 July 2014.*

*Citation: Wain-Hobson S (2014) The irrationality of GOF avian influenza virus research. Front. Public Health 2:77. doi: 10.3389/fpubh.2014.00077*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Wain-Hobson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Restricted science

## **Sonia Ben Ouagrham-Gormley \* and Shannon R. Fye**

Biodefense Program, School of Policy, Government and International Affairs, George Mason University, Fairfax, VA, USA \*Correspondence: sbenouag@gmu.edu

**Edited and reviewed by:**

Jonathan E. Suk, European Centre for Disease Prevention and Control, Sweden

**Keywords: H5N1, NSABB, regulating dual-use research, Venter Institute, M. mycoides, bioweapon**

## **INTRODUCTION**

In 2004, the National Science Advisory Board for Biosecurity (NSABB) was created as an independent federal advisory body. Its role was to advise the U.S. government on strategies to prevent the misuse of dual-use research. Since its inception, the NSABB has ruled on two cases: the 1918 flu-virus synthesis conducted by government scientists in 2005 and the H5N1 experiment conducted in 2011 by two separate university teams in the Netherlands and the United States. While in the first case, without much public debate, the NSABB quickly decided to support publication of the experiment's findings, in the second case, it initially requested a halt on publication and the removal of methodological details from the proposed articles for fear that they could be used by malevolent actors to create a pandemic among humans. The decision was reversed 6 months later, but it sparked a worldwide firestorm, engaging the scientific and security communities in a heated debate about whether the dissemination of scientific data should be regulated, and what types of research should be conducted. Yet, the key question that triggered the overall controversy remains largely ignored: under what conditions could the H5N1 experiment be reproduced, if at all, by malevolent actors using only published data?

The lack of attention to the issue of reproducibility stems from a widespread belief that science is inherently reproducible and published data are the primary tool allowing such replication. Empirical evidence suggests otherwise. Analysis of recent dual-use research projects and past bioweapons programs shows that reproducibility of past work faces stiff challenges, especially when using written protocols alone. Translating a scientific idea into a product that functions reliably is a challenge that is routinely encountered in the pharmaceutical industry, as well as in past bioweapons programs. In this article, we start by emphasizing the challenges associated with reproducing scientific experiments and their application to specific purposes based on empirical research conducted by the authors. We then suggest criteria to weigh security risks against the health benefits of dualuse research for the purpose of producing more accurate threat assessments, without imposing unnecessary restrictions on the diffusion of knowledge.

## **SOURCES OF REPRODUCIBILITY CHALLENGES IN SCIENCE**

While the H5N1 controversy was raging, the National Institutes of Health (NIH) revealed that much of its past funded research could not be reproduced. In 2012, for example, the drug company Amgen reported that it failed to reproduce 89% of the findings from 53 major cancer-related papers (1). The previous year, the pharmaceutical company Bayer in Germany indicated that it could not validate the results of two-thirds of its own preclinical studies (1). Interestingly, no connections were made between these revelations and the H5N1 experiment, also funded by the NIH.

Empirical research shows that some experiments are extremely difficult to replicate, due to the contingencies associated with experimental work and the nature of knowledge. First, replication of past work using published documents is problematic because scientific articles rarely provide a detailed account of all stages of an experiment and their associated contingencies. The methods section of scientific papers is usually brief and provides only an overview of the experimental methods to show that a concept has been implemented; it is not intended to be a step-by-step protocol (2). Second, scientific articles rarely delve into the problems that researchers encountered during the experiment nor do they explain how long it took to resolve such problems. For example, the article describing the 2010 creation of a self-replicating *Mycoplasma mycoides* cell by researchers at the J. Craig Venter Institute (JCVI) includes a two-sentence statement indicating that the team faced challenges with transplantation, which were eventually overcome (3). However, interviews with JCVI scientists reveal that transplantation attempts routinely failed for 2 years, leading the scientist responsible for transplantation to consider abandoning the project. As her supervisor explains:

After two years of just seven days a week [of continuous work], she came into my office saying she wanted to work on a new project; she couldn't do this anymore . . .. We tried lots and lots of different approaches. And we had suspicions of something we thought might work . . . but these were hard experiments to do with a lot of reagent prep for every experiment . . .. Everything you could possibly think of that might allow you to move a really big piece of DNA into a cell [we tried] (4).

Publications often play down the long and painstaking process of systematic problem solving that is often required to resolve difficulties involved in experimental work, leaving the false impression that problems can be readily overcome.

Experimental work also sometimes requires the development of new techniques and protocols that cannot easily be used for other purposes or by other individuals. In the *M. mycoides* transplantation case, a new protocol had to be designed for the experiment, and was published in 2007. Yet, 6 years later, the researchers were not able to use this protocol for

work with another organism (4). Additionally, the researchers worked with large pieces of DNA that break easily during pipetting, introducing an additional hurdle to replicating the experiment. To prevent damage, the team emphasized the importance of pipetting "gently" and using pipette tips with wide openings through which large pieces of DNA could pass unobstructed (5). Although pipetting is a common technique, not all scientists were able to pipette the *M. mycoides* DNA gently enough to keep it intact. As one researcher explains:

Our genome transplanters are really good at this [keeping supercoiled DNA intact]. . .I sat in the same hood . . .with Carole [Lartigue – the expert] and we used the same reagents. . . the only thing different was each of us had our own pipettes and plates, and I did a transplant in parallel with her. . .she got . . . 2,000 colonies [successful transplants] and I got 20. I thought I was doing exactly what she was doing in pipetting slowly. [But] doing these tricks is still very much a magic hand sort of thing (4).

This highlights a problem well known among practicing scientists but generally ignored in evaluations of the potential reproducibility of dual-use experiments: the importance of expertise acquired through years of practice in the laboratory. Much of this expertise involves tacit skills not easily translated into words, such as the muscle memory that allows a researcher to know what constitutes "gentle" pipetting, or acquired and replicated by others, even when a technique is demonstrated in person or an experiment is done in cooperation with the technique's designer (6–8). Moreover, laboratory disciplines and routines often contribute to the development of laboratoryspecific skills that cannot be standardized or transferred to a new location. The University of New York-Stony Brook virologists who synthesized poliovirus in 2002 emphasized the importance of maintaining"sameness" in their laboratory routines, materials, and technicians to ensure successful results. Tellingly, a post-doctoral fellow who spent 6 years in the New York laboratory could not replicate his

work in his home laboratory in Belgium (9).

Thus, the tacit, personal, and local nature of knowledge constitutes a strong barrier to reproducibility. Because knowhow does not easily translate into words, its importance for experimental success is frequently ignored in threat assessments.

#### **APPLICATION TO NEFARIOUS OBJECTIVES**

The NSABB's initial decision to edit the H5N1-related article before its publication was followed by the Dutch government's decision to impose export-control restrictions on the Dutch team's article. Dutch authorities claimed that the research fell under European Council Regulation EC 428/2009, which attempts to prevent the spread of nuclear, chemical, and biological weapons by requiring an export license before publication (10). These moves are based on the assumption that innovations achieved in the laboratory can be easily fashioned into a harmful agent or a bioweapon. Yet, past bioweapons work shows that transforming a scientific concept developed in the laboratory into a product that has a specific, applied purpose, and functions reliably and effectively can take several decades and require a variety of expertise. Specifically, the passage from laboratory concept to specific application faces the challenge of scaling-up fragile microorganisms for large-scale production and developing a delivery mechanism that will protect the agents from environmental degradation when released as a weapon. For example, within the Soviet bioweapons program, the development of an antibiotic-resistant strain of the bacterium that causes plague took 20 years to achieve and involved teams at three institutes. Scaling-up anthrax and smallpox weapons took Soviet researchers about 5 years to achieve and required the involvement of large teams of scientists, including the designers of the original strains. And within the U.S. bioweapons program, scientists discovered that the botulinum toxin weapon they had produced eventually lost some of its toxicity upon aerosol release. These examples demonstrate that laboratory successes do not necessarily lead to successful application to a specific purpose. Instead, specialized skills honed over years of practice in production and weaponization work are critical to success (11).

## **NEW ANALYTICAL FRAMEWORK**

Seen against this background, fears that the H5N1-related articles might support replication by malevolent actors seem exaggerated. They ignore the fact that science is a cumulative process where knowledge is acquired and built through many years of personal and collective experimentation. Therefore, it is neither easily acquired nor easily transferred, and even less so by means of published articles. More importantly, these fears also indicate that the NSABB's initial decision to edit the H5N1 article before its publication was not rooted in a risk/benefit analysis that considered the determinants of success in scientific work. Indeed, even though the Board interviewed the lead authors and a variety of influenza experts, it did not interview the scientists and technicians who actually conducted the laboratory work (12). In fact, important details about the experiment's difficulty were revealed after the Board issued its recommendation, and only as a result of the controversy, not as a result of the Board's inquiry.

Therefore, any future review of dual-use research should be based on a careful analysis of the tacit, personal, and laboratoryspecific skills required to perform scientific experiments. This implies that NSABB reviewers conduct face-to-face interviews with the scientists and technicians who executed the laboratory work to identify the hidden contingencies associated with key stages of an experiment, including the development of laboratory- or agentspecific techniques or protocols that may not transfer easily to a new location. A laboratory visit may also reveal hidden laboratory idiosyncrasies that contribute to experimental success and may prevent replication elsewhere. In order to improve the NSABB's ability to assess the ease of replication by terrorists or states, its reviewers should also include an expert who has hands-on experience working with the microorganism under consideration. In the H5N1 case, NSABB members had access to outside influenza experts, but their lack of experience working with the influenza virus itself, notwithstanding their expertise in other areas, did not allow some of them

to appreciate the importance of experimental details that could have impacted the ultimate threat assessment<sup>1</sup> . Without a major change in the NSABB's approach, future restrictions might result in two equally negative consequences. First, suspicions among foreign entities that restrictions on scientific work are hiding U.S. government bioweapons work might increase. Second, scientists may avoid U.S.-funded research for fear that the government might block their work from being published. To wit, the Dutch scientist who conducted the H5N1 research temporarily blocked by the NSABB recently published a follow-up study in the journal *Cell*. In the Section "Acknowledgment," he stipulated that the work was not funded by the NIH (13).

## **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 10 May 2014; paper pending published: 17 July 2014; accepted: 10 September 2014; published online: 24 September 2014.*

*Citation: Ben Ouagrham-Gormley S and Fye SR (2014) Restricted science. Front. Public Health 2:158. doi: 10.3389/fpubh.2014.00158*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Ben Ouagrham-Gormley and Fye. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

<sup>1</sup>A list of NSABB members can be found on the National Institute of Health website. A review of their biographies indicate that their respective expertise lies in areas different from the flu: http://osp.od.nih.gov/sites/default/files/resources/NSABB%20Voting%20Members%20Roster%20\_June%202014\_WEB\_0.pdf

### **Jo L. Husbands\***

Board on Life Sciences, U.S. National Academy of Sciences, Washington, DC, USA \*Correspondence: jhusband@nas.edu

**Edited by:**

Evangelos Giamarellos-Bourboulis, University of Athens Medical School, Greece

**Reviewed by:**

Efthymia Giannitsioti, University of Athens Medical School, Greece

**Keywords: biosecurity, bioterrorism, dual use, responsible science, non-proliferation**

## **INTRODUCTION**

Efforts to prevent the proliferation of biological weapons or bioterrorism face an increasingly complicated landscape characterized by rapid scientific and technological progress, growing global diffusion of research capacity, and an array of stakeholders with important potential roles. In 2002, the International Committee of the Red Cross (ICRC) introduced the concept of a "web of prevention" to underscore the need for a comprehensive and coordinated strategy that could engage gather the many communities necessary to address the challenges (1). The scientific community, broadly defined to encompass the many fields beyond biology that now make up the life sciences research enterprise, is recognized as essential to the success of any strategy. This is particularly true for addressing what has come to be called"dual use" research, which is undertaken for beneficial purposes but has the potential to be misused to cause deliberate harm (2–5). Programs for "scientific engagement" have expanded in the last decade to reflect this recognition and policy attention.

#### **APPROACHES TO ENGAGEMENT: FRAMING THE ISSUES**

Approaches to engaging scientists in biosecurity issues generally follow one of two approaches. The traditional framing starts with *requirements*, the legal obligations to which scientists are subject, broadly under international treaties as well as specifically under the national laws and regulations that either implement the agreements or are undertaken independently by countries for their own security purposes (6). In cases such as the European Union, scientists may also be subject to a significant regional regulatory framework. It should be no surprise that the natural inclination for those in the security and law enforcement communities, as well as for the diplomats who tend the treaties, is to begin with one's legal obligations, what a scientist "must" do.

An alternative framing that appears to be gaining momentum treats biosecurity within the broader context of the social responsibility of science as another example of the *responsibilities* that scientists are expected to fulfill. Science is not conducted in a social vacuum and scientists are subject to the effects of many broader forces (7). Among them, changing social attitudes clearly affect how science is carried out<sup>1</sup> . What scientists "should" do thus comes from norms of professional behavior as much, or in some cases perhaps more, as from legal requirements (8). It also allows scientific engagement on biosecurity to take advantage of the international attention to issues of research integrity and responsible conduct of science. This growing attention reflects the need for common understandings as the life sciences have become an increasing global enterprise<sup>2</sup> . High-level declarations and statements have underscored the ethical imperative that along with the fundamental principles of freedom in the conduct of science come responsibilities and the need to maintain public trust<sup>3</sup> .

An example of nesting security within the broader framing comes from a project of the InterAcademy Council (IAC) and IAP – The Global Network of Science Academies<sup>4</sup> . In its first phase, an international committee formed by the IAC and IAP produced a short policy report on research integrity (9). The report addresses a broad range of issues, including security. The report notes, for example, that Science and other forms of scholarship have been incredibly productive by seeking knowledge unfettered by tradition, ideology, and external pressure. At the same time, research can have a profound influence on the environment, human health and well-being, economic development, national security, and many other facets of human life. Many areas of science and technology can be used for destructive as well as constructive purposes and researchers have a special responsibility to understand and address issues of "dual use." Research on biological pathogens, for example, poses

<sup>1</sup>A clear example is the development of standards for the treatment of human subjects in experiments, which developed over time, particularly during the twentieth century in response to egregious abuses by researchers. The standards for the treatment of laboratory animals have continued to evolve as well.

<sup>2</sup>For example, the Global Research Council, created in 2012, is a virtual organization comprises national science and engineering funding bodies from about 50 countries that is devoted to promoting high quality research collaborations, including issues of research integrity. More information is available at http://www.globalresearchcouncil.org/. <sup>3</sup>A discussion of these developments may be found in a report from the National Research Council (10).

<sup>4</sup> IAP is a global network of more than 100 of the World's Science Academies, launched in 1993. Its primary goal is to help member academies work together to advise citizens and public officials on the scientific aspects of critical global issues. More information is available at http://www.interacademies.net/. The IAC produces reports on scientific, technological, and health issues related to the great global challenges of our time to provide knowledge and advice to national governments and international organizations. More information is available at http://www.interacademycouncil.net/

both risks and benefits for human health [Ref. (9): p. 15].

The report then concludes that "researchers should bear in mind the possible consequences of their work, including harmful consequences, in planning research projects" [Ref. (9): p. 16], which has clear implications for scientists' roles in addressing dual use issues.

## **CONCLUSION**

The two approaches to framing scientific engagement on biosecurity are not mutually exclusive. Many laws reflect social norms and science engagement programs using a framework of responsible science include discussions of laws and regulations, with the Biological Weapons Convention as the international legal embodiment of a fundamental norm against using disease as a weapon. And much more needs to be done to decide on and develop the appropriate mix of legal, regulatory, and policy measures to address the security challenges posed by globalizing science. The issue is where to begin and what works best to reach one group of essential stakeholders. Responsible conduct offers a foundation on which one can build and complements more detailed attention to security issues

and legal requirements needed by those in certain areas of research. It can also contribute to making scientists part of the solution to biosecurity challenges rather than part of the problem.

## **REFERENCES**


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 12 May 2014; accepted: 19 July 2014; published online: 11 August 2014.*

*Citation: Husbands JL (2014) Responsibilities or requirements: framing dual use issues for scientific engagement. Front. Public Health 2:107. doi: 10.3389/fpubh.2014.00107*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Husbands. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## The consequences of a lab escape of a potential pandemic pathogen

## **Lynn C. Klotz <sup>1</sup>\* and Edward J. Sylvester <sup>2</sup>**

<sup>1</sup> The Center for Arms Control and Non-Proliferation, Washington, DC, USA

<sup>2</sup> Science and Medical Journalism, Walter Cronkite School of Journalism and Mass Communication, Arizona State University, Phoenix, AZ, USA

\*Correspondence: lynnklotz@live.com

#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Simon Wain-Hobson, Institut Pasteur, France

**Keywords: potential pandemic pathogen, lab-acquired infection, H5N1, H7N9, gain of function**

In letters to the journals Science and Nature (1, 2), 22 virologists notified the research community of their interest in expanding research to develop strains of the already deadly H7N9 Asian influenza virus that would be transmissible via aerosols among mammals, thus creating potential pandemic pathogens. PPPs are defined as pathogens that are potentially highly contagious, potentially highly deadly, and not currently present in the human population. Mammalian contagious avian flu, the 1918 pandemic flu, and SARS are examples. The letter writers cite their scientific reasons for the need for such research, much the same reasons as given by those working on similar projects for the H5N1 avian flu virus (3, 4). This new proposed research signals wider interest in making dangerous influenza viruses (5, 6) contagious in mammals via respiratory aerosols. At present, there are no international regulations or guidelines in place to decide whether such a research project should proceed.

Now is the time to address the next critical question: what is the likelihood that one of these viruses will escape from a lab and seed the very pandemic the researchers claim they are trying to prevent? As we shall estimate, that probability could be as high as 27%, a risk too dangerous to live with.

First, from the calculations in two indepth pandemic risk analyses (7–9), there is a substantial probability that a pandemic with over a 100-million fatalities could be seeded from an undetected lab-acquired infection (LAI), if a single infected lab worker spreads infection as he moves about in the community. From the Klotz (2014) analysis,there is about a 1–30% probability, depending on assumptions, that, once

infected, the lab worker will seed a pandemic. This large probability spread arises from varying the average number of people infected by an infected person between 1.4 and 3.0 (*R*0, in standard epidemiology notation), varying the details of commutes to and from work on public transportation, and whether infected acquaintances are quarantined before spreading infection. The Merler (2013) study, based on a computer-generated population grid of size and varying density of the Netherlands, supports our concern over a lab escape not being detected until it is too late: "there is a non-negligible probability (5–15%), strongly dependent on reproduction number and probability of developing clinical symptoms, that the escape event is not detected at all."

Different methodologies were used in the Klotz (2014) and Merler (2013) risk analyses. Additional analyses are needed using other methodologies, such as the mathematical model employed for SARS (10), which hopefully will lead to some consensus on risk. The Klotz and Merler studies, however, are the first to raise these concerns and point to valid issues about the potential risks from a single LAI.

Given such a dire predicted outcome by the existing studies, the critical question is: what is the probability that a worker acquires an undetected infection in the lab in the first place? To answer this question, we reproduce here one part of the Klotz (2014) analysis: the probability of an escape through an LAI from at least one of the many labs expected to be involved in this research enterprise.

A 2013 Centers for Disease Control report is a significant source of recent data on LAIs (11). The report documents four undetected or unreported LAIs in registered US Select Agent, high-containment BSL-3 labs between 2004 and 2010. An undetected or unreported LAI implies an escape when the infected person leaves the lab. The report identifies an average of 292 registered Select Agent BSL-2, BSL-3, and BSL-4 labs operating over those 7 years, for a total of 292 × 7 = 2,044 lab years. Unfortunately, the study does not break down numbers into BSL-2, BSL-3, and BSL-4 labs or lab years.

Thus, the probability of escape for a single year, *p*1, can only be calculated as 4 LAIs/2,044 lab years = 0.002 or 0.2% per lab per year. This is clearly an underestimate since BSL-2 and BSL-4 labs contribute to the denominator. (The denominator used here, 2,004, equals the number of BSL-2 plus number of BSL-3 plus number of BSL-4 labs. But the denominator in our calculation should be just the number of BSL-3 labs, so the denominator is overestimated and the percent escape is then underestimated. Although requested, the CDC has not supplied us with the number of BSL-3 labs for us to do the exact calculation.) This basic probability is consistent with that for SARS escapes in Asia through LAIs (12) and with all known escapes from BSL-4 labs in the Soviet Union from LAIs and Great Britain from a mechanical failure (13).

To illustrate potential risk, the probability of no escape from a single lab in a single year is (1 − *p*1), so

$$
\rho\_{\rm no} = \left(1 - p\_1\right)^{N \times Y} \qquad (1)
$$

is the probability of no escape from *N* labs in *Y* years. And

$$p\_{\text{at least one}} = 1 - \left(1 - p\_1\right)^{N \times Y} \qquad \text{(2)}$$

is the probability of at least one escape from *N* labs in *Y* years.

Given the Science and Nature articles listed above (1, 2), it is reasonable to assume that at least 10 labs will undertake this research and that this work would continue for 10 years, so

$$p\_{\text{at leastone}} = 1 - (1 - 0.002)^{10 \times 10} = 0.18 \,\text{g}$$

or an 18% likelihood of at least one escape from at least one lab for the whole research enterprise, almost 100-times greater than the likelihood for a single lab in a single year.

We noted above that the probability *p*<sup>1</sup> = 0.2% is conservative, estimated from the CDC data alone. The first Department of Homeland Security risk assessment for the planned National Bio- and Agro-Defense Facility in Manhattan, Kansas estimated a significantly higher escape risk, over 70% likelihood for the 50-year life of the facility (14), which works out to be a basic probability of escape, *p*<sup>1</sup> = 2.4% per year. The National Research Council (14) overseeing the risk assessment remarked "The . . . estimates indicate that the probability of an infection resulting from a laboratory release of FMDv from the NBAF in Manhattan, Kansas approaches 70% over 50 years (see Figure 3-1) with an economic impact of \$9–50 billion. The committee finds that the risks and costs could well be significantly higher than that. . ." While the DHS subsequently lowered the escape risk to 0.11% for the 50-year lifetime (14), the NRC committee (14) was highly critical of the new calculations:"The committee finds that the extremely low probabilities of release are based on overly optimistic and unsupported estimates of human error rates, underestimates of infectious material available for release, and inappropriate treatment of dependencies, uncertainties, and sensitivities in calculating release probabilities." We have more trust in the NRC committee conclusions, as they have no skin in the game.

With this higher number, which we take as a worst-case scenario, the likelihood of at least one escape from 10 labs in 10 years becomes 91%, almost a certainty. It follows that, if the likelihood of one LAI leading to a pandemic is 30% in the worst-case scenario, the likelihood of an LAI-caused pandemic resulting from this whole research enterprise could be as high as 30 × 91% = 27%, a likelihood that is too dangerous to live with, as we noted. While this represents a worst-case scenario, it is not improbable.

Recent self-reported mistakes at the CDC (15), involving a particularly deadly strain of anthrax removed from BSL-3 containment and H5N1 Asian bird flu released from the CDC laboratories altogether, lend support to our concern that the probability of escape may be much greater than the 0.2% per lab per year from just LAIs. The CDC report spawned a congressional inquiry (16) and led to dozens of newspaper articles with concerns about lack of safety in high-containment laboratories.

Our concern is shared by many virologists and epidemiologists. A recent letter to the President of the European Commission (17) co-signed by 56 scientists from more than a dozen countries warned, "The probabilities of a lab accident that leads to a global spread of an escaped mutated virus are small but finite, while the impact of global spread could be catastrophic." The European Centre for Disease Prevention and Control (18) weighed-in with its concerns as well, as did the Cambridge Working Group (19). It must be noted that some of the signers of the European Commission letter and the Cambridge Working Group's consensus statement are the same.

The risk of a man-made pandemic from a lab escape is not hypothetical. Lab escapes of high-consequence pathogens resulting in transmission beyond lab personnel have occurred (20, 21). The historical record reveals lab-originated outbreaks and deaths due to the causative agents of the 1977 pandemic flu, smallpox escapes in Great Britain, Venezuelan equine encephalitis in 1995, SARS outbreaks after the SARS epidemic, and foot and mouth disease in the UK in 2007. Ironically, these labs were working with pathogens to prevent the very outbreaks that they ultimately caused.

Do benefits outweigh risks? Those who support PPP experiments either believe the probability of PPP escape is infinitesimal or the benefits in preventing a pandemic are great enough to justify the risk. In making decisions for what lines of research will lead to new knowledge, experts must rely on intuition honed by years of research in a particular field. In the case of this PPP research, in our opinion it would take extraordinary benefits and significant reduction of risk via extraordinary biosafety measures to correct such a massive overbalance of highly uncertain benefits to too-likely risks (Wain-Hobson, 2013).

Whatever number we are gambling with, it is clearly far too high a risk to human lives. This Asian bird flu virus research to develop strains transmissible via aerosols among mammals, and perhaps some other PPP research as well, should for the present be banned. We must emphasize that we have been considering only a very small subset of pathogen research. Most pathogen research should proceed unimpeded by unnecessary regulations.

Special precautions in BSL-4 laboratories for work with PPPs should be adopted (22). These would include:


We label BSL-4 laboratories with the special precautions, BSL-4+. While PPP experiments would be carried out primarily under BSL-4+ containment, BSL-3 containment with the special precautions might suffice for some work.

Given the global threat, the international community should insist on discussions leading to an international agreement that would require the strictest oversight to conduct this particular research anywhere. To place responsibility with the international community where it belongs and to provide maximum transparency, policy makers should require that international inspectors have access to facilities at any time on short notice.

As it stands, there is no proactive oversight nor regulations for this PPP research, so any and all of the world's nations can carry out this dangerous work without regard to consequences. But consequences would be shared by all of us. In the meantime, insurance companies who routinely provide insurance for biological research should consider excluding such risky research from coverage.

#### **REFERENCES**


2014/01/Human-Fatality-Economic-Burden-of-Gain-of-Function-Flu-Research-v1-5-14.pdf


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 May 2014; accepted: 28 July 2014; published online: 11 August 2014.*

*Citation: Klotz LC and Sylvester EJ (2014) The consequences of a lab escape of a potential pandemic pathogen. Front. Public Health 2:116. doi: 10.3389/fpubh.2014.00116*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Klotz and Sylvester. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Benefits of a European project on diagnostics of highly pathogenic agents and assessment of potential "dual use" issues

#### **Roland Grunow <sup>1</sup>\*, G. Ippolito<sup>2</sup> , D. Jacob<sup>1</sup> , U. Sauer <sup>1</sup> , A. Rohleder <sup>1</sup> , A. Di Caro<sup>2</sup> and R. Iacovino<sup>2</sup>**

<sup>1</sup> Robert Koch Institute, Berlin, Germany

<sup>2</sup> Spallanzani National Institute for Infectious Diseases, Rome, Italy

#### **Edited by:**

Evangelos Giamarellos-Bourboulis, University of Athens, Greece

#### **Reviewed by:**

Magdalena Chirila, Iuliu Ha¸tieganu University of Medicine and Pharmacy, Romania George Dimopoulos, University

Hospital Attikon, Greece **\*Correspondence:**

Roland Grunow, Robert Koch-Institut/ ZBS 2, Nordufer 20, 13353 Berlin, Germany e-mail: grunowr@rki.de

And on behalf of the QUANDHIP partners: http://www.quandhip.info/ Quandhip/EN/Home/Homepage\_node. html

Quality assurance exercises and networking on the detection of highly infectious pathogens (QUANDHIP) is a joint action initiative set up in 2011 that has successfully unified the primary objectives of the European Network on Highly Pathogenic Bacteria (ENHPB) and of P4-laboratories (ENP4-Lab) both of which aimed to improve the efficiency, effectiveness, and response capabilities of laboratories directed at protecting the health of European citizens against high consequence bacteria and viruses of significant public health concern. Both networks have established a common collaborative consortium of 37 nationally and internationally recognized institutions with laboratory facilities from 22 European countries. The specific objectives and achievements include the initiation and establishment of a recognized and acceptable quality assurance scheme, including practical external quality assurance exercises, comprising living agents, that aims to improve laboratory performance, accuracy, and detection capabilities in support of patient management and public health responses; recognized training schemes for diagnostics and handling of highly pathogenic agents; international repositories comprising highly pathogenic bacteria and viruses for the development of standardized reference material; a standardized and transparent Biosafety and Biosecurity strategy protecting healthcare personnel and the community in dealing with high consequence pathogens; the design and organization of response capabilities dealing with cross-border events with highly infectious pathogens including the consideration of diagnostic capabilities of individual European laboratories.The project tackled several sensitive issues regarding Biosafety, Biosecurity and "dual use" concerns. The article will give an overview of the project outcomes and discuss the assessment of potential "dual use" issues.

**Keywords: EQAE in diagnostic, anthrax, tularemia, plague, melioidosis, glanders, brucellosis, dual use research of concern**

### **INTRODUCTION**

Internationally accepted biological infectious agents are divided into four risk groups based on their virulence, potential of public health threat, and availability of adequate treatment. Risk group 1 poses the lowest and risk group 4 the highest level of threat. A complex risk assessment for handling these pathogens leads to the definition of corresponding biosafety levels 1–4 (BSL1–4 or P 1–4) including technical, organizational, and personal protective measures. Highly pathogenic bacteria of risk group 3, e.g. *Bacillus anthracis*, *Yersinia pestis*, or *Francisella tularensis*, and risk group 4 viruses, e.g. haemorrhagic fever viruses, could cause severe diseases in humans and animals and are suspected to be used in bioterrorism attacks (1–7). Although there are various endemic areas in Europe for some of these zoonotic agents causing outbreaks, many questions about the epidemiology and ecology of these bacteria still remain open. In the context with other highly frequent diseases, the impact of infections caused by these bacteria and viruses on public health in Europe was so far rather limited. This also seems to be one of the reasons why the commercialization

of diagnostic tests for these agents has not raised large interest and the microbiological laboratories are mostly forced to rely on their in-house assays. However, reliable diagnostics should be at hand for eventual natural outbreaks, for unpredictable imported cases and for the deliberate release of these agents, which poses an ongoing threat to the human population. Because of the different impacts and unpredictabilities of these agents to human health in different countries, networking of interested and/or appointed laboratories providing diagnostics in this field should be a logical consequence to exchange experiences, knowledge, and material supporting the laboratory response to outbreaks of these agents in single countries or cross-border events.

Diagnostic laboratories need to participate in quality assurance exercises to assess their diagnostic approaches and to define measures for improvement and maintenance of their diagnostic capacities and capabilities. One of the reasons for the establishment of the EU Joint Action (JA) Quality Assurance Exercises and Networking on the Detection of Highly Infectious Pathogens (QUANDHIP) was the fact that capacities and possibilities to conduct proficiency tests in this field are very limited in many European countries (8). The JA is running from August 2011 to July 2014 and aims to link and consolidate the objectives of two existing networks dealing with highly infectious bacteria and viruses: The bacterial network emerged from the EU funded project EQADeBa (EAHC n°2007 204), coordinated by the Robert Koch-Institut (RKI), Germany, and served as a basis for the European Network on Highly Pathogenic Bacteria (ENHPB). The other one is the European Network of P4-laboratories (ENP4- Lab) project (EAHC n°2006 208), coordinated by L. Spallanzani National Institute for Infectious Diseases (INMI), Italy. The primary objective of the JA is to stabilize both network activities, which link 37 highly specialized and advanced partner laboratories from 22 European countries (**Figure 1**).

The overall goal of the project was the improvement of the detection and diagnosis of highly pathogenic bacterial and viral agents as well as to provide and further develop the laboratory support to the EU in the management of biological cross-border events. The range of target agents to be diagnosed is given in **Table 1**.

The JA developed a supportive European infrastructure and strategy for external quality assurance exercises (EQAEs) in order to establish a universal exchange of best diagnostic strategies. The EQAEs included shipment of infectious reference material, bacterial antibiotic susceptibility testing, the development of international repositories of reference material, shipment of living

#### **Table 1 | Diagnostic target agents in QUANDHIP**.


bacterial cultures, training, and Biosafety and Biosecurity reviewing of current practices. A special work package (WP) was directed to describe the capacities and capabilities of European laboratories, which are responsible for the analysis of highly pathogenic infectious agents and to provide recommendations on the activation mechanisms and the support offered by the QUANDHIP partners and the network in case of biological cross-border events. Most of these activities and the related data required an assessment of bio-risks in terms of Biosafety and Biosecurity including dual-use research of concern (DURC), which will be discussed in this article together with the most important outcomes of the project.

The general discussion on DURC was renewed when two studies on transmissibility of the avian influenza virus H5N1 got published (9). The dual-use problem concerning research is described on the coordinator's (RKI) website as follows:"Research and development in the life sciences have crucially contributed to today's progress and improvement of living conditions. At the same time, findings in the life sciences often run the risk of being misused to the detriment of society and environment. This "double applicability" of scientific findings is described as the "dual use dilemma." The potential for misuse of scientific findings is especially obvious for research on pathogenic microorganisms and toxins: on the one hand, research results regarding transmissibility, pathogenesis, and genomics of pathogenic biological agents are indispensable to prevent the agents' spread and proliferation and to enable or improve the treatment of infection and exposure to toxins. On the other hand, these results can also potentially be misused to cause harm to humans, animals, or plants (9, 10). It should be considered that not only biological agents as tools of research but also information on the outcomes of research activities could be categorized as dualuse dilemma. Appropriate Biosafety and Biosecurity measures and conventions can contribute to prevent the harmful side of biological research. Our project contains three horizontal (coordination, dissemination, evaluation) and five core (EQAE, repository, training, Biosafety and Biosecurity, support in cross-border biological events) WPs, which will be illustrated and discussed in terms of Biosafety/Biosecurity and DURC issues.

## **MATERIAL AND METHODS**

#### **WP 1–3 PROJECT COORDINATION, EVALUATION, AND DISSEMINATION**

Before submitting the application for the JA, all potential partners were selected by an official "Letter of Intent" stating and confirming that these public or governmental institutes have been assigned the task to perform diagnostics on highly pathogenic agents under appropriate Biosafety and Biosecurity conditions. This was approved by the project controlling "Consumers, Health and Food Executive Agency" (CHAFEA). The JA was managed by the coordinator RKI and the co-coordinator INMI. A Steering Committee (SC), consisting of selected partners, had the function, in addition to the coordinators, to check the correct and timely implementation of the work program. An external Scientific Advisory Board (SAB), comprising representatives of the European Centre for Disease Prevention and Control (ECDC), the World Health Organization (WHO), and the European Commission, was set up for the evaluation of the project activities and gave advice for the optimization of contents and the course of

the project in general. Both coordinators kept close contact so that all decisions were agreed between the coordinators beforehand, and, where necessary, including the SC, SAB, and individual partners. The JA comprised common and separate actions for the bacterial and viral network, the Network on Highly Infectious Bacteria (abbreviated here as NIB), coordinated by the RKI, and the Network on Highly Infectious Viruses/P4-Laboratories (abbreviated here as NIV), coordinated by INMI. Altogether, three joint meetings of both bacterial and viral networks, combined with an SAB meeting, and three separate meetings of each of the networks were organized. The meetings were used to share scientific and administrative information. The coordinators are running a public website and an internal workspace on a secure official server to share sensitive information. The project was presented on several scientific conferences/meetings, and a number of publications were developed. Besides the continuous internal controlling, the external evaluation of the project was performed by the CHAFEA, also including an external review of the interim report, and by the SAB.

#### **WP 4 EXTERNAL QUALITY ASSURANCE EXERCISES Administrative preparation**

The providers of EQAEs, RKI and PUM, prepared the samples and took care of quality assurance and shipment. The partners carried out the analysis of the samples due to the given parameters. The shipment of samples was realized by the selected shipping agency fulfilling all regulations for transportation of dangerous goods and national import/export regulations (10–16). The EQAEs were conducted separately for the highly pathogenic bacteria, including living and inactivated bacterial samples of risk group 3, and for the risk group 4 viruses, only including non-infectious nucleic acid from risk group 4 viruses so far. According to the Consortium Agreement a Material Transfer Agreement (MTA) was signed for each EQAE by provider and recipient.

Beforehand, all partners were asked to provide and confirm officially that they are entitled to handle risk group 3 and/or 4 agents, respectively, and carry out the work under appropriate Biosafety and Biosecurity conditions using a questionnaire developed in the framework of the previous projects EQADeBa and ENP4 and further optimized during this JA (WP 7). Partners who are not yet or currently not able to handle risk group 3 bacteria agreed to receive only sets of inactivated samples. The prepared living and inactivated samples were suitable for the application of different methods like molecular genetic methods, immunological methods, biochemical methods, or microbiological methods. In case of risk group 4 viruses, nucleic acid samples were only delivered to participants who practically have the possibility to further analyze positive samples under BSL4 conditions, having direct access to those laboratories, or having established collaborations and agreements with such laboratories. The sample design, preparation, and quality control of the EQAEs are described under the Supplementary material.

The data analysis was performed by the providers of the EQAEs and recommendations given for further improvement. QuoData was chosen as subcontractor and developed a new software for data entry and analyses, which has been used for the evaluation of the second and third NIB-EQAE (17).

For EQAEs on viruses, the procedure for the preparation of samples sent for these exercises was (1) amplification of the virus in cell culture; (2) inactivation by gamma irradiation (implying fragmentation of viral nucleic acid); (3) verification of inactivation procedure, no addition of PCR inhibitors; (4) testing of stability of the sample; (5) serial dilution and testing via (RT)-PCR and q(RT)-PCR. In some cases, human sera, spiked with the virus and inactivated as above, were used as testing material.

#### **WP 5 REPOSITORY**

To establish an international bacterial RG3 repository, a number of strains were provided by the participants to the RKI, who has been setting up and is keeping this repository. All strains were confirmed for their identity and phenotypic and molecular characteristics. DNA and inactivated bacteria were developed as reference material. According to the procedure for usage of the repository, which has been agreed by all participants beforehand, a limited set of material was delivered to partners on request. The repository was also used for the development of EQAE samples.

The BSL4-laboratories developed a list of key reference viral strains located at individual laboratories. The exchange of material between partners was agreed and regulated. For security and administrative reasons the exchange of "living"risk group 4 viruses was reduced to a minimum.

#### **WP 6 TRAINING**

From the very first beginning of the project, several training programs, usually running for one week, were designed by the participating laboratories and listed and made accessible to all partners by the coordinator first by e-mail, later via the internal workspace. Partners could select and prioritize training programs of up to 10 days they considered most beneficial. For security and administrative reasons, usually only staff listed in the grant agreement was allowed to attend the training.

For risk group 4 viruses, rather theoretical courses were organized, covering various aspects of BSL4 work (basic knowledge, biosafety, management issues, competency, and scenario exercises).

#### **WP 7 BIOSAFETY, BIOSECURITY**

The infrastructure checklists for Biosafety and Biosecurity composed of the two existing networks (EQADeBa/ENHPB and ENP4) dealing with highly dangerous bacteria and viruses, respectively, have been compared, evaluated, reviewed, and exchanged.

In addition, the checklists and recommendations produced by other already completed European programs [e.g. Biosafety-Europe, European Training in Infectious Disease Emergencies – ETIDE (18), European Research Infrastructure on Highly Pathogenic Agents – ERINHA (19)] have been assimilated for review and impact on the outputs of this WP. This work has been supported by an external internationally recognized specialist for Biosafety and Biosecurity.

#### **WP 8 SUPPORT TO CROSS-BORDER EVENTS**

In order to comply with the objectives, the Project Coordinators created an expert working group, composed of both Coordinators, supported by external expert consultants and by staff from European Health Authorities involved in the SAB of the project. The aim

was to develop an operational document containing recommendations on laboratory management of biological events and defining the role and activation procedures for the QUANDHIP network in case of international biological cross-border events. The tasks were agreed with the representatives of the EC and worked out with the support indicated above. Possibly occurring real events will be used for the evaluation of the developed document.

## **RESULTS UNDER SPECIAL CONSIDERATION OF SECURITY ISSUES**

#### **MANAGING ISSUES RELATED TO THE SHARING OF SENSITIVE INFORMATION (REGARDING WP 1-3)**

The QUANDHIP JA activities require the sharing of sensitive information between the consortium partners. This is made possible by strong joint project coordination (RKI and INMI) and a culture of trust that has been built over many years through various European networking projects in the area of high containment laboratory work. In addition to a core coordination, the QUAND-HIP partnership relies on the inputs from a SC, consisting of the main activity leaders and an external SAB. To manage the collaborations within the consortium, collaborating partners, the SC, and the SAB, a number of specific agreements have been developed and signed by all parties. These agreements were approved by the legal departments of the coordinator and all participants. The agreements, which were signed by all partners, contained beyond administrative and financial issues several regulations also preventing any misuse of material, data and information like


The network websites and electronic mail transfer have ensured regular communication between participants (8). The internal website was provided by a secure German official provider.

For external communication beyond the consortium, it was important to establish a communication channel and coordination with other relevant European networks like the ECDC funded project European Network for Diagnostics of "Imported" Viral Diseases (ENIVD) (20) and the ERINHA to improve collaboration and exchange of relevant information and to avoid duplication in any international activities. To achieve this, bilateral meetings were arranged and letters of collaboration developed in order to clarify how information would be exchanged in a responsible manner. In addition, a dissemination plan was developed and all QUANDHIP JA deliverables were assessed in order to determine their confidentiality level. Except for the detailed

interim and final reports, which were restricted only to relevant EU stakeholders (defined by CHAFEA), most of the deliverables were supposed to be made available to the scientific community. As a matter of course, the detailed results of the EQAEs provided by the individual partners were treated confidentially by the evaluators of the EQAEs and were published only in an anonymized form. The primary target groups to be considered in the dissemination strategy were laboratory workers of the associated and collaborating partners dealing with the diagnostics of high threat pathogens, biosafety experts, first responders, clinical staff, and security forces. In the framework of this project, various documents including recommendations for diagnostics, Biosafety and Biosecurity, management of biological events, and risk assessment from the laboratory perspective were or will be developed until the end of the project. So far, 40 scientific presentations and publications were used for dissemination of information on the QUANDHIP JA.

The evaluation of the project was done by CHAFEA and reliable external specialists working together in the SAB who approved by their signatures the confidentiality rules and procedures of the Commission. Altogether, the internal and external management of information sharing benefits from a trusted network of experts, collaborating external partners, and transparency in terms of the documentation of the "code of practice" when information is produced through the work of the QUANDHIP JA. Ultimately, the management of dual-use risks associated with information sharing on these high containment pathogens is ensured through a strong project coordination team and close collaboration with the funding authorities.

### **DUAL-USE ISSUES CONCERNED WITH "EXTERNAL QUALITY ASSURANCE EXERCISES" (REGARDING WP 4)**

External Quality Assurances are part of a methodology aimed at assessing the quality of laboratory diagnostics and strategy at the participating laboratories. EQAEs can be helpful to identify 'best practices' for certain diagnostic approaches which can be exchanged between participants for improvement of own procedures. It also can be a reflection of the overall quality management systems of the laboratory. Within the QUANDHIP JA framework, EQAE rounds involve the preparation and shipment of a panel of coded samples (living or inactivated) to participating laboratories. The laboratories test the samples and then return the results to the central coordination place of the EQAE round. The coordination processes the results and provides feedback to all participants so that they can only identify their own proficiency scores but also in a way that they can compare it with the performance of all other participants in an anonymized way. This information enables the laboratories, which did not perform so well or have certain deficiencies, to perform internal checks of their diagnostic assays and quality systems and make adaptations to improve their performance in the next testing round. Also there is the opportunity for exchange of information, protocols, and event training of the partners to improve their proficiency. As this exercise involves the shipment of materials with dual-use potential and the reporting of sensitive information that could expose vulnerabilities in the capability to detect high containment pathogens, the QUANDHIP JA has developed strategies to mitigate this risk.

The results and lessons the QUANDHIP coordination team has learned from the experience of performing six EQAs (both within the NIB and NIV sub-networks of the QUANDHIP JA) are summarized here.

The first mitigation of risk was to ensure that the partners involved in the performance of the EQAs had the bio-risk management elements in place to receive non-attenuated or live strains of the pathogens belonging to the testing panel.

The agreed EQA objective was therefore to identify progress and best practices in the performances of the participating laboratories as well as to identify gaps to be filled. This rationale was of clear public health benefit for preparedness and strengthening capacity of laboratory response and, therefore, these benefits outweighed the risks of not performing such proficiency testing activities.

During this evaluation we are analyzing the exercise elements


The bacterial EQAEs were focused on *B. anthracis*, *Y. pestis*, *F. tularensis*,*Burkholderia pseudomallei*,*Burkholderia mallei*,*Brucella melitensis*-group, and only in Q-2 on *Coxiella burnetii*. From 7 to 15 samples, according to the exercise scenario, containing living bacteria (native samples), partially mixed with typical "contaminating" bacteria and inactivated bacteria in a variety of complex matrices and were provided by the RKI. Typical composition of EQAEs and summarized results are given in Tables S1–S8 in Supplementary Material.

Shipment was identified as a potentially very sensitive element in the EQAEs and was therefore prepared with a comprehensive effort involving the provider as sender, shipping agency, and the consignee considering the international regulations for air (IATA) and ground (ADR) transportation of dangerous goods of class 6.2.

According to the European Export Regulations of goods with dual-use potential (10), the provider has received a general export license for the set of defined biological agents relevant for this project from the German Federal Office of Economics and Export Control (BAFA), which has fully implemented the European Regulation article 9 (2) (10, 21). This license is required for EU exports to any non-EU Member States. Anyway each EQAE has been announced to the BAFA and any additional national regulations of the EU Member States have been followed by the partners receiving the material.

It was extremely important for Biosafety and Biosecurity reasons to select a reliable shipping agency. In this context, a market analysis was conducted and revealed only one appropriate provider, who could be identified as an appropriate shipper in terms of shipment quality and bio-risk management. At the beginning of the project, some partners intended to use alternative shipping approaches for ground transportation to reduce costs. However, it appeared that the cooperation with less experienced companies caused doubts concerning an appropriate risk management with all involved parties (sender,shipper,and consignee),and the reduced costs were compensated by an increased work load. Most problems appeared through inadequate knowledge of drivers and equipment of vehicles according to the ADR and through the deficient traceability of sent material. As a commonly agreed solution it was decided to use only certified and known shippers. This approach resulted in a high quality of shipment without any lack of traceability. Some rare delays at custom authorities were usually due to lack of provided information by the consignees and technical problems at the customs office and could be solved by the support of the shipper. During the project, all participants were learning from these administrative issues and cross-border problems did not really occur.

This was part of the reason why the deviation of time, required for the delivery of samples to the various participants, was significantly reduced. All in all, the transportation time was relatively short and border issues were an exception and could be solved without serious problems.

It becomes very obvious that the involved laboratories substantially improved their response time by about 70%. This important improvement is due to a better preparedness of the laboratories through training effects during the exercises and an improvement of diagnostic algorithms and methods. This practically relevant achievement is a very impressive outcome of the project and is rather discouraging for a third party who might have the intention to misuse a time delay in diagnostics of the agents. On the other hand, this information might be helpful for other laboratories to adapt their own procedures to this high level standard.

The submission of correct quantitative and qualitative results and the assessment of good performance of the laboratories are a major element of EQAEs. In addition, even more profound analyses have been performed to identify best practices for correct and to reveal reasons for incorrect results (data not shown), which were discussed with the participants. This included applied algorithms and methods. Important conclusions could be drawn and related recommendations for improvement could be given, where appropriate.

As a conclusion, PCR or immunological approaches were identified as best practices for sample analyses as a first step for preliminary identification of target bacteria, which should be followed by a confirmation by cultivation/isolation of bacteria and subsequent identification of growing germs by PCR or other applicable methods like MALDI-TOF (22).

From our study it can be concluded that the range of results on the quantitative reference samples is by far too broad.

All in all, it can be stated that the laboratories performed on a high level of diagnostic quality. However, if the sample composition varied and got more complex and challenging, the individual as well as the overall results fell off in quality. Together with the revealed problems that occurred in terms of correct identification of *Brucella* species and *F. tularensis* subspecies, the detection of "mixed" samples, the quantification of target bacteria, and the antimicrobial susceptibility testing (data not shown), quite sensitive information has been produced. As a conclusion of the results, topical working groups were set up in the framework of NIB profoundly tackling AST, MALDI-TOF, and the development of quantitative reference materials. As done here there is a need to provide the scientific and health community with technical data

on these issues to generate scientific and administrative input and support. Regarding AST, the European Committee on Antimicrobial Susceptibility Testing (EUCAST) is showing interest in our activities and a close contact could be established with the aim to approve our results and take those into consideration for the development of appropriate European standards. It can be assumed that this high standard of diagnostics is rather hindering a third party to misuse this information. On the other hand, the gained experience can be very helpful for other laboratories to improve their diagnostic approaches. Thus, without disclosing individual laboratory performances it seems to be of benefit to publish these data for the scientific community and policy decision makers in this area.

#### **DEVELOPMENT AND SHARING OF A REPOSITORY OF HIGH CONTAINMENT PATHOGENS AND MATERIALS DERIVED FROM IT (REGARDING WP 5)**

To perform EQA exercises and to validate existing and new diagnostic methods the consortium needs access to appropriate biological materials (i.e., living and inactivated strains, antibodies, reagents, etc.). During the previous EU projects EQADeBa and ENP4-Lab repositories have been established which could be extended and characterized as part of the QUANDHIP JA activities. One key question related to dual-use issues was how the information of the repository contents would be communicated and how requests from the different consortium members, collaborating partners, and external experts are dealt with. Internal data bases are available only on the restricted QUANDHIP JA workspace providing all available characteristics of the samples. An appropriate "MTA" for guiding the sharing and the use of strains has been developed and implemented (an example is provided as supplementary material). As for the EQAEs this MTA has been agreed between all partners beforehand including their legal departments. The MTA is intended to be used also for a rapid exchange of material in outbreak situations and as well for managing any use of the materials for subsequent research activities. Another use of the MTA is to advise good practices and to record the sharing of the materials as part of the bio-risk management aspect. All partners have been asked to provide relevant and characterized bacterial isolates and, both clinical and environmental samples. A procedure and recommendations for the transport of infectious material were developed, including the strict advice for consideration of national and international regulations, and according to the dangerous goods regulations.

Today, the QUANDHIP JA NIB repository consists of 148 different strains. From most of the bacterial isolates reference material has been produced as genomic DNA, as heat inactivated cells for non-spore-forming bacteria, as PAA inactivated cells for spore-forming bacteria, and as viable cells. This approach will be completed for all relevant strains. Moreover, further methods for inactivation, like gamma irradiation (23), and for storage of the material, like lyophilization, are under development. In the QUANDHIP JA NIV, the BSL4 laboratory partners have their own repository of viral agents are collaborating with the DG RTD funded European Viral Archives (EVA) project to support the quality, management, and distribution of viral strains and reagents (24).

In order to facilitate the safe and secure circulation of biological reference materials and procedures within the network, the following activities are in planning:


In addition, it was also planned to develop and verify quantitative nucleic acid standards for the comparison of different methods and instruments.

#### **CONTINUED PROFESSIONAL DEVELOPMENT TOWARD A COMPETENT AND RESPONSIBLE WORKFORCE (REGARDING WP 6 AND 7)**

Partners of the network have offered practical laboratory based training to other partners covering laboratory diagnostic response strategies in terms of preparation and analysis of samples within BSL3 and/or BSL4 facilities focused on best practices. Twelve training courses have been provided by selected partners at their institutions. All associated partners have agreed on the course contents and defined learning objectives and intended outcomes. The evaluation of the training courses, provided so far, has shown the high benefit for the participants, which has been implemented for the optimization of laboratory practices for both diagnostic and bio-risk management.

This exchange of experiences is one major aspect for the improvement of the performances shown during the EQAEs and for the setting up of new technical approaches. Another result of these courses is that partners may tackle questions,e.g.,of Biosafety and Biosecurity, on more familiar grounds, and find help by personal contacts in cases of emergent biological situations. From the perspective of dual-use one could argue that trainees get deep insight in best practices, capabilities, and capacities as well as security approaches on the trainer's side. However, all the staff were selected and responsible for laboratory bio-risk management in the partners' states. Moreover, we could create a culture of mutual trust and responsibility.

The training course within the NIV was organized in five sessions, covering various theoretical aspects of BSL4 work (basic knowledge, biosafety, management issues, competency, and scenario exercises). In addition, practical training was offered on BSL4 working conditions, handling a BSL3 glove box, sample storage, differential diagnosis of hemorrhagic fever viruses, and decontamination.

## **TOWARD HARMONIZING BIO-RISK MANAGEMENT PRACTICES (REGARDING WP 7)**

For the development of operational Biosafety and Biosecurity recommendations useful for self-evaluation and to be agreed among the project consortium, the work was designed to identify, agree, and disseminate key elements of structure and operation of primary and secondary containment, building design and infrastructure, integrated special equipment, disinfection strategies, biosecurity issues, etc. The previously developed check lists were compared with guidelines and recommendations derived from documents produced by EC, WHO, CDC, CEN workshops and national authorities. Based on the practical requirements of institutions running or developing high containment laboratories, BSL3 or BSL4 check lists will be further developed and validated considering external input. Currently, this is under further development.

Considerable collaboration and input has been provided by the European ENP4 laboratory and Biosafety (EBSA, ECDC) community to the development of the CWA 16393: 2012 (25) – Laboratory bio-risk management-Guidelines for the implementation of CWA 15793:2008 Laboratory Bio-risk Management Standard.

The check list is of almost general character and avoids any detailed description of the bio-risk management. This information belongs only to the partner of the project and can therefore not be misused by third parties if not disclosed.

## **FROM THEORY TO PRACTICE – IMPLEMENTING LABORATORY RESPONSE SUPPORT ACCORDING TO THE QUANDHIP CORE COMPETENCIES (REGARDING WP 8)**

All efforts and outputs of the JA are used to develop proposals to support the coordination of response to cross-border events with highly infectious pathogens is addressing the following areas:


A document, including a set of recommendations for all mentioned issues, has been drafted, disseminated to and discussed with all project participants. In addition, links with other European initiatives in the field of mobile lab, like DEVCO, have been considered. Moreover, the final version of the document will be disseminated through the project website and via scientific and project meetings (inviting, e.g., EpiSouth plus (26), ERINHA, ENIVD).

## **DISCUSSION**

It is now widely accepted in the scientific community to perform an assessment of the "dual-use" character of scientific projects. QUANDHIP is a JA aimed to perform quality assessments and improvement of the diagnoses of highly pathogenic bacterial and viral agents. Consequently, this also includes all measures for an appropriate bio-risk management considering the potential dualuse character of certain elements of the project. These elements consist of sharing sensitive information on diagnostic approaches, performance of diagnostic laboratories dealing with highly pathogenic microbiological agents, repositories of these agents, laboratory bio-risk management and, in addition, exchange of highly pathogenic biological agents between participating laboratories was practically performed.

Several research institutions and authorities, like the Robert Koch Institute, have developed a policy and recommendations for this assessment (27). The RKI policy on DURC which is the basis for the assessment of our projects and which is in line with officially published recommendations includes several criteria for the evaluation of research and development:


In addition, other policies, like those provided by the U.S. National Science Advisory Board for Biosecurity (NSABB) and EU European Export Regulations, include a principal list of agents and toxins which might be misused with high consequences for public health in addition to specific aims of research projects (10, 28, 29). The target pathogens focused in QUANDHIP are listed in these lists of agents to be controlled for export.

Using this set of criteria we assessed the JA QUANDHIP for elements of DURC and, when relevant, how it was prevented. The sensitive scientific procedures in general mentioned in both RKI and NSABB documents lead to the conclusion that no potential for DURC can be identified although, in contrast, the indicative list of agents published by the U.S. NSABB, the exchange of these materials between laboratories and the handling of sensitive information are raising the need to consider potentially DURC. However, the work of our project is not focused on research of these organisms but rather on best practices to detect and identify these microorganisms. Several international guidelines, regulations and recommendations as well as our own responsibility lead to the inevitable conclusion that this project is sensitive in terms of Biosafety and Biosecurity and measures had to be undertaken to prevent misuse of infectious agents and of information generated and used in the framework of this project. Under this premise we would like to illustrate some relevant issues in the project.

Most relevant in this context might be WPs 4 (EQAEs) and 5 (Repositories) where we exchanged highly pathogenic inactivated and living microorganisms between participating laboratories. Beforehand, we developed a strategy to minimize the risk of misuse or loss of these agents as already described in the previous sections. The strategy included


Taking this preparation into account three EQAEs including living bacteria for NIB and three EQAEs with inactivated material for NIV were carried out during the project. These exercises were technically most challenging for the participating laboratories. Together with a gap analysis of methods and procedures the EQAEs and training courses led to a substantial improvement of the performance of many laboratories. Collaboration between laboratories formed the basis for mutual support in emerging biological cross-border events but also for scientific collaboration. The provided material was also used as a means of assessment for further evaluation and improvement of laboratory techniques. Several participating laboratories were using the results of the EQAEs for accreditation purposes. In this respect, the QUANDHIP project had an exceptional international dimension because the organization of such EQAEs at national level is almost impossible and not cost-effective due to a too low number of participants. The check lists for Biosafety and Biosecurity were used by participants for self-evaluation but also in

some cases for evaluation of newly constructed high containment laboratories.

The gained experiences regarding shipping issues will also serve sample sharing during an acute outbreak situation. All samples delivered in the framework of this project were shipped according to international regulations for the transportation of dangerous goods (12, 13). In addition, national regulations for import and export of such material were considered and a certified shipping agency contracted. Consequently, no serious problems occurred in terms of transportation of the samples, nor during border transfer due to a well prepared action.

Most EU Member States do probably not carry out proficiency tests for diagnostics of highly pathogenic infectious agents at national level although this would be strongly required to ensure the necessary quality of diagnostics (30–32). It should be considered that all activities described above are not only important from the perspective of intentional release of these highly pathogenic microorganisms as these bacteria also occur naturally in the environment and require appropriate diagnostic tools on a regular basis (33–38). The European scope of the project offers the appropriate framework to evaluate, improve and sustain these diagnostics to have a broader platform to develop and exchange knowledge, methods and reference material on these often "neglected" but potentially very dangerous diseases.

Not only agents but also information could be misused. On the one hand, this was mainly prevented within the consortium by signed agreements, on the other, a dissemination plan was developed and the exchange of sensitive information was realized using a specially secured German governmental internet provider. The internal workspace was continuously administered and moderated by the coordinator. The individual results of the EQAEs were strongly confidential and visible for the single participant only. Only a restricted number of persons from the coordinators´ part could see all results and draw anonymized overviews and trend analyses. There is no other way than by the laboratories themselves to make their individual results available to the public. Confidentiality is not only important because of the good laboratory practice but also because of security issues, i.e., to prevent information on gaps in the analytical capability of a certain national laboratory which could be intentionally misused by third parties. In contrast, overviews on best practices and possibilities of optimization of diagnostic approaches, published in scientific journals and reports, and presented on scientific meetings, could be useful for non-participating laboratories and stakeholders. The more detailed reports of the project were restricted to CHAFEA for defining the further usage.

Also the recommendations and description of the activation procedure of the QUANDHIP network in biological cross-border events, drawn in WP 8, could be regarded as sensitive information. This document also includes a list of national laboratories appointed to perform diagnostics on highly pathogenic infectious agents. This operational document should be used by EU and national decision makers and be distributed to other laboratories conducting "first line" diagnoses. The negative side of the "dual-use" potential could consist of identifying laboratories with relatively low diagnostic performance by a penetrator. Moreover, the activation and collaboration procedures inside the network could be disturbed in case of emergent situations including bioterrorism. Yet, the document does not offer this type of information and the established procedures are robust enough not to be disturbed. Moreover, the document defines clear procedures for laboratory response in cross-border events and indicates possible supportive collaboration between laboratories in different Member States. So, the benefit clearly outweighs a probable misuse.

All in all, the following summary, including recommendations, may be given on the outcomes of the JA:


Finally, we came to the conclusion that the benefit for all participating laboratories and therefore for the health protection of the citizens was stronger than the minimized residual risk of our activities, which gave us the opportunity to carry out the project from the perspective of DURC assessment. It were taken all measure to minimize the misuse of exchanged biological material, to make these measures transparent to the legal agencies and authorities and to create a network of trusted and reliable laboratories, which professionally handle biological material and information to prevent any misuse of it.

## **ACKNOWLEDGMENTS**

We would like to thank all Project Partners for their cooperation: Austrian Agency for Health and Food Safety (Vienna, Austria), Veterinary and Agrochemical Research Institute (Brussels, Belgium), Public Health Agency of Sweden (Solna, Sweden), Public Health England (Porton Down, United Kingdom), National Center for Epidemiology (Budapest, Hungary), Friedrich-Loeffler-Institut (Jena, Germany), Norwegian Institute of Public Health (Oslo, Norway), Istituto Superiore di Sanità (Rome, Italy), National and Kapodistrian University of Athens – Medical School (Athens, Greece), National Public Health Surveillance Laboratory (Vilnius, Lithuania), National Institute for Public Health and the Environment (Bilthoven, The Netherlands), National Institute for Health and Welfare (Helsinki, Finland), Istituto Zooprofilattico Sperimentale della Puglia e della Basilicata (Foggia, Italy), Bundeswehr Institute of Microbiology (Munich, Germany), Instituto Nacional de Saúde Dr. Ricardo Jorge (Lisbon, Portugal), Istituto Zooprofilattico Sperimentale della Lombardia e dell'Emilia Romagna (Brescia, Italy), National Institute for Nuclear, Biological and Chemical Protection (Milin, Czech Republic), Instituto de Salud Carlos III (Madrid, Spain), Direction Générale de l'Armement – Maîtrise NRBC (Vert-le-Petit, France), National Veterinary Research Institute (Puławy, Poland), Riga Eastern Clinical University Hospital – Infectiology Center of Latvia (Riga, Latvia), National Center of Infectious and Parasitic Diseases (Sofia, Bulgaria), Fundación Vasca de Innovación e Investigación Sanitarias (Sondika, Spain), National Institute of Public Health (Warsaw, Poland), Technical University of Denmark – National Veterinary Institute (Copenhagen, Denmark), Health Board/Terviseamet (Tallinn, Estonia), L. Spallanzani National Institute for Infectious Diseases (Rome, Italy), Bernhard-Nocht-Institute for Tropical Medicine (Hamburg, Germany), Philipps-Universität Marburg (Marburg, Germany), Institut National de la Santé et de la Recherche Médicale (Lyon, France), Labor Spiez (Spiez, Switzerland), and Norwegian Veterinary Institute (Oslo, Norway). Of course, we are very grateful to the European Commission and the CHAFEA for financially and technically supporting this JA, and we would also like to thank the Scientific Advisory Board (ECDC, SANCO C3, GHSAG-LN, and WHO) for their very helpful and active comments and suggestions.

#### **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at http://www.frontiersin.org/Journal/10.3389/fpubh.2014. 00199/abstract

#### **REFERENCES**

1. Pappas G, Panagopoulou P, Akritidis N. Reclassifying bioterrorism risk: are we preparing for the proper pathogens? *J Infect Public Health* (2009) **2**(2):55–61. doi:10.1016/j.jiph.2009.03.002

2. *Centers for Disease Control and Prevention*. Available from: http://www.cdc.gov


including internal controls for nucleic acid extraction and amplification. *BMC Microbiol* (2010) **10**:314. doi:10.1186/1471-2180-10-314


for the diagnostics of viral diseases to enhance the emergency preparedness in Europe. *Euro Surveill* (2005) **10**(6):102–6.


**Conflict of Interest Statement:** This presentation has been produced with the support of the European Commission's Consumers, Health and Food Executive Agency (CHAFEA). Its content is the sole responsibility of the authors and can in no way be taken to reflect the views of the CHAFEA or any other body of the European Union.

*Received: 28 May 2014; accepted: 03 October 2014; published online: 11 November 2014.*

*Citation: Grunow R, Ippolito G, Jacob D, Sauer U, Rohleder A, Di Caro A and Iacovino R (2014) Benefits of a European project on diagnostics of highly pathogenic agents and assessment of potential "dual use" issues. Front. Public Health 2:199. doi: 10.3389/fpubh.2014.00199*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Grunow, Ippolito, Jacob, Sauer, Rohleder, Di Caro and Iacovino. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Biosecurity in emerging life sciences technologies, a Canadian public health perspective

## **Kirsten X. Jacobsen\*, Kirsten Mattison, Marianne Heisz and Sandra Fry**

Centre for Biosecurity, Public Health Agency of Canada, Ottawa, ON, Canada \*Correspondence: kirsten.jacobsen@phac-aspc.gc.ca

#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Robert P. Ellis, Colorado State University, USA Laura Kwinn Wolf, U.S. Department of Health and Human Services, USA

**Keywords: biosafety, biosecurity, pathogen, toxin, laboratory, dual use, risk management**

Driven by innovation, science and technology are continually evolving. Over the past several years, the global scientific community and the world have had the opportunity to see firsthand the significant strides that have been made in the area of life science research, and the corresponding ethical, safety, and security questions that arise as a result of this work. The idea that well-intended research could be used for nefarious purposes is not new. The "dual-use" potential of advancing technologies has driven the dialog in a variety of sectors, including biological, chemical, and nuclear. In Canada, the Public Health Agency of Canada (PHAC) administers the *Human Pathogens and Toxins Act* (HPTA), the principle legislative tool overseeing the biosafety and biosecurity of activities involving human pathogens and toxins in Canada.

The HPTA currently requires all persons conducting controlled activities (possessing, handling, using, producing, storing, permitting access to, transferring, importing, exporting, releasing, or otherwise abandoning) with human pathogens and toxins to take all reasonable precautions to protect the health and safety of the public. Proposed regulations (*Human Pathogens and Toxins Regulations –* HPTR) to support full implementation of the HPTA (in 2015) were published online for public consultation until September 4, 2014 (http://www.gazette.gc.ca/rp-pr/p1/ 2014/2014-06-21/html/reg2-eng.php) and will be made final over the coming year. The HPTA framework has been developed through extensive consultation with regulated parties in order to keep the public safe

and secure, while not inhibiting responsible scientific innovation and critical outbreak response activities. The key elements of the proposed framework are outlined here.

## **A PROPOSED LICENSING REGIME**

Socially responsible scientific innovation requires that the research community feels responsible for the outcomes of their research. Internationally, there are numerous approaches to establishing national biosafety and biosecurity accountability systems. The licensing regime that is proposed under the HPTA framework would marry these two perspectives by providing greater freedom to internally assess and manage risks with a corresponding increase in accountability for safe and responsible practices.

Under the proposed HPTR, a licensing scheme would be established for facilities conducting controlled activities with human pathogens and toxins. This riskbased scheme would impose more stringent biosafety and biosecurity requirements based on the inherent risks of the agents being handled and the nature of the activities being undertaken. Five elements of the proposed HPTA framework that are particularly relevant for the oversight of research with the potential for dual use and would be facilitated under the licensing scheme are as follows.

### **ENHANCING INTERNAL ACCOUNTABILITY SYSTEMS**

Certain stakeholder populations have risk management and oversight practices in place to support core business functions, such as quality control of their products, which also mitigate many biosafety risks. In other populations, institutional oversight can be highly variable. The proposed HPTR would require facilities conducting scientific research to submit information on how their facility administratively manages and controls biosafety and biosecurity risks, including information on roles and responsibilities of key biosafety personnel or committees. This is intended to enhance local oversight over pathogen research, the foundation of a "systemic" safety regime.

## **BIOLOGICAL SAFETY OFFICERS**

Under an HPTA license, a qualified biological safety officer (BSO) would be designated for each institution (licensed entity) and this individual would have a number of duties and powers, including


The BSO would be a powerful resource for both the license-holder and PHAC to help oversee biosafety and biosecurity within an institution.

#### **SECURITY REQUIREMENTS FOR "SECURITY SENSITIVE BIOLOGICAL AGENTS (SSBAS)"**

The proposed HPTR would establish a prescribed list of pathogens, including a subset of Risk Group 3 or 4 pathogens and prescribed toxins that are on the Australia Group common controls list (http://www.australiagroup.net/ en/human\_animal\_pathogens.html). The proposed framework would require anyone with access to SSBAs to receive a security clearance, unless accompanied and supervised or exempted under the HPTR. Work with SSBAs would also be subject to additional biosafety and biosecurity requirements in the *Canadian Biosafety Standard*.

#### **"GAIN OF FUNCTION" REPORTING**

The proposed HPTA framework would support institutional risk management by requiring notification for experiments that will increase the risks posed by a pathogen (e.g., increased pathogenicity or virulence).

#### **REPORTING EVENTS OF PUBLIC HEALTH SIGNIFICANCE**

The proposed HPTA framework would further require that license holders notify PHAC of events (incidents, accidents) involving human pathogens or toxins that have the potential to put the public at risk. This includes inadvertent release, inadvertent production (e.g., through synthetic biology), inadvertent possession, missing or stolen pathogens or toxins, and any exposure that has or may have caused disease. These reporting requirements allow PHAC to assist in investigations, identify biosafety and biosecurity issues, and follow-up on potential issues of public health concern. Most of these events will not require direct action on behalf of the agency, but will assist in the ongoing and open dialog between regulators and stakeholders.

#### **A COLLABORATIVE APPROACH TO BIOSAFETY AND BIOSECURITY**

While the proposed framework provides a range of compliance monitoring, verification, and enforcement tools, PHAC focuses heavily on compliance promotion. Through extensive outreach and engagement, PHAC provides opportunities for open communication between researchers and regulators, enhancing overall biosafety and biosecurity. For example:


PHAC has initiated a dialog with other Federal departments and agencies that have an interest in emerging life sciences and dual-use research. Together, we are examining where programs are robust, and where there are opportunities to improve oversight at all levels: among federal regulators, manufacturers, and distributors of enabling technologies, industry, researcher, the public, and anyone else with a stake in this complex issue. These conversations are happening in parallel around the world, providing further opportunities to look at the global context.

## **ESTABLISHING A CULTURE OF RESPONSIBILITY**

#### **PLANNING FOR SUCCESS**

For years, researchers have been trying to understand whether and how influenza A/H5N1 could become human-to-human transmissible by aerosols. A wide variety of approaches have been employed, but it was only when Drs. Kawaoka and Fouchier obtained relative success (1, 2) that the international community engaged in heated debate.

In 2006, when the National Institutes of Health recommended research on influenza viruses, including influenza virus transmission [National Institute of Allergy and Infectious Diseases (NIAID); (3)], there was an opportunity to initiate a dialog on the possible risks of such research. Had the international scientific community started to discuss the possible dual-use implications of actually succeeding in creating a mammalian transmissible highly pathogenic avian influenza virus in the laboratory, at the very least, we would have been more prepared when it occurred in 2011 (1,2). These and other examples tell us that, in the global arena, we have a way to go in planning for success from the perspective of biosafety and biosecurity, which may include early involvement of regulators and oversight bodies in the planning stages. As science and technology continue to advance, the challenges associated with "planning for success" will increase exponentially, and policy makers will need to determine how to adjust, for example, to a reality where one can create an entire biological system that has never been seen before.

## **A CULTURE OF RESPONSIBILITY**

In a 2011 report on strengthening the culture of responsibility in the context of biosecurity [National Science Advisory Board for Biosecurity (NSABB); (4)], the NSABB writes that "knowledge is rarely, if ever, neutral." Information of almost any type can be used for both positive and negative applications and thus, determining what knowledge presents the greatest risk for dual use is not only difficult but also highly subjective. Within the realm of life sciences research, external bodies, such as federal regulators, can play an important role in education and enforcing accountability. A culture of responsibility, however, cannot be legislated, but it can be cultivated. In this scenario, everyone with an interest in the great potential benefits and possible risks associated with cutting edge life sciences research has a role to play.

The NSABB report (4) details the role researchers have in understanding the possible implications and applications of their work, in championing good research practices, and in taking ownership of their own responsibility by holding themselves and their peers accountable. This philosophy could equally be extended beyond researchers to the wider community: to institutions, to the manufacturers and distributors of enabling technologies, to professional associations, and even to civil society and the public.

Social responsibility is much larger than individual researchers. It is widely accepted that working in silos is disruptive to institutional alignment and collaboration. Silo mentality has added disadvantage of diffusing safety, security, and ethical practices, leading to redundancies and possible gaps. There is a significant advantage to systemizing safety and security practices – increasing both the responsibility and the accountability of institutions for the oversight and the outcomes of the research done within. In Canada, the proposed framework would place responsibility on research institutions for administrative oversight, permitting a tailored approach to risk management to suit the unique research environment. Establishing biosafety and biosecurity programs such that risk management occurs in a collaborative and integrated environment is expected to increase the likelihood that the necessary conversations take place long

before research with potential for dual use is underway. This would then inform the wider dialog on the ethics, safety, and security issues related to emerging biological sciences.

#### **RISK–BENEFIT ANALYSIS**

In recent examples of research with the potential for dual use, international discussions focused on the risks of publication, with less emphasis on the need for systematic, scientific, evidence-based risk– benefit analysis in such research (5, 6). In the absence of concrete data, as is the case with emerging technologies, this risk– benefit analysis may be largely hypothetical. The risks of potential misuse (accidental or intentional) are weighed against the assumed potential benefits of scientific innovation. Within the scientific community, there is growing debate over the latter with respect to "gain-of-function" flu research. On October 16, 2013, a letter was sent to the President of the European Commission on behalf of the European Society for Virology (ESV), cautioning against prohibiting dual-use research because of the potential benefits (7). Two months later, a letter in response to the ESV appeal was sent to President Barroso on behalf of the Foundation for Vaccine Research, challenging many of the reputed benefits (8).

This underscores the need to take a critical look at the real benefits of research with potential for dual use and, in some way, measure them against the real risks. This will be very difficult as, in almost all cases, the "real" risks and benefits have not yet been realized, and there may be significant division within the scientific community on both counts. This is perhaps the area of discussion in which it is most important to involve all sectors, including the public, as "risk" and "benefit" are both highly influenced by perception. A qualitative risk–benefit analysis framework for assessing research with dual-use potential, if possible, would be the most decisive tool for asking the hardest and most important questions we currently face: "what happens if we do not do this research"but also"what happens if we do."

#### **REFERENCES**

1. Herfst S, Schrauwen EJ, Linster M, Chutinimitkul S, de Wit E, Munster VJ, et al. Airborne transmission of influenza A/H5N1 virus between ferrets. *Science* (2012) **336**(6088):1534–41. doi:10. 1126/science.1213362


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 28 July 2014; paper pending published: 04 September 2014; accepted: 02 October 2014; published online: 29 October 2014.*

*Citation: Jacobsen KX, Mattison K, Heisz M and Fry S (2014) Biosecurity in emerging life sciences technologies, a Canadian public health perspective. Front. Public Health 2:198. doi: 10.3389/fpubh.2014.00198*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Jacobsen, Mattison, Heisz and Fry. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

**Petra C. C. Sijnesael 1,2, Linda M. van den Berg<sup>1</sup> , Diederik A. Bleijs 1,3, Paul Odinot <sup>4</sup> , Carin de Hoog<sup>1</sup> , MiekeW. J. C. Jansen<sup>5</sup> , Evelien Kampert <sup>6</sup> , Saskia A. Rutjes 1,7, Martien Broekhuijsen<sup>8</sup> and Sander Banus <sup>1</sup>\***

<sup>1</sup> Biosecurity Office, Centre for Environmental Safety and Security, National Institute for Public Health and the Environment, Bilthoven, Netherlands

<sup>4</sup> Corporate Real Estate and Campus, Taskforce Safety and Environment, Utrecht University, Utrecht, Netherlands


#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Laura H. Kahn, Princeton University, USA

**Keywords: biosafety, biosecurity, biorisk, CWA15793, questionnaire, toolkit, awareness, self-assessment**

#### **INTRODUCTION**

Life sciences, biotechnology, and medical biology are indispensable research fields for public health and the development of therapeutics and vaccines. However, biological agents and information developed to better health, welfare, and safety, could be misused for harmful purposes to cause damage to public health, safety, and the environment (1–3), which is termed the "dual-use" aspect of research in the life sciences. Laboratory biosafety describes containment principles, technologies, and practices to protect people from biological agents, and prevent accidental release of biological agents (4). In addition to biosafety, laboratory biosecurity measures aim to prevent theft and intentional or malicious use of biological agents (4). Thus, both biosafety and biosecurity should be an integral part of program management of organizations handling dangerous pathogens, in order to prevent potential dual-use research, undesired spread, theft, malicious use, and bioterrorism.

#### **FROM BIOTERRORISM TO BIOSECURITY**

The biosecurity program of organizations should contain physical, personnel, transport, technology, and material security (5). In addition, personnel should be well educated and aware of the biorisks of handling dangerous pathogens (1, 4–6). Theft and malicious or terrorist use of biological agents could possibly be traced back to breaches or lacunas in the biosecurity program of an organization. The next three historical examples of malicious use of biological agents illustrate the importance of biosecurity measures within organizations. The first example is the intentional spread of *Salmonella typhimurium*, which led to more than 750 cases of gastroenteritis in Oregon, USA, 1984. Members of the Bhagwan Shree Rajneesh commune ordered *Salmonella* bacteria from a commercial supplier, cultured the bacteria in their laboratory, and contaminated 10 salad bars (7). Criminal investigation revealed that the *Salmonella* outbreak strain was indistinguishable from the strain that had been cultured in the laboratory at the commune. The source of the biological material was a legitimate, easily accessible source, which underlines the importance of biosecurity awareness and the proper functioning of the biosecurity program of organizations. The second example of malicious use of biological agents is a biological attack in Japan. The Japanese religious cult Aum Shinrikyo tried to produce large-scale botulinum toxin and spores of *Bacillus anthracis.* The members isolated harmless strains of *Clostridium botulinum* from soil and thereby failed to produce active botulinum toxin in 1990 (8). For the production of anthrax, the members unsuccessfully attempted to steal *B. anthracis* from a laboratory. Later, the cult received anthrax from an Aum Shinrikyo sympathizer that

had access to the biological agent within a university (8). However, this was an animal vaccine strain of anthrax, and not causing disease during dissemination, in 1993 (8). Thus, biosecurity pillars such as physical security, personnel screening, and personnel reliability are important in preventing theft of biological agents and bioterrorism. The last example describes the anthrax letters containing spores of *Bacillus anthracis* in 2001 in the USA. In total, 22 people were infected of which 5 people died. The source of the biological agent was a state laboratory involved in the national biodefense program of the USA (9). In addition to personnel screening and personnel reliability, material control and accountability might play an important role in preventing future malicious use of biological agents.

#### **DUTCH BIOSECURITY INITIATIVES**

The Dutch government recognizes the need to reduce biological threats and to prevent malicious use of biological agents. Therefore, the Dutch government and the Royal Dutch Academy of Arts and Sciences (KNAW) published the "Code of Conduct for Biosecurity" in 2007 (10). The code is intended to guide organizations and professionals that are, directly or indirectly, engaged in research or education in the life sciences, such as biology, medical biology, or biotechnology. Life sciences quickly evolve and new biosecurity and

<sup>2</sup> Stout Groep BV, Echteld, Netherlands

<sup>3</sup> Genetically Modified Organisms Office, Centre for Safety of Substances and Products, National Institute for Public Health and the Environment, Bilthoven, Netherlands

dual-use questions rise, such as the worldwide H5N1 biosecurity debate in 2011, 2012 (3). Therefore, the KNAW published the advisory report "Improving biosecurity, assessment of dual-use research" in 2013 (3). This report recommends biological threat analyses and an advisory board for research in the life sciences. Furthermore, the report emphasizes the importance of raising early awareness for the risks and potential misuse of research and knowledge in the life sciences.

In response to international biosecurity initiatives (11–15) and the evolving life sciences, the Dutch government initiated a biosecurity project to establish a coordinated Biosecurity Program for organizations handling hazardous biological agents and associated technology, in 2009. The purpose of this Biosecurity Program is to prevent proliferation of biological materials and associated knowledge for illegitimate purposes. As part of the Biosecurity Program, the Dutch Biosecurity Office was founded in 2012. The Biosecurity Office is the national knowledge and information center for biosecurity, and offers awareness raising workshops. The Biosecurity Office utilizes previously adopted good practices from both national and international initiatives, such as the BTWC, the EU CBRN Action Plan, CWA 15793, and the Dutch Biosecurity Code of Conduct. The Biosecurity Office cooperates with existing relevant organizations, such as the Dutch Platform of Biosafety Professionals. The biosecurity policy in the Netherlands reflects the current worldwide trend to combine biosafety and biosecurity into biorisk program management (4, 15).

#### **ONLINE "BIOSECURITY TOOLKIT"**

In close collaboration with the Dutch Platform of Biosafety Professionals and other experts, the Biosecurity Office developed the online "Biosecurity Toolkit," in 2012 and 2013. The Biosecurity Toolkit aims at enhancing biorisk management within organizations handling hazardous biological materials. The Toolkit is a selfassessment tool that is freely available via www.biosecuritytoolkit.com in Dutch and in English. The Toolkit is an easily accessible tool for professionals and organizations to analyze gaps in their institutional biosecurity management. The outcome of the Toolkit includes best practices per biosecurity pillar to improve the biosecurity level of the organization. The use of the Toolkit is anonymous and online results are not stored. The Toolkit helps organizations to assess their current level of biosecurity and combines biosafety and biosecurity into biorisk.

## **METHODS**

The Biosecurity Toolkit has specifically been developed for organizations handling hazardous or dangerous biological agents. Representatives from those organizations, governmental representatives, and biosafety/biosecurity experts were invited to participate in the development process of the Biosecurity Toolkit. This group of stakeholders and experts convened in several meetings to compose the toolkit and ascertain applicability of the Toolkit for the intended users.

#### **QUESTIONNAIRE**

The experts defined eight pillars of biosecurity risk management, namely awareness, personnel reliability, transport security, information security, accountability for materials, response, management, and physical security (5, 6, 16). The biosecurity experts added the eighth pillar "management" to the Biosecurity Toolkit, since the management of an organization should also be aware of biological risks, and commitment of the higher management is a prerequisite for successful implementation of the biorisk management program. A short description per biosecurity pillar is provided in **Figure 1A**. Per biosecurity pillar, the user needs to answer up to 10 questions with"yes"or"no"in the questionnaire (**Figure 1B**), and the relative score for each category is normalized to 100%. In case of doubt or uncertainty, the user is advised to fill in "no," so the associated suggestion for improvement will be addressed after fulfilling the Toolkit. Each question is accompanied with explanatory or background information, accessible via the information icon (**Figure 1B**). The questionnaire can be saved and interim results can be viewed between different pillars, at every convenient time for the user.

#### **LEGAL BASIS AND GOOD PRACTICES**

Supplemental information about legal basis and good practices is provided under the tab page "Good practices" (**Figure 1C**). The information under "Basis" refers to national and international laws, guidelines, standards, and other relevant documents that are available in the Netherlands, such as the Biosecurity Code of Conduct (10) and CWA 15793 (15). The column "Good Practices" lists specific biosecurity measures that may increase the biosecurity level of that particular biosecurity pillar. The good practices have been formulated in collaboration with experts from the field.

#### **RESULTS SECTION OF THE TOOLKIT**

After completing the questionnaire, the user is directed to the results section of the Toolkit and the outcome of the survey is automatically presented to the user. Relative scores for each category are calculated as a percentage (actual score as percentage of the maximum achievable score). Importantly, the overall score is not calculated as an average of the individual scores, but is equal to the *lowest* score obtained in the separate elements. The overall score is presented as lowest score since the aim of the survey is to identify gaps and strengthen the biosecurity program, which is most effectively obtained by improving the weakest element in an organization.

## **EXAMPLES AND CONCLUSION**

The type of organization, the biological agents handled by the organization, the risks associated with executing proceedings, the dual-use potential or likelihood that an agent can be misused, and many more variables are important for designing and implementing a biosecurity program within the organization (6, 16). To illustrate the use and possible gap analysis of the Biosecurity Toolkit, we hypothetically describe two types of organizations handling dangerous pathogens: a highcontainment diagnostic laboratory from a university medical center, and a highcontainment laboratory from a pharmaceutical company.

#### **HIGH-CONTAINMENT DIAGNOSTIC LABORATORY IN A UNIVERSITY MEDICAL CENTER**

For diagnosed or suspicious hazardous material, the university medical center has a BSL3 facility. Only authorized personnel are allowed to enter and conduct laboratory work in the BSL3 facility. Reference material and patient samples are stored within the containment of the BSL3 facility. Since the laboratory is part of a university medical center, knowledge is shared among different departments and potential hazardous samples may be used for research or scientific purposes. The employees are fully aware of biosafety risks; however, there is less awareness for biosecurity and dual-use risks of the samples. Entrance to the BSL3 facility has been restricted to authorized employees only, however, the medical center is a public, open organization and outsiders can easily enter the hospital. There is no security culture within the center. The medical center scores well on external transport security; however, internal transport of diagnostic samples that are used for scientific purposes is pore documented in procedures. The same applies for information security: the university medical center has guidelines for confidentiality of patient samples, but no guidelines for securing and following research samples. The hospital has procedures for emergency and crisis response, and has a clear policy of communication in case of emergencies. Thus, the fictive gap analyses for the medical center identified gaps for biorisk management system, physical measures, biosecurity awareness, and personnel reliability. The center scores well on material accountability and response, and scores average on information security and transport security.

#### **LABORATORY IN A PHARMACEUTICAL COMPANY**

This pharmaceutical company develops vaccines against airborne influenza viruses. The company has BSL3 animal facilities and laboratories for research purposes. The company is located in a rural area and has strict entrance security. Employees are background checked and research is well documented, since patents and intellectual property are important for the development of vaccines. The organization has high standards regarding general security, biosafety regulations and well-documented research, recorded in standard operating procedures, and procedures describing coding of materials. The fictive gap analyses for the pharmaceutical company identified gaps for biosecurity awareness and response, specifically in case of dual-use research awareness. Although personnel are well educated and trained for handling dangerous airborne pathogens, this training has been focused on biosafety and not on biosecurity awareness. The same applies to response and incidents: response in case of biosafety incidents and theft have been documented in procedures and covered in


**FIGURE 1 | The online Biosecurity Toolkit has eight biosecurity pillars**.

(Continued)


#### **FIGURE 1 | Continued**

**(A)** The eight biosecurity pillars were adapted from previous studies (5, 16) and were ascertained by the Dutch biosecurity expert group and biosecurity stakeholders. In the left column, the pillars are placed in the order of appearance in the online Toolkit. In the right column, a short description per biosecurity pillar is provided. **(B)** The pillars are placed in the tab pages on the top of the webpage where the questionnaire for "Awareness" is shown. By clicking on the subsequent pillar, the questions become visible and can be answered with "yes" or "no." The yellow "i" information button provides information about the specific question. The online questionnaire can be saved between pillars, and interim results can be viewed at any convenient time. By clicking the "reset" button, the form will be cleared from previously entered answers. **(C)**. The tab page "Good Practices" contains legal bases and good practices for biosecurity program improvement. By clicking on specific biosecurity pillars, a list with links, best practices, and information is available with suggestions for improvement of the biosecurity program within organizations.

the employee training; however, no procedures are present regarding emerging dualuse research. Thus, the company scores well on personnel reliability, physical measures, material accountability, and information security. The company scores less on response and biorisk management,since biosafety and biosecurity are not integrated in the company.

#### **CONCLUSION**

Here, we describe an online self-assessment "Biosecurity Toolkit," which was developed to strengthen awareness among laboratory employees, biosafety or biosecurity officers, the management team, or security managers of organizations handling dangerous biological agents. The web-based Biosecurity Toolkit offers a free and easily accessible tool and the resulting gap analysis of the questionnaire is for internal use only. The results are anonymous and not automatically uploaded or stored. The main purpose of the toolkit is to provide the user insight in the level of biosecurity within the organization, to create awareness and above all, to provide suggestions for improvement of the biosecurity level by focusing on the weakest elements.

### **AUTHOR CONTRIBUTIONS**

Petra C. C. Sijnesael, Linda M. van den Berg, Diederik A. Bleijs, and Martien Broekhuijsen wrote the manuscript. Petra C. C. Sijnesael, Linda M. van den Berg, Diederik A. Bleijs, Paul Odinot, Carin de Hoog, Mieke W. J. C. Jansen, Evelien Kampert, Saskia A. Rutjes, and Martien Broekhuijsen investigated, designed, and tested the Biosecurity Toolkit. Sander Banus supervised all aspects and execution of the project. All authors critically reviewed the manuscript for correct content.

#### **ACKNOWLEDGMENTS**

The authors are grateful to the members of the Biosecurity Office and the National Institute for Public Health and the Environment (Bilthoven, the Netherlands) for their valuable input. We would like to thank the Dutch Platform of Biosafety Professionals, the Dutch interdepartmental advisory committee, and the national biosecurity experts for their valuable input and support. The Biosecurity Toolkit was funded by a grant from the Dutch Government (Biosecurity Office). The authors declare to have no financial, personal, political, or academic interest.

#### **REFERENCES**

1. National Science Advisory Board for Biosecurity. *Proposed Framework for the Oversight of Dual Use* *Life Science Research: Strategies for Minimizing the Potential Misuse of Research Information*. Bethesda: NSABB (2007). 62 p.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 25 July 2014; accepted: 02 October 2014; published online: 20 October 2014.*

*Citation: Sijnesael PCC, van den Berg LM, Bleijs DA, Odinot P, de Hoog C, Jansen MWJC, Kampert E, Rutjes SA, Broekhuijsen M and Banus S (2014) Novel Dutch self-assessment Biosecurity Toolkit to identify biorisk gaps and to enhance biorisk awareness. Front. Public Health 2:197. doi: 10.3389/fpubh.2014.00197*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Sijnesael, van den Berg, Bleijs, Odinot, de Hoog, Jansen, Kampert, Rutjes, Broekhuijsen and Banus. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## **Frank L. Smith III \* and Adam Kamradt-Scott**

Centre for International Security Studies, University of Sydney, Sydney, NSW, Australia \*Correspondence: frank.smith@sydney.edu.au

#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Jennifer Gaudioso, Sandia National Laboratories, USA

**Keywords: dual use research of concern, biosecurity,World Health Organization, United States, Australia**

The creation of a virulent mousepox virus in Australia and publication of this experiment in 2001 are often argued to mark a dangerous turn in dual use research (1). After this experiment and – far more consequential – September 11 and the anthrax letters, the oversight of dual use research in the life sciences received considerable attention in the United States. We argue that the American experience provides valuable lessons for Australia, three of which are highlighted here.

First, the international community is illequipped to govern the life sciences. Like the United States, Australia should therefore help itself through national regulations and oversight. Second, like most special interest groups, scientists prefer selfregulation. While this may be a practical solution for scientific publications, federally funded research warrants independent review as a condition of funding. Third, in order to provide independent review, oversight should be truly multidisciplinary, including social, political, and biological expertise. A multidisciplinary approach stands the best chance of balancing the risks and rewards of dual use research.

#### **THINK GLOBAL, ACT NATIONAL**

The risks and rewards of dual use research have global implications. Despite repeated calls for international leadership, however, the World Health Organization (WHO) has followed rather than led member states, and it is unlikely to adopt a more assertive role. The WHO rarely issues advice on biological weapons (although it helps to monitor some smallpox research), and it said little about dual use until after the National Academies in the United States published

the influential Fink Report confronting this dilemma in 2004 (2). In 2010, the WHO published its own anodyne guidance on "responsible life sciences research," which acknowledges the "important role of WHO to lead" but then concedes to a "countrybased approach"(3). Moreover, when faced with experiments that increased the transmissibility of H5N1 influenza (4, 5), the WHO sought to distance itself from any suggestion that it might assume additional responsibilities for oversight, focusing instead on "*ad hoc* solutions" to this particular controversy (6). Simply put, the WHO lacks the resources and will necessary to govern research, so states must act on their own.

Of course, the United States was never waiting for the WHO. American guidance and oversight of dual use research draws on an older system devised for recombinant DNA research that dates back to the Asilomar Conference in 1975. The Fink Report, for instance, argued that "we now need to build upon the Asilomar experience" to manage the potential misuse of biotechnology (2). This report and the National Science Advisory Board for Biodefense (NSABB) that it inspired in turn proposed incorporating oversight of dual use research into the existing system of review by Institutional Biosafety Committees and the National Institutes of Health (NIH) Recombinant DNA Advisory Committee (7).

Though less explicit, similar logic is implied in the 2012 "Policy for Oversight of Life Sciences Dual Use Research of Concern" (8), as well as the 2013 "United States Government Policy for Institutional Oversight" (9). As it now stands, U.S. policy will address 7 types of experiments of concern that were identified by the Fink Report, along with 15 select agents that are regulated through the 2001 PATRIOT Act and the 2002 Bioterrorism Act. Oversight will be accomplished through institutional and federal review of dual use research that is funded by the government, which, if fully implemented, will probably resemble the system used for recombinant DNA research.

In Australia, at least two legislative instruments apply to the dual use dilemma. First, the *National Health Security Act 2007* regulates biosafety and biosecurity standards for handling "security sensitive biological agents." Second, the *Defence Trade Control Act 2012* applies to dual use technology, particularly through the *Defence and Strategic Goods List*. Like the American *Commerce Control List*, the Australian list designates various "materials, chemicals, micro-organisms, and toxins" as being dual use and subject to export control (10). Responsibility for research oversight in the life sciences was devolved to the National Health and Medical Research Council (NHMRC) in 2013. As "Australia's leading expert body promoting the development and maintenance of public and individual health standards" (11), the NHMRC is now proposing a supplement to the *Australian Code for the Responsible Conduct of Research* regarding dual use or "gain of function" research (12).

### **FOCUS ON FUNDING**

Moving forward, Australia will need to decide what steps it will take to actually oversee research of concern, and the United States still has a long way to go in implementing its evolving policies. The success of these national systems will depend on the active participation and support of the scientific community, which has special interests in how research is governed. Yet, special interests can diverge from the public interest, and this political fact has important implications for where oversight should occur and who should be involved.

Where should oversight occur? Of all the potential points of intervention, scientific communications are probably the least practical and most controversial to regulate, as illustrated by the H5N1 publishing controversy. Granted, the pre-publication review of these experiments could be considered a partial success, since they were not simply published without first evaluating the risks (as was the case in the Australian mousepox experiment and others). It is debatable, however, how much this review process affected the outcome. Plus, the surrounding controversy suggests that many scientists fear censorship more than malicious use of their research. Insisting on self-governed or unrestricted publications, scientists are quick to cry foul over the U.S. government encroaching on the norm of scientific transparency or openness.

Adherence to this norm is easy to overstate: scientists restrict information all the time, from nuclear and trade secrets to blind peer review. Nevertheless, it is impractical for the United States or Australia to restrict information that is often disseminated in different forms and venues throughout the course of research. It is far better to focus on government funding. Many life scientists depend on this funding, which gives the government significant leverage. Requiring review as a condition of funding also provides for early oversight. This may shape the trajectory of research, thereby increasing the potential benefits while reducing the risks before there are results to worry about publishing.

Tying funding to oversight is not a new idea nor is it a complete solution. But shaping research through federal funding is a relatively efficient option for the United States, especially since research of concern represents a small fraction of dual use research. For example, the NIH spends nearly \$30 billion each year on more than 50,000 projects. Of these, <800 involve agents covered by the government's new policy, and review by the NIH "designated 10 extramural and no intramural projects as dual use research of concern" (13). While NIH is not the only relevant sponsor, these numbers suggest that funding agencies can oversee research of concern without stigmatizing it or imposing an undue burden.

Attaching oversight to federal funding is potentially an even more efficient option for Australia. Compared to the United States, the Australian funding environment is more government-driven and centralized, with relatively few private, philanthropic foundations, and a smaller ecosystem of federal sponsors. As in the United States, some research will escape oversight tied to federal funding, but in Australia, both the supply of commercial research and total government support are smaller. For example, while a large fraction of Australian medical research is funded by the NHMRC, its total budget is only about \$1 billion per year. It is therefore feasible to oversee research of concern through federal funding in Australia.

Unfortunately, Australia appears poised to forgo this promising option, attempting instead to abdicate government responsibility. That may be a reasonable response to the publication problem, and the prospect that it would become an offense to"publish or otherwise disseminate" listed technology under the *Defence Trade Control Act* has prompted a proposed amendment to narrow the prohibition on publication to military goods (14). However, rather than conduct any review of the research that it funds, the NHMRC looks set to delegate this responsibility to individual researchers and their home institutions through the *Australian Code for the Responsible Conduct of Research*. Institutions may establish review bodies to ensure that their researchers' activities and publications do not breach the Code, but this overly decentralized approach stands to be burdensome to implement, difficult to enforce, and it probably fails to address core tensions between some research and the public good.

Australia would be better served by learning from the United States. This not only means establishing an advisory board like the NSABB but, perhaps more significant, also focusing on federal oversight of federally funded research. The risks – including the risk of agencies such as NHMRC looking negligent in the event of an accident or malicious use – are too great to delegate or ignore.

## **MULTIDISCIPLINARY REVIEW**

Finally, who should be involved with oversight? Independent review is easier said than done. Again, scientists prefer selfregulation and, when it comes to admissible expertise, the boundaries they draw around their profession often look strategic and narrow. The H5N1 publishing controversy demonstrates that even the NSABB and WHO, which supposedly represent a range of expertise, are vulnerable to the accusation that their recommendations suffer from conflicts of interest (15). Similarly, the NIH and other funding agencies risk losing public trust if their oversight systems are stacked with researchers from the same fields that they review.

Dual use research is – by definition – a social and political issue, so oversight that lacks considerable social and political expertise is suspicious. "Separating science from politics is impossible in the real world" (16), and a breadth of expertise and perspective is critical for independent review (17). So a multidisciplinary approach to oversight is best. We would include political science, given our profession, but not to the exclusion of fields ranging from sociology and history to economics and medical anthropology. Although NSABB has voting members with a background in law, for example, legal expertise is no more a substitute for political science than chemistry is for biology.

A multidisciplinary approach can also provide valuable perspectives on how to incorporate public participation into oversight. Some scientists bristle at the very idea, consistent with their special interest in self-regulation. But federal funding is tax-payer money, and so it is not outlandish to suggest that scientists should be accountable for the resources they use. Furthermore, it is hypocritical to tout the norm of scientific transparency when opposing restrictions on scientific publications and then decry more open oversight in favor of closed or cloistered peer review. A multidisciplinary approach to oversight is only one step toward adjudicating the tradeoffs involved. Yet, it is a critical step, since the positive and negative externalities of dual use research extend far beyond the laboratory.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 12 May 2014; paper pending published: 02 July 2014; accepted: 30 August 2014; published online: 16 September 2014.*

*Citation: Smith FL III and Kamradt-Scott A (2014) Antipodal biosecurity? Oversight of dual use research in the United States and Australia. Front. Public Health 2:142. doi: 10.3389/fpubh.2014.00142*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Smith and Kamradt-Scott . This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## H5N1: a cautionary tale

## **Susan A. Ehrlich\*†**

Judge (retired), Arizona Court of Appeals, Phoenix, AZ, USA \*Correspondence: ehrlich\_sa@cox.net

†The author was the member of the National Science Advisory Board for Biosecurity representing the "public perspective" from 2005 through 2012. During this time, the Board engaged in discussions about the communication of the results of H5N1 research conducted by Drs. Yoshihiro Kawaoka and Ron Fouchier, and she served on the Working Group formed to study the issues as well as on the Board. At the time of this article, the NSABB has not met after 2012.

**Edited by:** Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Gerald Epstein, U.S. Department of Homeland Security, USA James Revill, University of Sussex, UK

**Keywords: H5N1, dual use research of concern, biosecurity, biosecurity policy, bioterrorism, National Science Advisory Board for Biosecurity**

The history of the research with the highly pathogenic avian influenza virus H5N1 and the publication of the results of that research is reminiscent of the Buddhist "Parable of the Blind Men and the Elephant." In the parable, the Buddha relates a story of a raja who, when confronted with disputatious scholars, gathered blind men and presented each man with a part of an elephant, telling him "here is an elephant." The raja then asked each man to describe the animal. As each man explained "what sort of thing is an elephant," the men began arguing about whether the likeness was to a pot or a basket or a pillar, etcetera, eventually coming to blows over the question and prompting the Buddha to observe:"for, quarreling, each to his view they cling. Such folk see only one side of a thing"<sup>1</sup> . Fortunately, the H5N1 debates have not led to blows but instead have yielded important provocative discussions about the importance of the research and the issues and implications of communicating the results of the research (1). Unfortunately, there still is no consensus regarding the manner of the oversight of life-sciences dual-use research of concern (DURC).

In the fall of 2011, the National Science Advisory Board for Biosecurity (NSABB), a United States Government Advisory Committee, was charged with reviewing two manuscripts describing research funded by the National Institute of Allergy and Infectious Diseases (NIAID) in which genetically modified H5N1 was shown to have the potential for respiratory transmission between ferrets. The primary author of the manuscript intended for publication in *Nature* was Yoshihiro Kawaoka, Ph.D., DVM, of the University of Wisconsin-Madison and the University of Tokyo; the primary author of the manuscript intended for publication in *Science* was Ron A.M. Fouchier, Ph.D. of the Erasmus Medical Center in Rotterdam. The mandate was the result of the manuscripts having been shown to the White House National Security Staff, which referred its concerns that the publication of the manuscripts might have biosecurity implications to the Department of Health and Human Services, which in turn asked the NSABB for its recommendations.

The NSABB formed an H5N1 Working Group. The Working Group and the Board spent hundreds of hours discussing the issues presented by and swirling around the manuscripts, all of which conversations and meetings were closed due to the very nature of the purpose for which the NSABB had been convened, i.e., biosecurity concerns posed by the publication of the manuscripts as well as the interests of the authors and publishers that the manuscripts not be made public prematurely. On December 20, 2011, the NSABB announced its recommendation that the authors' "general conclusions highlighting the novel outcome be published but that the manuscripts not include the methodological and other details that could enable replication of the experiments by those who would seek to do harm" (2).

Subsequently, the authors' revised manuscripts were submitted to the NSABB and discussed at a meeting on March 29 and 30, 2012. This assembly included the perspectives of the governments of the Kingdom of the Netherlands and Japan<sup>2</sup> and from meetings sponsored by the World Health Organization and the American Society for Microbiology. Additionally, the "United States Government Policy for Oversight of Life Sciences Dual Use Research of Concern" was issued at the end of the first day of the 11/2-day conference (3). After a robust debate among its voting and ex officio members, which ex officio membership included NIAID's director, the NSABB unanimously recommended that the revised Kawaoka manuscript should be "communicated in full" (4). It also recommended, but in a 12-to-6 decision, "the communication of the data, methods, and conclusions presented in [the] revised Fouchier manuscript"<sup>3</sup> .

The NSABB path of H5N1 review was to varying degrees rocky from beginning to end. Some of the confusion could have been avoided; some of the commotion could not have been escaped. Pertinent

<sup>1</sup>*The Udana* 68–69.

<sup>2</sup>Other countries expressed their apprehensions about a precedent that the H5N1 research results would not be shared with them although it was they who had shared samples of those viruses with the researchers.

<sup>3</sup>The author was one of the six members who did not concur with the majority's recommendation with reference to the Fouchier manuscript.

questions include why the process was disordered, how confusion could have been or be avoided and whether another mechanism should replace that of the NSABB.

The NIAID-funded studies undoubtedly should have been highlighted for biosecurity concerns well before the results of the H5N1 research were being readied for publication. Questions whether the research might have the potential to be DURC should have been considered during the design, execution, and reviews of the research, and there should have been in place a communication plan given the potential for novel results. From early on, the research clearly was within the seven categories of experiments that could constitute DURC and warrant particular scrutiny according to the 2004 National Research Council report *Biotechnology Research in an Age of Terrorism* (5) and the 2007 NSABB report *Proposed Framework for the Oversight of Dual Use Life Sciences Research* (6). As a consequence of this failure, much of the NSABB discussion centered around the potential for the research to be used for malevolent purposes – the nature and results of the research, its complexity in terms of how readily the research could be replicated, by whom it could be reproduced and made more dangerous, what facilities and what conditions would be necessary, and how it could be disseminated – and a crucial question of whether the benefits for public health outweighed the risks.

As noted, the manuscripts necessarily were kept confidential because of the biosecurity concerns and the interests of the authors and publishers, but before the review of the revised manuscripts, the Dutch government invoked European export-control legislation as well. The secrecy surrounding this review process led to various ill-founded assumptions, misunderstandings, commentaries by individuals who did not know how the matter came to the NSABB and/or would not have read the manuscripts, leaks – some of which were not accurate reflections of the discussions, and reporting errors, all of which harmed the NSABB's credibility despite the NSABB lacking the ability to respond without violating the confidentiality to which its members were sworn. Some kind of equilibrium between transparency and consensus versus security thus was an

researcher and institution, and a Presidential Commission for the Oversight of Dual Use Life Sciences Research established by

> an Executive Order. The first element would be fivefold: (a) a requirement that the federal departments and agencies have personnel with sufficient expertise to screen research proposals for DURC potential with the concomitant requirement that if such potential exists, the researcher be asked to consider modifying the research to reduce risk; (b) a means to make sure that the facilities are adequate that the researcher and laboratory staff are knowledgeable about DURC issues and engage in a continuing review to minimize any necessary risk, and that the institution has a communication plan regarding novel techniques and/or novel results; (c) the mandatory education of each researcher in biosecurity in conjunction with biosafety for the purpose of recognizing and addressing DURC issues; (d) the requirement that the researcher attest to a review of the research for DURC potential at the beginning and on each occasion of a funder's review; and (e) the requirement that each institution have a committee that includes the institution's responsible official, additional experts and community members to review the conduct of research identified as having DURC potential.

> important issue for the NSABB when discussing how and to whom the results of the H5N1 research could be communicated, but a satisfactory balance, while ardently sought by the NSABB, was not found. Ultimately, the NSABB proved not to be an exemplary model and instead showed that a new, autonomous advisory commission must be created as its substitute. The better practical model would have two components: requirements for the federal funding agency and for a federally funded

The second element would be the creation of a Presidential Commission to serve as a truly independent expert advisory group. Its voting membership would be appointed and include individuals nominated by the federal departments and agencies that conduct, support, or have interests in life-sciences research, including the intelligence community, so as to be comprised of persons reflecting a diversity of scientific and other relevant expertise and interests, including that of the public, in order to provide divergent perspectives. The Commission would have a staff and budget separate from any federal department or agency, and it would be able to convene itself and set its own agenda. Its members would have access to security information, and it would be able to call upon experts from outside government. Ex officio members would represent the interested federal departments and agencies with the caveat that the department or agency that funded the research would be limited in participation in any discussion of that research because of the inherent conflict of interest possessed by a department's or agency's stake in promoting the communication of research that it has found sufficiently important to fund.

The Commission would be the authoritative voice to the United States Government. Federal departments and agencies would be expected to refer to its expertise in decisions whether to fund research identified as potential DURC at the outset and during continuing reviews; by this means, the Commission could provide needed consistency and integrated approaches among the departments and agencies. The Commission also would be available to an institution, whose DURCreview committee refers queries to it, and it also would be available to the editors and publishers of scientific journals who now by default are the arbiters of whether and what data and analyses are published. As would be true of the institutional committee, the Commission's responsibilities would not be unduly burdensome because as a practical matter, very little scientific research constitutes DURC.

Additionally, the Commission should undertake other duties: provide educational materials to institutions and vigorously promote their use; propose federal standards for personnel reliability; recommend approaches regarding the communication of the results of DURC research along the continuum of full disclosure to government classification, including restricted access; and undertake a review of the plethora of the federal statutes, regulations, rules, guidelines, and policies for the purpose of organizing the existing regulatory cacophony, which jumble burdens and discourages scientific research.

As a general, undisputed principle, the unrestricted dissemination of the results of scientific research is critical for the progress of science. When this must be compromised for reasons of national security, there has to be a means by which the results can be shared with trusted and responsible researchers and institutions. Among the unresolved issues, however, is who is responsible for the decision? Should the Commission's guidance control the agency's or department's funding decision subject to an appeal to the Cabinetlevel officer or the National Security Council? By what mechanism will compliance be enforced? The circumstances of the publication of the results of the H5N1 research will be repeated in a multitude of circumstances, e.g., ongoing gain-offunction research with highly pathogenic avian influenza viruses, a new botulinum toxin serotype. Only from an increasing mindfulness of biosecurity issues and DURC, in particular, will come a legitimate approach – and a legitimate approach must be found – to safeguarding public health, safety, and security without compromising the essential vitality of the scientific enterprise.

## **REFERENCES**


*the Potential Misuse of Research Information, 2007 NSABB*. Available from: http://osp.od.nih.gov/ sites/default/files/resources/Framework%20for% 20transmittal%20duplex%209-10-07.pdf

**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 22 May 2014; paper pending published: 09 July 2014; accepted: 29 July 2014; published online: 12 August 2014.*

*Citation: Ehrlich SA (2014) H5N1: a cautionary tale. Front. Public Health 2:117. doi: 10.3389/fpubh.2014.00117*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Ehrlich. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## **Gigi Kwik Gronvall \***

UPMC Center for Health Security, Baltimore, MD, USA \*Correspondence: ggronvall@upmc.edu

#### **Edited by:**

Amanda Jane Ozin, European Centre for Disease Control and Prevention, Sweden

**Reviewed by:**

Amanda Jane Ozin, European Centre for Disease Control and Prevention, Sweden Allan Bennett, Public Health England, UK

**Keywords: DURC, dual-use research, gain-of-function, biosafety, biological research norms**

Dual-use concerns – that legitimate research has the potential to be misused – are inherent in life sciences research. In the past 15 years, numerous scientific papers have raised security questions, and split opinions of scientists, ethicists, and policymakers on whether the research should have been performed or published. Some of the most high-profile examples include the addition of an immunomodulatory gene into the mousepox virus genome, which made the mousepox vaccine ineffective (and suggested that similar manipulations to the smallpox virus genome could make the smallpox vaccine ineffective); the synthesis of poliovirus, the 1918 influenza strain, and the synthesis of a bacterial cell; and "gain of function" (GOF) work on influenza viruses to explore whether H5N1 and other strains have the potential to become transmissible in humans (1–6). Given the increasing ease of manipulating and synthesizing genetic material, and the continued expansion of biological research globally, additional dual-use concerns are certain to arise in the future.

In response to these challenges, policies have been developed in the US to allow a thoughtful pause before beginning or publishing specific areas of research, to consider which aspects of the research are potentially problematic, and evaluate what can be done to mitigate concerns. (7, 8) Though such policies may become more common in biological research, it will be difficult to create hard-and-fast rules about whether or not to conduct potentially dualuse research and publish it in the open scientific literature, because what to do about the work is inextricably tied to the specifics of the research in question. Decisions of whether to fund, perform research on, or publish the next dual-use research of concern article, whether it involves influenza, a different pathogen, or something that is not a pathogen at all, will likely be tipped one way or another by a mix of qualities that are difficult to predict. Some of these qualities are the technical specifics of the research in question; the researchers involved; the urgency of the public health threat that the research is trying to address; an assessment of the danger that the information could be applied toward a biological weapon; and an assessment of the soundness or importance of the research. It is unlikely that in considering these factors, consensus will emerge about what the right course of action should be, or agreement about whether the work will yield important scientific or public health advances. The lack of consensus may lead to some types of work not being funded by one government but pursued by another, or journals with different standards for publication.

Some dual-use research raises concerns, however that can be more easily and broadly addressed than the potential for misuse: the potential for accident. The laboratories, which first demonstrated that H5N1 avian influenza has the potential to become transmissible in mammals have high levels of biosafety training, top-ofthe-line equipment, engineered controls, and health monitoring of the researchers performing the work. Yet as GOF influenza research is repeated elsewhere, or even becomes commonplace, how can people be assured that the same level of attention will be paid? Biosafety is particularly important in these cases because of the potential of an accident to spark a pandemic. Most accidents in biological laboratories

are likely to be limited to the researchers involved and possibly their close contacts, but laboratory acquired infections with transmissible pathogens, such as noncirculating human influenza strains, SARS, or engineered influenza strains could have consequences that go well beyond the laboratory (9, 10).

The good news is that safety is more objectively measured than dual-use research, and there are practical systems to put into place that could raise confidence that concerns are being addressed. There is excellent guidance available for individual researchers, laboratories, and research institutions to adhere to high biosafety practices, as well as biosafety professional training. There are international standards for BSL-1, BSL-2, BSL-3, and BSL-4 labs including what engineering controls should be in place in each level of biocontainment, as well as to manage biorisks within a research institution. (11, 12) The World Health Organization, the Centers for Disease Control and Prevention, professional organizations, and other institutions aim to bring technical information to practitioners, enhance laboratory safety practice, and promote biosafety standards (13–17).

Yet while technical guidance for researchers and institutions is in abundance, a key piece is missing: *national-level* norms for the safety systems necessary to perform such consequential research, to make biosafety a political priority. The next time there is concern about GOF or some other potentially concerning research, it would be helpful to know that the research took place in an environment where there are national standards for the work, including for equipment

maintenance, worker safety training, health monitoring, surveillance, and other myriad activities to help keep the researchers and the larger public safe, and that the nation has an adequate surveillance system in place to identify and limit potential outbreaks that could result from such accidents. Without national-level standards for biosafety and interest in making sure that research institutions that perform potentially high-consequence research adhere to those standards, there will remain insufficient incentives to commit the resources required to achieve high levels of biosafety in individual laboratories and institutions.

The problem of setting biosafety standards for GOF work and other, dual-use research with the potential for consequential accidents does not address the dual-use dilemma in the life sciences, in that such research may lower barriers toward making a biological weapon. But, for legitimate scientific research, increasing international accountability for safety could raise barriers to accidentally achieving the same, horrible result. Even the most dangerous pathogen cannot cause harm to populations if it does not escape containment. Nations which fund this type of scientific research should therefore have the systems in place to provide appropriate levels of safety.

#### **REFERENCES**

1. Jackson RJ, Ramsay AJ, Christensen CD, Beaton S, Hall DF, Ramshaw IA. Expression of mouse interleukin-4 by a recombinant ectromelia virus suppresses cytolytic lymphocyte responses and overcomes genetic resistance to mousepox. *J Virol* (2001) **75**(3):1205–10. doi:10.1128/JVI.75.3.1205- 1210.2001


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 16 May 2014; paper pending published: 02 June 2014; accepted: 02 July 2014; published online: 14 July 2014.*

*Citation: Gronvall GK (2014) National-level biosafety norms needed for dual-use research. Front. Public Health 2:84. doi: 10.3389/fpubh.2014.00084*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Gronvall. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Biosecurity policy in the US: a critical assessment

## **Ori Lev <sup>1</sup>\* and Limor Samimian-Darash<sup>2</sup>**

<sup>1</sup> Department of Public Policy and Administration and Masters Program in Public Policy, Sapir College, D.N. Hof Ashkelon, Israel <sup>2</sup> The Federmann School of Public Policy and Government, The Hebrew University of Jerusalem, Jerusalem, Israel

\*Correspondence: orilev22@sapir.ac.il

#### **Edited by:**

Kathleen Vogel, Cornell University, USA

#### **Reviewed by:**

Kathleen Vogel, Cornell University, USA Kavita M Berger, American Association for the Advancement of Science, USA

**Keywords: dual use research of concern, risk-benefit assessment, biosecurity policy, H5N1, US Government Policy**

This commentary will critically evaluate the *US Government Policy for Oversight of Life Sciences Dual Use Research of Concern* with a special focus on the process of assessing the risks and benefits of studies that are deemed to be dual use research of concern (DURC). Assessing the risks and benefits of DURC studies is probably the most complicated part in implementing the policy. Curiously, little attention has been paid to this complex process. This paper details how this process is conducted and points out a major challenge it faces. We will suggest that this challenge is difficult to resolve thereby requiring further policy development.

On March 29, 2012, the US Government issued *The United States Government Policy for Oversight of Life Sciences Dual Use Research of Concern* (1). The policy was published after months of controversy over the issue of whether studies that enhanced the transmissibility of the highly pathogenic avian influenza (HPAI) H5N1 viruses should be published and if so in what form (2). The main concern these studies have generated was that if they are to be published in full malevolent actors might misuse the information included in them to construct a deadly virus. Issuing the policy was, at least in part, a way for the government to demonstrate that it is taking control of the events and is pursuing steps that would mitigate some of the concerns that were raised about these studies.

Importantly, the policy gave the US Government tools; it lacked when the H5N1 controversy erupted. Examples of such tools are listed below.

The new policy, for example, provides the government with the authority to terminate funding of research that is deemed too risky (1). This is an extreme measure that is unlikely to be used; however, including it in the policy reveals not only the sense of pressure government officials felt given the circumstances, but also their belief that the government should have a very wide scope of tools that could be employed to govern this research. Other tools the policy provides are related to determining the biosafety conditions under which the research is done and a periodic assessment of the research for its potential to be DURC. This periodic assessment is a direct result of the H5N1 controversy, in which it seemed the US Government was caught by surprise by the ensuing crisis. The periodic review allows the government to be constantly updated on the state of the research portfolio it funds. These steps are crucial given the potential that more DURC studies are likely to be conducted.

To decide which of these steps should be applied the policy articulates a four-step process (1). The first step is to determine whether the research involves a pathogen from a list of 15 infectious agents and toxins that are deemed most lethal. The second step is to determine whether that research performs an experiment that falls under any of the seven categories of experiments listed in the policy.

If the study meets these two criteria, a third step is pursued, specifically, determining whether the study meets the DURC definition set out in the policy. The definition is as follows:

"DURC is life sciences research that, based on current understanding, can be reasonably anticipated to provide knowledge, information, products, or technologies that could be directly misapplied to pose a significant threat with broad potential consequences to public health and safety, agricultural crops and other plants, animals, the environment, materiel, or national security."

This definition was adopted with a few revisions from an earlier definition that the National Science Advisory Board for Biosecurity has articulated in its report "Proposed Framework for the Oversight of Dual Use Life Sciences Research: Strategies for Minimizing the Potential Misuse of Research Information" (3):

"Research that, based on current understanding, can be reasonably anticipated to provide knowledge, products, or technologies that could be directly misapplied by others to pose a threat to public health and safety, agricultural crops and other plants, animals, the environment, or materiel."

The definitions are similar but have two differences worth pointing out.

The first difference is that the phrase "by others" was eliminated from the new DURC definition. This phrase was originally intended to express the idea that scientists are well-intended when conducting research, while "others," malevolent actors, might misapply their research. Eliminating the phrase "by others" could be understood as suggesting that scientists themselves could misapply research findings. This is probably the result of the 2001 U.S. Anthrax attacks, which were allegedly undertaken by a scientist and resulted in 5 deaths and 17 injuries (4). Another possible

reason for this change is the acknowledgment that scientists might accidentally misapply their research.

The second difference is the addition of the phrase "a significant threat with broad potential consequences" to the new definition. This addition is intended to help those that assess particular studies; it aims to provide them with a more specific criterion with which they can assess the risks of misuse. However, this addition though intended to help is still vague and is likely to be interpreted in an inconsistent way thereby endangering the effectiveness of the policy. This vagueness, we propose, should be addressed. One way in which this could be achieved is by entirely eliminating the third step and moving directly to the fourth step, which calls for a robust risk benefit assessment. This assessment is the only way in which the magnitude of the risks and their likelihood will be determined; the third step is redundant and confuses the process.

As said, the fourth step of the policy calls for an assessment of the risks and benefits of the studies that were determined to be DURC. The risk benefit assessment is utilized to decide whether any of the tools the policy provides ought to be used: should the study design be modified, should it be done under different conditions, should its publication be subject to any limitations, should its funding be terminated?

However, the fourth step presents a serious challenge. A challenge that we would argue ought to be seriously considered and addressed if possible. To be clear, this challenge is related to the policy as it is currently set out. The policy places the responsibility for conducting the risk benefit assessment in the hands of scientists; however, generally speaking scientists lack the knowledge and capabilities required for assessing the risks of misuse.

A risk benefit assessment for DURC is unique in its focus on the risks of misuse by malevolent actors (5). In other words, a DURC assessment is essentially a biosecurity assessment. Yet, the scientific community is not equipped with the knowledge, expertise, and capabilities to conduct a security assessment (6).

The scientific community is well placed to assess the public health benefits of their research. They can provide sound assessments of the likelihood of the benefits and their magnitude. We would also argue that they are well placed to assess the magnitude of the harms if the research is misused. They might even be able to provide a sound assessment of the feasibility of misusing the information. That is, they are able to attest to the technical abilities needed and whether they are easy or difficult to acquire.

Yet, scientists are incapable of assessing the likelihood that a given study would be misused; they do not have access to such information. This kind of information is not publically available. In particular, they have no way of knowing if there is any group with the intention of misusing the research information or materiel. They also lack any information regarding the capabilities of any group that might have the intention to misuse research findings. Moreover, they do not have access to knowledge about efforts to prevent groups who intend on doing harm and the success of such efforts. Without this information, the scientific community cannot assess DURC.

The kind of information that is needed for a comprehensive assessment would only exist within the security and intelligence community. This kind of information is sensitive and thus it is unlikely that it will be shared with the scientific community unless a reliable mechanism to convey such information is established.

However, one might only imagine the difficulties of establishing such a mechanism. Scientists would have to get security clearances; they would also have to be trained on how to interpret such information reliably (6). This has been done to a limited extent through the creation of the Biological Sciences Experts Group (BSEG) in which a limited number of scientists and science administrators receive clearance and are briefed from time to time (7). Yet this model cannot meet the demands of the new policy as it is too limited in scope and authority. The security and intelligence communities, it is safe to say, are unlikely to agree to extend this type of mechanism. They would object to sharing sensitive and classified information with a growing number of people outside their institutions. The risks of such a mechanism are too high.

One might suggest a middle way in which security personnel would participate in the assessment process and provide input on whether a given study has high or low likelihood to be misused. But even this middle way would be problematic as scientists would probably demand greater transparency if they are to accept any limitations on their freedom to pursue scientific inquiries. Greater transparency, however, is unlikely to be forthcoming as providing more detailed information could have detrimental effects to the intelligence operations.

This divides between the interests of the scientific community and the security and intelligence communities must be bridged if we are to address the DURC challenge effectively. Leaving the policy as it currently stands seems unsustainable. This is because it would lead to problematic outcomes. Without information on the likelihood of misuse scientists would have to turn to"educated guesses,"under such conditions they are likely to make two kinds of mistakes. First, they might place low likelihood of misuse on studies that have a high chance to be misused, thereby endangering national security. Second, they might curtail important research on the grounds that it encapsulates high risks of misuse although in reality such research is unlikely to be misused thereby harming important advances that could benefit public health.

As suggested, to avoid these potential mistakes a way for the security establishment and the scientific community to collaborate must be sought. If such a mechanism is impossible to set up, policy makers must convey to the public that the DURC policy has limits. Moreover, scientists conducting DURC reviews must be aware that their determinations are subject to the kinds of mistakes that we pointed out. Should they then err on the side of caution or not is a difficult question that should receive close scrutiny (8, 9).

It is important to note that there are still policy tools available to the government if the dangers of misuse are increasing. It is the responsibility of the security establishment to constantly be on the lookout for malevolent actors who intend to misuse scientific information. If these risks are increasing dramatically they could demand that certain lines of research be done in a classified way. The scientific community in the US as well as in other countries is unlikely to easily endorse this approach, yet it did so already when it became clear that openly conducted nuclear physics research poses severe risks to society (10). This is not the situation we are facing with regard to life sciences research. Yet, it is important to realize that the limitations of the DURC policy can be addressed in an alternative way if warranted.

To conclude, the US Government policy for the oversight of DURC is an important step in attempting to balance the need for scientific progress and safeguarding our societies. Yet the policy is formulated in such a way that the risks of misuse cannot be accurately assessed. To fulfill its goal further policy development efforts are necessary.

### **REFERENCES**


*for Minimizing the Potential Misuse of Research Information*. National Institutes of Health (2007). Available at http://osp.od.nih.gov/sites/ default/files/resources/Framework%20for% 20transmittal%20duplex%209-10-07.pdf


10. Evans GE. Contrasting dual-use issues in biology and nuclear science. In: Rappert B, Selgelid MJ editors. *On the Dual Uses of Science and Ethics – Principles, Practices, and Prospects*. Anberra: ANU E Press (2013). p. 255–73.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 07 May 2014; accepted: 20 July 2014; published online: 04 August 2014.*

*Citation: Lev O and Samimian-Darash L (2014) Biosecurity policy inthe US: a critical assessment. Front. Public Health 2:110. doi: 10.3389/fpubh.2014.00110*

*This article was submitted to Infectious Diseases, a section of the journal Frontiers in Public Health.*

*Copyright © 2014 Lev and Samimian-Darash. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## ADVANTAGES OF PUBLISHING IN FRONTIERS

FAST PUBLICATION Average 90 days from submission to publication

#### COLLABORATIVE PEER-REVIEW

Designed to be rigorous – yet also collaborative, fair and constructive

RESEARCH NETWORK Our network increases readership for your article

## OPEN ACCESS

Articles are free to read, for greatest visibility

## TRANSPARENT

Editors and reviewers acknowledged by name on published articles

## GLOBAL SPREAD Six million monthly page views worldwide

## COPYRIGHT TO AUTHORS

No limit to article distribution and re-use

IMPACT METRICS Advanced metrics track your article's impact

## SUPPORT By our Swiss-based editorial team

EPFL Innovation Park · Building I · 1015 Lausanne · Switzerland T +41 21 510 17 00 · info@frontiersin.org · frontiersin.org