Please tell us what you think of this issue!  Feedback

Bulletin, October/November 2008


Special Section


Integration and Implementation Sciences: A Framework for Thinking about Standards in Complex, Problem-Based Research


by Caryn Anderson

Caryn Anderson is integration research manager for Program 4 of the ARC Centre of Excellence in Policing and Security at the Australian National University in Canberra, Australia (www.ceps.edu.au/). She has been contributing to the development of integration and implementation sciences (www.anu.edu.au/iisn/) since 2003. Most recently she was doctoral studies program manager and lecturer at the Simmons College Graduate School of Library and Information Science. She holds leadership roles in ASIS&T, including co-founding and managing the International Calendar of Information Science Conferences (http://icisc.neasist.org/). She can be reached at caryn.anderson<at>anu.edu.au.

I love standards – absolutely love ’em. Maybe it’s the hippy in me… I just want everyone to get along. Maybe it’s the accountant in me… I want everything to make sense, work together, be predictable and come out balanced with nothing left over. Maybe it’s the slacker in me… I don’t really want to have to think that hard about how things work or whether they will work, and I certainly don’t really want to DO any work if I don’t have to. Maybe it’s the commander in me… I want operations to be executed quickly, efficiently and completely. Theoretically, standards can facilitate all these things, whether they are standards for information exchange and organization or standards of social behavior. But they also have limits, and it is useful to zoom up periodically for a helicopter view of the landscapes in which standards operate. This perspective helps us to understand where and when standards work well and when alternative or multi-dimensional approaches may be called for. 

As information scientists, we are usually asked to develop better ways to facilitate the production, exchange and preservation of relatively defined types of information. Both technological and metadata standards are essential to our work – whether for facilitating Internet and data transfer (Z39.50) or for standardizing metadata (METS, MARC, Dublin Core, MPEG) or for electronic resource management (ERM) or any of a multitude of other critical functions. In recent years, many of us have become engaged with operations and organizations working on highly complex and multidisciplinary problems where the types, sources and uses of information are broadly defined or unpredictable including crisis response, international development and national security. To be effective in these situations, it is useful to view our roles in a broader context and take advantage of concepts and methods specifically designed to address these kinds of problems. In this way, we can expand our arsenal of approaches to managing the information needs of multidisciplinary problem-solving teams. 

Integration and implementation sciences (I2S) is an emerging discipline focused on the needs of researchers who seek to integrate knowledge across disciplines and stakeholder groups in approaching real-world problems. I2S brings together and develops concepts and methods in four critical domains that have been only partially and intermittently covered by different disciplinary and practice areas. I2S focuses on enhancing the following: 

  • Fresh thinking on intractable problems
  • Integration of disciplinary and stakeholder knowledge
  • Understanding and management of ignorance and uncertainty
  • The provision of research support for decision-making and practice change. [1, p.2]

When it comes to solving big, complex, thorny, “wicked” problems, there isn’t really anything routine or basic about them and thus it becomes difficult to come to agreement on a way forward or even how to define the problem in the first place. With problems like AIDS, sustainable development or terrorism there is not one community that needs to come to agreement, but many communities. Each of the disciplinary and stakeholder groups come at the problem with their own perspectives and their own set of epistemological, theoretical and practical standards of operation and behavior. For example, in the problem of how to develop land at the edges of cities, disciplinary experts may include ecologists, economists, hydrologists, sociologists, soil scientists, demographers and so on. Relevant stakeholders include those affected, such as farmers and recreational users of undeveloped land (whose activities are impinged on by the expansion of housing) and families requiring housing. Stakeholders also include those in a position to make decisions about the issue, such as government policy makers, local councilors, regulators and land developers. 

In order to carefully consider, weigh and integrate all these perspectives, it is important to step back and be clear about context and intentions of the problem-solving effort, the boundaries of the problem, the relationships among all the interconnected components, the degree of ignorance or uncertainty surrounding the issue (what don’t we know and what can we do about it) and issues around implementing any potential strategies that emerge from integrating the relevant research. Information scientists who can comprehend the full scope of these issues will have a significant advantage in designing effective information management and exchange strategies, including the development and/or use of standards. By understanding the aspects, demands and iterative phases of complex, multidisciplinary problem solving, they can craft information solutions that support these activities more effectively.

The reminder of this article briefly describes I2S and the diverse ways information scientists can think about standards in the context of multidisciplinary, problem-based information environments. Professor Gabriele Bammer leads the development of I2S, and the author is grateful to her for permission to quote extensively from her recent work “The case for a new discipline of integration and implementation sciences (I2S),” Integration Insights #6 [1]. Some material is also quoted from her earlier (2006) publication “A systematic approach to integration in research,” Integration Insights #1 [2]. Both articles are available at www.anu.edu.au/iisn.

In some cases the sections of these works quoted here contain material originally published by the author or her colleagues in earlier works. The interested reader can consult [1] or [2] for these citations. Material that I have added or interpolated is enclosed in brackets. 

Fresh Thinking
In “The case for the new discipline of integration and information sciences,” Bammer explains: 

Many of the real-world problems that societies face are intractable, so that sparking innovative thinking about them is essential. For example, how do we balance the rights of individuals, but prevent abuse of legal safeguards by criminals; how do we motivate young people to become engaged productive citizens; how do we encourage independence in medical research but restrict the development of potentially dangerous viruses? I2S develops concepts and methods that can catalyze innovative ways forward in thinking about such problems, leading to more effective policy and practice approaches. [1, p.3]

We often think of standards as utilitarian, but standards can also support creativity and innovation. If communities agree that they will all do the basic routine things in the same way, then more energy can be put into creative expression of the things that make us unique and interesting. The rule of law actually gives us more freedom because we can trust that others will behave in certain ways (or we can claim restitution for violations) rather than having to be on our guard about every interaction. A standard currency allows us to exchange goods and services easily with many people without spending great amounts of time and energy finding the one person who has something we want for something we have to give. Standards for information production, integration or exchange are usually designed in ways that both reduce the time and energy necessary for routine activities. But it is also worth exploring whether they can be designed in ways that enable easy and/or serendipitous recombination, juxtaposition or visualization of information to stimulate new ideas.

Integrating Knowledge
As described above, effective research related to complex problems usually requires integrating knowledge from a variety of sources. This activity is rooted in the concept of systems-based thinking – acknowledging the interconnectedness of the elements of a problem. Effective integration founded on system-based thinking involves three iterative activities according to I2S:

1. Improving scoping, problem framing and boundary setting, which define how a real-world problem will be approached and which perspectives will be included. 
2. Integrating knowledge effectively. There are five classes of methods:

  • Dialogue-based – Almost all integration processes involve people engaging directly with each other. There are many methods for bringing people and their ideas together. 
  • Model-based – The process of developing a model (system dynamics, agent-based, econometric, scenario, etc.) brings together different view points. 
  • Product-based – The atomic bomb is a product that brought together experts from many disciplines (from physics to engineering to military strategy) to combine their knowledge. 
  • Vision-based – The World Commission on Dams had a vision for more equitable and sustainable dam-building globally and integrated views of engineers, hydrologists, politicians, potentially displaced peoples, etc. to develop their recommended standards for dam development. 
  • Common-metric-based – By translating diverse types of data and perspectives to a common metric like the purchasing parity dollar or ecological footprint, strategies can be compared based on “cost” or “impact” on the society. 

3. Identifying and managing synergies and conflicts between, for example, different values, interests, and epistemologies. [1, p. 3; the examples of methods are drawn from 2, p. 3].

Many individuals or teams already integrate intuitively or on a semi-structured basis, but for integration to be truly effective (and able to be improved) the integration process itself must be made explicit. An agreed standard way for describing and analyzing an integration endeavor is based on a simple framework comprising six questions:

1. What is the integration aiming to achieve and who is intended to benefit?
2. What is being integrated?
3. Who is doing the integration?
4. How is the integration being undertaken?
5. What is the context for the integration?
6. What is the outcome of the integration? [2, p.1]

The process of developing a standard can be an excellent method for integrating knowledge across disciplines and stakeholders. In some ways it is like a conceptual model-based or product-based method of integration. The common experience of integrating multiple existing standards is also an integration process that can benefit from attention to the above concepts and methods. Successful standards (adopted widely) have usually considered and integrated technical requirements and restrictions as well as diverse user needs and preferences. They have also anticipated political, legal and economic responses in the immediate and long-term future. Standards for the Internet or air traffic radio communications address the physics of data exchange while simultaneously considering economic and human impacts and the associated political priorities in global contexts. 

By understanding the integration framework and using it to explicitly describe or develop information-related standards for a particular problem, information scientists can move beyond roles as information managers to contribute directly to problem-solving processes. Information scientists may also be able to recommend modifications to existing standards, crosswalks or the selection of more appropriate standards to better facilitate integration activities or avoid marginalization of certain groups or approaches.

Understanding Ignorance and Uncertainty
Bammer has this to say about managing ignorance and uncertainty (with my additions and examples):
Real world problems also have many different types and aspects of ignorance and uncertainty embedded within them and there is currently no systematic way of recognizing and dealing with all these attributes
[what we don’t know and what we do about it]. 

Managing unknowns is just as important as making maximum use of what is known. This involves concentrated attention to the nature of ignorance and uncertainty, including the irreducibility of some uncertainties. [For example, we will never know exactly how many members of Al Qaeda there are at any given time]. It also involves understanding how people think about and respond to uncertainty, for example through exploration of the metaphors they use, their motivations and even their moral orientations. [For example, uncertainty is an opportunity for discovery, creativity or change vs. uncertainty is blindness and bad and should be eliminated.] 

Further it involves examining different ways of coping and managing under uncertainty, especially in relation to meeting the adaptive challenges posed by uncertainty. The possibilities range from outright denial or banishment to acceptance and even exploitation of uncertainty. Each kind of response can be shown to have strengths and weaknesses that indicate when it is likely to be adaptive.

While different disciplines and practice areas have established ways of dealing with ignorance and uncertainty – for example, statisticians focus on probability-based approaches, intelligence analysts focus on distortion, historians take taboo into account and psychologists think about norms – no discipline or practice area has the role of bringing all of these different approaches together. [1, pp. 3-4] 

Technical or metadata standards by their very nature assume a fixed domain of information and operation. Some, like the Dublin Core metadata standard, try to be as simple and extensible as possible so as to allow for diversity and unknown developments in the future. This flexibility, however, also means that the way different archives use each metadata element may be inconsistent. This can be problematic and confusing for users accessing multiple archives [via OAI-PMH (Open Archives Initiative-Protocol for Metadata Harvesting) for example] because available metadata in results lists may be confusing or data may seem to be missing. The Moving Picture Expert Group’s MPEG21 standard, on the other hand, is so precise that very little is left unconsidered. This extreme specification, however, has compromised its utility, as it is only useful to a very narrow group in its present form. For others it is too complex and confusing for their needs and is like using a 747 to cross the street. In developing or utilizing information standards, it is useful to consider what has been ignored and why and how various types and degrees of uncertainty are accommodated, neutralized or could be exploited.

Translating Research into Policy and Practice Change
I2S also concerns itself with practical change. Bammer observes the following:

In terms of providing research support for decision making, over the last decade or so, there has been growing interest in the lack of impact much research has on policy making and how this can be remedied. This is a subset of a larger problem, namely how to increase consideration of research knowledge in decision making more generally, not only by policy makers, but also by business leaders, community activists, non-government organizations and professional groups. 

Such considerations have five elements: a) understanding decision making processes, for example, government policy making or business commercial decision making; b) appreciating the attributes of influential research; c) delineating different types of researcher-decision maker engagement – ranging from one-way communication to the co-production of knowledge – and their strengths and weaknesses; d) understanding how institutions can influence which research is taken up by decision makers and e) developing more effective ways to evaluate research support for decision making. [1, p. 4]

Often the work of information scientists is narrowed to one or a few small groups in the information cycle: knowledge producers such as researchers, disseminators such as publishers, managers/brokers such as librarians, and users such as the public or decision makers. Skilled information scientists spend a great deal of time identifying the primary users so that their tools and systems best serve those groups. It is also useful to consider whether the information, for which we are being asked to design better management systems, will also ultimately need to be translated for other users further down the line. If so, we can ask ourselves how we might be able to design the standards we need for our primary purpose to also facilitate that additional transformation.

Thinking about Information-Related Standards in a Larger Context
For information scientists developing, using or integrating information standards in the realm of complex, multidisciplinary research and development teams, it is useful to ask if the standards involved can indeed be designed to enhance the foci of I2S that we previously listed:

  • Fresh thinking on intractable problems;
  • Integration of disciplinary and stakeholder knowledge;
  • Understanding and management of ignorance and uncertainty; and
  • The provision of research support for decision making and practice change.

It may not be necessary to service all these goals in any one information project, but a conception of the big picture in multidisciplinary, problem-based research can only help information scientists to be more comprehensive, efficient and useful in their roles related to information standards.

Acknowledgement
Professor Gabriele Bammer leads the development of integration and implementation sciences, and the author is grateful for her thoughtful review of this article and her gracious permission to include the quoted descriptions of I2S, from the resources below.

Resources Mentioned in the Article
[1] Bammer, G. (2008, May). The case for a new discipline of integration and implementation sciences (I2S). Integration Insights #6, Retrieved August 29, 2008, from www.anu.edu.au/iisn.

[2] Bammer, G. (2006, September). A systematic approach to integration in research. Integration Insights #1. Retrieved August 29, 2008, from www.anu.edu.au/iisn