Kirstie L. BellmanAerospace Corporation · CSD AISC
Kirstie L. Bellman
Ph.D.
About
133
Publications
14,179
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
1,699
Citations
Introduction
I'm a Prin. Scientist at The Aerospace Corp., not-for-profit FFRDC.. Recently I was Chief Eng. for the DARPA META pgrm for new (formal) design methods. It drew upon work from several of my DARPA prgrms. I was a PM in '93 for Domain-Specific Software Archs, Prototech, Formal Foundations & the Computer-Aided Education and Training Initiative (CAETI). My work spans psych, neurophys, CS and math. My passion is "reflective architectures" that manage their own resources and adapt behavior.
Additional affiliations
August 1993 - August 1997
Position
- Manager
Description
- Worked largely in the areas of domain-specific languages and architectures, rapid prototyping, formal languages, and computer-aided education and training.
April 1983 - present
Description
- Design environments Advanced modeling methods for large complex adaptive systems; verification and validation virtual worlds domain specific languages Wrappings and reflective architectures
June 1979 - April 1983
Description
- Crayfish and guinea pig neurophysiology: especially adaptive conflict behavior and brain modeling Qualitative dynamics and adv math methods applied to neurophysiol
Publications
Publications (133)
Complex Adaptive Systems (CAS) are difficult to
understand and predict. Certain CAS phenomena can only be
observed at certain abstraction levels – e.g., swarm dynamics
cannot be analysed by only tracking one individual. In most
current systems the “right” abstraction levels for each question
asked, or each problem solved, are determined at design t...
The research initiative “self-improving system integration” (SISSY) was established with the goal to master the ever-changing demands of system organisation in the presence of autonomous subsystems, evolving architectures, and highly-dynamic open environments. It aims to move integration-related decisions from design-time to run-time, implying a fu...
“Self-improving system integration” (SISSY) is a research initiative that aims to master the ever-changing demands of system organisation in the presence of highly-dynamic autonomous subsystems, evolving architectures, and unpredictable open environments. In contrast to traditional system modelling, design and development – typically performed by e...
In this paper we make the case for the new class of Self-aware Cyber-physical Systems. By bringing together the two established fields
of cyber-physical systems and self-aware computing, we aim at creating systems with strongly increased yet managed autonomy,
which is a main requirement for many emerging and future applications and technologies. Se...
![Fig. 1. Conceptual view of an individual self-integrating system as in [9].](profile/Sven-Tomforde/publication/335074053/figure/fig1/AS:814924494286848@1571304548295/Conceptual-view-of-an-individual-self-integrating-system-as-in-9_Q320.jpg)
In human society, individuals have long voluntarily organized themselves in groups, which embody, provide and/or facilitate a range of different social concepts, such as governance, justice, or mutual aid. These social groups vary in form, size, and permanence, but in different ways provide benefits to their members. In turn, members of these group...
We define the notion of “self-aware computing” and the relationship of this term to related terms such as autonomic computing, self-management, and similar. The need for a new definition, driven by trends that are only partially addressed by existing areas of research, is motivated. The semantics of the provided definition are discussed in detail e...
Self-aware computing systems exhibit a number of characteristics (e.g., autonomy, social ability, and proactivity) which have already been studied in different research areas, such as artificial intelligence, organic computing, or autonomic and self-adaptive systems. This chapter provides an overview of strongly related concepts and areas of study...
In this chapter, we propose a methodology to analyse the different levels of self-awareness present in distinct types of computing systems and architectures, investigate the level of self-awareness that is already present in those systems and architectures, and describe some transition strategies to increase the level of self-awareness in these sys...
In this chapter, we review the state of the art in self-aware computing systems with a particular focus on software architectures. Therefore, we compare existing approaches targeting computing systems with similar characteristics as self-aware systems to the architectural concepts for single and collective self-aware systems discussed in the previo...
![Fig. 3.1 Lewis et al.'s [34] levels of computational self-awareness...](profile/Kyrre-Glette/publication/312597842/figure/fig1/AS:616386065874974@1523969298400/Lewis-et-als-34-levels-of-computational-self-awareness-right-describe-various_Q320.jpg)
Increased self-awareness in computing systems can be beneficial in several respects, including a greater capacity to adapt, to build potential for future adaptation in unknown environments, and to explain their behaviour to humans and other systems. When attempting to endow computing systems with a form of self-awareness, it is important to have a...
Making computing systems self-aware calls for appropriate architectural designs of such systems that allow developers to explicitly decide and reason about the system’s self-awareness capabilities. In this context, a critical issue is the development of appropriate reflections that enable self-awareness and that impact the architectural design. Thi...
This chapter aims to discuss the architectural aspects relevant to collectives of self-aware computing systems. Here, collectives consist of several self-aware computing systems that interact in some way. Their interactions may, potentially, lead to the formation of a self-aware collective of systems. Hence, the chapter defines different types of i...
The purpose of this chapter is to discuss why self-aware systems must pay special attention to self-modeling capabilities, clarify what is meant by both strong and weak self-modeling, and describe some of the defining characteristics of self-modeling. This chapter is also about self-management via run-time model creation by the operational system,...
This chapter discusses the importance of assessing self-awareness of a system and different approaches and aspects on how to enable a human as well as a machine to perform such an assessment. The chapter also elaborates on the different requirements and constraints for an assessment. Furthermore, this chapter outlines how these requirements and con...
Biological organisms show a remarkable flexibilityin how they organize their behavior and adapt it tochanged circumstances. In this paper, we apply some ofthe more interesting concepts from biological theory tocyber-physical systems, especially those in such remoteor hazardous environments that we cannot expect ourcontrol of them to be adequate for...
Current trends in information and communication technology show that systems are increasingly influencing each other - which is seldom completely anticipated at design-time. As a result, mastering system integration with traditional methods becomes infeasible due to the resulting complexity. In this paper we argue that self-improving system integra...
No system in the real world can compute an appropriate response in reaction to every situation it encounters, or even most situations it is likely to encounter. Biological systems address this issue with four strategies: (1) a repertoire of already computed responses tied to a situation recognition process, (2) organized in a response-time hierarch...
Space systems have always been model-based. The difference now is the depth and extent of the modeling and how much models and model analyses have begun to substitute for traditional design and development activities. In this paper, we consider some advances in model-based design, engineering and development (MBDED) that show great promise for comp...
In order for a self-organizing real time (SORT) system to produce real time behavior that is "good enough," it must have the ability to trade off among competing performance metrics, of which time is only one. In this paper we discuss what some of those trade-offs are at both "design time" and during operations, present some examples of how biologi...
Organic Computing (OC) and other research initiatives like Autonomic Computing or Proactive Computing have developed the vision of systems possessing life-like properties: they self-organize, adapt to their dynamically changing environments, and establish ...
This paper is about the requirements and architectural considerations that provide a SORT system with processes for observing, modeling, simulating, predicting, deciding, and acting in an external environment. For our purposes, ``real time'' (RT) means coordinated with an external source of time or with sequences of events over which the system has...
We describe an approach that potentially could dramatically improve the ability to test Fault Management System (FMS) specifications and software by re-expressing them as a rule-base, and then applying previously developed verification and validation methods for rule-based software systems. We then generalize the strategy underlying this approach t...
The term “emergence” is usually used to mean something surprising (and often unpleasant) in the behavior of a complex system,
without further qualification. Designers of OC systems want to manage emergence in complex engineered systems so that it can
contribute to, or even perhaps enable, accomplishing the system’s performance goals. That is, OC de...
Organic Computing (OC) has become a challenging vision for the design of future information processing systems: As they become increasingly powerful, cheaper and smaller, our environment will be filled with collections of autonomous systems equipped with sensors and actuators to be aware of their environment, to communicate, and to organize themsel...
Our knowledge of physics, chemistry, engineering, even psychology have all been used to develop weapons and approaches for
winning armed conflicts. Especially the information sciences have been developed with a tremendous amount of funding and much
of the initial motivation stemming from military needs and later developed with its funding. Robert T...
Organic Computing has emerged recently as a challenging vision for future information processing systems, based on the insight that we will soon be surrounded by systems with massive numbers of processing elements, sensors and actuators, many of which will be autonomous. Because of the size of these systems it is infeasible for us to monitor and co...
Organic Computing has emerged recently as a challenging vision for future information processing systems, based on the insight that we will soon be surrounded by large collections of autonomous systems equipped with sensors and actuators to be aware of their environment, to communicate freely, and to organize themselves in order to perform the acti...
In this paper, we offer an approach to software evolvability based on our wrapping infrastructure for integration in constructed complex systems. We believe that the self-modeling systems that we have built using wrappings may be able to manage their own evolution to some extent. In a wrapping-based system, the design decisions are visible in the r...
We are developing a distributed system for the tracking of people and objects in complex scenes and environments using biologically based algorithms. An important component of such a system is its ability to track targets from multiple cameras at multiple viewpoints. As such, our system must be able to extract and analyze the features of targets in...
Many sensor systems such as security cameras and satellite photography are faced with the problem of where they should point their sensors at any given time. With directional control of the sensor, the amount of space available to cover far exceeds the field-of-view of the sensor. Given a task domain and a set of constraints, we seek coverage strat...
Many vision research projects involve a sensor or camera doing one thing and doing it well. Fewer research projects have been done involving a sensor trying to satisfy simultaneous and conflicting tasks. Satisfying a task involves pointing the sensor in the direction demanded by the task. We seek ways to mitigate and select between competing tasks...
Summury
Increasingly diverse and complex computer-based support underlies critical human processes, such as education, commerce, medicine, science, defense, and government. These systems are fed massive amounts of data and due to the complexity and size of these systems, they are integrating and interfacing with each other with less human oversight...
Based on the strengths and weaknesses of many current applications, this chapter discusses how to make virtual worlds (VWs) “real-world capable.” With sufficiently realistic data and dynamic processing capabilities within VWs, we could do medical interventions, analysis, engineering, invention, and design. This will require creating systems with so...
One of the important components of a multi sensor "intelligent" room, which can observe, track and react to its occupants, is a multi camera system. This system involves the development of algorithms that enable a set of cameras to communicate and cooperate with each other effectively so that they can monitor the events happening in the room. To ac...
We discus a tool kit for usage in scene understanding where prior information about targets is not necessarily understood. As such, we give it a notion of connectivity such that it can classify features in an image for the purpose of tracking and identification. The tool VFAT (Visual Feature Analysis Tool) is designed to work in real time in an int...
In view of the growing complexity of computational tasks and their design, we propose that certain interactive systems may be better designed by utilizing computational strategies based on the study of the human brain. Compared with current engineering paradigms, brain theory offers the promise of improved self-organization and adaptation to the cu...
This paper is about strategies for developing new approaches to solving difficult problems in complex computing system design,
based on a very strong use of abstraction and reflection. We address problems in the design and development of Constructed
Complex Systems, which are large, heterogeneous, usually distributed, systems managed or mediated by...
The context of this paper is system engineering of the software needed in constructed complex systems, which are large systems managed or mediated by computing systems. In this paper, we describe an approach to systems engineering design and modeling that allows difficult design decisions to be delayed until enough information is available to make...
Integration Science is the study of the processes and phenomena required to build effective combined models from disparate component or partial models. Integration is usually treated very differently in different subject domains, but there are several commonalities across all of them: the explicit identification and processing of assumptions, colle...
The Theoretical Biology Program initiated by Robert Rosen is intended to identify the key theoretical characteristics of organisms, especially those that distinguish organisms from mechanisms, by looking for the proper abstractions and defining the appropriate relationships. There are strong claims about the distinctions in Rosen's book ``Life Itse...
This paper will review and examine the definitions of Self-Reflection and Active Middleware. Then it will illustrate a conceptual framework for understanding and enumerating the costs of Self-Reflection and Active Middleware at increasing levels of Application. Then it will review some application of Self-Reflection and Active Middleware to simulat...
Our work on integration of data and knowledge sources is based in a common theoretical treatment of 'Integration Science', which leads to systematic processes for combining formal logical and mathematical systems, computational and physical systems, and human systems and organizations. The theory is based on the processing of explicit meta-knowledg...
Enormous amounts of information are produced every day, all over the world, but very little of it is true. In this paper, we describe the modeling component of a current events analysis system that combines statistical and Knowledge-Based approaches to natural language analysis with new modeling and architecting techniques to produce a Knowledge-St...
System engineering practices have reduced the rate and effect of errors in complex, heterogeneous computing systems, but there are still areas in which they are insufficient. Space systems and other research systems have unique problems that preclude the use of conventional techniques, since the requirements are not all available until after the sy...
Investigates time management for distributed simulations in active networks. Time management is essentially the computation of the lower-bound time stamp (LBTS) across federates in a distributed simulation, including in-transit (in-flight) messages. We show that LBTS computation is an instance of the distributed termination detection (DTD) problem...
This paper is about systems with complete models of themselves (down to some very low level of detail). We explain how to
build such a system (using careful system engineering, and our Wrapping approach to fiexible integration infrastructures for
Constructed Complex Systems), and why we want to do so (it is at least interesting, and we believe it i...
Over the last seven years, we have been exploring the use of collaborative virtual environments called Virtual Worlds (VWs) as a new type of testbed for experimenting with ways of organizing and integrating diverse types of computational and human processes. In other papers, we have discussed a wide variety of applications to fields as diverse as e...
This paper is an overview of our research program in intelligent systems. Our object of study is constructed complex systems, which are software and hardware systems mediated or managed by computers. We describe how biological systems provide stiff competition for constructed complex systems in the areas of autonomy and intelligence, robustness, ad...
The purpose of this paper is to raise some hard and interesting questions about the new relationships possible between humans and their artifacts:
What happens when we can have collaborative relationships with our responsive and knowledge-bearing artifacts?
What happens when group minds are mediated through new types of computing system that can su...
In this paper, we describe the influence of Richard Bellman's work on our research. The influence is primarily felt in the attitude we take towards the research, which is that whenever the right mathematical tool for a problem that we are considering is not available, then some mathematical invention is required. We describe our research studies in...
It is well known that complex systems are difficult to design, implement, and analyze. Component-level verification has improved to the point that we can expect to produce formal or nearly formal verification analyses of all components of a complex system. What remains are the system-level verifications which we believe can be improved by our appro...
In this paper, we describe our wrapping technique for dynamic
integration in complex heterogeneous systems, which is based on
explicit, machine-interpretable descriptions of all software, hardware,
and other computational resources in a constructed complex system, and
active integration processes that select, adapt, and combine these
resources for...
We discuss the following measurable characteristics of intelligent
behavior in computing systems: (1) speed and scope of adaptability to
unforeseen situations, including recognition, assessment, proposals,
selection, and execution; (2) rate of effective learning of
observations, behavior patterns, facts, tools, methods, etc., which
requires identif...
This paper is about new technologies for complex model-based simulations: the use of Virtual Worlds (VWs) to build Virtual Simulation Environments (VSEs), that can contain the simulation developers as well as their tools, models, experiments, and partial resultss and the use of our knowledge-based integration infrastructure to make the model-based...
This paper describes lessons we have learned over the last ten
years from our “wrapping” approach to large-scale system
development. Our research program in integration for constructed complex
systems has led to several results with system engineering applications:
the wrapping expression notation wrex for communication among
distributed entities,...
Computational embodiment is the computer implementation of principles of autonomy that allows software systems to exist in and interact with complex environments. We restrict our attention here to symbolic environments MUDs , as an initial step towards understanding and constructing ''interaction spaces'' in which humans and computer programs can i...
In this paper, we describe a programming paradigm that changes the
focus of programming from solution methods for certain application
problems to the specification of the problems themselves, leaving the
mapping from the problem specification to the computational resources
that will provide or coordinate the solution to one or more separate
(and po...
This paper is about a particular technical problem in the study of
autonomy in constructed complex systems: how can a computing system
decide that it needs a new symbol system to re-express all of its own
processes? We have shown earlier that a constructed complex system will
need to change its symbol system, by proving two theorems that limit
what...
Synthesizes earlier work on collaboration and virtual worlds, and
proposes a new attitude about distributed heterogeneous systems that
extends both the shared artifacts of the World Wide Web and the shared
presence of multi-user virtual environments (MUVEs), and in addition
allows the use of any user-favored tools from within the same
environment....
We describe in this paper a new approach to Generic Programming
that combines our integration results with Partial Evaluation methods
for adaptation. Our approach supports Partial Evaluation by providing
much more information than is usually available, including explicit
meta-knowledge about the program fragments and their intended execution
enviro...
Virtual Worlds are computer-mediated environments, in which we can monitor the interactions among our tools, the human users of the tools, and the humans themselves. They thus provide a powerful new experimental platform for studying issues that were heretofore very difficult. In this paper, we describe an experiment to study two claims about langu...
This paper is about how developers will know whether intelligent virtual environments (IVEs) are appropriate for the tasks set to them. There are several important research questions that need to be answered before they can even begin to build IVEs for some of the more promising applications, such as entertainment, education, collaboration on resea...
This paper starts by expanding the scope of the Knowledge Sharing problem to the scale of space systems: hundreds of thousands of physical components and millions of lines of code, produced by hundreds or thousands of organizations, integrated into a system that is expected to last for decades. Our problem is to organize the associated information...
Constructed complex systems are heterogeneous software and
hardware systems that have to function in complex environments. Building
and managing such a system requires explicit infrastructure that
includes models of the system, its architecture, and its environment. We
describe wrapping, our knowledge-based integration infrastructure, and
show by e...
Questions
Questions (8)
I've developed some new metrics that attempt to gauge the effectiveness of different analytic methods (especially in their ability to discover new findings) and now want to do a more complete literature search. Does anyone have some recommendations for good articles on the development of new metrics for evaluating the effectiveness or power of different analytic/mathematical methods to discover new patterns or relationships?
I am interested in any papers on estimating the size of a graph. I'm particularly interested in what assumptions one would have to make to do the estimation (if one can at all under different scenarios for how one is constructing or determining different types of networks or graphs). Thank-you for any pointers or for better phrasing on this question.
I would love some pointers to existing work/papers on sparsity in large data sets. What I am looking for are not the (important) usual papers on how to compute given sparsity in large data sets; I am instead thinking about how one might use the fact of sparsity and a non-homogeneous distribution of features and relationships to characterize overall solution spaces/data set spaces into regions of greater interest and less interest.
Recently I have been thinking about the challenges of suitably integrating self-optimization processes within large dynamic (changing components and relationships) complex systems. Especially in Systems of Systems, one will have optimizing processes occurring at many levels and sectors of such systems. I have an approach that sidesteps for now hard questions on how to map for example ongoing optimizations at one level into another level, but I would like to eventually have many approaches to handle these issues. Does anyone know some current or past work in this general area?
Many of the current methods for assessing risks depends on probabilities and events as randomly distributed occurences. This does not fit with intentional acts that can be persistent, reoccuring, focused etc. I am looking for any new work on providing new methods for assessing events that are not randomly distributed and for aggregating such risks. A closely related topic would be any new work on aggregating very low probability or rare events. Does anyone have a pointer or suggestions for who to look at and where to look?
Anyone have recent studies correlating Discrepancy Reports or Software bugs/failures / errors with different SW code metrics?
The investigators here are searching for a way of saying that the test matched its requirements, met the expectations of the standards that are applicable, and that the test was high quality. They drew the term from HW tests. All obviously overlapping with sense of test validation, test coverages (of many sorts), and other test effectiveness measures. Have any of you developed a metric on the fit of a test against requirements, standards, etc.? Any references very welcomed.
