ArticlePDF Available

Abstract and Figures

This paper introduces a new metamodel-based knowledge representation that significantly improves autonomous learning and adaptation. While interest in hybrid machine learning / symbolic AI systems leveraging, for example, reasoning and knowledge graphs, is gaining popularity, we find there remains a need for both a clear definition of knowledge and a metamodel to guide the creation and manipulation of knowledge. Some of the benefits of the metamodel we introduce in this paper include a solution to the symbol grounding problem, cumulative learning, and federated learning. We have applied the metamodel to problems ranging from time series analysis, computer vision, and natural language understanding and have found that the metamodel enables a wide variety of learning mechanisms ranging from machine learning, to graph network analysis and learning by reasoning engines to interoperate in a highly synergistic way. Our metamodel-based projects have consistently exhibited unprecedented accuracy, performance , and ability to generalize. This paper is inspired by the state-of-the-art approaches to AGI, recent AGI-aspiring work, the granular computing community, as well as Alfred Korzybski's general semantics. One surprising consequence of the metamodel is that it not only enables a new level of autonomous learning and optimal functioning for machine intelligences, but may also shed light on a path to better understanding how to improve human cognition.
Content may be subject to copyright.
February 12, 2021 1:30 MetamodelAGI
Journal of Artificial Intelligence and Consciousness
©World Scientific Publishing Company
A Metamodel and Framework for Artificial General Intelligence
From Theory to Practice
Hugo Latapie, Ozkan Kilic, Gaowen Liu, Yan Yan, Ramana Kompella, Pei Wang, Kristinn R.
Th´orisson, Adam Lawrence, Yuhong Sun, Jayanth Srinivasa
hlatapie@cisco.com, okilic@cisco.com, gaoliu@cisco.com, yyan34@iit.edu, rkompell@cisco.com,
pei.wang@temple.edu, thorisson@ru.is, adamlawr@cisco.com, yuhosun@cisco.com,
jasriniv@cisco.com
Received 15th December 2020
Revised ... December 2020
This paper introduces a new metamodel-based knowledge representation that signifi-
cantly improves autonomous learning and adaptation. While interest in hybrid machine
learning / symbolic AI systems leveraging, for example, reasoning and knowledge graphs,
is gaining popularity, we find there remains a need for both a clear definition of knowl-
edge and a metamodel to guide the creation and manipulation of knowledge. Some
of the benefits of the metamodel we introduce in this paper include a solution to the
symbol grounding problem, cumulative learning, and federated learning. We have ap-
plied the metamodel to problems ranging from time series analysis, computer vision,
and natural language understanding and have found that the metamodel enables a wide
variety of learning mechanisms ranging from machine learning, to graph network anal-
ysis and learning by reasoning engines to interoperate in a highly synergistic way. Our
metamodel-based projects have consistently exhibited unprecedented accuracy, perfor-
mance, and ability to generalize. This paper is inspired by the state-of-the-art approaches
to AGI, recent AGI-aspiring work, the granular computing community, as well as Alfred
Korzybski’s general semantics. One surprising consequence of the metamodel is that it
not only enables a new level of autonomous learning and optimal functioning for machine
intelligences, but may also shed light on a path to better understanding how to improve
human cognition.
Keywords: Artificial Intelligence and AI and AGI and General Semantics and Levels of
Abstraction and Neurosymbolic and Cognitive Architecture
1. Introduction
The 1field of artificial intelligence has advanced considerably since its inception in
1956 at the Dartmouth Conference organized by Marvin Minsky, John McCarthy,
Claude Shannon, and Nathan Rochester. The exponential growth in compute power
and data, along with advances in machine learning and in particular, deep learning,
1Preprint of an article submitted for consideration in [Journal of Artificial Intelli-
gence and Consciousness] ©[2021] [copyright World Scienti!c Publishing Company]
[https://www.worldscientific.com/worldscinet/jaic]
1
arXiv:2102.06112v1 [cs.AI] 11 Feb 2021
February 12, 2021 1:30 MetamodelAGI
2Latapie et al., 2020
have resulted in remarkable pattern recognition capabilities. For example, real-time
detection of pedestrians has recently achieved an average precision of over 55%
[Tan et al., 2019]. Natural language understanding systems are achieving superhu-
man performance on some tasks such as yes/no question answering [Wang et al.,
2019]. However, as Yan [LeCun, 2020] wrote,“trying to build intelligent machines by
scaling up language models is like building high-altitude planes to go to the moon.
You might beat altitude records, but going to the moon will require a completely
different approach.” Beyond the need to improve the accuracy of pattern recognition
beyond current levels, current deep learning approaches suffer from susceptibility
to adversarial attacks, a need for copious amounts of labeled training data, and an
inability to meaningfully generalize.
After over 30 years of intense effort, the AGI community has developed the theo-
retical underpinnings for AGI and affiliated working software systems [Wang, 2005,
2006]. While achieving human-level AGI is arguably years to decades away, some of
the currently available AGI subsystems are ready to be incorporated into non-profit
and for-profit products and services. Some of the most promising AGI systems we
have encountered are OpenNARS [Wang, 2006, 2010], OpenCog [Goertzel, 2009;
Goertzel et al., 2013], and AERA [Th´orisson, 2012]. We are collaborating with all
three teams and have developed video analytics and Smart City applications that
leverage both OpenCog and OpenNARS [Hammer et al., 2019].
After several years of applying these AGI technologies to complex, real-world
problems in IoT, networking, and security at scale, we have encountered a few
stumbling blocks largely related to real-time performance on large datasets and
cumulative learning [Th´orisson et al., 2019]. In order to progress from successful
proofs of concept and demos to scalable products, we have developed the Deep
Fusion Reasoning Engine (DFRE) metamodel and associated DFRE framework,
which is the focus of this paper. We have used this metamodel and framework to
bring together a wide array of technologies ranging from machine learning, deep
learning, and probabilistic programming to the reasoning engines operating under
the assumption of insufficient knowledge and resources (AIKR) [Wang, 2005]. As
discussed below, we believe the initial results are promising: The data show a dra-
matic increase in system accuracy, ability to generalize, resource utilization, and
real-time performance when compared to state-of-the-art AI systems.
The following sections will cover related theories and technologies, the meta-
model itself, empirical results, and discussions as well as future work. Appendix A
contains some background information that may help readers gain a deeper under-
standing of this material.
2. Related Theories and Technologies
2.1. Korzybski
After living through WWI, Korzybki, was concerned about the future trajectory of
mankind. He focused his research on the creation of a non-metaphysical definition
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 3
of man that was both descriptive and predictive from a scientific and engineering
perspective. He focused on what he termed the “time-binding property” that en-
abled human societies to advance exponentially from a technological perspective.
As his objective was to discover the source of humanity’s self-destructive tendencies,
he created a model of the human nervous system he called “the structural differen-
tial”, which is the primary inspiration for our metamodel. Korzybski developed a
theory that explains the power of the human nervous system, the weaknesses that
cause many of humanity’s major problems such as world wars, and a path to op-
timal/correct functioning of the human nervous system. The Institute of General
Semantics, which Korzybski founded, continues to train of educators around the
world.
Korzybski focused on helping people better utilize the considerable power of the
human nervous system in part because the combination of exponential advancement
of technology and a primitive way of using it could lead to large-scale destruction.
Given that we have far more powerful compute capability and weapons, the sane
operation of all autonomous learning systems, human or machine, is of even greater
importance.
2.2. OpenNARS
OpenNARS (see [Hammer et al., 2019]) is a Java implementation of a Non-
Axiomatic Reasoning System (NARS). NARS is a general-purpose reasoning system
that works under the Assumption of Insufficient Knowledge and Resources (AIKR).
As described in Wang [2009], this means the system works in Real-Time, is always
open to new input, and operates with a constant information processing ability and
storage space. An important part is the Non-Axiomatic Logic (see Wang [2010] and
Wang [2006]) which allows the system to deal with uncertainty. To our knowledge,
our solution is the first to apply NARS to a real-time visual reasoning task.
2.3. Embeddings
Graph embedding [Cui et al., 2018; Hamilton et al., 2017] is a technique used to
represent graph nodes, edges, and sub-graphs in vector space that other machine
learning algorithms can use. Graph neural networks use graph embeddings to ag-
gregate information from graph structures in non-Euclidean ways. This allows the
DFRE Framework to use the embeddings to learn from different data sources that
are in the form of graphs, such as Concept Net [Speer et al., 2017]. Despite its per-
formance across different domains, the graph neural networks suffer from scalability
issues [Ying et al., 2018; Zhou et al., 2018] because calculating the Laplacian matrix
for all nodes in a large network may not be feasible. The levels of abstraction and
the focus of attention mechanisms used by an Agent resolve these scalability issues
in a systematic way.
February 12, 2021 1:30 MetamodelAGI
4Latapie et al., 2020
3. The DFRE Metamodel
The DFRE metamodel and framework are based on the idea that knowledge is a
hierarchical structure, where the levels in the hierarchy correspond to levels of ab-
straction. The DFRE metamodel refers to the way that knowledge is hierarchically
structured while a model refers to knowledge stored in a manner that complies to
the DFRE metamodel. It is based on non-Aristotelian, non-elementalistic systems
of thinking. The backbone of its hierarchical structure is based on difference, a.k.a.
antisymmetric relations, while the offshoots of such relations are based on symmet-
ric relations. As in figure 1, even a simple amoeba has to differentiate distinctions
and similarities because preserving symmetric and antisymmetric relations is fatally
important.
Fig. 1. Amoeba distinguishing between distinctions and similarities.
Korzybski [Korzybski, 1994] dedicated the majority of his professional life to
analyzing and studying the nature of this hierarchical structure. While it is well
beyond the scope of this paper to discuss the details of his analysis, our initial focus
was to incorporate these fundamental principles.
K1 – the core framework of knowledge is based on anti-symmetric relations
Spatial understanding: right/left/top/bottom
Temporal understanding: before/after
Corporal understanding: pain/satiation
Emotional understanding: happy/sad
Social understanding: friend/foe
Causal understanding: X causes Y
K2 – symmetric relations add further structure
A and B are friends
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 5
A is like B
K3 – knowledge is layered
Sensor data is on a different layer than high level symbolic information
Symbolic information B, which expands or provides context to sym-
bolic information A, is at a higher layer / level of abstraction
In the symbolic space, there are theoretically an infinite number of
layers, i.e., it is always possible to refer to a symbol and expand upon
it, thus creating yet another level of abstraction
K4 – since knowledge is structure, any structure destroying operations
such as confusing levels of abstraction, treating an anti-symmetric relation
as symmetric, or vice-versa, can, if inadvertently applied, be knowledge-
corrupting and/or a knowledge-destroying operation.1However it should
be noted that creative problem solving and other adaptive behaviors may
require mixing levels of abstraction. The key is to ensure that the long-
term structure of the metamodel is meticulously maintained and that these
operations occur by design and not by accident.
The DFRE Knowledge Graph (DFRE KG) groups information into four levels as
shown in Figure 2. These are labeled L0, L1, L2, and L* and represent different levels
of abstraction with L0 being closest to the raw data collected from various sensors
and external systems, and L2 representing the highest levels of abstraction, typically
obtained via mathematical methods, i.e. statistical learning and reasoning. The
layer L2 can theoretically have infinitely many sub-layers. L* represents the layer
where the high-level goals and motivations, such as self-monitoring, self-adjusting
and self-repair, are stored. There is no global absolute level for a concept and
all sub-levels in L2 are relative. However, L0, L1, L2 and L* are global concepts
themselves. For example, an Agent, which is basically a computer program that
performs various tasks autonomously, can be instantiated to troubleshoot a problem,
such as one related to object recognition or computer networking. The framework
promotes cognitive synergy and metalearning, which refer to the use of different
computational techniques (e.g., probabilistic programming, Machine Learning/Deep
Learning, and such) to enrich its knowledge and address combinatorial explosion
issues.
One advantage of the DFRE Framework is its integration of human domain
expertise, ontologies, prior learnings by the current DFRE KG-based system and
other similar systems, and additional sources of prior knowledge through the mid-
dleware services. It provides a set of services that an Agent can utilize as shown in
1Korzybski argues that what is currently limiting humanity’s advancement is the general lack
of understanding of how our own abstracting mechanisms work [Korzybski, 1949]. He considers
mankind to currently be in the childhood of humanity and the day, if it should come, that humanity
becomes generally aware of the metamodel, is the day humanity enters into the “manhood of
humanity” [Korzybski, 1921]
February 12, 2021 1:30 MetamodelAGI
6Latapie et al., 2020
Fig. 2. DFRE Framework with four levels of abstraction.
Figure 3.
Fig. 3. DFRE Framework.
The Sensor Data Services are used to digitize any real world data, such as
video recordings. Similarly, the Data Structuring Services restructure data ,e.g.,
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 7
rectifying an image, if needed. These two services are the basis for Image Processing
Services which provide a set of supervised and unsupervised algorithms to detect
objects, colors, lines, and other visual criteria. The Sensor Data Analytic Services
analyze objects and create object boundaries enriched with local properties, such
as an object’s size and coordinates, which create a 2D symbolic representation
of the world. Spatial Semantic Services then uses this representation to construct
the initial knowledge graph that captures the spatial relations of the object as a
relational graph. Any L2- or high-level reasoning is performed on this knowledge
graph.
Graph-based knowledge representation provides a system with the ability to:
Effectively capture the relations in the sub-symbolic world in a world of
symbols,
Keep a fluid data structure independent of programming language, in which
Agents running on different platforms can share and contribute,
Use algorithms based on the graph neural networks to allow preservation
of topological dependency of information [Scarselli et al., 2009] on nodes.
All processes are fully orchestrated by the Agent that catalogues knowledge by
strictly preserving the structure while evolving new structures and levels of abstrac-
tion in its knowledge graph because, for DFRE KG, knowledge is structure. Multiple
Agents can have not only individual knowledge graphs but also a single knowledge
graph on which all can cooperate and contribute. In other words, multiple Agents
can work toward the same goal by sharing the same knowledge graph synchronously
or asynchronously. Different Agents can have partially or fully different knowledge
graphs depending on their experience, and share those entire graphs or their frag-
ments through the communication channel provided by the DFRE Framework. Note
that although the framework can provide supervised machine learning algorithms
if needed, the current IoT use case is based on a retail store which requires unsu-
pervised methods as explained in the next section.
4. Experimental Results
The DFRE Framework was previously tested in the Smart City domain [Hammer et
al., 2019], in which the system learns directly from experience with no initial training
required (one-shot), based on a fusion of sub-symbolic (object tracker) and symbolic
(ontology and reasoning) information. The current use case is based on object-
class recognition in a retail store. Shelf space monitoring, inventory management
and alerts for potential stock shortages are crucial tasks for retailers who want to
maintain an effective supply chain management. In order to expedite and automate
these processes, and reduce both the requisite for human labor and the risk of
human error, several machine learning and deep learning-based techniques have
been utilized [Baz et al., 2016; Franco et al., 2017; George and Floerkemeier, 2014;
Tonioni and Stefano, 2017]. Despite the high success rates, the main problems for
February 12, 2021 1:30 MetamodelAGI
8Latapie et al., 2020
such systems are the requirements for a broad training set, including compiling
images of the same product with different lighting and from different angles, and
retraining when a new product is introduced or an existing product is visually
updated. The current use case does not demonstrate an artificial neural network-
based learning. The DFRE Framework has an artificial general intelligence-based
approach to these problems.
Fig. 4. Retail use case for DFRE Framework.
Before a reasoning engine operates on symbolic data within the context of the
DFRE Framework, several services must be run, as shown in Figure 4. The flow
starts with a still image captured from a video camera that constantly records the
retail shelves by the Sensor Data Services as in Figure 4.a, which corresponds to
L0 in Figure 2. Next, the image is rectified by the Data Structuring Services in
Figure 4.b for better line detection by the Image Processing Services, as displayed
in Figure 4.c. The Image Processing Services in the retail case are unsupervised
algorithms used for color-based pixel clustering and line detection, such as proba-
bilistic Hough transform [Kiryati et al., 1991]. The Sensor Data Analytics Services
in Figure 4.d create the bounding boxes which represent the input in a 2D world of
rectangles, as shown in Figure 4.e. The sole aim of all these services is to provide
the DFRE KG with the best symbolic representation of the sub-symbolic world
in rectangles. Finally, the Spatial Semantics Services operate on the rectangles to
construct a knowledge graph, which preserves not only the symbolic representation
of the world, but also the structures within it in terms of relations, as shown in Fig-
ure 4.f. This constitutes the L1 level abstraction in the DFRE KG. L1 knowledge
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 9
graph representation also recognizes and preserves the attributes of each bounding
box, such as the top-left xand ycoordinates, and the center’s coordinates: height,
width,area and circumference. The relations used for the current use case are in-
side,aligned,contains,above,below,on left of,on right of,on top of,under and
floating. Since the relations in the DFRE KG are by default antisymmetrical, the
system does not know that aligned(a,b) means aligned(b,a), or on left of and on
right of are inverse relations unless such terms are input as expert knowledge or are
learned by the system through experience or simulations. The only innate relations
in the DFRE metamodel are distinctions, which are anti-symmetric and similarity
relations; and the rest is learned by experience.
The system’s ultimate aim is to dynamically determine shelves,products and
unknown/others, as illustrated in Figure 5, and to monitor the results with times-
tamps.
While L2 identifies only the concepts of shelf,product and unknown, and the
possible relations among them, the reasoning engine, NARS [Wang, 2006, 2010;
Wang et al., 2018], creates their L1 intensions as an evidence-based truth system
in which there is no absolute knowledge. This is useful in the retail use case sce-
nario because the noise in L0 data causes both overlapping regions and conflicting
premises at L1. This noise results from not only the projection of the 3D world
input data into a 2D framework, but also the unsupervised algorithms used by L1
services. The system has only four rules for L2 level reasoning:
If a rectangle contains another rectangle that is not floating, the outer rect-
angle can be a shelf while the inner one can be a product.
If a rectangle is aligned with a shelf, it can be a shelf too.
If a rectangle is aligned with a product horizontally, it can be a product too.
If a floating rectangle is stacked on a product, it can be a product too.
Note that applying levels of abstraction gives the DFRE Framework the power to
perform reasoning based on the expert knowledge in L2 level mostly independent
of L1 level knowledge. In other words, the system does not need to be trained for
different input; it is unsupervised in that sense. The system has a metalearning ob-
jective which continuously attempts to improve its knowledge representation. The
current use case had 152 rectangles of various shapes and locations, of which 107
were products, 16 were shelves, and the remaining 29 were other objects. When
the knowledge graph in L1 is converted into Narsese, 1,478 lines of premises that
represent both the relations and attributes were obtained and sent to the reasoner.
Such a large amount of input with the conflicting evidence caused the reasoning
engine to perform poorly. Furthermore, the symmetry and transitivity properties
associated with the reasoner resulted in the scrambling of the existing structure in
the knowledge graph. Therefore, the DFRE Framework employed a Focus of At-
tention (FoA) mechanism. The FoA creates overlapping covers of knowledge graphs
for the reasoner to work on this limited context. Later, the framework combines the
results from the covers to finally determine the intensional category. For example,
February 12, 2021 1:30 MetamodelAGI
10 Latapie et al., 2020
Fig. 5. LoA for retail use case.
when the FoA utilizes the reasoner on a region, a rectangle can be recognized as a
shelf. However, when the same rectangle is processed in another cover, it may be
classified as a product. The result with higher frequency and confidence wins. The
FoA mechanism is inspired by the human visual attention system, which manages
input flow and recollects evidence as needed in case of a conflicting reasoning re-
sult. A FoA mechanism can be based on objects’ attributes, such as color or size,
with awareness of proximity. For this use case, the FoA determined the contexts by
picking the largest non-empty rectangle, and traversing its neighbors based on their
sizes in decreasing order.
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 11
The framework is tested in various settings with different camera angles and
products placements as shown in Figure 6.
Fig. 6. DFRE Framework qualitative result in retails use cases with different environments
In Figure 6, each row represents samples from different settings: rectified frames,
bounding boxes, and instantaneous output of the reasoner. We would like to em-
phasize that the system does not require any retraining or any change in order to
adapt to the new setting. It requires only a camera to be pointed to the scene; then
it automatically generalizes.
DFRE Framework was tested 10 times in 4 different settings with and without
the FoA. The precision, recall and f-1 scores are exhibited in Table 1.
Table 1. DFRE Framework experimental results.
Category without FoA (%) with FoA (%)
precision recall f1-score precision recall f1-score
product 80.70 29.32 52.88 96.36 99.07 97.70
shelf 8.82 18.75 12.00 82.35 87.50 88.85
other 36.61 89.66 52.00 96.00 82.76 88.89
overall accuracy 46.30 (min/max: 30.13/84.65) 94.73 (min/max: 88.10/100.00)
The results indicate that the FoA mechanism improves the success of our AGI-
based framework significantly by allowing the reasoner to utilize all of its computing
resources in a limited but controlled context. The results are accumulated by the
framework, and the reasoner makes a final decision. This approach not only allows
us to perform reasoning on the intension sets of L1 knowledge, which are retrieved
through unsupervised methods, but also resolves the combinatorial explosion prob-
lem whose threshold depends on the limits of available resources. In addition, one
February 12, 2021 1:30 MetamodelAGI
12 Latapie et al., 2020
can easily extend this retail use case to include prior knowledge of product types
and other visual objects, such as tables, chairs, people and shelves, as allowed by
the DFRE KG.
4.1. Graph Embedding for Link Predictions
Recall, a graph G(V, E ), where Vis the set of all vertices, or nodes, in G, and E, is
the set of paired nodes, called edges. |V| ∈ Zis the order of the graph, or the total
number of nodes.
As mentioned before, DFRE takes advantage of graph embedding, a transfor-
mation that constructs a non-dimensional knowledge graph Ginto a d-dimensional
vectors space SR|Vd. Among the many benefits, such as creating a Euclidean
distance measurement for G, link predictions can be established between node vec-
tors in S. Preliminary experimental results have given a great deal of insight into
the relationships between nodes that might otherwise not be present from the graph
space.
The main algorithm used by DFRE to transform our knowledge graph Gto a
2-dimensional vector space is Node2Vec. [Grover and Leskovec, 2016]. This frame-
work is a representation learning based approach that learns continuous feature
representations for all nodes in a given knowledge graph G. The benefits from this
algorithm, and the motivation for use in DFRE, construct the graph embedding
space Swhere link prediction, and other methods of measurement, can be used
while preserving relevant network properties from the original knowledge graph.
The functionality behind Node2Vec is similar to most other embedding pro-
cesses, by use of the Skip-Gram model, and a sampling-strategy. Four arguments
are input into the framework: the number of walks, the length of the walks, p, and
q, where pis referred to as the return hyper-parameter, and qis the I/O hyper-
parameter.
Once a 2-dimensional vector representation has been assigned to every node
nV, our embedding vectors space SR|V2can provide additional metrics used
for machine learning and prediction measures. One such measure is link prediction
used to understand the relationship between nodes in a graph that might not be
obvious from the graph space.
Consider the nodes n1, n2G(V, E) such that n1and n2are not similar ideas
in the graph (e.g. the probability (n1, n2)E(G) is low). Once the nodes are
represented in vector form ˆn1,ˆn2R2S, we establish a linear relationship
between the two such that a line y=ax +bis satisfied, where a=ˆn22ˆn21
ˆn12ˆn11 and
b= ˆn21 a( ˆn11).
Let  > 0 , then ˆnkthat lies within the range of y±, we consider these node
vectors to be associated hidden links between two the two ideas ˆn1and ˆn2.
Additionally, if the line yis divited into four evenly distributed quadrants y1..y4
and grown by small perturbations where y0=y±0such that 0=+γand
γ(0,1] until there exists at least one ˆnkin every quadrant. We call this set of
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 13
node vectors Sn.
This set of node vectors Sngathered within range y0provide DFRE a rela-
tionship that might not be immediately obvious from the graph space alone. The
distribution of the vectors along the quadrants is revealing in away such that, for
example, consider the two disjoint subsets ( ˆnk)iand ( ˆnl)jof Sn. Without loss of
generality, if ( ˆnk)iy0
1and ( ˆnk)jy0
4, where i << j, we see that the relationship
skews towards the set of node vectors that lie within the range of y0
4.
Additionally, within the embedding space, consider a finite set of clusters
{C1, C2, . . . }, each corresponding to its own central idea. For any arbitrary cluster
Ci, if a new node vector ˆ
n0is introduced in Ssuch that ˆ
n0Ci, then we can easily
leverage this proximity into our sub-symbolic space to identify any additional node
vectors.
We find the main benefit to graph embedding is that we now have an unsuper-
vised method for correlating the graph embedding space with additional embedding
spaces that are generated using unsupervised machine learning techniques.
5. Philosophical Implications
The nativism-versus-empiricism debate, which posits that some knowledge is in-
nate and some is learned through experience, was ascribed in the ancient world by
the Greek philosophers, including Plato and Epicurus. Today, Descartes is widely
accepted as a pioneering philosopher working on the mind as he furthered and refor-
mulated the debate in the 17th century with new arguments. Perception, memory,
and reasoning are three fundamental cognitive faculties that enhance this debate by
explicating the building blocks of natural intelligence. We perceive the sub-symbolic
world, and abstract it in memory, and reason on this symbolic world representa-
tion. All three place concept learning and categorization at the center of the human
mind.
The process of concept learning and categorization continues to be an active
research topic related to the human mind since it is essential to natural intelligence
[Lakoff, 1984] and cognitively inspired robotics research [Chella et al., 2006; Lieto
et al., 2017]. It is widely accepted that this process is based more on interactional
properties and relationships among Agents, as well as between an Agent and its
environment, than objective features such as color, shape and size [Johnson, 1987;
Lakoff, 1984]. This makes the distinction of anti-symmetric and symmetric relations
crucial in the DFRE Framework, which assumes that the levels of abstraction are
part of innate knowledge. In other words, an Agent has L0, L1 and pre-existing
L2 by default. This constitutes a common a priori metamodel shared by all DFRE
Agents. Each Agent instantiated from the framework has the abstraction skill based
on interactional features and relationships. If the concepts in real life exist in inter-
actional systems, natural intelligence needs to capture these systems of interactions
with its own tools, such as abstraction. These tools should also be based on in-
teractional features by strictly preserving the distinction between symmetry and
February 12, 2021 1:30 MetamodelAGI
14 Latapie et al., 2020
anti-symmetry.
The mind is a system as well. Modern cognitive psychologists agree that con-
cepts and their relations in memory function as the fundamental data structures
to higher level system operations, such as problem solving, planning, reasoning and
language. Concepts are abstractions that have evolved from a conceptual primitive.
An ideal candidate for a conceptual primitive would be something that is a step
away from a sensorimotor experience [G¨ardenfors, 2000], but is still an abstraction
of experience [Cohen, 1997]. For example, a dog fails the mirror test but exhibits
intelligence when olfactory skills are needed to complete a task [Horowitz, 2017]. A
baby’s mouthing behavior is not only a requisite for developing oral skills but also
for discovering the surrounding environment through one of its expert sensorimotor
skills related to its survival. The baby is probably abstracting many objects into ed-
ible versus inedible higher categories given its insufficient knowledge and resources.
What is astonishing about a natural intelligence system is that it does not need a
plethora of training input and experiments to learn the abstraction. It quickly and
automatically fits new information into an existing abstraction or evolves it into a
new one, if needed. This is nature’s way of managing combinatorial explosion. Ob-
jects and their interconnected relationships within the world can be chaotic. Natural
intelligence’s solution to this problem becomes its strength: context. The concept of
‘sand’ has different abstractions depending on whether it is on a beach, on a cam-
era, or on leaves. An Agent in these three different contexts must abstract the sand
in relation to its interaction with world in its short-term memory. This cumulative
set of experiences can later become part of long-term memory, more specifically,
episodic memory. The DFRE Framework uses a Focus of Attention (FoA) mecha-
nism that provides the context while addressing the combinatorial explosion prob-
lem. The DFRE metamodel’s new way of representing practically all knowledge as
temporally evolving (i.e. time series) can be viewed as the metamodel’s conceptual
space. For example, the retail use case given in Section 2 starts with a 3D world
of pixels that is abstracted as lines and rectangles in 2D. The framework produces
spatial semantics using the rectangles in the 2D world. Based on this situation, a
few hundred rectangles produce thousands of semantic relations, which present a
combinatorial explosion for most AGI reasoning engines. For each scene, the DFRE
KG creates contexts, such candidate shelves, runs reasoners for each context, and
merges knowledge in an incremental way. This not only addresses the combinatorial
explosion issue, but also increases the success rate of reasoning, provided that the
levels of abstraction are computed properly [Gorban and Tyukin, 2018].
Abstracting concepts in relation to their contexts also allows a natural intel-
ligence to perform mental experiments, which is a crucial part of planning and
problem solving. The DFRE Framework can integrate with various simulators, re-
run a previous example together with its context, and alter what is known for
the purposes of experimentation to gain new knowledge, which is relationships and
interactions of concepts.
Having granular structures provides structured thinking, structured problem
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 15
solving, and structured information processing [Yao and Deng, 2012]. The DFRE
Framework has granular structures but emphasizes the preservation of structures in
knowledge. When a genuine problem that cannot be solved by the current knowledge
arises, it requires scrambling the structures and running simulations on the new
structures in order to provide an Agent with creativity. Note that this knowledge
scrambling is performed in a separate sandbox. The DFRE Framework ensures that
the primary DFRE KG is not corrupted by these creative, synthetic, “knowledge-
scrambling” activities.
6. Discussion on Metamodel and Consciousness
For the purposes of this discussion, we will define consciousness as autonomous self-
aware adaptation to the environment. This means that an abstraction of the self as
well as the environment of the self is learned autonomously. Human consciousness
builds on the prior capabilities of chemistry-binders (plants) and space-binders (an-
imals) with the unique ability of infinite levels of abstraction [Korzybski, 1994]. For
any concept, one can envision creating a higher-level meta-concept. We are able to
formulate symbolic representations that can be externalized and shared. The hu-
man model of the self evolves not only via direct interaction with the environment
and cogitation, but also by watching other humans and modeling them. Human con-
sciousness as an implementation of the metamodel appears to be dynamic in nature.
The concept of self can grow to encompass family, friends, social/work groups, and
beyond. Understanding our nature as time-binders that form a collective conscious-
ness of ever-increasing power (cognitive and physical) over time, civilizations, and
generations, appears to lead to higher cognitive functioning of the individual.
Human societies consisting of billions of people networked together in real-time,
with petabytes of shared storage and petaflops of compute, may see the evolution of
exo-cortical consciousness. In fact, many argue that this exo-cortical consciousness
already exists with the growing number of autonomous self-healing systems deployed
and connected throughout the world.
Since the metamodel hypothesizes the existence of an exo-cortical consciousness,
it consequently yields to the possibility of implementing artificial consciousness, e.g.,
in robots. Artificial consciousness, which is also known as machine consciousness,
is a field designed to mimic the aspects of human cognition that are related to
human consciousness [Aleksander, 2008; Chella and Manzotti, 2009]. In the 1950s,
consciousness was seen as a vague term, and inseparable from intelligence. [Searle,
1992; Chalmers, 1996]. Fortunately, the improvements in technology, and compu-
tational and cognitive sciences have created new interest in the field. [Chella and
Manzotti, 2011] reviews that the most important gap between artificial and biolog-
ical consciousness studies is engineering autonomy, semantic capabilities, intention-
ality, self-motivation, resilience and information integration. The Agents based on
the metamodel have autonomy. They also have semantic capabilities and intentions
to seek solutions or communicate with other Agents for knowledge sharing, which
February 12, 2021 1:30 MetamodelAGI
16 Latapie et al., 2020
are set by self-motivation.
Chella and Manzotti [2011] emphasize that consciousness is a real physical phe-
nomenon, can be artificially replicated, is either a computational phenomenon or
more. We bring forth the metamodel as an enabler for the achievement of artificial
consciousness. The abstraction mechanism constantly and automatically creates the
abstractions of the sensor data and the system’s own experience. The metamodel
is based on generation of new knowledge using self-perception and experience, and
shares knowledge among the Agents of similar nature to support collective con-
sciousness and resilience. The creation of self through experience gives the meta-
model the ability of enhanced generalization and autonomy. Similarly, focus of the
attention mechanism to segment complex problems semantically is also related to
consciousness because attention and consciousness are interrelated [Taylor, 2007,
2009]. Implementation of attention is important because control theory is related
to consciousness and plays a leading role in the intentional mechanism of an Agent.
7. Conclusion
Several mathematical models and formal semantics [D¨untsch and Gediga, 2002; Be-
lohl´avek, 2004; Wang and Liu, 2008; Wille, 1980; Ma et al., 2007] are proposed to
specify the meanings of real world objects as concept structures and lattices. How-
ever, they are computationally expensive [Jinhai et al., 2015]. One way to overcome
this issue is with granular computing [Yao and Deng, 2012]. The extension of a con-
cept can be considered a granule, and the intension of the concept is the description
of the granule. Assuming that concepts share granular common parts with varying
derivational and compositional stages, categorization, abstraction and approxima-
tion occur at multiple levels of granularity which plays an important role in human
perception [Hobbs, 1985; Yao, 2001, 2009]. The DFRE Framework has granular
structures but emphasizes the preservation of structures in knowledge. Being in the
extension of a concept does not necessarily grant the granular concept the right
to have similar relations and interactions of its intensional concept up to a certain
degree or probability. Each level must preserve its inter-concept relationships and
its symmetry or anti-symmetry in a hierarchically structured way.
We have outlined the fundamental principles of the DFRE Framework. DFRE
takes a neurosymbolic approach leveraging state-of-the-art subsymbolic algorithms
(e.g. ML/DL/Matrix Profile) and state-of-the-art symbolic processing (e.g. reason-
ing, probabilistic programming, and graph analysis) in a synergistic way. The DFRE
metamodel can be thought of as a knowledge graph with some additional structure,
which includes both a formalized means of handling anti-symmetric and symmet-
ric relations, as well as a model of abstraction. This additional structure enables
DFRE-based systems to maintain the structure of knowledge and seamlessly sup-
port cumulative and distributed learning. Although this paper provides highlights
of one experiment in the visual domain employing an unsupervised approach, we
have also run similar experiments on time series and natural language data with
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 17
similar promising results.
8. Future work
Nowadays there is a rapid transition in the AI research field from single modality
tasks, such as image classification and machine translation, to more challenging
tasks that involve multiple modalities of data and subtle reasoning, such as visual
question answering (VQA) [Agrawal et al., 2015; Andersonet al., 2018; Zhuet al.,
2020] and visual dialog [Vishvak et al., 2019]. A meaningful and informative conver-
sation, either between human-computer or computer-computer, is an appropriate
task to demonstrate such a reasoning process given the complex information ex-
change mechanism during the dialog. However, most existing research focuses on
the dialog itself and involves only a single Agent. We plan to design a more reliable
DFRE system with implicit information sources. To this end, we propose a novel
natural and challenging task with implicit information sources: describe an unseen
video mainly based on the dialog between two cooperative Agents.
The entire process can be described in three phases: In the preparation phase,
two Agents are provided with different information. Agent A1 is able to see the com-
plete information from different modalities (i.e., video, audio, text), while Agent A2
is only given limited information. In the second phase, A2 has several opportuni-
ties to ask A1 relevant questions about the video, such as the person involved, the
event happened, etc. A2 is encouraged to ask questions that help to accomplish the
ultimate video description objective, and A1 is expected to give informative and
constructive answers that not only provide the needed information but also moti-
vate A2 to ask additional useful questions in the next conversation round. After
several rounds of question-answer interactions, A2 is asked to describe the unseen
video based on the limited information and the dialog history with A1. In this task
setup, our DFRE system accomplishes a multi-modal task even without direct ac-
cess to the original information, but learns to filter and extract useful information
from a less sensitive information source, i.e., the dialog. It is highly difficult for AI
systems to identify people based on the natural language descriptions. Therefore,
such task settings and reasoning ability based on implicit information sources have
great potential to be applied in a wide practical context, such as the smart hospital
systems, improving current systems.
The key aspect to consider in this future work is the effective knowledge transfer
from A1 to A2. A1 plays the role of humans, with full access to all the information,
while A2 has only an ambiguous understanding of the surrounding environment
from two static video frames after the first phase. In order to describe the video
with details that are not included in the initial input, A2 needs to extract useful
information from the dialog interactions with A1. Therefore, we will propose a QA-
Cooperative network that involves two agents with the ability to process multiple
modalities of data. We further propose a cooperative learning method that enables
us to jointly train the network with a dynamic dialog history update mechanism.
February 12, 2021 1:30 MetamodelAGI
18 Latapie et al., 2020
The knowledge gap and transfer process are both experimentally demonstrated.
The novelties of the proposed future work can be summarized as follows: (i) We
propose a novel and challenging video description task via two multi-modal dialog
agents, whose ultimate goal is for one Agent to describe an unseen video based on the
interactive dialog history. This task establishes a more reliable setting by providing
implicit information sources to the metamodel. (ii) We propose a QA-Cooperative
network and the goal-driven learning method with a dynamic dialog history update
mechanism, which helps to effectively transfer knowledge between two agents. (iii)
With the proposed network and cooperative learning method, our A2 Agent with
limited information can be expected to achieve promising performance, comparable
to the strong baseline situation where full ground truth dialog is provided.
Appendix A. Background Material
This paper presents a few simple, yet potentially revolutionary ideas regarding the
nature of knowledge. Unfortunately, we find that current habits of cognition can
make understanding this paper difficult for some people irrespective of level of
formal education, intelligence, and so forth. This preface is intended to highlight
some of the potential blockages to understanding, along with causes, in an attempt
to assist readers to potentially benefit from this paper.
We begin by first diving into what may for many appear to be an oxymoronic
concept: “simple yet revolutionary.” In general, although this is largely an unstated
belief/feeling, it would make much more sense that the term revolutionary would
be associated with a certain amount of complexity and/ or dramatic, immediate
and obvious impact, e.g., a meteor killing the dinosaurs, Nikola Tesla inventing
the induction motor, or Einstein formulating a new theory of space-time. People
generally have difficulty gaining an intuitive understanding for small, incremental,
simple changes that still have tremendous world changing impact over longer periods
of time.
This particular cognitive blockage, as pointed out by Roberto Unger [Unger and
Smolin, 2015], is probably at the heart of why society has not yet learned how to
continually transform and adapt to changing technological, political, and economic
circumstances. Societal changes that are incremental are discounted as insignificant,
and those that are dramatic are considered too dangerous, thus locking society into
a mode where tectonic political pressures build to monumental levels leading inex-
orably to world wars and the like. However, for the purpose of understanding this
paper, it may help to keep in mind the rice and the chessboard problem [Wikipedia,
2019] which demonstrates how, in an exponential growth situation, a single grain
of rice quickly grows to several times the world production of rice [Unger, 2014].
In order to understand how exponential growth laws apply to systems, such as
the AI systems which are the focus of this paper, or societies which we mention
only in passing, we remind the reader that technological and knowledge progress at
an exponential growth as highlighted in the references (pg. 73 of [Korzybski, 1921],
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 19
and [Swanson, 2020]).
Another major impediment to understanding this paper is what Korzybski
termed elementalistic thinking. Elementalistic thinking is the mental habit of at-
tempting to understand systems by focusing primarily on the elements of the sys-
tems rather than the interactions between the elements. In complex systems consist-
ing of a large number of elements, understanding individual elements does little to
help understand the overall system. For example, in software consisting of millions
of lines of code, understanding the individual bits and bytes is of little to no benefit
in understanding the nature of the entire software/hardware system.
Korzybski identified many fallacies in our thought processes largely stemming
from cultures and educational systems based on modes of thinking originating in
Aristotle’s time circa 300 BC. While our scientific understanding has progressed by
leaps and bounds, we remain largely bound by incorrect legacy beliefs. The meta-
model we present in this paper is intended to create intelligent systems that exhibit
new levels of autonomous learning and operation. The irony is that to truly under-
stand these concepts, one must be at least partially unhindered by the problems
the metamodel solves.
The metamodel is based on the addition of very simple structure to a basic
knowledge graph consisting of nodes and links. We add structure to represent dis-
tinction relations (i.e. this is not that). These distinction relations we represent as
anti-symmetric and form the backbone of our knowledge structure. We then add
structure to represent similarity relations. Finally, we formalize Korzybski’s struc-
tural differential to encode different levels of abstraction [Korzybski, 1949].
While at the atomic level, these changes are trivial, at the system level, the
effects we have seen are revolutionary. The metamodel enables systems to learn and
reason without inadvertently mixing levels of abstraction. This actually resolves all
of the issues mentioned in this section as “cognitive blockages.” It should be noted
that Korzybski created the field of General Semantics to help humanity evolve to a
higher level of cognitive functioning. He spent many years creating a foundation and
working with students and educators. He found that younger children were able to
fully absorb and adapt to these ideas extremely quickly. Unfortunately for adults,
it is a longer process but always successful when there is real interest.
We recommend keeping Baudrillard’s idea from Simulacra and Simulation
in mind [Baudrillard, 1994]:“ The simulacrum is never that which conceals the
truth—it is the truth which conceals that there is none. The simulacrum is true.”
In the sense that the mental models we humans create define our individual worlds
and what is possible within those worlds. When these mental models are incorrect,
as they are bound to be at times (e.g. the belief that something cannot be both
simple/incremental and revolutionary), it has far reaching implications. At best,
our mental models are useful abstractions, e.g. our simplification of the quantum
scale world. If we keep this in mind, perhaps we will not attempt to touch the
seemingly solid disk mental model we create for a dangerous spinning object like
February 12, 2021 1:30 MetamodelAGI
20 Latapie et al., 2020
a spinning metal object [Korzybski, 1949]. More importantly, we will continually
question whether our views are based on valid abstractions for the current context.
As has been pointed out in the machine learning community: “all models are wrong
however some are useful.” [Box, 1976].
References
Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Batra, D., Parikh,
D. VQA: Visual Question Answering. Proceedings of the IEEE International
Conference on Computer Vision (ICCV).
Aleksander, I. 2008. Machine consciousness. In Scholarpedia.3(2):4162.
Anderson, P. and He, X. and Buehler, C. and Teney, D. and Johnson, M. and Gould,
S. and Zhang, L. Bottom-Up and Top-Down Attention for Image Captioning
and Visual Question Answering. Conference on Computer Vision and Pattern
Recognition (CVPR).
Baudrillard, J. 1994. Simulacra & Simulation. University of Michigan Press.
Baz, I., Yoruk, E., Cetin, M. 2016. Context-aware hybrid classification system for
fine-grained retail product recognition. In Image, Video, and Multidimensional
Signal Processing Workshop (IVMSP), 2016 IEEE 12th IEEE. 1–5.
Belohl´avek, R. 2004. Concept lattice and order in fuzzy logic. Annals of Pure
Applied Logic. 128(1-3):277–298.
Bono, E. D. 1969. The Mechanism of Mind. Jonathan Cape.
Box, G. E. P. 1976. Science and Statistics. In Journal of the American Statistical
Association.71(356):791-–799.
Chella, A., Dindo, H., Infantino, I.. 2006. A cognitive framework for imitation
learning. In The Social Mechanisms of Robot Programming by Demonstration,
Robotics and Autonomous Systems. 5:403—408.
Chella, A., Manzotti, R. 2009. Machine Consciousness: A Manifesto for Robotics.
In International Journal of Machine Consciousness.1(1):33—51.
Chella, A., Manzotti, R. 2011. Artificial Consciousness. In Perception-Action Cycle.
637—671.
Chalmers, D. J. 1996. The Conscious Mind: In Search of a Fundamental Theory.
New York, Oxford University Press.
Cohen, P. 1997. Pro jections as Concepts. Computer Science Department Faculty
Publication Series (194), https://scholarworks.umass.edu/cs/_faculty/
_pubs/194. Last accessed 18 May 2020.
Cui, P., Wang, X., Pei, J., Zhu, W. 2018. A survey on network embedding. IEEE
Transactions on Knowledge and Data Engineering.
untsch, I., Gediga, G. 2002. Modal-style operators in qualitative data analy-
sis. In Proceedings of the 2002 IEEE International Conference on Data Mining,
Washington, D.C. 155—162. Springer.
Franco, A., Maltoni, D., Papi, S. 2017. Grocery product detection and recognition.
Expert Systems with Applications. 81:163—176.
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 21
ardenfors, P. 2000. Conceptual Spaces: the Geometry of Thought. MIT
Press.Cambridge, Massachusetts.
George, M., Floerkemeier, C. 2014. Recognizing products: A perexemplar multi-
label image classification approach. In Proceedings of the European Conference
on Computer Vision. 440–455. Springer.
Goertzel, B. 2009. OpenCogPrime: A Cognitive Synergy Based Architecture for
General Intelligence. In Proceedings of the International Conference on Cognitive
Informatics, Hong King.
Goertzel, B., et al. 2013. The CogPrime Architecture for Embodied Artificial
General Intelligence. In Proceedings of the IEEE Symposium on Human-Level
AI, Singapore.
Gorban, A. N., and Tyukin, I. Y. 2018. Blessing of dimensionality: mathematical
foundations of the statistical physics of data. Philosophical Transactions of the
Royal Society A: Mathematical, Physical and Engineering Sciences, 376.2118.
Grover, A., Leskovec, J. node2vec: Scalable Feature Learning for Networks. ACM
SIGKDD Internation Conference on Knowledge Discovery and Data Mining
(KDD).
Hamilton, W. L., Ying, R., Leskovec, J. 2017. Representation learning on graphs:
Methods and applications. IEEE Data(base) Engineering Bulletin. 40:52-–74.
Hammer, P., Lofthouse, T., Fenoglio, E., Latapie, H. 2019. A reasoning based model
for anomaly detection in the SmartCity domain. In NARS Workshop at AGI-
19, Shenzhen, China. https://cis.temple.edu/tagit/events/papers/Hammer.pdf.
Last accessed 19 May 2020.
Hobbs, J. R. 1985. Granularity. In Proceedings of the 9th International Joint
Conference on Artificial Intelligence. 432–435.
Horowitz, A. 2017. Smelling themselves: dogs investigate their own odours longer
when modified in an olfactory mirror test. Behav. Proc.. 143:17–24.
Johnson, M. 1987. The Body in the Mind. University of Chicago Press.
Jinhai, L., Mei, C., Xu, W., Qian, Y. 2015. Concept learning via granular comput-
ing: A cognitive viewpoint. Information Sciences. 298:447—467.
Kiryati, N., Eldar, Y., Bruckstein, A. M. 1991. A probabilistic Hough transform.
Pattern Recognition. 24(4):303–316.
Korzybski, A. 1921. Manhood Of Humanity, The Science and Art of Human Engi-
neering. NY: E. P. Dutton & Company.
Korzybski, A. 1949. Videos
with Alfred Korzybski. https://www.thisisnotthat.com/korzybski-videos/. Last
accessed 14 Dec 2020.
Korzybski, A. 1994. Science and Sanity: An Introduction to Non-Aristotelian Sys-
tems, 5th Edition. NY: Institute of General Semantics.
Lakoff, G. 1984. Women, Fire, and Dangerous Things. University of Chicago Press.
LeCun, Y. 2020. Facebook’s chief AI scientist says GPT-3 is ‘not very good’
as a dialog system. https://thenextweb.com/neural/2020/10/28/facebooks-yann-
lecun-says-gpt-3-is-not-very-good-as-a-qa-or-dialog-system/. Last accessed 13 Dec
February 12, 2021 1:30 MetamodelAGI
22 Latapie et al., 2020
2020.
Lieto, A., Chella, A., Frixione, M. 2017. Conceptual spaces for cognitive architec-
tures: A lingua franca for different levels of representation. Biologically, Inspired
Cognitive Architectures.19:1-–9.
Ma, J. M., Zhang, W. X., Leung, Y., Song, X. X. 2007. Granular computing and
dual Galois connection. Information Sciences. 177(23):5365-–5377.
Murahari, V. and Chattopadhyay, P. and Batra, D. and Parikh, D. and Das, A..
Improving Generative Visual Dialog by Answering Diverse Questions. Proceed-
ings of the Conference on Empirical Methods in Natural Language Processing
(EMNLP).
Pauli, W. 1935. Part I. General: (A) theory. Some relations between electro-
chemical behaviour and the structure of colloids. Transactions of the Faraday
Society.31:11-–27.
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., Monfardini, G. 2009. The
graph neural network model. IEEE Transactions on Neural Networks. IEEE
Transactions on Neural Networks. 20(23):61—80.
Searle, J. R. 1992. The Rediscovery of the Mind Cambridge (Mass), MIT.
Speer, R., Chin, J., Havasi, C. 2017. ConceptNet 5.5: An Open Multilingual Graph
of General Knowledge. In Proceedings of the AAAI 31.
Swainson, B. 2020. The Exponential
Internet. https://www.uschamberfoundation.org/bhq/exponential-internet. Last
accessed 13 Dec 2020.
Tan, M., Pang, R., Le, Q. V. 2019. EfficientDet: Scalable and Efficient Object
Detection. https://arxiv.org/pdf/1911.09070v7.pdf. Last accessed 27 Jul 2020.
Taylor, J. G. 2007. CODAM: A neural network model of consciousness. In Neural
Networks. 20:983–992.
Taylor, J. G. 2009. Beyond Consciousness?. In International Journal of Machine
Consciousness.1(1):11-–22.
Th´orisson, K. R. 2007. Integrated A.I. Systems. Systems. Minds & Machines.
17:11–25.
Th´orisson, K. R. 2012. A New Constructivist AI: From Manual Construction to Self-
Constructive Systems. In: P. Wang & B. Goertzel (Eds.)Theoretical Foundations
of Artificial General Intelligence. 145–171. New York: Springer.
Th´orisson, K. R., Bieger, J., Li, X., Wang, P. 2019. Cumulative Learning. In
Proceedings of 12th International Conference on Artificial General Intelligence
(AGI-19), Shenzhen, China, 198–209.
Tonioni, A., Di Stefano, L. 2017. Product recognition in store shelves as a sub-
graph isomorphism problem. In International Conference on Image Analysis and
Processing, 682-–693.
Unger, R. M., Smolin, L. 2015. The singular universe and the reality of time.
Cambridge University Press.
Unger, R. M. 2014. Roberto Unger: Free Classical Social Theory from Illu-
sions of False Necessity. Online Lecture. Retrieved on 13 December 2020 from
February 12, 2021 1:30 MetamodelAGI
A Metamodel and Framework for Artificial General Intelligence 23
https://www.youtube.com/watch?v=yYOOwNRFTcY
Wang, L. D., Liu, X. D. 2008. Concept analysis via rough set and AFS algebra.
Information Sciences. 178(21):4125-–4137.
Wang, P. 2005. Experience-grounded semantics: a theory for intelligent systems.
Cognitive System Research. 6:282—302. doi:10.1016/j.cogsys.2004.08.003
Wang, P. 2006. Rigid Flexibility: The Logic of Intelligence. Dordrecht: Springer.
Wang, P. 2010. Non-axiomatic logic (nal) specification. Retrieved on October 22,
2020 from: https://cis.temple.edu/ pwang/Writing/NAL-Specification.pdf
Wang, P. Insufficient Knowledge and Resources—A Biological Constraint and Its
Functional Implications In Proceedings of 2009 AAAI Fall Symposium Series.
Wang, P. 2019. On Defining Artificial Intelligence. Journal of Artificial General
Intelligence. 10(2):1–37.
Wang, P., Li, X., Hammer, P. 2018. Self in NARS, an AGI System. Frontiers
Robotics and AI. 5(20). doi: 10.3389/frobt.2018.00020
Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O.,
Bowman, S. R. 2019. SuperGLUE: A Stickier Benchmark for General-Purpose
Language Understanding Systems. https://arxiv.org/pdf/1905.00537.pdf. Last
accessed 13 Feb 2020.
Wikipedia. 2019. Wheat and chessboard problem.
https://en.wikipedia.org/wiki/Wheat and chessboard problem. Last accessed 14
Nov 2020.
Wille, R. 1980. Restructuring lattice theory: an approach based on hierarchies of
concepts. In: I. Rival (Ed.),Ordered Sets, 445–470. Dordrecht-Boston: Reidel
Yao, Y.Y. 2001. Information granulation and rough set approximation. Interna-
tional Journal of Intelligent Systems. 16:87–104.
Yao, Y.Y. 2009. Integrative levels of granularity. In Bargiela A and Pedrycz W
(Eds.),HumanCentric Information Processing Through Granular Modelling. 31–
47. Berlin: Springer.
Yao, Y.Y., Deng, X.F. 2012. A Granular Computing Paradigm for Concept Learn-
ing. In Ramanna, S., Jain, L., Howlett, R.J. (Eds.),Emerging Paradigms in Ma-
chine Learning. 307–326. London: Springer.
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., Leskovec, J. 2018.
Graph convolutional neural networks for web-scale recommender systems. In
Proceedings of SIGKDD2018.
Zhou, J., Cui, G., Zhang, Z., Yang, C., Liu, Z., Sun, M. 2018. Graph neural networks:
A review of methods and applications. https://arxiv.org/pdf/1812.08434.pdf.
Last accessed 24 May 2020
Zhu, Y. and Wu, Y. and Yang, Y. and Yan, Y.; Describing Unseen Videos via
Multi-Modal Cooperative Dialog Agents. The European Conference on Computer
Vision (ECCV).
... Graph and Hypergraph based Integration: The integration via graph and hypergraph provides an effective capturing relationship between symbolic and sub-symbolic (neural network level), independency on programming language, and most importantly, preserves the topological dependency of information [158]. Graph representations are inherently compositional in nature and allow us to capture entities, attributes and relations in a scalable manner. ...
... Layer-wise Integration and Attention: In the study [41], varieties of stressing (attention) mechanisms are employed between the symbolic and subsymbolic levels. Similarly, different layers of abstraction have been utilized to integrate high and low level information in terms of symbolic and subsymbolic [158]. The attention mechanism among these layers (L 0 , L 1 , ..., L n ) should be cognitively synergized among all the layers. ...
... The study [158] proposed a deep fusion based reasoning engine (DFRE) to represent the knowledge into four layers, where L 0 is close to the raw sensor data obtained from physical system. This study attempts to represent, integrate, and reason knowledge mimicking human approach. ...
Preprint
Full-text available
Neural networks have been rapidly expanding in recent years, with novel strategies and applications. However, challenges such as interpretability, explainability, robustness, safety, trust, and sensibility remain unsolved in neural network technologies, despite the fact that they will unavoidably be addressed for critical applications. Attempts have been made to overcome the challenges in neural network computing by representing and embedding domain knowledge in terms of symbolic representations. Thus, the neuro-symbolic learning (NeSyL) notion emerged, which incorporates aspects of symbolic representation and bringing common sense into neural networks (NeSyL). In domains where interpretability, reasoning, and explainability are crucial, such as video and image captioning, question-answering and reasoning, health informatics, and genomics, NeSyL has shown promising outcomes. This review presents a comprehensive survey on the state-of-the-art NeSyL approaches, their principles, advances in machine and deep learning algorithms, applications such as opthalmology, and most importantly, future perspectives of this emerging field.
... However, the authors believe that there is a need for a metamodel which will accommodate hierarchical knowledge representations. Latapie et al. (2021) proposed such a model inspired by Korzybski's (1994) idea about levels of abstraction. Their model promotes cognitive synergy and metalearning, which refer to the use of different computational techniques and AGI approaches, e.g., probabilistic programming, machine learning/Deep Learning, AERA Thórisson, 2020), NARS 4 (Wang, 2006(Wang, , 2013 to enrich its knowledge and address combinatorial explosion issues. ...
... The metamodel with the level of abstraction was actually achieved fully in the retail domain (see Latapie et al., 2021 for details). The flow of the retail use case with the metamodel is shown in Figure 2. The example for the levels of abstraction using the results of the retail use case is shown in Figure 3. Latapie et al. (2021) emphasized that no Deep Learning model was trained with product or shelf images for the retail use case. The system used for the retail use case is solely based on representing the subsymbolic information in a world of bounding boxes with spatial semantics. ...
... This initial attempt at a commercial neurosymbolic system dramatically improved the ability of the system to generalize and learn behaviors of interest, which in this case were all related to safety. In essence the objective of the system was to raise alerts if any two moving objects either made contact FIGURE 3 | Levels of abstraction for retail use case (from Latapie et al., 2021). or were predicted to make contact as well as to learn other dangerous behaviors such as jay walking, wrong-way driving, and such. ...
Article
Full-text available
A cognitive architecture aimed at cumulative learning must provide the necessary information and control structures to allow agents to learn incrementally and autonomously from their experience. This involves managing an agent's goals as well as continuously relating sensory information to these in its perception-cognition information processing stack. The more varied the environment of a learning agent is, the more general and flexible must be these mechanisms to handle a wider variety of relevant patterns, tasks, and goal structures. While many researchers agree that information at different levels of abstraction likely differs in its makeup and structure and processing mechanisms, agreement on the particulars of such differences is not generally shared in the research community. A dual processing architecture (often referred to as System-1 and System-2) has been proposed as a model of cognitive processing, and they are often considered as responsible for low- and high-level information, respectively. We posit that cognition is not binary in this way and that knowledge at any level of abstraction involves what we refer to as neurosymbolic information, meaning that data at both high and low levels must contain both symbolic and subsymbolic information. Further, we argue that the main differentiating factor between the processing of high and low levels of data abstraction can be largely attributed to the nature of the involved attention mechanisms. We describe the key arguments behind this view and review relevant evidence from the literature.
... However, the authors believe that there is a need for a metamodel which will accommodate hierarchical knowledge representations. Latapie et al. (2021) proposed such a model inspired by Korzybski's (1994) idea about levels of abstraction. Their model promotes cognitive synergy and metalearning, which refer to the use of different computational techniques and AGI approaches, e.g., probabilistic programming, machine learning/Deep Learning, AERA (Thórisson, 2020, OpenNARS 1 (Wang, 2006;Wang, 2010) to enrich its knowledge and address combinatorial explosion issues. ...
... The metamodel with the level of abstraction was actually achieved fully in the retail domain (see Latapie et al., 2021 for details). The flow of the retail use case with the metamodel is shown in Figure 2. The example for the levels of abstraction using the results of the retail use case is shown in Figure 3. Latapie et al. (2021) emphasized that no Deep Learning model was trained with product or shelf images for the retail use case. The system used for the retail use case is solely based on representing the subsymbolic information in a world of bounding boxes with spatial semantics. ...
... For detecting anomalies and finding regime change locations, Matrix Profile algorithms are used (see Yeh et al., 2016;Gharhabi et al. 2017 for Matrix Profile and Semantic Segmentation). Similar to the retail use case, millions of sensory data points are reduced to a much smaller Category Figure 3. Levels of Abstraction for Retail Use Case (from Latapie et al., 2021) number of events based on the semantic segmentation points. These points are used to form a histogram of regime changes as shown in Figure 4. ...
Preprint
Full-text available
A cognitive architecture aimed at cumulative learning must provide the necessary information and control structures to allow agents to learn incrementally and autonomously from their experience. This involves managing an agent's goals as well as continuously relating sensory information to these in its perception-cognition information stack. The more varied the environment of a learning agent is, the more general and flexible must be these mechanisms to handle a wider variety of relevant patterns, tasks, and goal structures. While many researchers agree that information at different levels of abstraction likely differs in its makeup and structure and processing mechanisms, agreement on the particulars of such differences is not generally shared in the research community. A binary processing architecture (often referred to as System-1 and System-2) has been proposed as a model of cognitive processing for low- and high-level information, respectively. We posit that cognition is not binary in this way and that knowledge at any level of abstraction involves what we refer to as neurosymbolic information, meaning that data at both high and low levels must contain both symbolic and subsymbolic information. Further, we argue that the main differentiating factor between the processing of high and low levels of data abstraction can be largely attributed to the nature of the involved attention mechanisms. We describe the key arguments behind this view and review relevant evidence from the literature.
... Meta Learning: For our purposes, the concepts of Meta Learning deal with how a machine learns Machine Learning, i.e., what methods with what hyperparameters lead to good results [31], [32]. The general concepts also apply to other fields; for example in Natural Science, applications that learn to transfer simulation settings from metadata save computational time and energy [33], [34], [35]. ...
Conference Paper
Full-text available
Transfer Learning methods aim to reuse previously acquired knowledge about a source task to facilitate learning of a target task. In this paper, we present a Meta Learning approach to find optimal hyperparameters for Transfer Learning processes given previously known metadata about the source task, the target task, and the pre-trained model. We collected metadata and model parameters from more than 15,000 Transfer Learning processes in a dataset, which we use to learn metamodels that predict a Transfer Learning process result in terms of accuracy on the validation sets, given prior information such as the number of epochs, learning rates, optimizers, etc. Using feed multilayer perceptrons (MLP), we show that and how our approach finds efficient hyperparameters for Transfer Learning for image classification.
Conference Paper
Full-text available
Using a proprietary visual scene object tracker and the Open-NARS reasoning system we demonstrate how to predict and detect various anomaly classes. The approach combines an object tracker with a base ontology and the OpenNARS reasoning system to learn to classify scene regions based on accumulating evidence from typical entity class (tracked object) behaviours. The system can autonomously satisfy goals related to anomaly detection and respond to user Q&A in real time. The system learns directly from experience with no initial training required (one-shot). The solution is a fusion of sub-symbolic (object tracker) and symbolic (ontology and reasoning).
Article
Full-text available
This article systematically analyzes the problem of defining “artificial intelligence.” It starts by pointing out that a definition influences the path of the research, then establishes four criteria of a good working definition of a notion: being similar to its common usage, drawing a sharp boundary, leading to fruitful research, and as simple as possible. According to these criteria, the representative definitions in the field are analyzed. A new definition is proposed, according to it intelligence means “adaptation with insufficient knowledge and resources.” The implications of this definition are discussed, and it is compared with the other definitions. It is claimed that this definition sheds light on the solution of many existing problems and sets a sound foundation for the field.
Chapter
Full-text available
An important feature of human learning is the ability to continuously accept new information and unify it with existing knowledge, a process that proceeds largely automatically and without catastrophic side-effects. A generally intelligent machine (AGI) should be able to learn a wide range of tasks in a variety of environments. Knowledge acquisition in partially-known and dynamic task-environments cannot happen all-at-once, and AGI-aspiring systems must thus be capable of cumulative learning: efficiently making use of existing knowledge while learning new things, increasing the scope of ability and knowledge incrementally—without catastrophic forgetting or damaging existing skills. Many aspects of such learning have been addressed in artificial intelligence (AI) research, but relatively few examples of cumulative learning have been demonstrated to date and no generally accepted explicit definition exists of this category of learning. Here we provide a general definition of cumulative learning and describe how it relates to other concepts frequently used in the AI literature.
Preprint
Full-text available
Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains a challenge. Here we describe a large-scale deep recommendation engine that we developed and deployed at Pinterest. We develop a data-efficient Graph Convolutional Network (GCN) algorithm PinSage, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information. Compared to prior GCN approaches, we develop a novel method based on highly efficient random walks to structure the convolutions and design a novel training strategy that relies on harder-and-harder training examples to improve robustness and convergence of the model. We also develop an efficient MapReduce model inference algorithm to generate embeddings using a trained model. We deploy PinSage at Pinterest and train it on 7.5 billion examples on a graph with 3 billion nodes representing pins and boards, and 18 billion edges. According to offline metrics, user studies and A/B tests, PinSage generates higher-quality recommendations than comparable deep learning and graph-based alternatives. To our knowledge, this is the largest application of deep graph embeddings to date and paves the way for a new generation of web-scale recommender systems based on graph convolutional architectures.
Article
Full-text available
This article describes and discusses the self-related mechanisms of a general-purpose intelligent system, NARS. This system is designed to be adaptive and to work with insufficient knowledge and resources. The system’s various cognitive functions are uniformly carried out by a central reasoning-learning process following a “non-axiomatic” logic. This logic captures the regularities of human empirical reasoning, where all beliefs are revisable according to evidence, and the meaning of concepts are grounded in the system’s experience. NARS perceives its internal environment basically in the same way as how it perceives its external environment although the sensors involved are completely different. Consequently, its self-knowledge is mostly acquired and constructive, while being incomplete and subjective. Similarly, self-control in NARS is realized using mental operations, which supplement and adjust the automatic inference control routine. It is argued that a general-purpose intelligent system needs the notion of a “self,” and the related knowledge and functions are developed gradually according to the system’s experience. Such a mechanism has been implemented in NARS in a preliminary form.