Content uploaded by Luc P. Beaudoin
Author content
All content in this area was uploaded by Luc P. Beaudoin on Aug 23, 2023
Content may be subject to copyright.
Beaudoin, Spring 2023 (contextual information-retrieval and consciousness)
1
How contextual information-retrieval software can support
knowledge-building by extending intermediate and long-term
consciousness
Luc Beaudoin, Simon Fraser University, Faculty of Education, 8888 University Drive, Burnaby BC V5A 1S6,
Canada, lpb@sfu.ca(1) and co-founder at CogSci Apps Corp.
Abstract: To do high-caliber knowledge work typically involves processing manifold knowledge
resources related to the information one is trying to understand and the conceptual artefacts one is
constructing. For instance, writing a paper like this one requires consulting a project plan, an
outline, some emails, PDFs, figures, web pages, and other information. Pertinent information is
often scattered across many apps and repositories. The ability to rapidly access pertinent
information is a significant under-acknowledged challenge to knowledge workers of all levels of
domain expertise. Current operating systems do not sufficiently facilitate such information access.
I contend that most knowledge workers fail to understand the information-retrieval problems they
face and how to solve them. This paper describes the contextual information-retrieval (CIR)
problems and opportunities knowledge workers face and how they can be addressed with software
such as Hookmark, by CogSci Apps Corp. of which I am a founder, software that interacts with
‘link-friendly’ software to serve contextually relevant links not merely to web items but
information with custom URL schemes. I argue that CIR software can support knowledge-
building by scaffolding human-consciousness.
Introduction
Knowledge building, such as writing an academic paper, is cognitively demanding. It often involves integrating
manifold objective information from multiple virtual locations —the local file system, and various apps (such as
task managers, email apps, outliners, concept mapping apps and web browsers). To access information, users
primarily rely on search tools (such as Spotlight® and web search engines) or folders. However, searching for
information is more cognitively demanding than it may seem. One needs to mentally formulate a search query, type
the query, issue the search command, look at search results, comprehend them, scroll, tap, and select. Each of these
partly physical actions involves multiple, complex neural and mental (world 2’) processes. These ancillary activities
place demands on working memory and divert other world 2’ resources from their super-ordinate goal. I believe
these ancillary processes interfere with the cognitive productivity of knowledge workers more than is publicly
acknowledged, and more than is necessary.
The most direct solution to this problem is contextual information-retrieval (CIR) software that supports
the creation, suggestion, and usage of contextually hyper-relevant links (hyperlinks) to information across multiple
apps. We are all familiar with web recommendation systems — for instance on Amazon or news sites. The quality
of their recommendations tends to be so low, in my opinion, that they hardly even serve the needs of the owners of
the websites that provide them; and of course they do not tend to be designed to serve the needs of consumers either
(typically recommendation systems are meant to keep people on websites and to monetize their presence).
Knowledge work requires information that is hyper-relevant to one’s knowledge building goals. Let’s call software
that supports creating and using links to hyper-relevant information HR-CIR software.
None of the main operating systems (Windows®, macOS®, Android, iOS® or iPadOS) ubiquitiously
facilitate cross-app linking in a manner that can be directly leveraged by the user. For instance, whereas ‘Copy’ and
‘Paste’ are first-class commands that are nearly always provided in the same location in a graphical user interface
(the Edit menu) with the$ same$ name$and$ same$keyboard$ shortcuts,$ most$apps$ do$ not$provide$ a$‘Copy$ Link’$
command$ —$ let$ alone$ provide$ it$ in$ the$ same$ place$ with$ the$ same$ keyboard$ shortcut.$ Apple’s$ own$ Notes,$
Messages$and$Mail$apps,$for$instance,$do$not$currently$contain$a$ ‘Copy$ Link’$command.$When$a$command
$is
$
provided$by$an$app$developer$for$copying$links,$it$has$an$unpredictable$name$and$location,$and$often$merely$
yields$a$ URL,$not$a$complete$link$(a$complete$link$has$both$ a$name$and$a$URL).$For$instance,$Apple’s$macOS$
Podcasts$and$Music$ apps$have$‘copy$ link’$ contextual$menu$items$which$ are$ not$replicated$in$the$ Edit$ menu$
and$do$not$yield$a$named$link,$just$a$URL.$$
As$its$general$editor,$I$led$the$creation$of$the$Manifesto$for$Ubiquitous$Linking.$This$manifesto,$based$
on$(Beaudoin & Jomphe, 2020),$was$initially$signed$by$24$professors,$software$developers,$and$‘influencers’.$It$
Beaudoin, Spring 2023 (contextual information-retrieval and consciousness)
2
has$since$been$signed$by$many$others.$This$manifesto$is$modeled$on$the$highly$impactful$Agile$manifesto.$Our$
manifesto$ argues$ that$ user-facing$ knowledge-building$ software$should$ provide$ a$ user$ interface$ (UI)$ and$
application$ programming$ (API)$ interface$ for$ linking.$ $ We$ hope$ this$ manifesto$ will$ have$ an$ even$ greater$
impact$on$computing$than$the$Agile$manifesto$had.$You,$the$reader$are$invited$to$read$sign,$and$comment$on$
the$manifesto.
We at CogSci Apps Corp. have developed software for macOS, called Hookmark, to address the need for
ubiquitous linking. Please refer to my conflict of interest disclosure as I am a director of CogSci Apps Corp.
Hookmark leverages the graphical user interface and API of manifold software (such as email apps, note-taking
apps, reference management software, and task managers) to provide inter-app link services to Mac users. It
provides a floating contextual information window that enables users to copy a link (name + URL) to the item that is
currently open or selected in the foreground app. For instance, in Microsoft Word a user might invoke Hookmark
and type ⌘ C to get a robust link to the foreground document. “Robust” here means that unlike file:// links, the
links still normally work even if the file moves (a bit like Finder aliases). These links can be pasted anywhere that
supports pasting including in the contextual window itself, where they act as bi-directional linking (meaning that
when one invokes the context window on the other side of the link Hookmark displays a link back to the source).
In the next section, I present the ‘Cognitive Productivity’ meta-effectiveness framework for using
knowledge resources. I then argue that (a) HR-CIR software can support ‘surfing’ and delving PDFs; (b) HR-CIR
software can support the production of new objective knowledge; and (c) HR-CIR can effectively extend
intermediate and long-term consciousness. I use ‘consciousness’ in the sense of Merlin Donald's Multi-component
Convergent theory of consciousness (Donald, 2001; Donald, 2004).
A Meta-effectiveness framework
It is helpful to situate CIR within a broader framework of meta-effectiveness. Meta-effectiveness consists of
the skills and dispositions (interpreted as mindware (Perkins, 1995; Stanovich, 2009; Beaudoin, 2014a)) involved in
using knowledge (‘world 3’) to become more effective at solving problems, and building knowledge and other
cultural artefacts. Mindware is a concept about World 2’ (Beaudoin, 2014a). World 2’ in turn is a modernization of
the World 2 concept developed by (Popper, 1979) and (Bereiter, 2002); World 2’ consists of the virtual machinery.
Mindware is not to be confused with (Clark, 2013)’s usurpation of ‘mindware’ as a World 3 concept. In contrast
with (Bereiter, 2002), Perkins, Stanovich, and I view mindware as causal layers of mental reality that are important
objects of cognitive science.
The “Cognitive Productivity” framework for meta-effectiveness (Beaudoin, 2014a; Beaudoin, 2018),
summarized in Figure 1 below, consists of seven practical principles for the self-regulation of knowledge building,
problem-solving and personal (‘mindware’) development. This framework also is expository in that it specifies a
framework for understanding ourselves and each other.
Beaudoin, Spring 2023 (contextual information-retrieval and consciousness)
3
Figure 1. “Cognitive Productivity” framework for meta-effectiveness
Principle 1 suggests people lead themselves in relation to objective knowledge. Principle 2 specifies how to
manage one’s life: managing one’s projects (world 3), one’s knowledge sources (world 3) and one’s attention (a
subset of World 2’). Principle 3 suggests assessing knowledge in terms of the “CUP’A” framework, distinguishing
between the caliber, utility and potency of information while being careful not to overvalue its appeal. Principle 4
describes surfing: superficial but important interactions with information, such as skimming, filing and tagging
information. Principle 5, ‘delve deeply’, involves deep interaction with knowledge sources in the hope of
comprehending the information well enough to use it (to solve problems, create products and/or improve oneself).
This can be supported with various cognitive tools, such as a categorical tagging tool that enables one to indicate
one’s relationship to the information (e.g., “i.e., don’t understand this”, a solid basis for bridging knowledge gaps),
or the information in relation to the document and other conceptual artifacts (“this is a major claim”, “this is a
hypothesis”). Principle 6, ‘practice productively’ (Beaudoin, 2014b) applies some of the most important findings in
cognitive science (from test-enhanced learning, deliberate practice and memory retrieval) to foster lasting
understanding (in Bereiter’s sense of the term) and improvements to one’s mindware; this may develop long-term
consciousness and the transfer of learning (principle 7). One can practice productively by using sophisticated
flashcard software such as Anki in sophisticated ways, thinking in terms of challenges rather than cards or questions.
Principle 7, ‘apply knowledge’, deals with the difficult challenge of applying knowledge when it is relevant
(‘transfer of learning’). This entire framework was developed to deal with working with all kinds of knowledge
resources: videos, screencasts, audiobooks, presentations, interactive meetings, etc. This is reflected not merely in
choice of terms (‘delving’ rather than ‘deep reading’), but a long and varied collection of examples involving
manifold IT.
Beaudoin, Spring 2023 (contextual information-retrieval and consciousness)
4
CIR applies most directly to principle 4 “surf strategically”. That it is to say that it pertains to superficial
information-processing. The point of CIR is to remove the friction between the intention of retrieving a specific
(contextually relevant) information item and its retrieval. However, these principles are all intertwined. Applying
each principle requires the efficient, discriminative retrieval of contextually relevant information.
How HR-CIR software can support surfing and delving PDFs
To deeply understand a demanding document (or set of documents) can require formulating explicit goals, taking
textual notes about the document, consulting other resources, engaging in productive practice (principle 6), and so
forth. Here we focus on delivng PDFs as an illustration since the highest caliber scientific content is in PDF. PDF
and PDF apps are not conducive to detailed note-taking. Rather than taking extensive notes inside a PDF, one should
normally take notes in one’s favorite note-taking app. However, most people do not know how (or so I claim), at a
random future moment, to quickly navigate, bidirectionally (back and forth) between the PDF and notes about the
PDF. I say ‘future moment’ because while it might be easy to navigate between a PDF and an external note
document (say in Obsidian or Apple Notes) when one has just created the note, it is a different matter days or weeks
later. One can’t necessarily store all related information in the same location (say folder). Moreover, many apps are
not file-based, they have their own data containers. Without ubiquitous hyperlinks, in order to quickly find their
notes about a PDF, users would need to be very systematic about where they store their notes, how they name their
notes, how they search for them, etc. To do this efficiently (without much searching) requires HR-CIR software.
This same navigation principle applies to navigating between a PDF and other types of information: one’s
tasks, emails about the PDF, a concept map about it, an outline about it, a photo of handwritten notes one has taken
about it, related PDFs, etc.
Figure 2 below illustrates the general case.
Figure 2. Several types of information items one should be able to create and bidirectionally link to a PDF for
instance bidirectional access between the information items.
Beaudoin, Spring 2023 (contextual information-retrieval and consciousness)
5
I contend that often knowledge workers simply decide not to take persistent notes, draw diagrams, etc.,
because they are stymied by the CIR problem. CIR may thus be an extremely significant barrier towards knowledge
building.
Using HR-CIR software can address all of these problems. Ideally, one can hook a PDF to any document in
any link-friendly software. A ‘hook’ is a bidirectional link, meaning one can navigate between the source and
destination (more generally, either side of a hook can be considered the source or destination).
Figure 3. Hookmark (HR-CIR app) invoked on a PDF in Skim PDF reader.
Figure 3 shows Hookmark’s contextual (floating) window invoked on a PDF file open in Skim (PDF
reader). The title and content of the contextual window changes automatically as the foreground app’s foreground
item changes. The contextual PDF and window are named “Lemyre-2020-Pre-sleep cognitive activity in adults”).
Hooked to the PDF are an email, a task list (in TaskPaper app of a blog post to write about the paper), a concept
map, an infographic about it in PDF format downloaded from the web), an Apple Notes note about the PDF, a text
file, an outline, the Bookends link to this file, an ebook on sleep, and a DEVONthink group about sleep papers. One
can double-click on a hooked link to access the destination. Of course, one would not normally hook so much
information to a given PDF. Most PDFs would have no hooks, some would have a few, some many. The point of the
figure is to show that manifold information can be hooked to a PDF or other item. These links are not stored in the
contextual document; they are stored in a database that references the document.
How HR-CIR software can support writing papers (and creating other conceptual
artefacts)
When writing a knowledge-intense document, or creating other conceptual artefacts, it is essential to be able to
quickly reference other resources. If the artefact is collaborative, one may need to be able to quickly access emails,
forum topics, and private messages about the artefact. Without HR-CIR software this is quite difficult. One can
imagine that passive AI recommendation systems in the future will address this challenge; but none to date seems to
address all the requirements specified here (e.g., working across apps). Moreover, arguably there is a ‘desirable
difficulty’ benefit to being intentional about what key information is connected to the current context. These are
hypotheses that can now be tested given that HR-CIR software exists (Hookmark).
Beaudoin, Spring 2023 (contextual information-retrieval and consciousness)
6
Figure 4. HR-CIR software (Hookmark) invoked in the context of a draft Word document.
Figure 4 above shows Hookmark’s contextual (floating) window invoked on this paper that I am writing.
The title of the contextual window reflects the file name of the paper. The top item in the list is the project plan for
this document. This paper is also linked to a relevant email from the program committee. The source of the Figures
used in this document are also linked, so that I can quickly go back to them when I want to edit them, without
needing to navigate folders on the file system. My draft is also hooked to the conference’s program committee’s web
page, and to a Grammarly page. On those web pages, Hookmark presents me a link back to this document on my file
system. Only I can see my hooks and access my local information, of course. The template from the program
committee is also conveniently hooked to this draft. Not only do all these hooks (again: bidirectional links) let me
get to the linked items from my draft, in the future I can access this draft through any hooked item. More generally,
this provides multiple contextual retrieval routes.
Thus, CIR software is a type of associative memory. Whenever users copy links or hook information
together with Hookmark, Hookmark adds the link to its database as a private bookmark (URL, name, etc.). My
hypothesis is that once a user gets into the habit of copying links and hooking information, their bookmark database
becomes a discriminative database of information worth consuming; search engines such as Spotlight® and
launchers index more information. Their search results are not highly context sensitive. This is not to deny the
Beaudoin, Spring 2023 (contextual information-retrieval and consciousness)
7
importance of random access searching for information. It is rather to emphasize the importance of a complementary
type of information access.
Deep PDF links, deep ebook links, and deep multimedia links
Thus far in this paper, I have focused on HR-CIR from links that are presented in a context window. However, often
the context for a link is in a document or object outside of the HR-CIR software. The reader, being familiar with
web links, will have no difficulty appreciating that just as it is useful to store web links in one’s local files, tasks,
notes and other local data, it would be quite useful to store links to arbitrary data (local or not) in one’s local files,
tasks, notes and other local data. Those links also qualify as contextual. Ubiquitous linking is also important for
gathering links to arbitrary information and pasting them outside the HR-CIR software. As noted, many apps have a
Copy Link command, and many apps do not. As long as the target app has an API for linking (described in the
technical requirements section of the Manifesto for Ubiquitous Linking), then HR-CIR software (like Hookmark)
can provide a uniform UI to copy and manipulate links. These links can then be pasted anywhere.
The manifesto specifies that software should provide deep linking services when possible. A deep link is to
a specific location within a document (or other complex structure). This applies most obviously to read-only
documents like PDF documents, ebooks, and multimedia. For example, when a user selects text in compatible PDF
software, invokes Hookmark’s floating window, and chooses the `Copy Link` function, Hookmark may return a
deep PDF link. This link contains identifying information about the file (akin to a information underlying a symbolic
link in UNIX or a Finder alias on macOS), the page number of the selection, and the start and end positions of the
selection. When the user clicks on this link, Hookmark attempts to locate the PDF document (even if it has moved or
been renamed on the user’s file system), and asks the default PDF reader to open the PDF file and to scroll to the
specific location. Hookmark can also copy the quoted text. Here is an example taken from the peer-reviewed paper
shown in Figure 1 (Lemyre et al., 2020):
Ø Clearly, cognitive activity is present both in normal transition to sleep and in insomnia episodes. Thus, the
question arises whether cognitive activity in insomnia is qualitatively or quantitatively different.
Ø Lemyre-2020-Pre-sleep cognitive activity in adults A systematic review-Lemyre2020fu-
10.1016j.smrv.2019.101253.pdf
The foregoing link is
hook://file/f5tc5dzSL?p=Qm9va2VuZHMvQXR0YWNobWVudHM=&n=Lemyre%2D2020%2DPre%2Dsleep%20
cognitive%20activity%20in%20adults%20A%20systematic%20review%2DLemyre2020fu%2D10%2E1016j%2Es
mrv%2E2019%2E101253%2Epdf#p=10&x=0&y=0&s=1692&e=1903
The user can then paste this link in his notes about the paper, for whatever knowledge-building purpose he
might have, such as to question the statement, review it, discuss it with peers (see next section), etc. HR-CIR
software like Hookmark supports deep linking to PDFs (in many PDF apps), Kindle, and QuickTime with partial
support for YouTube . (YouTube does not currently provide an API to obtain the timestamp of the current video,
only a UI is available.) Being able to easily copy and use links to specific locations in target content is expected to
be protective of working memory and intermediate consciousness. It means that when the user needs to process
information, he can jump specifically to the destination without needing to search for the document, open the
document, search for the target information in the document, and so forth.
Social aspects of HR-CIR
In order to be useful, knowledge-building HR-CIR software also needs to support sharing and discussion
knowledge. Hookmark provides extensive support for sharing and resolving links between knowledge workers on
different computers, though currently the public version works only on macOS. Here are some social requirements
for HR-CIR software.
First, it needs to be possible for users to easily share links to emails. If I send you the reader an email, I
should be able in the future to refer to this email using a link that I can send in any medium (email, text, forum, etc.)
Of course, only people who have access to the email can resolve an email link. (The entire discussion in this paper is
about sharing links to information, not sharing the target documents.) The Internet standard for RFC-5322
(Resnick, 2008) means this is technically possible. In sum, with Hookmark if one is using software that conforms to
the Manifesto for Ubiquitous Linking in compatible software, one can in principle copy a link to the currently
selected or open email. Through any available means, one can send this to the recipient. If the recipient has access to
the email in compliant software (such as Apple Mail for Ventura) and Hookmark (free or other version), then the
Beaudoin, Spring 2023 (contextual information-retrieval and consciousness)
8
recipient can click on the link and open the message. Otherwise, the sender either needs to copy parts of the original
email or reference it in ordinary language; and the recipient needs to parse and use this information manually.
Secondly, it should be possible for users to copy a link to any locally represented file that is (a)
automatically synced through cloud syncing service such as Dropbox or iCloud; or (b) synced through a version
control system such as Git or Subversion. Also it should under certain assumptions be possible for the recipient who
has access to the shared file to click on this link and reveal it on their file system. This should not require usage of a
web link (https://) etc or a web browser as that would interfere with psychological flow. Unlike resolving emails,
resolving such links can only be done on a heuristic basis because the notion of identical file in this type of situation
is not sufficiently precise. In contrast to this heuristic case, collaborative synchronous web editing software can have
the advantage of having a globally unique URL that is unambiguous.
By reducing the friction involved in referencing files, emails and other information, and resolving these
references, knowledge workers may be able to help themselves and each other stay in psychological flow.
Consciousness and knowledge building
According to Donald’s multi-component convergent theory of consciousness, human consciousness involves a
capacity to integrate information from multiple sources not merely in short time frames involved in sensory binding
and working memory, but over periods of minutes, hours and longer. These multiple sources include multiple mind-
brain systems with an individual, the brains of others, and conceptual artifacts. The ability to rapidly retrieve and
understand information from multiple sources is a critical feature of human consciousness. Technologies such as
writing and electronic computing devices can extend our consciousness.
The relatively durable encoding of information that is characteristic of expertise requires frequent rapid
access to the same information over time. The slower the access, the less information one can process and ultimately
master. As we noted in the Manifesto for Ubiquitous Linking’s rationale, Donald expounded upon the basic
principle of information technology extending consciousness:
Ø For instance, if we sit in front of a computer, the screen becomes a temporary external working memory
field. Anything we display in it is processed in consciousness, and the viewer is locked into an interactive
loop with the display while creating, writing or thinking. This changes the traditional function of the brain’s
biological working memory system. (Donald, 2004) (p. 57)
Ubiquitous linking capabilities tighten this loop, making it easier to rapidly access and integrate manifold
information from various external stores into the virtual machinery of mind.
The experience of psychological flow has been defined as “a gratifying state of deep involvement and
absorption that individuals report when facing a challenging activity and they perceive adequate abilities to cope
with it” (Peifer et al., 2022). I believe that using HR-CIR can facilitate the experience of psychological flow in
knowledge work. This may have multiple benefits, such as being enjoyable, leading to better quality work, and
being conducive to the development of self-efficacy.
Ignorance of the friction caused by lack of HR-CIR software and skills is reminiscent of how before the
iPhone® was first launched in 2006 most knowledge workers failed to realize they even needed a convergence
device like the iPhone. The advent of ubiquitous hyperlinking software may lead countless knowledge workers to a
similar ‘aha’ experience in felt need for CIR tools and skills, and an equally impressive revolution in knowledge
building. Time and future research are required to test this theory.
Endnotes
(1). COI disclosure: Dr. Luc P. Beaudoin an Adjunct Professor at Simon Fraser University, and he is a director of
CogSci Apps Corp., owner of CogZest and co-founder of Somnolence+ Inc. These Canadian businesses develop
products based on his research, including Hookmark, mySleepButton®, books, and training services.
References
Beaudoin, L., & Jomphe, D. (2020). A manifesto for user and automation interfaces for hyperlinking. In (pp. 266-
269).
Beaudoin, L. P. (2014a). Cognitive productivity: Using knowledge to become profoundly effective. CogZest.
https://leanpub.com/cognitiveproductivity/
Beaudoin, Spring 2023 (contextual information-retrieval and consciousness)
9
Beaudoin, L. P. (2014b). Developing expertise with objective knowledge: Motive generators and productive
practice. In J. Wyatt & D. Petters (Eds.), (pp. 161-189). Springer.
Beaudoin, L. P. (Ed.) Linking Manifesto – Manifesto for Ubiquitous Linking. https://linkingmanifesto.org
Beaudoin, L. P. (2018). Cognitive Productivity with macOS: 7 Principles for Getting Smarter with Knowledge.
CogZest. https://leanpub.com/cognitive-productivity-macos
Bereiter, C. (2002). Education and mind in the knowledge age. Laurence Earlbaum Associates.
http://www.amazon.com/Education-Mind-Knowledge-Carl-Bereiter/dp/0805839429
Clark, A. (2013). Mindware. OUP USA.
Donald, M. (2001). A mind so rare: The evolution of human consciousness. W. W. Norton & Company.
Donald, M. (2004). The definition of human nature in the context of modern neurobiology. In D. A. Rees & S. P. R.
Rose (Eds.), (pp. 34-58). http://www.cadmusjournal.org/article/volume-2/issue-2-part-2/digital-era-ch
allenges-modern-mind
Lemyre, A., Belzile, F., Landry, M., Bastien, C. H., & Beaudoin, L. P. (2020). Pre-sleep cognitive activity in adults:
A systematic review. Sleep Medicine Reviews, 50, 101253. https://doi.org/10.1016/j.smrv.2019.101253
Peifer, C., Wolters, G., Harmat, L., Heutte, J., Tan, J., Freire, T., Tavares, D., Fonte, C., Andersen, F. O., van den
Hout, J., Šimleša, M., Pola, L., Ceja, L., & Triberti, S. (2022). A Scoping Review of Flow Research.
Frontiers in Psychology Front. Psychol., 13, 815665. https://doi.org/10.3389/fpsyg.2022.815665
Perkins, D. (1995). Outsmarting IQ: The emerging science of learnable intelligence. Free Press.
Popper, K. R. (1979). Objective knowledge. Oxford University Press.
Resnick, P. (Ed.), P. R. (2008). RFC-5322: Internet Message Format. https://www.rfc-editor.org/rfc/rfc5322
Stanovich, K. E. (2009). What intelligence tests miss: The psychology of rational thought. Yale University Press.
Acknowledgments
I’d like to thank the software development team of CogSci Apps Corp. for helping bring Hookmark to market.
Thanks also to Merlin Donald for personal communications about his theory.
