Liane Gabora's Indra's Net Project Temporary Website

Overview of the Representational System

The representational system has three components. The first is based on optics, the second is based on electromagnetism, and the third is based on quantum mechanics. This document only provides a peek into rudimentary elements of the optics part of the representational system. (If you go to my Publications page and check out the papers on quantum models of cognition they may give you a glimpse into some of the other components). (Note: I am writing a series of fictional stories and a novel based on this representational system. You can meet the characters in the novel here.)

The principles of optics provide an elegant and intuitive metaphor that renders thoughts, motives, and interactions visible and thereby more concrete and manageable. A person’s prana--the Hindu term for creative life force, also known as chi, vital energy, or Inner Light (IL)--is portrayed as light of a unique spectrum. Individual differences in prana are indicated by the intensity and shade of the IL; see Fig. 1a versus 1b. The mind is portrayed as one or more concentric shells that surround the IL. The IL and shells constitute the user’s vessel. Arousal level is depicted by flickering sparks that vary with respect to their density and speed of flickering; see Fig. 1b versus Fig. 1c. An avatar generates a unique Inner Music (IM) using Shawn Bell's self-organizing NetWorks music generation program, which will translate avatar characteristics into node settings.

Fig. 1. Depictions with user’s vessel in foreground, with other vessels in background. Prana is represented by intensity of IL. Arousal is represented by flickering sparks. a) High IL, lower arousal. b). Low IL, low arousal. c) Low IL, high Arousal.

Arousal

Arousal level is indicated by IL and IM strength and consistency. High arousal results in erratic flickering IL and energetic IM. Low arousal results in a consistent IL and calm IM. Arousal level is depicted by flickering sparks that vary with respect to their density and speed of flickering; see second and third image above.

Thought

A stream of thought is depicted as a beam of light reflecting within the vessel, such that it becomes more focused each time it reflects (Fig. 2). This represents how, as you 'bounce an idea around', view it from different perspectives, or 'reflect' on it, you gain a richer understanding of it. The greater the perceived importance or emotional valence of a thought, that stronger the beam's intensity. The greater the extent to which a thought deviates from the 'here and now', i.e., lies in the realm of imagination, the more diffuse the beam

Conscious vs Unconscious Thought

Unconscious thoughts (e.g., ideas that remain below the threshold of awareness) are depicted as beams that do not stray beyond the central cavity of the vessel, i.e., do not reach the shells. Conscious thoughts are depicted as beams that penetrate, and are altered by, interactions with the shells.

Cognitive operations are depicted as alterations of the beam as it interacts with a shell. Inner shells (not shown) perform simple operations (e.g., categorizing something/someone as good or bad), while outer shells perform sophisticated cognitive operations (e.g., moral reasoning). A stream of thought is dynamically depicted as reflection of a beam of light within the vessel

Fig. 2. Though represented as reflecting beam of light

Expressed Thought

Thoughts that are put into words or expressed through another means such as gesture, music, or art—are depicted as beams that pass through the vessel such that they are detectable by others, as in Fig. 3a. The inability to articulate an idea clearly is represented as divergence of the beam as it passes through the vessel (not shown). Each person who considers an idea may subtly affect it, i.e., ‘put their own spin on it’, as in Fig. 3b. Absence of thought—i.e., being ‘in the moment,’ or in a meditative state---is represented as the absence of a beam of light; the vessel is lit up by the IL only, as in Fig. 3c.

Fig. 3. (a) Externally expressed thought. (b) Transformation of an idea as it moves from one vessel to another. c) Vessel lit up by only IL (meditative state). d) (In progress) Since the surface of a sphere is concave, a ray of light diverges, or becomes less focused, after passing through a sphere. The more focused it is, the less it diverges. Note: Arrows show direction of travel. In this and all subsequent figures, for clarity, only the minimum number of vessels needed to illustrate the point are shown.

Centered vs Uncentered

Feeling centered is portrayed as the IL coming to a focal point at the center of the vessel, such that it emanates in every direction without refraction, as in Fig. 4a; thus, the individual comes across in an undistorted, authentic way. Conversely, feeling uncentered is portrayed as the IL coming to a focal point at the periphery of the vessel, as in Fig. 4b. In this case, there is only a narrow range of angles by which it can exit the sphere without undergoing refraction. Any irregularity in the vessel distorts the natural flow of light such that the IL doesn’t come to a focus at the center. (Below we show depictions of how hiding aspects of yourself can make you feel off-center, inauthentic, or insincere.)

Fig. 4. The IL comes to a focus at a) the center of the vessel when the user feels centered, and b) the periphery of the vessel when the user feels uncentered.

Repression & Rumination

The platform can depict impediments to the natural flow of thought. Repression is depicted by an enclosure that interferes with the flow of light. Avoidance of repressed material (wherein one tries to avoid speaking or thinking about something painful or traumatic) is depicted as deflection of beams of light off the enclosure, as in Fig. 5a. Rumination, or the obsessive dwelling upon a traumatic or regretful past event or a dreaded future event, is represented by repeated reflection of a beam on the inner surface of an enclosure, as in Fig. 5b. Confrontation of the traumatic material (e.g., through creative expression, as in writing a song about the traumatic event, or through psychotherapy) is represented by a bridge that vents the traumatic contents, as in Fig. 5c, and eventual dissolution of the enclosure, as in Fig. 5d.

Fig. 5. (a) Repression of traumatic material. Note how presence of enclosures shifts where IL comes to a focus; it converges near periphery. (b) Rumination on traumatic event. (c) Confrontation of repressed material. (d) Dissolution of repressed material; IL shifts toward center.

Creative Thought

Like rumination, creative thought is depicted as repeated reflection off a bounded surface, but in other respects it differs from rumination. First, since creators may use everything at their disposal as ingredients for their creativity, creative thought needn’t be delimited to a tiny enclosure (though it can be, as when creative thought serves as a means of destroying the bubble and gaining mastery over the traumatic material by ‘releasing’ it). Second, creativity often begins with a question or something not well-understood, represented by diffuse light. ‘Reflection’ on a creative idea is represented by a beam of light that gets less diffuse and more focused each time it reflects off a shell, as illustrated in Fig. 6b. (This is what light actually does when it reflects off a concave surface; it converges.) This represents how, as you ‘bounce an idea around’, viewing it from different perspectives, you gain a richer understanding of it. The more ill-defined the idea, the more diffuse the beam of light that represents it. A very diffuse beam represents a thought that is as-of-yet not well-understood, i.e., it lies in the realm of imagination.

Fig. 6. Reflection on a creative idea. a) The idea starts with a problem or something that is not well-understood or that needs to be expressed. b) By reflecting on it, it comes into focus, and a creative resolution may be achieved.

Obscuring The Truth (or One’s True Self)

Openness

Openness is associated with coming across as more accessible to others, but it can also heighten vulnerability; the individual is more of an ‘open book.’ Openness is depicted as having a transparent vessel, as in all panels of Figs 1 through 6. Conversely, aloofness or evasiveness i.e., someone who is hiding their true nature or intensions, is depicted with an opaque, light-reflecting shell that makes it more difficult to be seen, as in Fig. 7. Being closed can provide self-protection or space to focus, but at the cost of reduced closeness to others. Thus, degree of openness can vary across different settings, from transparent in unthreatening situations to opaque in threatening situations, and the platform enables users to adjust their degree of opacity to explore what degree of openness works best for them in different situations.

Fig. 7. Aloofness or evasiveness

Dishonesty

Dishonesty, ‘bending the truth,’ and other forms of deception are represented by light-refracting irregularities, as in Figs 8 and 9. Deliberate deception of others is depicted as the refraction or bending of light at an irregularity in the surface of the vessel, as in Fig. 8. This alters the direction and frequency of the light, which represents distortion of the information content of a particular thought. Habitual deception may lead to self-deception, which is depicted as refraction of light at an irregularity in the interior of the vessel, as in Fig. 9.

Fig. 8. Deliberate deception

Psychological Fragmentation

Psychological fragmentation is represented as the fleeting, temporary, or permanent separation of the vessel into fragmented ‘pieces of self’ (not shown). Lack of integrity is depicted as using the ‘self-fragmentation’ capacity to intentionally distort the flow of thought; the platform can depict how bias can creep into ideation or decision-making by avoiding certain relevant points of view, e.g., prioritizing one’s own needs over the needs of others (not shown).

Fig. 9. Self-Deception

The Indra's Net Visualizer

Capitalizing on the universality of the metaphor between light and insight, the Visual Diary -- or simply, Visualizer -- project, is an interactive digital platform that will provide a systematic way for people to depict and express their interior life and interactions with others using the 'language of light' metaphor. It will enable people to explore and better understand their inner lives by engaging in a form of playful art therapy that requires no artistic skill. Our innovative approach meaningfully integrates psychological attributes and indicators of wellbeing into a single display that allows users to gain self-awareness and explore avenues for self-change.

The platform will enable people to can explore alternative life paths, or view situations from different perspectives (e.g., from a further removed or more distant perspective, or from the perspective of someone else). By depicting how their lives are unfolding, where their thought patterns originated, and what behavior they lead to, people can make sense of subtle and intangible aspects of their mental lives, and put personal trials in perspective by seeing how they fit into the tapestry of human experience.

A short video showing an initial proof of concept is shown below

Overview

According to the Buddhist allegory of Indra’s Net, humanity consists of a web made of threads of light stretching horizontally through space and vertically through time. At every intersection dwells an individual, and in every individual dwells a bead of light. This and other light metaphors have been woven into the human psyche since the “dawn” of civilization. It permeates, for one, the English language, evidenced by the phrases: “creative spark,” “beam with enthusiasm,” “moment of illumination,” “brilliant idea,” and “her face lit up.”

Our first step in the Indra’s Net research is to develop tools that enable users to visualize, explore, and better understand their inner worlds. The tools are expected to have transformative value as a form of art therapy based on principles of cognitive science that requires no artistic skill, wherein outputs have straightforwardly interpretable meanings. While aiming for scientific accuracy, the tools will educate users primarily through narratives of personal growth. This document describes the first 2D digital prototype which will serve as a testing ground to help fine-tune the development 3D visualizer available as a mobile app and desktop software.

If successful, users of this basic form of Indra’s Net will find psychosocial processes and problems less elusive, and thus more manageable, by deploying the power of visualization, and better understand their capacity for psychosocial growth. The prototype will include a brief tutorial that teaches users the basic mechanics of the game and what parts of the psychosocial self they represent.

Design

The vessel represents layers of a person’s mind and psyche, a vessel containing a person’s Inner Light (IL), or inner life force. This is the user’s avatar that they can customize in different ways, including the colour of the layers and the Inner Light within, and different animations/shapes for different types of thoughts and interactions that originate inside the vessel. The components of the vessel are:

Inner layer: represents the person’s inner self and mind, and the space within houses the IL and depicts the realm in which the user thinks. The qualities of a person’s inner layer impact how they think about things. Perfect smoothness produces no change in the angle of refraction or reflection and allows thoughts to be more clearly reflected on or sent out in their intended direction
Inner Light: represents the user’s chi, or life-force. It is where the user’s resources in the game come from, and it is the object of their protection. It is treated as a renewable energy source and its strength is quantified by three Inner Light stats, which will be discussed later.
Outer layer: represents the image of one’s self they project to others, however truthful. The layer can be made more transparent or opaque to allow the user to express thoughts or receive thoughts from others, or to reflect on thoughts.
Thought: is represented by beams of light. They are literally “reflected on”; when done against a completely opaque and smooth layer, thoughts will reflect perfectly and become less diffuse and more focused. The beams, then, would go from wider to narrower as they travel. Thoughts can also be expressed by making the layers more transparent so that the beam one is bouncing around can be expressed and leave the sphere. To the extent that the layers are made transparent, the light will be able to refract out more fully without as much being reflected off the layer’s surface, meaning the user will be able to express a thought to someone else more successfully. The beams can also be of different intensities, the higher intensity representing a thought perceived as more personally important and/or emotionally intense.

Implementation

The following sections describe the technical details on how the visualizer is being created, the underlying platform usd and its usage.

Development Platform

The first version of Indra's Net was built in Visual Studio but the requirements of the project needed us to move to a stronger platform. Since then multiple versions on different platforms like Unity and Unreal Engine were built. The Unity prototype was limited and had only a 2D representation of the visualizer. The most extensive work was done on the Unreal Engine 4 platform with an undergrad researcher Josh Cooper working on it from Aug 2018 to Mar 2020. The psychological research aspect was handled by another researcher Connor Gibbs.

The current implementation is built on the Unreal Engine 5.3 platform

Introductory Level

A session begins with the user seeing a vast web of interconnected vessels, each of which represents an individual. There is a ‘buzz’ of light within each vessel, representing its mental processes, and light traveling amongst vessels, representing lines of communication. The user zooms in on the vessel that depicts themselves in the foreground, and others nearby that the user has depicted in past sessions. This ritual zoom-in is intended to provide perspective (the user is merely one individual in a sea of interconnected individuals, each of whose behavior affects the others around them, which in turn affects the others around them, and so forth). During the first session, the user encounters an interactive guide that explains what the Indra’s Net platform is, and how it is used (i.e., how psychological states are depicted and interpreted), with detailed use case examples. After the first session this can be skipped, though the guide remains readily accessible.

Scenario Building Level

In the next level the user will be able to create specific scenarios that they want to visualize and understand better. The user will be able to add multiple people (mental models) and build interactions between them.

Building a Mental Model (Adding a Person)

Using the "Add Person" button you can start by either loading a previously saved mental model or create a new on in the current scenario. Once added the context menu can be used to update name, description and other model properties. The model can be saved on the local storage to be fetched and added later.

Users can create multiple models for themselves and other people in their life (in different situations and timelines). These models can be added to different scenarios they are trying to visualize

Designing a Scenario

Using the "Add Scenario" button you can start by either loading a previously saved scenario or create a new one. Users can create as many scenarios as they want and save them to the local storage. In future users will also be able to share or invite other users to collaborate in creating common scenarios to identify different perspectives

‘In Progress’ and Future Features

Artificial Intelligence Integration

While initially the platform was conceived as a tool for user-generated mental state depictions, it has been re-envisioned to be more conversational, and to play a greater role in the depiction generation process, capitalizing on recent advances in AI, as shown in figure belwo. This process flow ensures that the platform becomes increasingly adept at asking useful questions, offering helpful suggestions, and generating bespoke depictions of users’ mental states, all while maintaining strict privacy and anonymity as no personally identifiable data is collected.

Fig. 10. Process flow diagram. a) Training phase: using data from beta users, the platform will learn mappings between natural language and the ‘language of light’ (how psychological issues are depicted as vessel properties), as well as correlations between issue-depiction mappings, and user effectiveness ratings. (b) Deployment phase: the platform uses these mappings to guide the user through a session, both textually and visually. The platform is now an active co-creator of the depictions.

The machine learning element of this project is just getting underway, but here is the general approach.

Default Version

In our current working version of the platform, depictions are generated solely by the user in real time using Unreal Engine.

AI-enhanced Version Training Phase

Our transformer model is using data from beta users to learn parameter settings that correlate text inputs with user-generated depictions. The text inputs include: (i) the user’s stated problem(s) or issue(s), (ii) users’ responses to predefined questions asked by the platform aimed at gaining more clarity into the issues they struggle with, as well as (iii) any other relevant input such as personality profile information. The user-generated depictions of their issues include (i) the depictions themselves, including any subsequent updates, and (ii) user-generated tags or captions. Thus, for example, if a user moves the focal point toward the periphery of the vessel and tags this as ‘feeling uncentred,’ this strengthens the association in the transformer between an off-center focal point and the word ‘uncentred.’

The reason we use transformers is that they incorporate an attention mechanism that selectively focuses on relevant parts of the input for more effective output generation. Also, transformers employ a self-attention mechanism to identify and interpret complex relationships within the input itself and between the elements of the output, which facilitates a nuanced understanding of the data's underlying structure and dynamics. In this way, the transformer learns, with unlabeled data, to respond to user descriptions of their psychological issues with the corresponding static or dynamic depictions.
The platform is then trained to increase the effectiveness of its responses. This is accomplished using reinforcement learning, with ‘reward signals’ coming from the beta users’ assessments of the effectiveness of the platform- generated depiction and accompanying platform-generated text for dealing with a specific issue.
We will tinker with the model on an as-needed basis to ensure that the platform reliably generates depictions that match user-stated issues. We may potentially need to use supervised learning with labeled data (depiction labeled with corresponding user issue, e.g., a beam of light that bends at an irregularity on the surface of the vessel is labeled ‘deception’).

AI-enhanced Version Deployment Phase

Now that the trained transformer knows how to translate natural language words, phrases, and longer descriptions (such as ‘deception’ or ‘I want to hide’) into parameter settings for the generation of corresponding depictions using the Indra’s Net ‘language of light’ (such as a depiction of deception as refraction at vessel surface), depictions are no longer generated solely by the user; they can now also be generated by the platform itself.
Subsequent additions and refinements to these platform-generated depictions can be created interactively by the user in real time using Unreal Engine. (Unreal uses, not the depictions generated by the transformer, but the parameter settings that were used to create those depictions.)

Monitoring and Maintenance

We will continuously monitor the model to avoid unwanted drift in performance.

Updating and Iteration

To improve the quality of responses and the effectiveness of the platform, it will be refined and improved using data from new users as it becomes available. Users have the option of allowing data from their sessions to be included in this training data, or not. The outputs of the transformer are the platform’s responses to specific issues. Responses have two components: (i) Text-based suggestions based on what previous users have tried, and (ii) static or animated depictions of scenes that portray resolutions to the issue.

Color and Sound

Since color plays an important role in visualization, we will continue to provide new opportunities to individualize user depictions using color and sound (the Inner Music) in ways that are personally meaningful. Building further on the concept of Inner Music described in the paper we plan to add sounds pertaining to specific mental states and interactions. For e.g. when a thought hits a repressed material, the user might hear a sound signifying pain and they might hear a sizzling sound accompanying the dissolution of the repressed material. Such enhancements would bring in more sensory engagement to the platform.

Multiplayer Version

To enable users to explore situations involving multiple individuals together, we will develop an online multiplayer version of the IN platform.

Biosensor Integration

The integration of biosensing data into the platform will be accomplished by transmitting the data from the device via Bluetooth or a direct connection, converting it into an Unreal Engine compatible format, and then using it on the computer for real-time visualization. Specialized plugins that use a protocol such as Open Sound Control (OSC) are used to interpret and reformat this data to control the vessel properties in the Indra’s Net platform. By setting up appropriate connectors between the visualizer and the biosensing device, we can drive various visual elements or interactive experiences, to further individualize depictions to match the state of the user.

Ethical Considerations

Because the platform deals with sensitive personal and psychological data, ethical responsibility and user safety have been our highest priority throughout the development process. Additional measures to mitigate any risks were established when the research proposal was examined and approved by the Behavioral Research Ethics Board at The University of British Columbia (Okanagan campus). Special care is taken to maintain stringent data management practices. Users have the option to opt out of data collection. To ensure data privacy, test and training data is collected with informed consent, and securely stored on the on-premise UBC servers, ensuring protection against unauthorized access.

Although AI can have inherent biases, the impact here is limited by the fact that the data used for platform-generated depictions comes strictly from previous platform users as opposed to scraping the entire web (specifically, correlations between a users’ reported issue and their visualizations). Moreover, ‘checks’ are being put in place to ensure that these correlations make psychological sense. (Thus, for example, if a user attempted to intentionally distort the data by rating a transition from integrated to fragmented as beneficial, this user’s data would be flagged for inspection.)

The risk of unintentional emotional manipulation is minimal, if present at all, because the platform is designed not to give advice, but to inspire self-reflection: to suggest new interpretations and perspectives (like seeing one’s house from a drone), and new avenues for shaping one’s worldview in a way that aligns with user priorities and is in harmony with the worldviews of others, and society at large. For example, if the user depicts a competitor as bathing in the light of others, and labels this depiction with text that indicates jealousy toward the competitor, the platform does not judge the user’s response. Rather, it provides depictions (and accompanying text) that illustrate how the competitor’s success might (a) inspire the user to new heights, (b) make them a valuable ally or partner, or (c) foster reflective practices that make them more intrinsically motivated and less dependent on external markers of success. The platform is not expected to pose more emotional risk to the user than, for example, writing their concerns in a diary or discussing them with a friend. Users are advised to interpret suggestions from the platform with caution and, when in doubt, review them with a healthcare professional to ensure responsible and informed use.

Indra's Net

An Interactive Visualization Platform for Understanding our Inner Lives

Introduction

Objectives

Approach

History of the Metaphor between Light and Cognitive Processes