Cybernetics Approach to Virtual Emotional Spaces An electrodermal activity actuated adaptive space

In contrast to reductionist investigating of interrelation between emotion and architecture, we have proposed a new concept for creating an adaptive architecture system that employs biosensors and virtual reality (VR). We have generated a dynamic audio-visual Virtual Environment (VE) that has the potential of manipulating the emotional arousal level of the users measured via electrodermal activity (EDA) of skin. Much like the second-order cybernetics system, our simulations have actuators, sensors, and an adaptation mechanism, whereby participant's real-time biofeedback is interpreted and loops back into the simulation to moderate the user experience. The results of our preliminary test show that our system is capable of manipulating the emotional arousal level of the participants by using its dynamic VE.


INTRODUCTION
Built environments are imbued with emotions; the qualities of spaces in the built environment impact profoundly on how we exist within them.While architects intuitively negotiate with emotional qualities in the context of architecture, we are far away from a comprehensive understanding of how the built environment is mapped to our mental and emotional states (Eberhard 2009).
As people spend most of their time indoors (Robinson et al. 1972), it is crucial to observe how the qualities of interior spaces impact on a person's psychophysiological and emotional state.Simultaneously, there are many pieces of evidence that emo-tions and health are interrelated.In fact, architectural design decisions are influential factors in regards to health, efficiency, and functionality of occupants (Lehman 2016).Moreover, it has been suggested that emotions are closely linked to the behaviour of inhabitants of a space (Mehrabian and Russell 1974).
Emotion is a complex phenomenon (Berrios 2019), and studying emotion is considered very subjective; therefore, the investigation and conclusive reporting on emotion are challenging (Schreuder et al. 2016).It is only recently that psychologists and neuroscientists have delved into a quantitative understanding of the exact bodily reactions to vari-ous stimuli in a built environment using advanced biosensors (Bower et al. 2019;Homolja et al. 2020).
These studies provide valuable contributions for a better understanding of the interrelationship between humans and their surrounding environment.Alongside that, they develop methodologies for studying human's emotion in the context of architecture from a quantitative perspective.However, these studies are not capable of providing practical guidelines and criteria for architects and designers to engineer emotions into their designs.Their reductionist approach to understanding the impact of architecture on emotions is incapable of addressing the complex and multi-sensory nature of architectural experience (Pallasmaa 2018).
With all these in our mind, we have initiated a research project to develop a conceptual framework that tries to tackle the concept of emotion in the context of architecture from an alternative perspective.This perspective employs theoretical concepts from cybernetics, system theories, and adaptive architecture to develop a system that adapts an environment (a virtual environment) to its inhabitants based on predefined emotional needs of the building beholders.

CYBERNETICS AND ADAPTIVE SYSTEMS
In the 1930s, the concept of 'General System Theory' was introduced by Bertalanffy to formulate a field of science that can be applied to systems of all types (Bertalanffy 1969).This topic was later followed by the works of Weiner (Wiener 1965), who coined the word 'Cybernetic'; this influenced early thinking about human and machine communications and became one the new fields of science in the 20th century.
Cybernetics encompasses topics from various fields such as psychology, computer science, engineering, management (Stanley-Jones and Stanley-Jones 2014).While conventional sciences try to explain the world by studying the parts of it, Cybernetics is concerned about how complex systems function rather than what systems consist of (Heylighen and Joslyn 2001); it focuses on how systems use sensors and actuators to steer towards a goal or maintain it.
In the early 1970s, motivations for studying the role of 'observer' in modelling self-regulating systems initiated a movement known as second-order cybernetics (Glanville 2004).This paradigm was highly influenced by constructivism philosophy originated from the works of Jean Piaget (Fischer and Herr 2019).
Pask was one of the second-order cyberneticians whose work shaped many ideas in architecture.According to Pask, a way cybernetics could impact architecture is that it provides a conceptual framework and meta-language for a new theory of architecture (Pask 1969).As Pask suggest (1969), a cybernetics design paradigm has five stages: 1.Specification of the goal, 2. Choice of environmental materials, 3. Selection of invariants, 4. Specification of what the environment will learn from inhabitants and how it will adopt, and 5. Choice of a plan for adaptation and development.
This paradigm overlaps with the definition of 'adaptive systems' .Although It is challenging to define adaptive systems in just one statement, a definition could be: "an adaptive control system is defined as a feedback control system intelligent enough to adjust its characteristics in a changing environment so as to operate in an optimum manner according to some specified criterion" (Narendra and Annaswamy 1989).
The definition of adaptive systems emphasizes that the first step to designing an adaptive system is to set a 'goal' .A goal in a system (a set of related variables), is a state or states (specific values for those variables at an instance of time) that are selected in preference to others (Morlidge and Player 2010).
Furthermore, an adaptive system can automatically act to seek the goal or the selected state; this process is called regulation.Likewise, cybernetics focuses on self-regulation mechanisms using the system information (Morlidge and Player 2010).As described in cybernetics, this process needs two com-ponents; a sensing component that reports on the current state of the system, and an actuation component that steers the system toward the goal based on the current state of the system.

ADAPTIVE ARCHITECTURE
Although frameworks for adaptive architecture that have been conceptualized previously (Barrett 2017;Schnädelbach 2015), adaptive architecture is not a well-established topic too.It is due to the multidisciplinary nature adaptive architecture that encompasses various fields of study such as architecture, computer science, system engineering, psychology, and social sciences (Schnädelbach 2015).
Buildings can be designed to react to various data sources, including data of the environment itself or data related to the inhabitants.A significant portion of literature about adaptive architecture is about designing architectural elements that adapt themselves to environmental settings such as weather or solar radiations intending to minimize energy consumption.However, few projects have tried to create spaces that adapt themselves directly to human factors such as the emotional state.
Adaptation of architecture could be a manual process, an automatic process or a mixture of both.Moreover, a building can be adapted to its inhabitants, its exterior space, or its interior conditions.For this project, we want to adapt interior architecture to its inhabitants in regards to their emotional state through an automatic process.Comparably, the second-order cybernetics approach emphasizes the importance of the role of the observer as an integral part of the adaptive architecture, as presented in the 'Fun Palace' by Cedric Price, an early cybernetics project (Mathews 2006).
A reason for the difficulty of implementation of adaptive architecture could be the limitations in the physical world for creating dynamic spaces that adopt its physical characteristics to its occupants.Similarly, Fun Palace was a concept for a dynamic environment that provides different experiences of the same place.Price proposed an architecture that could be responsive to inhabitant's needs by being an adaptive system.But this project never became a reality due to technical limitations.However, Extended reality (Virtual reality and Augmented reality) that provides an immersive experience of a virtual environment can provide an opportunity to realize the concept of adaptive architecture.

EMOTION REGULATING SYSTEMS
Emotions could be reflected in all modes of human communication such as word choice, tone of voice, facial expression, gestural behaviour, posture, skin temperature, respiration, muscle tension, and more (Picard et al. 2001).Despite the ambiguity in the definition of emotion, dominant theories of emotion such as the James-Lange theory, Cannon-Bard theory, and Schachter-Singer theory all emphasise that emotional responses are closely related to autonomous physiological responses.Despite all the differences between these models, what is constant is that emotion felt never precedes the physiological reactions.
This understanding of emotion provides great possibilities for measuring and quantifying emotion by measuring and quantifying physiological responses.Already, there are a variety of non-intrusive mobile data acquisition devices that can be used for collecting various numerical physiological responses while experiencing a condition or being exposed to a stimulus.These commercially available biosensors for real-time measurement of physiological data includes, but is not limited to, EEG (Electroencephalogram), GSR (Galvanic Skin Response), fNIRS (Functional near-infrared spectroscopy), and Eye Trackers.
An advantage of measuring physiological reactions to predict emotional state is that no manual cognitive processes (which are slow and subjective) are needed to produce those reactions.It is because physiological responses, which are an integral component of emotion, are mostly the result of unconscious processes associated with the autonomic nervous system, and sympathetic nervous system.
There are several research projects that try to use a similar understanding of emotion to create adaptive emotional systems.Yamaguchi [3] used Twitter as a platform to explore the "possibilities of designing systems of interaction for current standard communication methods such as social media, to facilitate a process of self-reflection leading to deeper selfawareness" [3].This study uses @heyhexx, a "robotic puppet theatre installation that exists in the physical world, but facilitates a two-way conversation through Twitter" [3].
Vidyarthi et al. studied immersive meditation of participants by shaping a "peaceful soundscape using only their respiration" (2012).The iterative design study aimed to address the regulation of emotions, particularly the management of stress.Based around a two-way system, participants create a soundscape based off of their immediate respiration patterns, creating a sense of self-awareness.Vidyarthi et al. use the medium of sound to allow participants to construct and be immersed in an environment which they curate, reiterating the therapeutic qualities of feedback loops in adaptive environments.
As we mentioned earlier, the environments we inhabit can impact our emotions, and every decision made by architects can have short-term and longlasting effects on inhabitants.In fact, sometimes architects intend to design buildings to evoke certain feelings, particularly when designing specific building typologies; health centres, educational buildings, memorials and sacred buildings all require care in the design of different sets of characteristics and functions.Louis Kahn emphasises this responsibility while defining architecture as "the creating of spaces that evoke a feeling of appropriate use" [2].
Interestingly, the possibility to infer emotional state from the interpretation of biodata provides potentials for architecture (Schnädelbach 2015).Schnädelbach (2015) proposes a model of physiologically driven adaptive architecture that employs physiological data from inhabitants.In this model, to achieve adaptive architecture, real-time physiological data collection is fed into actuators.Schnädel-bach et al. (2010) describe ExoBuilding as "a tentlike structure that externalises a person's physiological data in an immersive and visceral way" which is achieved "by mapping abdominal breathing to its shape and size, displaying heartbeat through sound and light effects and mapping EDA to a projection on the tent fabric".Using empirical and subjective feedback, the study implies that adaptive architecture may be understood as a socio-technical system.
Similarly, a study conducted by Ghandi (2019) aims to transform biofeedback into "actionable changes, " with the goal "to blur the lines between the physical, digital, and biological spheres and create cyber-physical spaces that can "feel" and be controlled by the user's mind and feelings" (Ghandi 2019).The intention is to address mental health and wellbeing; Ghandi uses artificial intelligence to create a system through which a responsive environment can be altered through the emotional states of users.Along with Schnädelbach et al. (2010), Ghandi presents a data-driven experiment as a "framework to transform the built environment into a living organism that is networked, intelligent, sympathetic, sensitive, adaptive, and yet under the comprehensive control of the user" (Ghandi 2019).

ELECTRODERMAL ACTIVITY ACTUATED ADAPTIVE SPACE (EAAAS)
Based on the points mentioned above, we started to design a system to take initial steps toward realizing the concept of adaptive architecture.EAAAS is dependent on cutting edge technologies, namely VR and biosensors, that provide a condition for achieving a multi-sensory adaptive architecture.Furthermore, the second-order cybernetics theory provides a theoretical framework for our system by defining rationale behind goals, sensors, and actuators of our system.

The Goal
Finding what can be a suitable goal for an emotional regulating architecture is challenging?We argue that, in an adaptive architectural system, the goal should be first determined based on the needs of the users and also based on the design program.
Moreover, based on the valence/arousal emotion model (Mehrabian and Russell 1974), every emotional state can be mapped to a two-dimensional map constructed from two axes, namely valence and arousal.This model of emotion discusses that a state of high emotional arousal is when someone is astonished or alarmed (Stangor and Walinga 2014), and the minimum level of emotional arousal is where someone may feel sleepy or droopy (Figure 1).
Figure 1 The arousal/valence space of emotions (Mehrabian and Russell 1974) (the position of the emotional states are only approximated).
In the context of architecture, the aim of an emotive adaptive architectural system could be to maximize emotional arousal (for amusement parks, sacred buildings, memorial buildings) or minimize emotional arousal level (for health centres).This indicates that there is no ultimate goal in terms of emotiveness of a building without considering the context and the program.
However, for the purpose of this research, we propose an abstract architectural emotional goal that could be used to test the effectiveness of the system.Achieving a dynamic equilibrium or maintaining a state of balance (Morlidge and Player 2010) is the goal for this adaptive system at this stage.Here, the goal of the system is to maintain an emotional equilibrium for the users of the virtual space.
To link this understanding to our system, we have defined an equilibrium state, and we call this state a 'neutral state' (Figure 1).The equilibrium state is a state between the most emotionally aroused con-dition and the least emotionally aroused condition, regardless of the valence of the emotional state (positive or negative feelings).The rationale behind this goal is to minimize the occurrence of extreme emotional arousal (very high and very low that makes) while experiencing an environment.

Actuators
Various interior architectural features can manipulate physiological arousal.It includes multisensory variables of space, such as visual or non-visual properties of the built environment.Using Mehrabian and Russel's An approach to environmental psychology (1974) and other supporting literature, we established a number of parameters which we identified as particularly useful to this study.The motive was to employ interior features that are highly potential for manipulating emotional arousal level, and use the combination of all those features to create a potential "stress-inducing" environment and a potential "stress-reducing" environment.It will help us to use those variables to lead the emotional arousal level of users to the direction of interest with a higher chance.
Based on the literature, we selected five interior design features to manipulate emotional arousal level in our virtual environment: 1. Colour (Küller et al. 2009), 2. Geometry (linear and nonlinear) (Vartanian et al. 2015;Banaei et al. 2017), 3. Size (Vartanian et al. 2015;Schnädelbach et al., 2010), 4. Biophilic (Yin et al. 2019), and5. Sound (Vijayalakshmi et al. 2010) For each of these five features, we created two extreme situations, and we named them 'stressinducing' and 'stress-reducing' conditions (Table 1).We parameterized our space for each design feature.It allowed us to create a dynamic space that can morph from one extreme condition to another extreme condition with smooth transitions for each of the five variables.
We mapped extreme points to the value of -1 and 1, and then we created a linear transition between these two extreme conditions (Figure 2).For instance, a condition 0 for the colour feature is where the colour of the space is white.To simplify our first step of this research, we linked all of the five conditions to a master variable that assigns the same value to all of them.It means, the system condition would be a number between -1 to 1 for all of the five interior design features.We called that value the Master Value or 'MV' .Since the manipulation of some of those features in the physical world is very challenging, we take advantage of VR and VE to create our dynamic and adaptive system.By using VE, we can parameterize various features of the environment and manipulate that in regards to our goal and adaptation mechanism.Moreover, the capabilities of VR as a proxy of reality for situations where manipulation of the physical world is challenging or it is highly costly (for example using smart materials), has been widely mentioned in recent studies.
Furthermore, empirically studying human emotion is a difficult task; utilizing VR can aid this process.VR creates a sense of presence which is an integral component for a system that relies on automatic physiological responses.Roberts et al. (2019) suggest that VR tests are comparable to real-life testing, demonstrating the validity of this medium for psychological assessment studies.Additional studies indicate the suitability of using VR for simulating the sense of presence in a corresponding real physical space (Kuliga et al. 2015;Maghool et al. 2018, Yeom et al. 2019).It supports the notion that the virtual environments, in combination with VR, could be considered as an alternative method when investigating the impact of spatial stimuli.
To manage the morphing of the design features from extreme points, we used a combination of Blender and Unreal Engine 4 software.Later, by using the Blueprint function in Unreal Engine 4, we linked the value of MV to the morphing state of the virtual space.Also, We employed Unreal Engine 4 for immersing users into the VE.

Sensor
Various sensors provide measures for emotional arousal.In the first part of this project, we used a GSR sensor to record EDA during exposure to the VE.In the simplest terms; "GSR provides a measure of the resistance of the skin by [...] passing a negligible current through the body.This resistance decreases due to an increase of perspiration, which usually occurs when one is experiencing emotions such as stress or surprise" (Soleymani et al. 2012).
To interpret emotional arousal from GSR signals, and use that in our adaptive system, we followed the guideline provided by iMotions [1].First, our system stores real-time data collected from the GSR device (Via Bluetooth) to a data frame with the frequency of 5Hz.Then the system automatically creates a buffer of the measurements of skin conductance during the last minute.Later the system decomposes the data to its 'phasic' and 'tonic' components and detects peaks in the phasic component.Then, the system counts the number of peaks for each epoch of time with a duration of 5 seconds.Finally, the system applies a basic real-time linear regression analysis on the num-Figure 2 'Stress-inducing' , 'stress-reducing' , and 'morphing' conditions.ber of peaks detected during the recent minute.The coefficient of the linear regression model is treated as an indicator of tendency in emotional arousal level of the users.For example, if the coefficient is negative, then we interpret that the user's emotional arousal state tendency (EAST) is toward a less aroused state.
Real-time data were collected and analyzed by a custom Python 3 module and using a Bluetooth communication method.We used 'Serial' and 'Pandas' libraries for Python 3 to handle the real-time data acquisition and analyses.

Adaptation mechanism
To maintain the goal of the system, we propose a mechanism that sends commands to the system's actuators based on the value of EAST.The actuation in this mechanism is morphing between design features.To elaborate, when the counters a disturbance (Heylighen and Joslyn 2001) in the emotional state of the user (indicated by the EAST) it will counteract by changing virtual interior design features (the MV value).It is anticipated that this will restore equilibrium.For instance, for a positive value of EAST, the EASTA changes MV incrementally toward -1 value to compensate that change in the emotional arousal and maintain the equilibrium.The system will achieve that by real-time sensing (using biosensors) and actuation (using stress-inducing and stressreducing actuators).
The EASTA mechanism is also implemented by using 'pandas' and 'NumPy' libraries of Python 3.Moreover, the MV value of the virtual environment is exposed to our Python 3 module by using a Transmission Control Protocol (TCP) connection and Blueprint tool provided in Unreal Engine 4. In this way, we managed to change that value automatically in real-time.

PRELIMINARY EXPERIMENT
While until this point, every step is based on theory and informed assumptions, we moved from designing the prototype to a pilot study to test the system's robustness.The goal here is to test how the extreme conditions in our system impact emotional arousal level.The experiment aimed to test what extent our conceptual system is practical and whether reliable recordings can be acquired for a cybernetics loop.Additionally, we wanted to assess whether morphing and transition produce the emotional reactions that we were speculating.Moreover, we planned to use the collected data for debugging the EASTA module.
Therefore, we conducted a preliminary experiment on four of our lab members.However, the result of our research thus far is a limited pilot experiment in the form of VR simulation, in which participants' galvanic skin response was recorded during various calm and stressful scenarios.
The test procedure is based on our literature review, together with recommendations we received from a colleague of the School of Psychology.This test protocol helped us to ensure a correct empirical approach was implemented to recording physiological data.
In this test, we used a wireless non-invasive EDA Shimmer sensor worn as a wristband on the distal forearm, with PPG sensor attached to the index finger and GSR diodes attached to the middle and ring fingers of subjects.Prior to participants arriving, the GSR was calibrated, with a sampling rate frequency of 5 Hz.The temperature was taken in the room, and the wireless HTC Vive Pro headset was prepared for the test.Participants were asked not to wash their hands immediately before the test in order to gain accurate readings of their electrodermal activity.
Then, the Shimmer set was undocked from the calibration dock, and logging activity was checked.The Shimmer GSR (worn as a wristband on the distal forearm) was placed on the participants' wrist and fingers, ensuring that everything is firmly strapped.
Participants were asked to sit in the testing chair.The HTC Vive Pro headset was placed on their head, ensuring it is seated firmly and is comfortable.Participants were asked to take a deep breath (this should show in the GSR data and is a good indication of GSR activity).
We immersed participants in three virtual environments (VEs), referred to as "stress-inducing", "stress-reducing", and "morphing" (Figure 1).Initially, participants were exposed to the "stress-inducing" VE for a period of time and then they were shown the "stress-reducing" VE.Finally, participants were immersed in the Morphing VE, where VE morphs from one extreme condition to another extreme condition and reverses this process.After the VEs were complete, the headset and Shimmer GSR were removed from the participants.The Shimmer GSR was re-docked to stop logging of data.

OBSERVATIONS AND DISCUSSION
In this project, we argue that real-time biofeedback collected from participants who are immersed in a VR environment could be used to create an adaptive architectural system that is developed based on the concept of cybernetics.To achieve this, we designed a system that works in this manner.To test this system, we planned a pilot study to check the technical aspect of this idea and also gather some empirical data to increase our understanding of the nature of data we are dealing with and to test our data analysing methods.
Our observation of the preliminary test highlights that raw skin conductance level measured in the 'stress-inducing' VE is averagely higher than emotional arousal measured in the 'stress-reducing' VE.However, it is not a good practice to average raw GSR data of participants for interpreting the emotional arousal.Therefore, we extracted the phasic component of the data, and we applied our analysing mechanism to calculate the number of peaks in data.It seems that the module is capable of automatically extracting phasic from raw GSR data.Also, it is also capable of automatically detecting peaks in the phasic component of the GSR signal and calculating our EAST value.Moreover, our preliminary observation again highlights that the number of peaks measured in the 'stress-inducing' VE is a bit higher than emotional arousal measured in the 'stress-reducing' VE.
Unfortunately, we are not able to provide any specific interpretation for physiological data collected during the morphing scenario.It could be mainly due to the fact that we conducted tests on a few participants.However, through verbal feedback, we found that the morphing transition seems odd to participants.We think that the speed of transition should be lowered for the next testing phase.
Overly, the results of our experiment suggest that our adaptive virtual architecture system is capable of manipulating the emotional arousal level of the participants by using its actuators.However, more experiments are required to pass the speculation phase.Also, our preliminary results are inconclusive in terms of the effectiveness of directing emotional arousal by morphing to various spatial conditions.We suspect that morphing and transition itself can also cause some emotional reactions.Similarly, a study conducted by Ojha et al. (2019) also highlights the importance of this point (variation of visual stimuli can cause emotional reactions).
Finally, since we only focused on emotional arousal, our adaptive system is not concerned about whether an architectural experience is pleasing or not.The experience of our system may be pleasing or not for a beholder based on many other influential factors.To elaborate, it is still the responsibility of designers to steer the cognitive processes of users toward liking or not liking an architectural experience in regards to other influential factors.

LIMITATIONS AND CONSIDERATIONS
Our sample size for this test was very small, given that it was a pilot test.This lowered the level of empirical evidence but was the initial dataset that we needed to test whether the system is working and a significant GSR response could be acquired.
The results are based on the preliminary test procedure, which is timed, curated, and behaves based on predetermined patterns.However, for realizing an actual adaptive system, we should examine the behaviour of the system when EASTA is linked to MV.
At this stage, the VEs represent the parameters (scale, colour, light intensity, biophilic elements and sound) which we thought are capable of manipulating emotional arousal.The future of this project aims to include other design features such as style, texture and visual complexity.
Also, using a single biosensor could be limiting accurate decoding of emotional responses.In fact, emotional analyses systems are most likely to be accurate when they combine multiple kinds of signals (Picard et al. 2001).To address this, in the next step, we plan to employ other biosensors such as EEG and Eye trackers to improve the sensing component of our system.

CONCLUSION
Our project investigates how to create an adaptive VE that is self-regulating toward a predefined goal by employing concepts from cybernetic and adaptive systems.It implements the concept of secondorder cybernetic in the context of architecture by providing a mechanism that uses real-time biofeedback in a loop with actuators to maintain a predefined goal.The actuators of the system include various visual and non-visual properties of space that previous studies suggest that have potential to impact on physiological reactions.We have designed a VE that has parameterized interior architecture features that can be controlled through a feedback loop dynamically.The subject or observer is part of this self-regulating system since we feed back its realtime emotional reactions to the system by utilizing a data acquisition module, a data interpretation module, and actuators.
To achieve the aim of this project, we employed a GSR device that provides real-time measurement of skin conductance to quantify the emotional reactions of the subject that are exposed to our predesigned virtual conditions.The findings suggest that our VR setup is capable of manipulating the emotional arousal level measured via GSR signal.However, to establish that more research studies are needed.
In the next steps, we will develop our system further to meet other stages of an adaptive architectural system.We plan to achieve a more intelligent 'self-adaptive' system, that implement a kind of memory mechanism that stores the systems actions and the responses it evokes in beholders.This capability requires implementing machine learning methods.This will help to create a system that can adapt itself automatically to its individual users over time without any need for linking architectural features to emotional responses prior to designing the system.