W06
A proposal for a virtual garden you can see, hear and feel.
| Year | 2019–20 |
| Place | Interaction Design, The Glasgow School of Art |
| Type | Final-year project |
| Tools | Unity, VR |
| Thesis | Implementation of Interactive Content in Virtual Garden with Application of Personal Data (2020) |
Starting from the picturesque English landscape garden, the project proposes a virtual space for multisensory experience: visual, auditory and tactile. It draws on immersive media and on research into brain–computer interfaces.
9 blog posts · Sep 2019 – Feb 2020
– My interests
This proposal is, “Can future tangible content such as virtual reality be used as a form of art?”, “How to use reaction data such as mind and emotion about human nature in art to provide viewers with a unique experience? ” It started with the question. These questions include personal opinions that want to inform the future in which individual data is transformed into immersive content using the five senses of the viewer, rather than the interaction on the two-dimensional screen.
And in the era of the 4th Industrial Revolution, the application of immersive content technology such as virtual reality to the art field is essential, and the theme was based on the realisation that interest and interest in human nature and the importance of nature can be increased.
In addition, personal interests were reflected in the project. I am personally interested in the development of technology. As Samsung celebrates 2020 years, they will refer to this decade as ‘the era of experience’ until 2030.
“The new 10 years will go to the ‘age of experience’, the emergence of ‘life partners’ devices that understand users, and Samsung’s innovation will help create a better world. Accordingly, what does the era of experience mean in art in design?
In the age of experience, customised technology that meets individual needs will be the main focus. This means that each device will understand its own individual users, the boundary between the real world and digital space will be blurred, and the way people communicate with cities and communities will change.
When the era of experience arrives, we will connect with cutting-edge technology in an optimised form for each individual. Instead of users learning how to use more complex and versatile features, devices will work for themselves. Imagine your device as a smart life partner who will respond and help you respond immediately to what you are trying to do.
However, the meeting of art, science and technology has not happened recently, it was existed from past. Since the beginning of modern science experiments in the 15th and 16th centuries, painters have been interested in science. The works of Filippo Brunelleschi and Masaccio, Leonardo da Vinci, Albrecht Dürer, Pieter Bruegel the Elder, and Arnold Schönberg This is an example. As Da Vinci used human anatomy knowledge to describe the human body in a more overwhelming and accurate way. Modern artists have also created new forms of art and other ways of expressing creativity, based on a new understanding of the biology of perception and the biology of emotion and empathy.
I think that since 2020, the area of art expression will beyond the two-dimensional screen based, and will appear as a three-dimensional or fully-immersive content that the user can arbitrarily adjust and interact with. This mean is the story that the viewer’s experience is shifting from the “appreciate-interpretation-feeling ” process to they can participate themselves to the internal content of the art works.
We need to understand what the audience or consumers want in order to understand the future more clearly. Today’s society is rapidly changing due to urbanisation, population changes, and especially technological advances. This change brings together opportunities to create new values for consumers and society. Consumers want more. It’s not just about material things, it’s about wanting a variety of experiences, one that allows new technologies to add more meaning to life.
Experience Economy and Age of Experience
– Experience Economy
“After 25 years of the world wide web birth, the information age is over and we are entering the era of experience.” This sentence which seems to reflect our current appearance is from the Tech Crunch website. (The information Age is over, welcome to the Experience Age)
In the age of information, communication started with information. It is the age of information to share the “me” created through writing, photos, and videos. However, as we enter the era of experience, we share the “experience with each other”. It can be said that sharing all the moments in real time is an age of experience.
As the new generation exerts a strong influence on consumption trends, it has become a difficult time for companies to survive simply by selling things. For these generations, it is more important to do what kind of experience they are getting continuously than to use them right away. The company where provide an engaging experience is becoming a famous. The era of empirical economy has arrived.
In particular, the importance of experience in consumption is growing on social media and streaming video platforms. This is because you have to try other people’s experiences and share what others don’t. The Internet is a showcase of experience for the younger generation. Providing a good experience is essential to keep the younger generations who get feeling insecure to be behind the fashion.
As we change into the era of experience, the standard of good content will be ‘what kind of experience and how effectively it is delivered’. Efforts are needed to provide digital services in line with the age of future experiences.
– Age of Experience
There is a space where you can see and experience the trends and technologies of the experience economy and experience era with your own eyes. That is ‘CES 2020’. ‘CES’ is a technology conference held by thousands of global conglomerates and startups held in Las Vegas every January. Companies will come out with technologies and products that will lead the future, making them open to the audience and making them experience.
The slogan of this year is ‘AI in everyday life’. The core technologies of this CES2020 are ‘Artificial Intelligence’ and ‘Next Generation Transportation using V2-X’, ‘IOT (Change from Internet of Thing to Intelligence of Thing)’, and the new next generation communication standard ‘5G’ and ‘Robot Technology’.
The overall aspect of the CES focused on ‘personalisation of experience’. The future development of artificial intelligence technology collects and analyses individual data to provide an optimised experience for each individual.
There are many new technologies introduced to us. But in this CES 2020, I would like to introduce only the areas that personalisation of data and areas of virtual reality and mixed reality that I want to work with in the 4th year project.
1. Intelligence of Things
IOT is changing from the ‘Internet of Things’ to the ‘Intelligence of Things’. ‘Intelligence of Things’ will help us by learning users emotions and life pattern data by themselves, and having their own intelligence.
Samsung Electronics’ ‘Ballie’, which received a lot of people’s attention at the CES 2020, is not a product aimed to the public. It is a realisation of a pet-like robot that can read my feelings. In other words, it was produced with a focus on personalisation.
As things have the power to think for themselves, “Ballie,” which serves as a companion for personal life, can be seen as an example of the direction of robot research pursuing human-centred innovation. This “Ballie” can be seen as the first step into a personal home robot.
2. Future relation between Me-Home-City
In the future, it is expected that the urban public domain will be transformed into a form of consumer experience tailored to individual lives. The relationship between ‘Me and Home and City (Me-Home-City)’ is changing. When the three are well balanced, we will be able to expect a higher standard of living.
At Home-LG’s Plant Grower-Plants can be grown for personal preference and used for cooking.
In the city-Smart mobility-Self-driving car based on shared platform and modern flight taxi ‘S-A1’, ‘V2-X’.
3. 8K display
Popularisation of 8K display – It is expected that the current 8K display market will grow from this year from SAMSUNG and LG. 8K (7680×4320) provides 4 times more vivid image quality than the existing 4k (3840×2160).
In VR, theoretically, if you can experience virtual reality with 8K per eyeball, you can get a resolution similar to that of reality. Currently, there is a ‘Fimax VR’ headset with a total quality of 5K. In the near future, if the spread of small 8K displays and the production cost of 4K displays go down in the near future, it could be applied to virtual reality devices.
This is because the resolution of the virtual reality device at a price point that is currently accessible to the public and the general consumer, is slightly higher than QHD (2560×1440). Because of this, the pixels are still visible when wearing a virtual reality device to create a mosquito net-like plaid, which limits the more immersive experience.
4. Evolution of AR/VR – emergence streaming service and XR innovation
What is indispensable thing at the CES 2020 is the fast growth of the “5G” market and the emergence of streaming services. Streaming service is a technique that downloads and plays files such as video and sound-animation from the Internet on the hard drive and plays them in real time without downloading. It was named “streaming” because the transmitted data was processed as if it were flowing like water. As the Internet develops, it is an increasingly important technology. As “5G” began to be available in some countries in earnest from last year, high-quality or low-latency streaming games are being provided.
This streaming service can be easily applied to virtual reality. The cloud game streaming service is a kind of cloud service, and means a service that stores a game on a server and immediately streams it to a terminal requiring game use. Depending on the streaming method, it can be divided into ‘video streaming’ based cloud games and ‘file streaming (pixel streaming)’ based cloud games.
The structure of the cloud game is as follows. First, the game is run on a remote cloud server with the connection of the gamer, and the output audio / video data is captured. The data is transmitted as network packets through an encoding process through a codec, and is received and played by the terminals of the client and gamer. And the game operation signal is input through the input device connected to the terminal, and it is transmitted to the server again. The input signal of the gamer transmitted to the server is processed as the input of the game driven by the server, and the operation of the gamer is reflected in the actual game.
In addition, you can receive and use performance from a cloud server, not just streaming a game. The cloud game service can be implemented by receiving high-level computation or graphics from the network of the cloud server main PC even if the specifications of the virtual reality device itself and the PC are insufficient. This is because the cloud game service puts the computer resources necessary to run the game, such as graphics cards, memory, and processors, on the cloud server, and performs all calculations related to the calculation on the server.
There are two ways to operate virtual reality devices currently on the market. The first method is a wired cable that connects directly to a computer (desktop PC) graphics card and receives data, and the second method is a wireless product equipped with a built-in battery and a mobile processor that used in a flagship smartphone. Both products have disadvantages. The first method has cable that makes you feel interrupted when you have an immersive experience. The second method has the advantage of wireless, but it is difficult to implement a high quality of graphics due to the limited performance of the mobile processor. In addition, there is a time limitation to the built-in battery life.
Personally, from the perspective of owning and using both products, I believe that the development of wireless virtual reality devices will be the mainstream in the future. Why I thought like that is the limitation of the limited built-in processor can be supplemented by using the cloud game streaming service on the aforementioned 5G network. In the case of battery life, I think that short-range wireless charging technology will be added to virtual reality devices, or battery capacity and charging technology development can be solved naturally.
– First intention
In terms of my interests and perspectives above, the core of this project will be ‘Me’ and ‘Personal Data’ and ‘Virtual Reality’ that a device that embodies this space into reality.
Also, I think that mixed reality, such as virtual reality, can reconsider our perception of our space in many ways. About the ownership of place and publicisation of space and the boundary between virtual and reality.
And I think that the reinterpretation of this space can be used to create and experience a utopian space that fits the individual’s ideals for humans. In the past, these ideal spaces were limited to arts such as text and painting. I think that the change of space and possession according to appearance of virtual reality would be a device that can show everyone that about ideal and utopian space.
‘Landscape’ is a sign of culture and the creative vision is contained in the garden where the human’s intention is involved. A designed garden is a cultural product, representing the ideals and values of its creator and located in its own social environment. The process in which the garden is created by the combination of nature and artificial elements in different times and places inspires modern designers and influences their work.
The goal of Y4 project is for the viewer to have an immersive (visual, auditory, tactile) experience of the “garden” through spatial content. Based on this experience, I hope that viewer will discover their own stories.
From a long time ago, most people dreamed of an ideal and livable society, and they developed to achieve it. In particular, the West describes this world as a mythical world called ‘Arcadia’. To embody the imaginary ideal in the present world, many people have been thinking creatively, and as a result, “garden” that imitates nature appears. The garden is usually considered a heaven on earth. In this project, I would like to discuss a garden in a virtual space beyond a physical space.
The theoretical background of my work is as follows.
Jean Baudrillard’s ‘simulacre’
Jean Baudrillard defines this era as a society dominated by simulation and imagery, and as a perfect virtual world where reality no longer exists. According to his logic, all contents such as movies, broadcasts, advertisements, and games are the hyper-reality created by simulations. In reviewing Marshall McLuhan’s thesis, “The medium is the message,” he argues that it is not real, not just the content of the media, but also the media itself. His argument is a critical discussion of the reality as the images constantly presented by the media replace the real world.
Pierre Levy’s “virtual”
The virtual solves the problem of reality, and the reality creatively produces another virtual problem. Virtual becomes the driving force for creating reality. As a constant loop in which virtual leads to reality and reality leads to virtual, and this process of circulation is called the Mobius effect. And he notes that viewers can produce their own meanings and create new cultures in space. Since spatial content is not a reality but a virtual space, viewers can break down all daily boundaries and experience new amusements in the space. Through this potential of virtualisation, he finds the possibility of new humanisation beyond human alienation or dehumanisation.
The following methods will be used in this project. First, I will visit the personally impressive gardens, referring to Humphrey Lepton’s “Red Book,” which contains drawing on the English landscape garden. Second, I will implement spatial content work dealing with the auditory and tactile elements with the visual element as a top priority. Third, the work will use VR devices, monitor screens and projectors or etc. Fourth, this project is aimed at all viewers who are interested in spatial arts such as garden, environment, media arts, architecture, urban design.
The basic concept of virtual reality means ‘an artificial environment similar to reality but not real.’ Therefore, in a broad sense, visual media such as games such as simulator games and second life can also be included in virtual reality. However, generally speaking, virtual reality means technology that enables spatial and temporal experiences close to reality by directly acting on the five senses of the user, beyond simply implementing a virtual space. The sub-concept is ‘Simulated Reality’, which is a simulation of virtual reality as close as possible to reality. Virtual reality can distinguish between “real” and “virtual,” but there is a difference that simulated reality is so realistic that makes impossible to distinguish what is real or virtual.
When you investigate the word ‘virtual reality’, you can see a somewhat unique meaning. ‘Reality’ means ‘a real (looks like) world in which an object actually exists, or a phenomenon unfolds’. However, ‘virtual’ means ‘things that are not real but falsely built’. Putting them together, it becomes ‘a real world that is not real but has been built in a false sense’, and a ‘real world that does not exist but exists’ by if we removed the intermediate modifier. Considering on this point, some people use that concept as a ‘virtual world’.
However, from a different perspective, it is a convincing term. Consider the fact that ‘virtual reality’ is a virtual world that allows you to see, hear and feel. Although, the substance is a faked world, but on the one hand it is clearly visible and heard, and as technology advances it may be felt by our senses. This is “real world” (reality) because it is felt through the senses, not the user’s illusion. In this interpretation, “virtual reality” is not a strange word. Virtual also means ‘substantially effective’ or ‘false.’ It also means “a reality has no substance but, that provides an experience to user that can be sensed practically by a illusion.”
The concept of virtual reality dates back to the 19th century. The first prototype of virtual reality was developed in 1852, it was called a “stereoscope”. That is an origin of 3D display technology today. In literature, we can find similar concept of virtual reality in the ‘Brave New World’, an SF dystopian novel published by Aldous Huxley in 1932, something like as ‘haptic technology screen’. And, the more advanced virtual reality concept was introduced from the ‘Pygmalion’s Spectacles’. The book was written in 1935 by American science fiction writer Stanley G. Weinbaum. In this book includes a direct depiction of holographic technology, virtual realisation of touch and smell.
The world’s first virtual reality original technology source was beginning in 1968 where U.S. University of Utah’s Ivan Sunderland studied of head mounted displays (HMD). A year later, when the NASA began its Apollo project, virtual reality technology developed fast and it used as a computer interaction reaction system to train its crew in 1969. However, in the 1985 ‘virtual reality’ word was became spread to the public widely.
After this, in the late 1980s and late 1990s, the public’s attention in virtual reality technology was extremely high. Starting with the appearance of the science fiction masterpiece ‘Ghost in the Shell’, which dealt with virtual reality in 1989, in the 1990s, works featuring virtual reality in popular culture such as ‘Matrix’ and ‘Total Recall’ continued to appear. The image of “virtual reality enjoyed using HMD, tactile gloves, and olfactory simulators” was also established at this time. It can be said that the technical and social image of virtual reality was actually established at this time.
Nevertheless, virtual reality technology still hard to overcome in the hardware performance and software area at that time and there were many restricts also. Ignoring technical conditions, devices released reason only by the public’s expectations were nothing more than a failure case like ‘Virtual Boy’. For this reason, in the early 2000s, interest in virtual reality technology diminished for a while.
The resurgence of public demand and interest in virtual reality technology is due to the development of subculture. Since 2007, with Crysis Engine as the starting point, the rapid development of 3D game graphics, and broadcasting Sword Art Online anime, etc., can be seen as the steadily demand for virtual reality games. Crucially, the most influential thing is that in 2015, an environment where consumers can purchase virtual reality devices such as Oculus, HTC Vive, PlayStation VR, and Gear VR at a reasonable price point.
The three elements necessary for realising virtual reality are three-dimensional spatiality, real-time interaction, and immersion. To implement each element, a number of input / output devices that stimulate the five senses, such as computer graphics technology, network communication technology, and HMD, are required. VR can be established and developed by developing technologies for these things more. Considering that the network communication technology has virtually no difficulty in realising virtual reality even with current technology, it can be considered that the technologies that actually require development to implement virtual reality are computer graphic technology and input / output device technology.
3.1) Computational Instrument
3.1.1) Graphics
There are many factors that determine the realism and performance of real-time graphics technology, but basically three factors are ‘resolution, modelling, and rendering’.
3.1.2) Resolution
Indicates the degree of sharpness of the output image. It can be determined by how many pixels, which are the basic units of the screen, are used. The higher the resolution require more computational power for the sharper the output.
With existing real-time rendering technology, it is difficult to stably output higher resolution results than FULL HD in home devices such as PCs and game consoles.
On the other hand, the human eye has a resolution of up to 120 million megapixels per side. This is about 14.5 times better than resolution of 4K and 58 times better than resolution of Full HD. If you correspond to the 16.9 ratio, which is the ratio used in the HD standard, the resolution is close to 16K resolution (15360 x 8640, 132.27 million pixels), which is 64 times that of Full HD.
If this is applied to VR, in the case of existing 1280x1440x2 and 1080x1200x2 resolution VR devices that are currently 90 ~ 110 degrees, the mosquito net phenomenon will be less felt when an upgraded VR HMD device of 2160x2160x2 appears, but the viewing angle problem remains. In the range of the viewing angle of the current generation, a resolution of 6144x6144x2 or higher is required even at 100 degrees, and in the case of a viewing angle range of 180 degrees horizontal and 180 degrees horizontal and 180 degrees vertical for both eyes, the resolution must expected be 10956x10956x2 to fit the eye level of a user familiar with Full HD. However, in addition to the resolution problem, there is also a view angle realisation problem, so in order to realistically compromise with the price, it is better to raise the resolution of the screen, which has to be saw without moving the eyeballing.
3.1.3) Modelling
As the number of polygons entering modelling increases, detailed expression becomes possible. However, the real-time computational power is also required. A polygon is a basic unit that composes a 3D object in computer graphics, and is a triangle made up of three coordinates.
However, the future real-time 3D rendering technology cannot be considered to consist of polygon-based modelling, and we should be considered when using a voxel (Volume + Pixel) engine.
3.1.4) Rendering
Virtual reality basically uses interaction with the user as a basic element, and non-rendering image technology using a camera or a pre-rendering technique that requires a long computation time cannot be considered. Therefore, 3D spatial realisation of virtual reality relies entirely on real-time rendering technology. Therefore, the development of shading and mapping-related technologies supporting this is essential
3.2.1) Audio
Binaural Effect – It means the effect that causes the listener to feel as if they are listening to the actual sound due to the phase difference of the sound source.
3.2.2) Input / Output Devices
As an input device for virtual reality, unlike traditional input tools such as a keyboard and a mouse, devices that can directly recognise a user’s physical activity are used.
Input
3.2.2.1) Motion Capture
Motion capture (sometimes referred as mo-cap or mocap, for short) is the process of recording the movement of objects or people. It is used in military, entertainment, sports, medical applications, and for validation of computer vision and robotics.
3.2.2.2) Omnidirectional Treadmills
Omnidirectional Treadmills refers to a device that allows a user to input a walking or jumping movement into a virtual reality device even in a small space by using a treadmill capable of moving in all directions 360 degrees. Some products can detect jump motions. Almost all products have a footrest and a lumbar support, because the user’s movement can be fixed so that the foot moves only on the footrest.
The above-described motion capture device can be used only in a range recognisable by the motion capture camera. However, omnidirectional treadmills may be a solution to space constraints.
Since it is a device that provides users with a virtual reality device to experience the space of virtual reality, it is essential to create an immersive feeling through the use of HMD. Buying an omnidirectional treadmill and playing a game looking at the monitor is just playing with exercise equipment. However, it has not yet reached the activation stage due to concerns about floor noise, high prices, and insufficient content.
The treadmill is a passive treadmill designed so that the sole of the foot slides on the footrest. ‘Vertuix Omni’ requires special shoes, and ‘Cyberith Virtualizer’ and ‘Kat Walk’ do not have special shoes. In the case of ‘Vertuix Omni’, it works on the principle of realising movement through the movement of the wheels under the dedicated shoes. In the case of ‘Kat Walk’, there are no special shoes, but there are additional devices to be worn on the shoes. In the case of ‘Strider VR’, a number of marbles are embedded in the foothold, and the movement of the marbles is used to grasp the user’s movement while the foothold itself rotates.
Unlike the above products, Infinadeck is an active treadmill with a moving foothold. There is no problem in motion input even when walking diagonally or curvedly because the rotation axis installed on each footboard is orthogonal. At the same time, the user can feel the feeling of stepping on the floor and pushing it back, so it conveys the feeling of walking more clearly than other treadmills. The problem is a big problem due to the complex mechanical structure. The product is still in development, so the size and weight issues can be improved.
3.2.2.3) Brain Computer Interface
Output
3.2.2.4) HMD (Visual and Hearing)
A head-mounted display (HMD) is a display device, worn on the head or as part of a helmet (See Helmet-mounted display for aviation applications), that has a small display optic in front of one (monocular HMD) or each eye (binocular HMD). An HMD has many uses including gaming, aviation, engineering, and medicine .Virtual reality headsets are HMDs combined with IMUs. There is also an optical head-mounted display (OHMD), which is a wearable display that can reflect projected images and allows a user to see through it.
Low-cost HMD devices are available for use with 3D games and entertainment applications. One of the first commercially available HMDs was the Forte VFX1 which was announced at Consumer Electronics Show (CES) in 1994. The VFX-1 had stereoscopic displays, 3-axis head-tracking, and stereo headphones. Another pioneer in this field was Sony, which released the Glasstron in 1997. It had as an optional accessory a positional sensor which permitted the user to view the surroundings, with the perspective moving as the head moved, providing a deep sense of immersion. One novel application of this technology was in the game MechWarrior 2, which permitted users of the Sony Glasstron or Virtual I/O’s iGlasses to adopt a new visual perspective from inside the cockpit of the craft, using their own eyes as visual and seeing the battlefield through their craft’s own cockpit.
Many brands of video glasses can be connected to modern video and DSLR cameras, making them applicable as a new age monitor. As a result of the glasses ability to block out ambient light, filmmakers and photographers are able to see clearer presentations of their live images.
The Oculus Rift is a virtual reality (VR) head-mounted display created by Palmer Luckey that the company Oculus VR is developing for virtual reality simulations and video game]. The HTC Vive is a virtual reality head-mounted display. The headset is produced by a collaboration between Valve and HTC, with its defining feature being precision room-scale tracking, and high-precision motion controllers. The PlayStation VR is the only virtual reality headset for gaming consoles, dedicated for the PlayStation 4. Windows Mixed Reality is a platform developed by Microsoft which includes a wide range of headsets produced by HP, Samsung, and others and is capable of playing most HTC Vive games. It uses only inside-out tracking for its controllers.
3.2.2.5) Touch
It is an output device that allows you to feel the feeling, temperature, and sensation of holding a real object.
3.2.2.6) Smell and Taste
Since 2015, consumer VR devices such as Gear VR, Oculus Rift, and HTC VIVE have been released, and the VR market has begun to approach normal people in real. Also, as time passed, hardware performance was sacrificed, such as PlayStation VR and Oculus Quest, but products that did not require complex sensors to be pre-installed or require expensive high-performance PCs have appeared.
There are many skeptics about VR game markets. VR HMD requires a separate hardware expenditure and requires relatively large space. Cause of the reasons, the VR game’s directing contents will not as new, it may end up at the amusement park’s attraction level. Of course, there have been many claims that the judgment is too early, as it has only been 3-4 years since the consumer VR market was born. However, considering the case of 3D TV, “still judgment is too early” comments only has been repeated, and the market has finally collapsed. If there is a good TV in the house, there was no reason to pay extra for expensive 3D function.
However, most people are satisfied with PC, mobile, and console games, so if they are willing to pay more money and can’t give a reason to buy VR equipment, the market will hard to expand. Gone are the days when everyone is excited about new IT products. A few decades ago, everyone imagined that they would ride in a flying car, but just as they have not escaped from tires yet, VR depicted in science fiction movies and the like can be just an imagination.
At first glance, about the drawbacks that 3D TVs were inferior to that of regular TVs may seem to be same from those of current VR devices. And, in the pessimistic opinions about the existing VR, unusually there are many cases in which a competitor of a virtual reality device is viewed as a monitor or a VR device is considered to be an evolution of a flat-panel monitor.
This is may reminiscent of the confrontation between “head-mounted display and flat-panel display” as in the composition of “3D TV vs 2D TV”. Thus, they are expected that VR devices will also fail, like as 3D TV was defeated in the market a few years ago.
However, VR devices do not aim onto replace the role of flat panel displays. VR devices can coexist with current computer equipment. VR devices can more effectively imitate the real world that the current computer doing.
Currently we were almost blinded by familiarity, we are so skilled in the combination of monitors, keyboards and mice that we are free to handle the keyboard and mouse as if an experienced car racer is driving his own car. However, the keyboard and mouse are never peripherals optimised for human physical interaction. The same is true for monitors. Digital content, especially games, constantly tried to imitate the interaction of reality in different ways, and the process was repeated for decades and consumers accepted them.
However now, is there anyone who questioning about “I am moving and rotating a mouse but why inside game character is moved?” and “Are there any people who now question the ‘WASD’ movement method?”. Some of male players in their 20’s and 40’s long been the dominant population of games, they were really good at shooting with a mouse in FPS games, even they are well with a joystick. However, gamer who are accustomed to hard play are unaware of the fact that they rarely encounter games of that genre, or that those who do not play computer games are not able to adapt to the non-intuitiveness of the fast and sophisticated mouse aiming method. In this situation, I think if VR interaction technology can be refined with the help of various developers, it can play a role in providing a much more human-friendly interface in the long term.
The VR game market has a small share, so it is meaningless to seek the statistics compared to the PC market. Most gamer prefer to play games that sit comfortably on the sofa / lay on the sofa, rather than having to wear a heavy VR device and move around to taste realistic graphics. Moreover, most of Steam’s current VR contents are not of high quality. Of course, even when Steam opened its doors to indie developers, the quality of the overall content fell for a few years. Clearly, many VR content now has a fundamental flaw that is difficult to fix. So, there are only a few high-quality games like ‘Job Simulator’, ‘SUPERHOT VR’, ‘Arizona Sunshine’ and ‘Beat Saber’.
Perhaps the interface hasn’t been studied enough, so VR game developers can’t figure out exactly how far VR devices game can do it. Even AAA game developers just move the same control method from PC games to VR games without any further consideration. So, it is natural happen that 3D motion sickness occurs simply by pressing a button and moving the location.
Although it has become much cheaper, the price of expensive VR devices still holds ankle for the average consumer. The price of HTC VIVE and Oculus Rift has dropped to less than $ 500. But PC VR games required at least 90 frames per second. Because, over that frames people does not feel VR sickness. And, It is also a burden for consumers. Due to the above, sales of VR headsets in 2018 fell by a third of 2017. In addition, the VR game production cost is about twice that of a regular game, so it is difficult for the game maker to leave a margin.
In order to solve the two chronic problems of demanding a high-end PC and being expensive, Facebook founder Mark Zuckerberg poured capital from the SNS business into VR device and content development and spared no effort to support development for VR popularisation. The new plan for Zuckerberg’s business expansion is the VR industry. As part of that effort, Oculus Quest, which does not require a PC, was released in 2019 for $ 399. And also HTC ViVE released HTC VIVE Cosmos in 2019.
It is a historical fact of in the IT world somthing called “failed technologies” is that always cannot beyond from the first generation. In order for virtual reality devices to survive in the future, it is necessary to overcome not only technical factors such as weight and performance, but also external factors such as price, content provision, and marketing.
In addition, VR developers will have to abandon the neglect of bringing the PC interaction method as it is and build a new VR-only interface to provide an irreplaceable experience. Like as ‘Doom’ and ‘Half Life’. Only then will VR technology become a fascinating success story by establishing a new incomplete link between the real world and virtual reality.
However, there is one genre that has been properly applied and settled. This is the racing / driving genre. Due to the nature of the driving game, it is a game where you sit on the seat and, so it is easy to use the HMD, and it is possible to completely solve the problem that the viewing angle is narrow when playing in the first person mode. In addition, it is possible to completely solve the problem of a narrow viewing angle when playing in the first-person mode in VR. In addition, many of the famous driving games support VR mode. Playing in VR is one of the few genres that is more beneficial than watching a flat monitor. No matter how bad the VR business is, there is a possibility of surviving in these areas with strong simulation tendencies.
Space based content is virtual. ‘Virtual is simulacre as a non-existent symbols’ according to Jean Baudrillard’s discussion. According to Pierre Levy, ‘it is opposed with reality, but can be interpreted as a concept for the creation of reality’. The museum provides artistic inspiration through exhibits that are not easily accessible, theme parks are full of dreams and fantasy stories, and festivals enable deviation through a variety of features and entertainment that cannot be seen in everyday life.
‘Virtual’ Spatial content
In the space based content, cultural materials are embodied using space as a medium, and inside of space visitors can do various experience. Visitors are ‘go there’, where museums, theme parks, and festivals. That space seems to exist in the reality. Visitors walk in the space, and in some cases experience using touch, but the essence aspect is virtual. The term ‘virtual’ means to a false phenomenon that subjectively seems real but objectively does not exist. Virtual is an unusual phenomenon.
This means that since the content is an experiential products, it is an object that cannot be actually owned and is an object that is cognitively satisfied only. Their experiences progress in the real spaces such as land and buildings, but this is not ‘reality’ or ‘real existence, that is, reality’. A series of experiences, such as artificially produced exhibitions, video reproduction, attractions, parades, shows, etc., consist of museums, theme parks, festivals, etc., but they are not real. In other words, although their experiences are fascinating, unusual and immersive, they are only temporary, illusions that appear to really exist.
Jean Baudrillard’s ‘simulacre’
Jean Baudrillard defines this era as a society dominated by simulation and imagery, and as a perfect virtual world where reality no longer exists. Furthermore, it claims that reality is a perfect virtual world that no longer exists. This ‘virtual’ is the concept that there is no original that can be indicated, and only the simulation logic is works and exist. This is the concept of simulacre proposed by Baudrillard.
However, Simulacre is not unrealistic. Rather, it becomes hyperrealism. This is because it tends to be more realistic than reality, or it takes the form of a particular manufactured product (Connor, 1997, p.65). In other words, the Baudrillard simulacre refers to a situation where the virtual symbol replaces the object of reality, and the relationship between the imitation and the real object to indicate the object becomes ambiguous. He argues that in this situation, a virtual symbol that is more realistic than reality replaces reality.
Baudrillard also recently discussed most of the popular culture called content from this perspective. According to his logic, all contents such as movies, broadcasts, advertisements, and games are the hyper-reality created by simulations. n reviewing Marshall McLuhan’s thesis, “The medium is the message,” he argues that it is not real, not just the content of the media, but also the media itself.
Baudrillard’s argument is a critical discussion of the realisation of media and the reorganisation of communication and consumption patterns through media. In connection with the virtuality of spatial content, Baudrillard reveals that Disneyland is the perfect model for simulacre and it is for a play of illusion and fantasy. In particular, he argues that Disneyland is being presented as an imaginary world to make real world is truth (Baudrillard, 1981, pp. 39-40).
What visitors experience at theme parks, museums, and festivals is a virtual experience, not a real experience. These spaces offer a variety of things to see and enjoy, such as various images and videos, but their experiences are not real, but virtual experiences expressed on a specific story or theme.
Pierre Levy’s “virtual”
According to Pierre Levy, imagination is not a confrontation with existence, but a confrontation with reality. Virtualisation not only solves the problem by shifting to reality, but also shifts reality to virtuality to creatively produce another problem. In other words, virtual is the driving force to create reality. Therefore, Levy’s concept is that virtuality is not as emptiness as Baudrillard discussed, but as a new possibility, it allows society members to play a creative role.
The virtual solves the problem of reality, and the reality creatively produces another virtual problem. Virtual becomes the driving force for creating reality. As a constant loop in which virtual leads to reality and reality leads to virtual, and this process of circulation is called the Mobius effect. It is discussed that this effect appears in the relationship between the public and the private, the relationship between the unique and the common, the relationship between the subjective and the objective, the relationship between the map and the territory, and the relationship between the author and the reader (Levy , 1988, p. 32). And he notes that viewers can produce their own meanings and create new cultures in space. Through this potential of virtualisation, he finds the possibility of new humanisation beyond human alienation or dehumanisation.
Levy’s discussion focuses on the fact that in the space, visitors can derive their own meaning of cultural material and create new cultures accordingly. In other words, the virtuality of spatial content escapes the reality of everyday life, and visitors can explore the meaning of the past and human heritage or enjoy intellectual play (museum). It has the possibility of experiencing a fantasy (theme park) that is isolated from reality, or discovering new humanity (festival) through experience of tradition and various features, and socialising among people. Since spatial content is not a reality but a virtual space, viewers can break down all daily boundaries and experience new amusements in the space.
A ‘virtual’ amusement experience of spatial content
The virtual experience provided by spatial content is based on unusual daily routines. Already ‘Huizinga’ have discussed that play is a free, temporary activity, not a routine or real life (Huizinga, 1955, p.20). Roger Caillois also said that one of the characteristics of play is “to make you believe”, which is to make you aware of secondary reality, not real reality (Caillois, 2000, p.10).
In light of this point of view, first of all, the unusual daily routines of the museum is related to the experience of aura through the exhibition of a unique “original” that cannot be seen in reality. You must go to the place (museum) where the object is placed to see it. Exhibits are preserved through various machines, and through this, visitors can check the artistic values that cannot be felt in reality, and experience intellectual play accordingly.
Theme Park builds a virtual environment in more detail. First of all, the theme park artificially emphasises reality and disconnection through natural objects and characters. There are fictional stories on the subject of didactic morality, dreams and love. In addition, the concept of time in reality is forgotten by the composition of day and night or seasonal changes, restoration of the past, manifestation of the future, and retro nostalgia devices of unknown time. In addition, all visual elements such as the character’s manifestation, various props, writing, stationery, and detailed design are also unusual things. In particular, directing designs through the use of colourful colours are strategies to make the world of animation, not reality, appear to be real.
The same goes for festivals. The festival provides visitors with a variety of viewing and experience elements that they could not see in their daily life, as well as experiencing local culture. In particular, experiential festivals are designed to escape from the rules and to escape from existing social relations, or to stimulate the nostalgia of the past to forget the desolate daily life of reality.
On the other hand, the festival is not always held, but only during a specific period or time. On the other hand, theme parks can be used almost all year round, and only the opening and closing times are fixed. Therefore, the theme park heightens the fence in the space design to remove the everyday constant feeling, and pays more attention to the exterior and blocking devices.
However, since the festival is held only at a specific time or period, the effect of the unusual experience can be enhanced even if there is no physical restriction on the composition of the space. This is because after a certain period of time, the space is ‘disappeared’, and it remains a normal space again. This spatio-temporal arrangement is a strategy to make spatial content unrealistic, thereby maximising the immersion of the viewers.
I used Oculus Quest to conduct my first virtual reality project. With Oculus Quest, I will be developing for about a year, so I decided to work with a highly stable LTS version.
Before proceeding with the project, I have done download Oculus Quest application on my phone. This is due to the setting of developer mode. After that, I can register as a developer on the Oculus site.
Return to Unity, create a project and I downloaded ‘Oculus Integration’, an Oculus support tool from the Asset Store.
Into the sample scene I located objects around, and set ‘OVRPlayerController’ in the centre of the scene.
And I have to change the platform to Android. Because, Oculus Quest run with Android-based mobile platform. In addition, in Preferences – XR settings should be changed to Oculus.
Then, if I connect the computer to the Oculus Quest and allow debugging, Unity project is installed as the ‘unknown application’ into the Oculus Quest.
And I found a virtual reality lecture provided by Oculus in the Unity Learn, Unity’s educational programme. In the future, I will refer to this online course as a reference when proceeding with my project.
We often use the term “picture-like scenery” when looking at a great scenery or a great view. What is the ‘picture-like landscape’? Can we just simply say ‘beautiful’ and ‘it is like a picture’? If there is a difference between the two what is different? and which kind of picture corresponds to “picture” expressed in “picturesque”?
The word ‘Picturesque’, which can be interpreted as ‘as pictures’ or ‘pictureable’, was written by William Gilpin in ‘Observations on the River Wye, and Several Parts of South Wales, etc. Relative Chiefly to Picturesque Beauty; made in the Summer of the Year, 1770’. This book is a guide for people going on a hobby trip to England, and he wants to explain ‘The face of a country by the rules of picturesque beauty’ in this book. It expanded and became more common with the cultural nostalgia for medieval Gothic and Celtic culture, along with the 18th-19th century romanticism.
Whitby Abbey is a place often used and drawn by romanticism painters who liked the ruins of medieval Gothic monasteries. Walter Crane, William Turner visited Whitby Abbey and painted the ruins there.
In fact, in order to understand the concept of ‘picturesque’, understanding of two aesthetic concepts, sublime and beauty, must be preceded. We often fall in love with the various infinite forms of nature. The landscape created by nature changes every moment, and at one point we see it and express it as ‘beautiful as a picture’. At this moment of fascination with nature, we have awe of nature, and this awe we feel can be called ‘noble beauty’ in another expression.
According to Edmund Burke and Kant’s definition, seductive beauty and sublime is different, and this sublime is not the result of rational judgment. Rene Loi Girardin, who wrote a book on the composition of the landscape, said “from the ‘picturesque’ scenery that charms the human eye, a philosophical scenery that charms the soul is created.” And, 18th century painter Joshua Reynolds defined picturesque as a “Picturesque is probably a synonym of ‘preference’.”
Claude Lorrain, who worked first in Rome in the 17th century, was the first painter to be faithful to the definition of ‘picturesque’ in the history of painting. Unlike Nicolas Poussin, who strictly and accurately portrays the landscape, Claude Lorrain effectively expresses the light that can be obtained by observing nature directly and harmoniously depicts the entire landscape through midtone tones.
His paintings represent Arcadia, the ideal and harmonious world spoken by Vergilius Maro, featuring classicism and lyrical atmosphere. After the 17th century, Claude Lorrain’s landscape has become a standard form of landscape painting to many artists, and his painting also called ‘becoming as like a picture’. In particular, his influence in landscape painting in England was strong, and Joseph Malrod William Turner’s tribute to Claude Laurent was famous enough to be passed down through numerous episodes.
The ancient culture tour called ‘Grand Tour’, which was popular in the 18th century, can be said to be the result of interest in this sublime and beauty. While royals or nobles travelled abroad and sought the romance of these ancient civilisations, in the general public class ‘Picturesque Tour’ was popular that leave the city and chase the local nature and scenery. The romantic tribute to the beauty of nature and its wild landscape which developed from literature, has developed greatly in the painting along with this boom in the public travel.
Like this, people tried to go with a guidebook wherever there was a beautiful scenery. Publishers wanted to make these guidelines more plausible, like the look of ancient Arcadia, and painters painted ‘picturesque landscape’ to meet the demands of those publishers. These painters and amateur writers who were called picturesque hunters, all gathered in the Lake District then holding ‘Claude Glasses’ looking for beautiful scenery.
‘Claude glass’ is a coated black convex mirror designed by Claude Lorrain. When we look at the landscapes through it makes overall colour looks low tone and the focus is slightly blurred. British people who experienced ‘Beautiful Scenery’ through Claude Laurent’s gradation technique and fantastic paintings were wandering all over the country in search of ‘Picturesque’ with this glass to find the same landscape painting he drew. Gilpin, a provider of ‘Claude Glass’ said the glass is “This glass lets us look at things in nature in a softer, more subdued colour. Like the colour of a great painter …”
In the 18th and 19th centuries began the industrialisation of England in earnest. This rapid change in society quickly changed people’s views. Those who sought comfort in the beauty of nature, now they quickly adapted to the city where their lives were made. The painters escaped the static beauty of nature and found the city’s vitality and beauty in it. Many of these phenomena perspectives were seen from the painters who often travelled abroad.
Through Turner’s later work, the Venetian landscape, we can feel a little more romantic from the well-known city of Venice. Turner painted portray the steamship and he expressed as a detailed description, the steam, sky, and horizon emitted from the steamship are harmonised and expressed as if they were sucked into the picture. This style of painting was enough to signal the appearance of a new landscape.
In the long human history, people all dreamed of an ideal society, a good time to live, and continued to develop to achieve it. In particular, in the West, this world is represented as a mythical world called ‘Arcadia’. The development of ‘Picturesque’, which was the basis for the development of Western landscape painting, especially the modern British landscape, can be seen as the essence of this ‘Arcadia’ exploration.
However, this ‘Arcadia’ is not a stationary world. Just as the human desire to constantly pursue beautiful things has been transformed through every moment, the ideal society ‘Arcadia’ and the ‘Picturesque’ landscape that resembles him have also changed. Picturesque was limited to the typical form of romantic landscape painting including England in the 18th century, but this research for ‘Arcadia’ was inherited by impressionist painters of the 19th century. As Joshua Reynolds said, it still shows a ‘flavour of beauty’ that continues to us until today.
7. Show and Tell – 01
In the Show and Tell, which was held for the first time in the semester, was a place to present my themes and work progress. Where did my idea originate from, and plans for the future. I made a short presentation with my proposal and presented it. The presentation was started with the introduction of what is virtual garden, and talked about I am plan to work with Virtual Reality. I also talked about why I chose the theme ‘Garden’.
And the reason I worked project with virtual reality is that the user’s experience becomes more important in the form of art in the future and I think it can be done through a fully immersive virtual reality. And I am also described Jean Baudrillard and Pierre Levy, who discussed the topic of “virtual” that supports this theory of virtual reality. In addition, I briefly introduced the theme I chose, ‘Garden’. We also introduced the theory of gardening by HirschFeld’s and Humphrey Lepton’s. Finally, I was talked about the virtual reality device to I will used and my future studio project planning.
Initial Concept of Y4 Essay
Topic Analysis
This summer ‘Oculus’ launched ‘Oculus Quest’, a wireless virtual reality device. The high-end PC is not needed and the price is low, so it is receiving positive responses from the market. Now, with the advent of the second-generation family of wireless VR, and the positive evaluation of the market, virtual reality devices have gained momentum and can be reached by everyone in the next five years. In the future, the new visual media will not only be viewed, and will be mixed with user experience and participation. In this respect, I believe that Mixed Reality or Extended Reality will be the Immersive Experience media, which is a content mediated through space. This study will examine the limitations of images, the most popular visual medium in modern society, and the leap into new media.
The motivation for this research is that ‘beauty’ is a very important issue in visual culture, and ‘beauty’ is also an essential element in immersive visual culture. In addition, I think that the ‘beauty’ of immersive visual culture can be achieved by the discussion of ‘Computeresque’, a modern reinterpretation of the ‘picturesque’ aesthetic of the 18th and 19th centuries, which discussed the ‘beauty’ of space. And four big tracks in interaction design, “user experience, user interface, motion graphics, and data visualization,” are expected to contribute significantly to researching user experience and data visualisation. The production and exhibition of this study as concrete content in 2020 / June will enable public communication about the immersive visual culture.
Since the development of print media and the advent of line perspective, visual culture has developed at a rapid pace. Later, with the development of optical media, photography and cinematography emerged, which were alternatives to fine art. And the visual media that enables interactive communication such as the internet appeared, and Instagram and YouTube are the representative. In the future, virtual reality and mixed reality will become a realistic content medium beyond the current visually dependent medium, and will largely replace our visual experience. However, rather than accepting a new visual culture unconditionally, I will discuss critically through Jean Baudrillard’s simulation and Deleuze’s Simulacre theory.
The specific content of the output will be “garden,” which is highly related to the “picturesque” and immersive content mentioned above. The garden is a paradise on earth and looks like Arcadia (Utopia). People experience various things through materials in the real space of the garden. However, the experience through the content of the garden is an object that cannot be owned in reality, and it is cognitively satisfied, so the garden can be treated the same as the virtual reality. While these experiences are fascinating, different and immersive, they are only temporary illusions that seem to exist. ‘Hirschfeld’ said that garden art is a multi-sensory art that allows people to experience the aesthetic correlation of emotion through movement and to use the five senses. When it is combined with a computer, I think it can create a unique aesthetic called ‘computeresque’.
The research subjects are divided into two categories. The first is spatial content such as virtual reality, and the second is ‘garden’, which will be the subject of spatial content. Virtual reality will be reviewed by Lev Manovich and Kevin Walker’s papers, and Jean Baudrillard’s Simulacre theory. For the garden, I will examine the garden list in Humphrey Lepton’s Red Book, the garden theory of Hirschfeld and the Picturesque theory.
In the case of the virtual reality, the investigation will be conducted at the same time by producing a demo or a work on the virtual reality of the existing interaction design. In the case of “garden,” a field observation of the Scottish gardens will be carried out of the 400 gardens listed in the Red Book.
The results of the study will be computer demonstrations and sketches. For computer demonstrations, the device will be used the Oculus Quest and the Oculus Link (released in December 2011), and for the program, Unity 3D will be used.
Expected Index
Introduction
Theorical Background etc
1) What is visual culture?
– Limitations of Modern Visual Culture: Unilateral Communication and Two-dimensionality
– Immersive Content: Reconsideration of Virtual Reality and Augmented Reality and Enterprise Investment Trends
2) Development History of Visual Culture-From Painting to Spatial Content
3) Picturesque and Computeresque
4) Garden – Hirschfeld’s Theory of Gardens and Humphrey Lepton’s Red Book
5) Virtual Space – Jean Baudrillard’s Simulacre Theory
6) Garden an Arcadia – The Image of Utopia and the Truth of Virtual Reality
1) literature survey
2) field survey
– Brain-Computer Interface (BCI)
– Brain-Machine Interface (BMI)
– Mind-Machine Interface (MMI)
– Direct Neural Interface (DNI)
1. Brain Computer Interface (BCI) is an interface system that directly connects a brain and a machine to directly operate a computer or machine. There are several names, but it is commonly used as BCI. It can be largely divided into Invasive BCI and non-Invasive BCI. The above fields are broadly part of Human Computer Interaction technology. Because it is a technology that connects the human brain and the machine, it is deeply related to neuroscience and medicine in addition to computer science and robotics.
BCI technology is also called Brain-Machine Interface (BMI) because it can also operate machines such as wheelchairs, prosthetic hand or leg through electrical signals from brain waves or brain cells. It is attracting attention as the core of cybernetics technology, which is the basis of cyborg technology, and in the long term, grafting with virtual reality is also being considered.
The implementation of BCI technology is realised by sensing and recognising brain activity through a mechanical device, and then analysing it through a signalling process and instructing the input / output device. For example, in the case of an EEG-sensing BCI that detects and reads brain waves, which are the ‘results’ of electrical activity of the brain, 1. After receiving the EEG through a device that recognises EEG stimulation, 2. Signalling process after that, the brainwaves are analysed to issue commands to the input / output devices. In the case of the BCI of the neural signal detection method (direct neural connection method) that is more directly connected to the nervous system and detects and reads the neural signal, the ’cause’ of the brain’s electrical activity by reading the electrical neural signals of the brain cells directly instead of the brain waves. There is a slight difference in that it detects.
The BCI under study is not a miracle device to read the mind. Of course, depending on the application, it can be used to read a person’s mind, emotion, or mental state, but it is only a secondary use. Because it is essentially a device for reading human thoughts, it is used as a form of reading electrical nerve signals of brain waves or brain cells and reading certain patterns among them as input signals. Currently, still need to make better speed of the accuracy and signal analysis.
It is not difficult to read brain signals from outside. The technical difficulty is low and the EEG detection type BCI that can be implemented in various forms is sufficient. Conversely, it is difficult to transmit external signals to the brain. This is because the technical difficulty is high and the neural signal detection method (direct neural connection method) BCI that must be implemented in the form of an embedded BCI is required. Also, even if it is realised, it is difficult to know how it will affect the brain, so it may not guarantee safety. Currently, the degree of implantable BCI for patients who have lost sight or touch has been limitedly studied.
2. Brain Computer Interface (BCI) Development
In reality, in the 1970’s first experiment is that succeeded in operating a machine using a monkey. This technology has evolved significantly in the 1990’s and has achieved some results as the 2010’s.
Brainwave (EEG) is a spontaneous electrical activity that can be measured on the human scalp, is a means to visually and spatially grasp changes in brain activity. For this reason, since the first brainwave (EEG) was recorded by Hans Berger in 1929, it has been widely used in clinical and brain function research.
Brainwave as a computer interface means has all the advantages and disadvantages. First, the advantage is that it is cheaper to measure than a large device such as fMRI, and the non-invasive method that does not invade the sensor into the scalp is harmless to the human body and enables real-time analysis of information responses in the brain. But, non-invasive type BCI method has difficult to sensitive analyse due to inevitable mixing of miscellaneous waves and loss of information. However, in the invasive type BCI method has high surgical procedure risk.
BCI technology was used mainly for medical purposes, such as the control of ADHD (attention deficit hyperactivity disorder) children or severely disabled people in the early stages of development. However, in the past BCI machine weight was heavy and also hard to wear them due to the large number of sensors. However, recently many companies like as Neurosky, Emotive, and InteraXon have released lightweight and easy to wear devices in the form of headsets at low prices, and they are also used for various purposes such as games and concentration improvement exercises.
If BCI research and technology development accelerate, it is expected to be used as a next-generation interface connecting touch screens and augmented reality in the future. Classic methods such as keyboards, mice, and keypads used as input interfaces for computers and smartphones have recently evolving into touch pads and motion recognition, and if BCI technology development accelerates, it is highly likely to be used as a next-generation interface. In particular, BCI is able to give orders naturally without using hands or other bodies, so it is judged to be suitable for fields such as virtual reality, image and photo recognition.
2.1) distinction of Brain Computer Interface technical method
The BCI technology method is classified into an invasive type (inserted type) and a non-invasive type (non-inserted type) according to a site for measuring electrical neural signals of brain waves or brain cells.
– Invasive BCI
The implantable BCI opens the skull and implants them into the brain. Through this, the electric signal or movement of the motor cortex is read, and the signal connected to the implant line is got them up and reflected on the external machine. In addition to BCI, an EEG detection method that detects and reads brain waves, which are ‘results’ of brain electrical activity, instead of brain waves, it reads nerve signals from brain cells directly and detects the ’cause’ of electrical activity in the brain. Since the BCI of the neural signal detection method is a method that directly connects to the nervous system and detects and reads a neural signal (direct neural connection method), it can be implemented only in the form of such an embedded BCI.
This method has the advantage of being the most sensitive and capable of reading nerve signals from brain cells in addition to brain waves, and also being able to respond to direct information input and output to the brain through more direct access to the nervous system (BCI without the aid of other interface devices). There is a risk of side effects caused by scarring in the brain where it comes into contact with the electrode. Over time, there could be problems such as a natural healing of the scar area or a weakening of the signal while.
Implantable BCI focuses on research to return vision to people who have lost sight in an accident, and to provide self-manipulating machines to patients who has general paralysis. Indeed, among those who have had medical procedure, some people can get limited visual information was obtained to reach the driving at a slow speed. General paralysis patient also has had an invasive BCI medical procedure.
– Partially invasive BCI
Partial insertion BCI refers to the extent to which an implant is inserted inside the skull, but not to reach the grey matter of the brain. BCI, an EEG detection method that detects and reads EEG, which is mainly the ‘result’ of electrical activity of the brain, is implemented in this form.
The basic principle of reading EEG is the same as the method of recording and interpreting EEG potential as in non-invasive BCI. However, a thin plastic plate is planted under the dura mater to measure the EEG by directly contacting the electrode with the cerebral cortex.
Compared to the non-invasive type, the signal not be halved caused by the skull, so the resolution is higher, and also the scarring problem in the brain is not generated compared to the invasive type.
– Non-invasive BCI
Non-invasive BCI is a method that does not insert an interface into the brain. As the same case of the partially invasive BCI, the EEG detection type BCI, which mainly detects and reads the EEG, which is the ‘result’ form of electrical activity of the brain, is implemented.
Currently, the most influential method is an EEG (Electroencephalography) socket that reads brainwave potential, which is attached to the scalp to read brainwave. In addition, magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI) to read brain magnetism, and a method using ELF / SLF / ULF waves, which are weak frequency bands that brain neurons can absorb, are being studied.
Also, “Magnetoencephalography (MEG)” and “functional magnetic resonance imaging (fMRI)” to read brain magnetic fields, and “ELF / SLF / ULF wave”, a weak frequency band that brain neurons can absorb, are being studied.
The EEG socket is a device that has already been used for medical purposes. Using this, it is commercialised in a relatively low-cost band. In particular, it gained momentum with the emergence of a dry EEG terminal that does not need to be applied with a gel to increase connection efficiency.
Invasive-type and partially invasive-type BCI are require to surgery, so it is difficult to get suppliment to the public. Non-invasive BCI is an advantageous form for commercialisation for the general public and is a promising field.
However, the non-invasive type has disadvantages. First of all, the non-invasive type cannot absolutely follow the precision of the invasive type. It is also a problem that the BCI of the neural signal detection method (direct neural connection method) cannot be implemented in a non-insertion type, so that it can respond only to inputs during input and output of information, and cannot respond to outputs. In fact, this is more of a problem than precision. There is no problem with precision, they already has been improved so much to usable level. But the fact of that the usage can be limited, because the neural signal detection method cannot be implemented. This is because it is fundamentally difficult to resolve.
Of course, if it is used as a means to assist other interface devices, such as the use of EEG-sensing BCI, it is evaluated that there is considerable commercial value despite the above-mentioned technical disadvantages because the non-invasive type is sufficiently useful. There are limitations to use for applications such as cyborg technology or fully-immersive virtual reality, but it can still be useful for applications such as operation of a computer or mobile device. It is also possible to use other interface devices to convey the senses to the user in order to compensate for the fact that it does not respond to the output of information.
3. Brain Computer Interface Process
The essence of BCI is ‘analyse what humans think they are trying to do and draw corresponding results.’ Therefore, a neurologically meaningful signal is obtained and BCI is constructed using the signal. In general, EEG is the most commonly used. This is because EEG is the cheapest, has good mobility, is actively researched, has many cases, and is easy to apply.
In addition to EEG, various methods are emerging, such as processing nerve signals through NIRS, which measures changes in blood flow in the brain using near-infrared rays. The problem is that the cost of constructing an NIRS system is higher than that of an EEG, and that much research data has not yet been accumulated.
The brainwave detection type BCI system measures brainwave signals of a specific state through EEG measurement devices, extracts singularities or features, classifies them, converts them into general control signals, and controls computers or devices.
1. After attaching an electrode to the user’s head, EEG data is measured using a measuring instrument.
2. The measured brainwave data is converted into a digital signal through an AD converter and input to a computer.
3. After processing the input EEG data using various algorithms, it is recognised and classified to generalise it as a control signal.
4. The final output signal is applied to various terminal devices such as computers, game machines, and medical devices.
4. Cases and Prospect
4.1) Brain Computer Interface cases
It can be used for mobile applications, toys, and fully-immersive interactive content.
4.2) Headset Type Brain Computer Interface
BCI measurement equipment or neurofeedback-related equipment that was used in the existing experiments has the advantage of being capable of relatively sophisticated and accurate measurements, but it is expensive and too bulky to be practically used as a personal equipment. However, “NeuroSky”, “InteraXon”, “Emotive” and “OpenBCI” have recently released headset-type BCI measurement devices at relatively low prices.
4.3) Brain Computer Interface Game
Recently, research and commercialisation attempts to utilise BCI as a game interface have been actively conducted. The game’s initial interface was in the form of a joystick, joypad, and Nintendo’s console game Wii started using a motion controller, and the game interface began to evolve in earnest. Since then, the era of popularisation of motion controllers, which are used to manipulate games by recognising human movements, from PlayStation 3 to Move and Xbox 360’s Kinect, has begun.
BCI is currently in the early stages of technology and is expected to be actively used in functional games such as strengthening children’s concentration, preventing dementia for the elderly, and training mental relaxation. As such a functional game, the significance of BCI technology is an example that can overcome the controversy over the harmfulness of the game, which to be making a positive contribution to the game industry.
– MindBalance
MindBalance is a video game using BCI jointly produced by researchers at University College Dublin and MediaLabEurope. In this game, the movement of the 3D avatar character in the virtual reality game is controlled by BCI. As shown in the image below, this game is people need to be balancing on the single rope using gorilla avatar.
This game uses SSVEP to control gorilla’s movement. The frequency of signals generated in response to a visual stimulus at a specific frequency of SSVEP is different, and we are using that difference. On both sides of the gorilla avatar, there are checkered patterns with different patterns. If you look at the right checkered pattern, the gorilla balances to the right and the left checkered pattern balances to the left.
According to the results of six experimenters playing the game, the result was accurate enough to succeed 41 times out of 48 gameplays. In addition, the accuracy of the real-time control shown by the experimenters reached 89 percent, which proved that BCI could be used as a real-time game control.
– Smart Brain Games
Smart Brain Game is a BCI control system developed by Cyber Learning Technology of the United States. The user plays the game by wearing a helmet with three sensors, and collects and analyses the user’s brainwaves through a device called a smart box connected to the game console, and uses it to control the game.
This system supports existing games and is limited in that it is a level that replaces the input of a specific button with BCI instead of the front, rear, left, and right arrow keys among controllers.
– Mindball
Mindball is a BCI game developed by Sweden’s Interactive Productline, where two users sit at both ends of a table that with a transparent tube containing a small ball and play the game.
The game’s rule of thumb is to win by pushing the ball away from you and sending it to the other side. The more physically and mentally stable you are during the game, the more the ball will move towards your opponent.
This game broke the old rule that ordinary games can only win if they competed in a more excited state, and took the reverse idea of winning by making a more stable relaxed state. Developers are promoting the game as a way to train players to take a physically and mentally stable or relaxed way, which helps to improve their mental and physical health.
5. Brain Computer Interface studies
The University of Tech in Sydney, Australia is conducting a Mind Switch study aimed at overcoming physical disabilities and using future homes. Using the principle that alpha waves appear when normal people close their eyes and maintain a stable state, and alpha waves decrease when they open their eyes, experiments were conducted to turn on and off electrical appliances.
In Japan, the Institute for Brain Function conducted a study on the distinction between positive and negative doctors for the purpose of developing emotional interfaces. It is said that a positive / negative doctor is distinguished by focusing consciousness on the left ear in case of “yes” and on the right ear in case of “no” as an EEG-inducing method.
IBVA, a US venture company, conducted a study on biofeedback for the purpose of application to virtual reality. EEG is applied to computer games, and in car racing games, the direction is controlled by the joystick and the speed is adjusted by the EEG.
In 1999, the Eberhard Karls University of Tübingen in Germany developed a word processor for the disabled using SCP (Slow Cortical Potential). This is a method of finally selecting one character by continuously selecting one of the two characters on the screen. It showed a typing speed of two characters per minute.
The ERP Lab Donchin team at the University of Illinois in the United States studied a word entry interface for people with disabilities using ERP. If the average value is extracted through repetitive practice for a specific stimulus, BCI control through ERP is possible. A character matrix as shown at the right of the image below is given, and the subject selects characters one by one in the vertical and horizontal directions through ERP activation to input a specific character.
The Salk Institute for Biological Studies and Naval Health Research Center in the United States are jointly working on a portable real-time awakening monitoring system that measures the level of arousal through brain waves and issues an appropriate alert when it falls below a certain level.
This technology can be used as a system to respond to drowsy driving or emergency situations by monitoring the level of alertness of truck drivers and aircraft pilots.
The team at the University of Duke of the United States, Miguel Nicolelis, succeeded in moving the robot arm by inserting an electrode into the monkey’s brain to capture signals related to arm movement. The results of this experiment were published in the Nature journal.
The State University of New York, Jonathan R. Wolpaw research group conducted an experiment using the EEG to move the cursor left and right on a monitor. However, this method has a drawback that requires long training of the subject.
5.1) Recent research trend
– If previous studies have focused mainly on the development of the interface for the disable people or the basic processing of the EEG and the use of the interface, more practical and commercial use studies are in recent progress. In addition to those mentioned, there are approaches such as image grasping classification, manipulation in virtual reality, and neural feedback.
For example, attention monitoring studies can be used for workers in occupations that require long-term attention. Through BCI, it is possible to install a device that recommends or warns of rest by identifying the degree of awakening and installs it in trucks, airplanes, and airport facilities. Mercedes-Benz conducted a study on a system that monitors the driver’s condition through the Mind-Lab project and detects and informs when drowsy driving occurs.
– Neuralink Corporation is an American neurotechnology company founded by Elon Musk and others, developing implantable brain–machine interfaces (BMIs).
5.2) Corruption cases in Brain Computer Interface technology
Ex) China: Students Undergo Trials For New Brainwave-detecting Headbands
6. Present Brain Computer Interface technology and comparison with the fiction
Virtual reality in fiction or movie is described as a fully immersive virtual reality. That is, all input / output processes are processed only with the Brain-Computer Interface (BCI) without any other interface device, and thus, virtually all input / output is performed using only the brain and the computer.
However, the current virtual reality technology is pursuing limited immersive virtual reality rather than full immersive virtual reality. Development has been conducted in a direction to implement immersive virtual reality to a limited extent by utilising existing interface devices such as HMD. It is pursuing directionality also using with common interface devices such as displays (monitors, HMDs, etc.), speakers, keyboards, and mouse. Still we cannot pursue a completely immersive virtual reality like in fiction or movie.
Why are you pursuing “limited immersive virtual reality” rather than “completely immersive virtual reality”? This is because in order to realise a fully immersive virtual reality such as that depicted in fiction, the user’s brain must not only issue commands to the computer, but also have to assume that the computer can read what is being delivered. In other words, the direct and smooth data communication between the computer and the user’s brain must be achieved, but the actual brain-computer interface technology has not progressed to such an extent. Although little by little progress has been made, the technology to create a cheap and convenient device that most people will be satisfied with has not yet emerged.
If the safety of fully immersive virtual reality, such as fiction, is proven and popularised to be widely used at a low price, it is expected to quickly replace limited access virtual reality.
6.1) Technical limitation
In order to realise a fully immersive virtual reality like in fiction, a technology capable of acting not only as an input device but also as an output device is required. It should be able to detect and interpret the ’cause’ of electrical activity in the brain by reading the nerve signals directly beyond the level of reading brain waves, which are simply ‘results’ of electrical activity in the brain. In addition, since it is necessary to cope with both input and output, a technical system that can directly input and output information to the brain is required. In addition to this, it is also necessary to temporarily stop the functions of the sensory and motor nerves between the user’s brain and the body, and to make the state in which only virtual reality data interacts.
The user’s brain only receives data from the virtual reality. Similarly, the electrical signal from the user’s brain is transmitted only to the virtual reality, and the signal transmitted from the sensory nerves of the body does not affect the user’s consciousness. Likewise, the brain needs to create a state where it cannot give orders to the motor nerves. If we want to realise that, we need to be able to interfere with the brain and directly induce the electrical activity of the brain, not just to sense the electrical activity of the brain.
However, the real brain-computer interface (BCI) technology is still insufficient to realise a fully immersive virtual reality. Currently, a field mainly studied in the field of brain-computer interface is a non-invasive BCI that is not inserted into a user’s body. However, most of these non-invasive BCIs employ an EEG detection method that detects and reads EEG, which is the ‘result’ of the brain’s electrical activity. It does not detect the ’cause’ of electrical activity in the brain, such as by directly reading nerve signals. So, it responds to inputting brain information into a computer, but it is common to not be able to directly output information to the brain.
It can serve as an input device such as a keyboard or mouse, but it cannot play the role of an output device such as a display or speaker (as a monitor and HMD). As such, the non-invasive BCI cannot completely replace the existing interface devices because it only supports the input and not the output. Even though the role of the input device is possible, it cannot function as an output device that directly shows or hears the screen or sound in the head, so a fully immersive virtual reality cannot be realised yet. As a means to assist the existing interface devices, the degree of utilisation in the implementation of limited immersive virtual reality is limited.
Meanwhile, in the case of an invasive BCI that is inserted into the user’s body and directly connected to the brain nerve, it is largely divided into two methods. Whether or not a fully immersive virtual reality can be realised depends on the method employed. The invasive BCI is divided by two “EEG detection method that detects and reads brainwaves” and “when a direct neural connection method that can directly read nerve signals and send information to the brain nerves is adopted”.
In the former case, the neural signal cannot be read directly, and it cannot cope with direct information input / output to the brain, which has the same limitations as the non-invasive BCI. It may be utilised in the implementation of limited-immersive virtual reality but cannot adapted fully immersive virtual reality. In the latter case, although it is a technology that can read neural signals directly and cope with direct input and output of information to the brain, it is a rudimentary stage in which research is currently being conducted with priority to application to cyborg technologies represented as prosthetic hand and foot. It still takes a lot of time to develop to the point where it can be applied to the realisation of a fully immersive virtual reality by being applied to the virtual reality field.
In order to realise a fully immersive virtual reality, the technical limitations of the brain-computer interface need to be resolved. Therefore, a much higher level of technology is needed than now. BCI can be used as an output device as well as an input device to realise a fully immersive virtual reality, but this is difficult to realise in the near future.
In addition, although the realisation of fully immersive virtual reality is possible through the development of the brain-computer interface, there are still many barriers to be overcome. The fully immersive virtual reality presupposes that the user’s body is blocked in the five senses and the motor ability is stopped and becomes a de facto vegetative state while accessing the virtual reality. Therefore, there are problems such as reducing the user’s ability to respond to unexpected errors or increasing the likelihood of an accident. As it directly affects the brain nerve of the user, there is high risk and the safety problem cannot be ignored. In addition, there is considerable concern about exploitation as information is transmitted and output from the outside.
6.2) Stability problem
As mentioned above, a fully immersive virtual reality such as in the fiction must halt the user’s body being. Directly interferes with the detection of electrical activity in the brain and induces electrical activity. Without moving the body, the brain interacts with data from virtual reality.
However, even if the above-mentioned concept solves technical problems, there is another problem that ethical and legal problems must also be overcome. As you may have noticed from the above description, blocking the five senses and stopping the body while accessing the virtual reality is in fact a plant state. What if there is an earthquake or fire in this situation? What if you have to get up and run away suddenly from emergence situation? Such a suspension of the body can increase the possibility of unexpected errors or accidents, and greatly reduces the user’s ability to respond. Although it is necessary to discuss whether the responsibility for the accident occurred in the virtual reality provider or the individual, the reality is still in the beginning stage due to controversy related to autonomous vehicles. In addition, if a more advanced approach is allowed, there is a high possibility of causing a specific risk or sequela as it directly affects the user’s brain nerve, and safety issues must be raised when dealing with this.
This safety problem is not only the suspension of the body, but also the direct output to the brain. The fully immersive virtual reality in fiction shows a direct effect on the user’s brain and nervous system, for example, feeling a pain similar to the real thing, or calming or manipulating the opponent’s consciousness in the name of skill. These inputs and outputs are problems in themselves, but as they actually act on the brain and nervous system, they are very likely to cause certain risks or sequela unless safety devices are provided. here fore, in order for virtual reality such as fiction or movie to be realised, it is necessary to pass the problem. What is the safety policy, and what is the ethical permission? Should it be understood as a continuation of existing law? Do you have to approach with different perspectives and perceptions? etc…
6.3) Ethical and social problem
There are many set-ups in the fiction that fully immersive virtual reality is a medium to satisfy the deficiency in reality. For example, in the scene of the Minority Report, many people are portraying the eruption of honour, sexuality, and violence that are not satisfied in reality through virtual reality.
In this way, the immersive virtual reality in the fiction includes as a simple game, a difficult aspect to be considered as a particular hobby, and even an unconsciousness aspect. Therefore, if the reality of immersive virtual reality develops in a form that satisfies the deficiency of reality, it is necessary to discuss how to manage addiction and over-immersion and manage it. Also, depending on the type of addiction, it is necessary to discuss whether it should be approached as a different field or whether it is managed by one special law. However, the image of virtual reality addiction is currently the known as for expressed same as an image of game addiction.
But is it right for the state to control individual needs in the name of over-immersion and addiction? That kind of question also exists. The above paragraph is stigmatising that it is ‘not right’ to meet a need in virtual reality that is not met in reality. The standards of right and wrong vary from person to person, and it is fascism to set the standard of right and wrong in the country and to control the needs of all people accordingly.
For some people, it may be right to fulfil their needs in virtual reality. However, from the perspective of the government, people are working all day long and making money for their needs. But when if people can easily satisfy their needs in virtual reality and spend only a minimum amount of time for working it is may not good to them. However, there is no reason for an individual to give up his hobby for the benefit of the country as a whole.
In addition, virtual reality in fiction implements various ideals and environments, and a setting is introduced that consumes less cost and time than reality. This, in turn, means that even inhuman situations can be easily achieved. As of now, even the problems of reality have not been properly solved, so it can be to worry that this may happen.
On the other hand, it is a story related to artificial intelligence that cannot be left out of fiction’s virtual reality. Artificial intelligence in the fiction’s virtual reality world seems to behave like a person and even looks like a human being. However, these artificial intelligences appear to be consumed like tools for the happiness of others in virtual reality. However, as much as human beings, their rights will need to be issue are bound to be discussed.
Excessive addiction and the collapse of life can hinder the pursuit of justice in human society and the maintenance of self-identity. It would also be right to limit people to keep them safe from mental and emotional deterioration. Indeed, in situations where the flow of thought and memory is under human control due to technology, in order to be truly free, it is necessary to stop danger by limiting power.
Of course, the above problems are only seen from the perspective of modern society, so in the optimistic future, there is a possibility that it will end with misunderstanding like digital dementia. However, after a problem arises there is no way to stop it, we need to start worrying that. It can be an efficient and effective way to replace the experience. However, if confusion with reality, it has a fatal effect and increase the social problem.
What is a brainwave?
Neural oscillations, or brainwaves, are rhythmic or repetitive patterns of neural activity in the central nervous system. Neural tissue can generate oscillatory activity in many ways, driven either by mechanisms within individual neurons or by interactions between neurons. In individual neurons, oscillations can appear either as oscillations in membrane potential or as rhythmic patterns of action potentials, which then produce oscillatory activation of post-synaptic neurons. At the level of neural ensembles, synchronised activity of large numbers of neurons can give rise to macroscopic oscillations, which can be observed in an electroencephalogram. Oscillatory activity in groups of neurons generally arises from feedback connections between the neurons that result in the synchronisation of their firing patterns. The interaction between neurons can give rise to oscillations at a different frequency than the firing frequency of individual neurons. A well-known example of macroscopic neural oscillations is alpha activity.
Basically, five types of brainwaves are well known, starting from the lower frequency band.
Delta wave – brainwave havingbetween 0.5 ~ 4Hz. A delta wave is a high amplitude brain wave with a frequency of oscillation between 0.5 and 4 hertz. Delta waves, like other brain waves, are recorded with an electroencephalogram (EEG) and are usually associated with the deep stage 3 of NREM sleep, also known as slow-wave sleep (SWS), and aid in characterising the depth of sleep.
Usually that comes out when people are sleeping. In addition, it is a signal that is emitted from all over the brain. The amplitude is very high compared to the low frequency. The frequency band is close to 0 Hz, the band is directly affected by the direct current component (DC), and is often removed by filtering when measuring EEG except sleeping EEG. It is a frequency band that is affected by human body movement or other action during measurement. It is often written that the delta is contaminated.
Theta wave – brainwave having between 4 ~ 7Hz. It refers to an EEG that comes out of hypnosis and drowsiness. Uniquely, theta wave is active in species like rodents. In humans, alpha and beta waves are surprising compared to their predominant activity. The limbic system is considered to occur mainly in the hippocampus and the thalamus among brain structures. That is, it is known to occur in the mammalian brain. It is said that it occurs more in childhood and in childhood than in adults.
Two types of theta rhythm have been described. The hippocampal theta rhythm is a strong oscillation that can be observed in the hippocampus and other brain structures in numerous species of mammals including rodents, rabbits, dogs, cats, bats, and marsupials. “Cortical theta rhythms” are low-frequency components of scalp EEG, usually recorded from humans.
Cortical theta rhythms observed in human scalp EEG are a different phenomenon with no clear relationship to the hippocampus. In human EEG studies, the term theta refers to frequency components in the 4–7 Hz range, regardless of their source. Cortical theta is observed frequently in young children. In older children and adults, it tends to appear during meditative, drowsy, hypnotic or sleeping states, but not during the deepest stages of sleep. Several types of brain pathology can give rise to abnormally strong or persistent cortical theta waves.
Alpha wave – Brainwave having between 8 ~ 12Hz. If the delta and theta waves coming from an un-awakening state, the alpha wave coming from an awakening state. It is an EEG in a relatively relaxed state among the awakened states, and is particularly noticeable when the eyes are closed. Therefore, it is closely related to the visual area. The brain waves belonging to the alpha wave include mu wave (μ wave) and SMR wave (sensorimoter wave).
They predominantly originate from the occipital lobe during wakeful relaxation with closed eyes. Alpha waves are reduced with open eyes, drowsiness and sleep. Historically, they were thought to represent the activity of the visual cortex in an idle state. More recent papers have argued that they inhibit areas of the cortex not in use, or alternatively that they play an active role in network coordination and communication. Occipital alpha waves during periods of eyes closed are the strongest EEG brain signals.
Beta wave – brainwave having between 12.5 ~ 30Hz. Among the brain waves in the awakening state, Beta states are the states associated with normal waking consciousness and thinking activities. In addition, it is the most powerful brainwave normally active. Compared to the delta wave, the amplitude itself is low compared to the high frequency, and is divided into beta 1, beta 2, and beta 3 according to the frequency band : Low Beta Waves (12.5–16 Hz, “Beta 1 power”); Beta Waves (16.5–20 Hz, “Beta 2 power”); and High Beta Waves (20.5–28 Hz, “Beta 3 power”).
Gamma wave – brainwave with a frequency of between 25 Hz to 140Hz, the 40-Hz point being of particular interest. Gamma rhythms are correlated with large scale brain network activity and cognitive phenomena such as working memory, attention, and perceptual grouping, and can be increased in amplitude via meditatio or neurostimulation
Also, it is a high frequency brainwave that comes out of an extremely tense or excited state. Since gamma waves are difficult to measure and maintain, that is not actively being studied. Therefore, the known fact is less than other brain waves, and it is generally known that the brain waves appear when the concentration is very deep.
And altered gamma activity has been observed in many mood and cognitive disorders such as Alzheimer’s disease, epilepsy, and schizophrenia.
How to measure?
Electroencephalography (EEG) is an electrophysiological monitoring method to record electrical activity of the brain. It is typically noninvasive, with the electrodes placed along the scalp, although invasive electrodes are sometimes used, as in electrocorticography. EEG measures voltage fluctuations resulting from ionic current within the neurons of the brain. Clinically, EEG refers to the recording of the brain’s spontaneous electrical activity over a period of time, as recorded from multiple electrodes placed on the scalp. Diagnostic applications generally focus either on event-related potentials or on the spectral content of EEG. The former investigates potential fluctuations time locked to an event, such as ‘stimulus onset’ or ‘button press’. The latter analyses the type of neural oscillations (popularly called “brain waves”) that can be observed in EEG signals in the frequency domain.
In conventional scalp EEG, the recording is obtained by placing electrodes on the scalp with a conductive gel or paste, usually after preparing the scalp area by light abrasion to reduce impedance due to dead skin cells. Many systems typically use electrodes, each of which is attached to an individual wire. Some systems use caps or nets into which electrodes are embedded; this is particularly common when high-density arrays of electrodes are needed.
Electrode locations and names are specified by the International 10–20 system for most clinical and research applications (except when high-density arrays are used). This system ensures that the naming of electrodes is consistent across laboratories. In most clinical applications, 19 recording electrodes (plus ground and system reference) are used. A smaller number of electrodes are typically used when recording EEG from neonates. Additional electrodes can be added to the standard set-up when a clinical or research application demands increased spatial resolution for a particular area of the brain. High-density arrays (typically via cap or net) can contain up to 256 electrodes more-or-less evenly spaced around the scalp.
– What if we applied this brainwaves study to the art?
How do we respond to the art?
Prior to the emergence of systematic brain science in the late 20th century, researchers studied how the human mind works based on psychology and visual perception, which has just begun to be understood. The human’s essential activities were also the subject of inquiry. That was ‘How do you perceive and create art works?’ An interesting question was raised from here. Can we objectively study various aspects of art, a creative and subjective experience? To answer this question, we must first find out what we know about how the mind responds to the art.
Beholder’s share
How do we respond to the art? The first people to explore this question were the ‘Alois Riegl’, Ernst Kris’ and ‘Ernst Gombrich’ of the Vienna School of Art History. Wrigley, Chris and Gombrich gained international fame by striving to establish art history as a science field based on psychological principles around the 20th century.
Alois Riegl (1858-1905) emphasised a psychological aspect of art that was obvious but has been ignored so far. That is, it is incomplete without the perceptual-emotional participation of the viewers. We cooperate with the painter by converting the 2D concept image contained in the canvas into a 3D depiction of the visual world. In addition, it interprets what you see in the canvas from a personal point of view and adds meaning to the picture. Alois Riegl called this phenomenon “beholder’s involvement.” Based on the concepts derived from Riegl’s research and insights beginning to emerge from cognitive psychology, visual perception biology, and psychoanalysis, Kris and Gombrich further developed this concept. Gombrich called it “beholder’s share.”
Kris (1900 ~ 1957), who later became a psychoanalyst, began by studying the ambiguity of perception. He responds to this ambiguity in terms of conflicts and experiences that the artist experiences throughout his life, and as a painting, he recreates the experience of the artist who created the image to some extent. For the painter, the creative process is also an interpretive process, and for the viewer, the interpretive process is also a creative process.
Inverse optics problem: Inherent limitations of visual perception
Gombrich accepted Kris’ concept of ‘the viewer’s reaction to the ambiguity of the painting’ and extended it to all visual perception. In the process, he came to understand one important principle of brain function. It was to take incomplete information about the outside world from the eyes and our brain would complete it.
The image reflected on the retina is first deconstructed by electrical signals describing lines and contours, creating the contours of the face or object. These signals are delivered to the brain and reorganised, and the Gestalt rules involved in organisation (The brain tends to grasp the perceptual object in detail, and then not from the whole, but from the overall perspective according to certain rules such as shape, background, similarity, and continuity. This is called the Gestalt principle.) and it is reconstructed and elaborated based on prior experience. Thus, it becomes the image we perceive. Surprisingly, each of us can create an image of the outside world that is surprisingly similar to the image seen by others, but rich and meaningful. In the process of reconstructing the inner representation of this visual world, we see the brain’s creative process in action.
The image projected on the retina of the eye can be interpreted innumerably. George Berkeley, Bishop of Cloyne and Anglo-Irish Anglican philosopher, discovered the key issue of this vision as early on 1709. He wrote that we are not looking at the material object, but the light reflected there. As a result, the two-dimensional image projected on the glass retina cannot directly point to the three-dimensional structure of the object one by one. When we try to understand how we perceive images, this is a challenge. This is called the ‘inverse optics problem’.
The inverse optics problem refers to the fundamentally ambiguous mapping between sources of retinal stimulation and the retinal images that are caused by those sources.
For example, the size of an object, the orientation of the object, and its distance from the observer are conflated in the retinal image. For any given projection on the retina there are an infinite number of pairings of object size, orientation and distance that could have given rise to that projection on the retina. Because the image on the retina does not specify which pairing did in fact cause the image, this and other aspects of vision qualify as an inverse problem.
Gombrich understood this problem properly and quoted George Berkeley’s view that “the world we see is a construct that each of us has slowly built through years of experimentation.”
Our brain is constantly reconstructing it even if it does not receive enough information to reconstruct it accurately. Moreover, the images reconstructed by individuals are surprisingly similar to each other. How can that be? Prominent doctor and physicist of the 19th century, Hermann von Helmholtz, argued that we solve the problem of inverse optics by adding two pieces of information. It is ‘bottom-up information’ and ‘Top-down information’.
The ‘bottom-up information’ is provided through computational activities in the circuits of our brain. Through this calculation, the brain can extract key elements such as contours, boundaries, and intersections of lines from the image of the physical world.
‘Top-down information’ refers to cognitive effects and higher-level mental functions such as attention, imagery, expectations, and learned visual association. Based on our experience, we guess the meaning of the image in front of us. The brain does this by building and testing hypotheses.
Perception is the process by which the brain integrates the information it receives from the outside world with knowledge learned through previous experience and hypothesis verification. We attach this knowledge to every image we see. So when looking at a work of art, we associate it with what we have experienced in the physical world over and over again. It also connects with the memories of all the other works of art we’ve encountered so far.
In this way, when we look at art, we need to study what causes emotional reactions. Emotion is an instinctive process. Emotions help you cope with fundamental challenges such as making life colourful, avoiding pain and pursuing pleasure. The emotions we think of in the picture are essentially the same as those we think of in everything else in our daily lives. So, it can be said that art raises countless questions about perceptions and emotions that we are only now beginning to notice and identify.
Prototype test
After researching the above content, I applied the EEG I had to the Unity project. First, I connected InteraXon Muse2 with a mobile phone, and then connected to the computer’s Muse Lab through OSC Streaming. Then I opened a new OSC port in the Muse Lab and connected to Unity.
In this test, differents brainwaves data were received, and EEG data was connected to the cube, and then the cube was rotated and the colour was changed.
Overview
The gardens in the world are different from country to country and from region to region and from time to time, and differ according to the profusion and affluence of flowers and leaves that bloom and fall according to the season, but they are all one in all, given that they are gifts with a thousand faces given by nature entering the garden of the house.
These gardens, which are grouped according to different regions and cultures around the world, maintained a consistent appearance that was formatted in a certain pattern, as they had almost consistent basic principles based on the problems of vegetation and irrigation in accordance with the climate climate and cultural characteristics unique to the culture.
British Garden
Renaissance garden of geometric order
England is today famous for its English Garden, but it was one of Europe’s poorest garden cultures until the arrival of the English Landscape Garden in the 18th century. In the Middle Ages, when frequent wars and black plagues were prevalent, functional gardens were created in some limited places, such as monasteries, and gardens for beauty were difficult to imagine. The turmoil of the Middle Ages began to stabilise as it moved to the Tudor Dynasty (late 15th-early 17th century) and later laid the foundation for the British Empire. The 100-year war ended with the victory of France, and people returned to their homeland. The nobles who returned to the mainland of England once again experienced a 30-year rose war. What was gained through two wars over a long period of time was rather the weakening of their forces. Henry VII took advantage of this social atmosphere to strengthen his power, establishing the Tudor dynasty, and by the time of Henry VIII, the Tudor dynasty was in its prime. Henry VIII built a garden as a symbol of his power. In 1525, after the death of Thomas Wolsey (1475-1530), Henry VIII confiscated his mansion and remodeled it into the Hampton Court palace, competing against France’s Palace of Fontainebleau. The gardens generally followed the French style, and later the Knot Garden was created and the statues were placed all over the garden, adding to the Italian style.
The history of the modern gardens in England was authentic from the Hampton Court, a symbol of the mighty kingship, but on the other hand, there was a garden created by ordinary people who had to nourish food to survive. Since the Middle Ages, the early cottage gardens, which have been developed as a means of living for the family, have been filled with vegetables. For as much production as possible, plants were also planted in the small soil around the house, and this tradition gradually changed into a garden made of flowers from the beginning of social stability when England entered modern society. The cottage gardens have been forgotten for many years and then settled in a unique English garden style with the influence of the 20th-century art and craft movement, the Anne Hathaway garden, in particular, is a place where you can meet the cottage garden of the Tudor era in the 16th century.
The typical garden created by a British garden where you can see something beautiful when you care for something fiercely is called the Cottage Garden. In the early days, they cultivated vegetables in their backyard to survive, most faithfully following the motif of “growning garden” found in the Bible. (Nietzsche said that the process expresses each other’s unconsciousness,) This can be explained by the process of gardening and grooming, which allows me to feel the beauty of unconsciousness that I did not realise. In this sense, the Cottage Garden in England is a garden that gives you the best sense of the beauty of the process.
Landscape Garden
By the era of Elizabeth I, Britain had formed a strong monarchy since Henry VIII, and a high level of culture, such as the literature of William Shakespeare, had blossomed. Britain, which has not been able to make clear its successor after the death of Queen Elizabeth I, is once again in turmoil, and a major change in the garden occurred as well. The French garden style, which was abundant until the Tudor dynasty, has become a target of criticism, and calls for breaking the mould of the garden locked in the formal style are beginning to emerge through literature.
The appearance of landscape gardens began in the late 17th century, the biggest change in English gardens. The background of the landscape garden can be discussed with the basis of understanding Alexander Pope, ‘Alexander Pope is considered to be one of the best of the neoclassical poets in England, and the one who had the greatest influence on the birth of the picturesque gardens. In 1713, he claimed that the gardens that resemble nature are more beautiful than formal gardens, as saying ‘People of the Roman era enjoyed the light, cheerfulness, and yet simplicity of nature in their gardens. This pleasure makes our minds more noble and serene,’ the poet of Neo-classicalism, ‘Neoclassical Augustan was a cultural style that began in England in an effort to recreate the era of Augustine, the Roman emperor who was peaceful after the civil war. As a stable society in the Tudor era entered the 17th century, many social problems arose, and neoclassical Augustan emerged to solve these problems, and the second golden age was celebrated. During this period, neoclassical Augustan was able to quickly spread its artistic tastes and styles throughout Europe, with artists, merchants, designers and architects forming a close network’.
With the Grand Tour, ‘The interest in ancient Roman culture increased explosively due to the excavation of Pompeii ruins in Italy, and British aristocrats were exposed to books on Roman thought and knowledge through Grand Tours. And as a large number of works of art entered the UK, the neoclassical Augustine style grew rapidly. British historian Lawrence Eachard (1670–1730) wrote, “August made the happiest world and was the happiest king in the world,” and inspired people to admire the Roman era. King George I of England wished to be himself a new Augustine Emperor by wisely solving many of the social problems that were raided at the time. Many writers, painters, architects, and gardeners have tried to bring material and mental happiness and glory of the Roman era in a chaotic British society. In the 18th century, Palladio-style architecture and neoclassical Augustine-style gardens were works that reproduced the glory of the Roman period.’
In 1727 Sir. John Clerk (1676-1755) sang the ideal beauty of enjoying the landscape in the countryside through the poem The Country Sheet, and Alexander Pope (1688-1744) claimed that French formal garden shows low beauty and true beauty is the beauty that resembles nature.
The desire for political freedom and the tiredness felt in the stereotyped French garden led to the desire for a new garden, and this desire made an important turning point for the English garden.
Grand tour, which was popular with the young, wealthy aristocrats at the time, made many Britons interested in Italian gardens, wanted to create landscapes in front of their homes, and created the most natural but artificial British landscape gardens.
In the early 18th century, there was a growing interest in the Italian landscape expressed in landscape paintings by artists such as Claude Lorrain and Nicolas Poussin and attempts to create a new culture by interpreting these landscapes in a classical way. The tiredness of strictly controlled formal gardens and yearning for nature-like spaces began to make landscape paintings that were no more than flat into three-dimensional ones when they needed the creation of new forms of gardens. This result created the Landscape Garden in England.
William Shenstone (1714-1763) who regarded the landscape garden as an art that expanded the scope of landscape painting trapped in a frame, and classified the beauty of landscape painting into three categories. These three elements later developed into an important concept of landscape gardening. The three beauties are first, Sublime, which means absolute beauty including natural beauty, secondly “Beautiful,” which means beauty that ordinary people can easily feel, and lastly, “Melancholy,” a sad beauty often expressed in a desolate form, a poet who sang the scenery of the countryside, said the garden resembled a dramatic poem, and created a view point where visitors could see the scenery while expressing the beautiful scenery in his garden work. Experience the scenery through the view has become the main way to appreciate landscape gardens, and various landscape sequences have been created using curved walkways.
The Ha-ha technique also attracted scenic views outside of the territory into the garden. The landscape garden, which was born and started to be created by writers, was established as a genre by William Kent (1684-1784) in the 1730s, and Kent, an ordinary artist, began to make landscape gardens using his painting techniques. This means that the landscape painters began to jump directly into the garden building, and although landscape artists had designed the garden through paintings so far, they later played a role in making garden designers appear in society.
Roman sculptures and garden buildings were the first to be used in Chatsworth House garden, but they became the most popular garden in England, such as Rousham House Garden, Stowe Landscape Garden, and Stourhead Garden. The vast landscape garden was also open to the public, and Richmond Park, designed and created by Charles Bridgeman (1680-1783) was a classic example of the transformation into a public park.
Kent’s disciple Lancelot Brown (Lancelot Brown, 1716-1783) plays a role in spreading landscape gardens across Britain and growing the garden industry. Because they were able to create new gardens at a relatively small cost compared to remodeling buildings, many nobles carried out projects to change their gardens, and many landscape gardens spread throughout England as a means of showing their wealth and power. He became more famous under the nickname “Capability Brown” for emphasising the capability that the landscape has to persuade customers.
A Stowe garden is a rare garden where three changes coexist trimmed by the designer Charles Bridgeman, William Kent, and Capability Brown.
Born in the English Baroque style Garden, it influenced many aristocrats and politicians, playing a pioneering role in the English landscape garden, which in fact was recorded as the first English garden to produce garden guidebooks.
By visiting the garden according to the brochure and introducing similar elements in their own garden, the landscape garden played a major role in spreading quickly across the UK. Stowe Garden is also known for bringing about an innovative change in English landscape gardens.
Landscape gardens have been designed to appreciate gardens in one of the best views, but Stowe Garden is designed to stroll through the gardens or take a tour of connected copper lines to enjoy natural, beautiful, and connected scenery. Since then, many gardens have developed into landscape gardens with sequences along these patterns.
Traditions of Flower garden and Cottage garden
Brown’s disciple Humphrey Repton (1752-1818) first added expertise to the realm of garden designers through the name Landscape Gardener. In particular, he is famous for persuading customers with the Red Book, which compares pre-construction and post-construction images through paintings. He proposed a popular landscape garden compared to Brown, and as he moved on to modern society, it could expand into the concept of the park. On the other hand, vegetable gardens or flower beds are often hidden in the wall due to landscape gardens.
Since none of the landscape paintings from Italy to England includes flower beds or garden, flowers or vegetables were hidden behind the fence as a natural procedure to imitate the painting. This created another type of garden called the Walled Garden in this era.
In the 1830s, there was a big change in society as the middle class grew in England. Changes in the garden began to increase interest in horticulture as various plants from the colonies began to be planted in Britain, and the flowers that had hidden over the fence began to come to the fore again. In 1832, John Claudius Loudon claimed a gardenesque created by the ‘principle of recognition’ rather than a picturesque garden that was used to create a landscape garden. The Principle of Recognition is that no creation that is perceived as an artwork should go against the laws of nature, and Loudon has created quite strict rules to apply this principle to the garden. For example, when planting trees, the rules were strictly applied to plant trees independently at sufficient intervals to prevent interference from one another while growing, and the gardener had to manage at least to promote the order they created themselves; artificially created lakes and streams must be made to look natural by planting on the waterfront; and the original plants should be carefully moved elsewhere. This theory researched ways to overcome the problems that trends of the time, which developed from the 17th century, had to plant rare plants such as tropical plants to make them look like art gardens, hindering their own growth.
The English garden, which has come within reach of the public interest, has established the beauty of the roughness and process created by the hands of non-mechanical production throughout the garden culture influenced by the Art and Craft movement, initiated by William Morris (1834-1896), and With further development, it has been becoming an important cultural heritage of the English garden by garden designers such as William Robinson and Gertude Jekyll, Vita Sackville-West.
It is not easy to answer the question of what the English garden is. It is not easy to find what British design is because European history is so complicated and interacts with one another. But the struggle to survive in barren nature and the British passion for creating gardens as a comfortable haven in a chaotic world is still creating a living garden. A garden that focuses on the pleasure of self-caring, rather than on seeing to boast of power or wealth, is a real English garden.
Moving visuality in the garden
The picturesque garden, which was the dominant garden form throughout Europe in the late 18th century and early 19th century, and its theoretical discussion, picturesque aesthetics, reflected ‘moving visuality’. As analysed by Hirschfeld, picturesque garden using serpentine lines as a design principle have been upgraded to be superior to other arts thanks to the “motion” required of the subject when experiencing space. Also, picturesque theorists valued subtle and complex variations in painting and gardens rather than monotonous aspects of vision and they were discovering that garden art is a multisensory art that is completed through mutual coordination with other senses, not just visual.
From formal garden to picturesque garden
The French formal garden, which was popular in the 17th century, has an axis and a centre point, and is designed to look most ideal view from the viewpoint. At this time, the view point was located on a relatively high level of the site, and this view is similar to the ‘bird’s eye view’ sketch at that time.
By the 18th century, the fashion of Bird Eye View ended. In picturesque gardens, the point of view descended to ground level (Hunt, 2000: 42). As the level of the viewpoint decreased, there was less chance to look at the scenery at a glance (Hunt, 2000: 47). In addition, the axis of garden, which was created by the perspective method, was changed to a serpentine line considered to be a natural form. As the viewer walks through the curved road, he experiences changes in the landscape that is obscured by planting or viscous objects. The garden consisted of a series of picturesque spaces as if unfolding a narrative, and the visitor participated in the space physically and mentally through association.
Picturesque garden: art of motion and emotion
Most picturesque theorists have sought to pursue its diversity rather than the monotony of vision (Lee and Pae, 2012). This is because the irregularities, sudden changes, and variations of contrast, which are considered to be the attributes of picturesque, are designed to create various aspects of vision through continuous stimulation.
From this we can see that the garden is an art that encompasses not only sight but other senses to create aesthetic experiences. German garden theorist Hirschfeld judges that garden is an art that produces motion and emotion, and in this respect it is superior to any other pure art form. Hirschfeld denounces France’s formal garden as an “old style” and instead advocates English picturesque style. This is because a formal garden with a strict symmetry and a stationary flat exterior was considered monotonous and contrasted with the ideals of nature (Parshall, 2001: 14). Instead, he prefers the serpentine line, the design principle of picturesque gardens. This is because the serpentine lines can create an illusion of motion, and these characteristics are considered to be similar to nature’s attributes (Parshall, 2003: 42).
According to Hirschfeld, garden art is a multisensory art that allows you to experience e-motion through motion (Hirschfeld, 2001: 137-146). The types of motions are as follows. First, there is a motion of natural elements such as leaves swaying in the wind. Second, there is a motion of visitors. Third, there is a relatively slow motion caused by changes in daytime, season, and weather. Fourth, there is an implicit and metaphorical motion created by statues and monuments. The motions listed above result in an “emotion” that touches the mind and heart of the visitor (Parshall, 2003: 40-44) 6).
Since painting relies on a two-dimensional plane, there is a limit to creating an illusion of motion, while a garden creates an aesthetic experience by organising a sequence rather than a single scene. Also, if the appreciation of painting is limited to visual pleasure, the garden experience touches all senses (Parshall, 2003: 46). Architecture and sculpture are three-dimensional, but there is no motion of its own, and the illusion is only partially created through the motion of the appreciator, whereas in garden art, not only the motion of the appreciating subject, but also the motion of the natural elements itself occurs. In these points, Hirschfeld said, “No imitation art is more concerned with nature than garden art. That is, it cannot be more natural”(Hirschfeld, 2001: 145).
Humphry Repton: Landscape gardener designed various experiences
Repton admitted to being the successor to Brown, a celebrity in the English gardens, and wanted to emphasise his distinction from his disciples. He devised an original sketchbook called ‘Red Books’ to sketch landscapes before and after design and use them as a visual means of expressing and promoting his opinions to customers, and achieving huge success with this sketchbook.
The ‘Red Book’ has acquired its own artistry independently from his garden heritage and can be regarded as an art genre in this respect (Rogger, 2007). It is known that the sketch of ‘Red Book’ was drawn differently from the actual landscape, but Repton reproduced the landscape he considered ideal.
Repton knew not only the pictorial effect of a single sketch, but also the effect of the illusion of motion created by arranging several sketches, and utilised it in his sketch to make the reader experience it. In particular, the ‘cover’, which creates the effect of covering and re-exposing a part of the sketch, was not only suitable for contrasting before and after the improvement of the private property, but also creates the aesthetics of disappearance and revelation experienced while moving through the garden. It was also an excellent device.
At the base of his design principle, the intention is to diversify the viewpoint and organically connect them to give the viewer aesthetic experience of colourful scenery. Also, when the viewer was in a stationary state, the motion of the landscape elements was enjoyed. The design strategy in consideration of these motions reminds us of the “motion of landscape elements” of Hirschfeld described earlier.
Repton systematically designed the road by separating the carriage path and the walking path, which shows that Repton recognised that the speed of carriage and walking was different, and designed the landscape in consideration of the difference in aesthetic experience.
Implications
The garden aesthetics of this period provide implications for contemporary landscape design. First, in a situation where storytelling is becoming important as a strategy for creating memories of a recent place, the process of appreciating the place’s memories through an associative method as the viewer moves and encounters objects is the same as the spatial design principle of picturesque aesthetics. Second, picturesque garden and picturesque aesthetic reflect the visuality of motion, and it is a multisensory art that uses not only vision but also hearing and smell. In this context, the urban landscape can reflect the “aesthetics of motion,” where urban experience and visual culture are linked.
Development History of Visual Culture-From Painting to Immersive Content
I discussed ‘Data Visualization and Digital Communication: Designing Data’ in my source review. I discussed contemporary visual culture and media, and briefly discussed the visual culture for the next five years through source review.
With the development of print media and the advent of linear perspective, visual culture developed rapidly. Later, caused by development of optical media, photography and cinematography (film) appeared, which emerged as an alternative to fine art. In the modern times, a visual medium capable of two-way communication called the ‘Internet’ has emerged. This beyond the limitations of visual media tied to specific places, times and spaces. Since then, the development of codecs of images and videos, as well as the development of network speed and latency, have resulted in ‘streaming services’ in company with the real-time sharing and play. The YouTube is best example of that.
The development of these technologies is expected to accelerate the change of visual culture. This is because technology development advances not as a linear curve but as an exponential growth curve. If I look at this paradigm here as a logarithmic graph, we’ve already got on this trajectory. I think now we are in turbulent times until we reach the ‘technical singularities’ of 2045 that are predicted by futurists like ‘Raymond “Ray” Kurzweil’. ‘Technology Singularity’ mean is a hypothetical future point in time when technological growth becomes uncontrollable and irreversible, resulting in unfathomable changes to human civilisation. Accordingly, art must take various forms in order to reflect and critically view the situation of the times.
In the past, Nam June Paik used a TV monitor as a painter’s canvas, and the work of the artists to paint the paints using brush on the canvas was replaced by the synthesiser. Now it goes beyond just flat TV screens to immersive three-dimensional spaces.
Immersive Content: Reconsideration of Virtual Reality and Augmented Reality and Limitations of Modern Visual Culture
Currently, what I am paying attention to as a new form of art is virtual reality. VR, AR, and MR technologies that provide an immersive experience for users are collectively referred to as immersive media and have recently been referred to as Extended Reality (XR). Virtual reality is a technology that enables users to realistically experience situations that are difficult to experience in the real world by expanding and sharing the user’s sensory information in a computer-generated virtual environment. Augmented reality is a technology that provides users with a communication environment and information by synthesising virtual contents in the real world and real objects.
My interest in the immersive experience was began in 2012 when a start-up called ‘Oculus’ founded by Palmer Lucky and a handful of developers. And they launched ‘Oculus Rift DK1’ in 2012 for developers. And in the second year of interaction design in 2018, I worked on a project called “Mixed Reality”, through this project I was confident that it could be used as a form of art in the future.
This summer ‘Oculus’ launched ‘Oculus Quest’, a wireless virtual reality device. The high-end PC is not needed and the price is low, so it is receiving positive responses from the market. Now, with the advent of the second-generation family of wireless VR, and the positive evaluation of the market, virtual reality devices have gained momentum and can be reached by everyone in the next five years.
And at 11 pm on April 3, 2019, world first commercialisation of 5G will bring more attention to VR. The development of networks speed and latency can have a major impact on the development of consumer services and experiences. For example, a cloud streaming service can be used to play graphics calculations (such as resorts and museums) calculated by high-performance computers on low-end devices. If cloud streaming services and virtual reality (or MR) are combined in the future, it is expected that many people will have a more realistic form of content experience at a lower cost. You can see the future of cloud streaming through ‘STADIA’, which is currently being prepared for commercial launched by Google.
If the current visual-based virtual information range is extended to the five senses, in a single user environment, multiple users can share and communicate with the same virtual space without limiting distance. As following this develops into a multi-user environment, it is expected to be further maximised.
In addition, in this year’s ‘Top 10 Strategic Technology For 2019’, Gartner selected technology for immersive experience as a key future technology for building the next generation digital business ecosystem. And IDC estimated the world’s AR / VR SW market size to grow about 44 times from about $ 21 billion on 2016 to about $ 926 billion by 2021. The device-driven hardware market is also expected to grow about five times in 2021 compared to 2017, with total device shipments of more than 90 million, according to CCS Insight forecasts.
Compared to computer games or immersive content that can be immersed in the sophisticated graphics and intense narratives of today, manual viewing of stories on large screens through a home monitor or laptop is being a thing of the past.
Brain-Computer Interface and Virtual Reality
In terms of immersion, no matter how good a video game can be, it can be argued that it cannot be compared with the pleasure or escape of a great story (Dickens or Tolstoy’s novel). But when we look at the data, things are different. In particular, the younger generation today will oppose this claim. Given the dramatic shift in audiences seeking entertainment and rich cultures with interactive games. How long will it take for leading artists to turn into this interesting medium for their works? If full-immersive, wide-area, and 3D motion tracking virtual reality is available to everyone at an affordable price, the trend towards interactive computers in entertainment, education and culture will accelerate.
In addition, as we crave higher levels of intervention and behaviour in games, new devices beyond the keyboard and simple compact controllers are required for access to immersive content in the future. I think the answer is a combination of Brain-Computer Interface and Virtual Reality. Known as BCI (Brain-Computer Interface), an interface system that directly connects a brain and a machine to operate a computer or machine. The BCI technology is implemented by sensing, recognising, and accepting brain activity through a mechanical device, then analysing the signal through a signalling process and giving commands to input and output devices. The BCI, currently under study, is not a miracle device to read the mind. Depending on the application, it can be used to read a person’s mind, emotion or mental state. Because it is essentially a device for reading human thoughts, it is used in the form of reading the electric nerve signals of brain waves or brain cells and receiving specific patterns as input signals.
Attempts are now being made to fuse with various industries using BCI. The most commercially advanced form now is probably ‘Neuralink’, founded by Elon Musk. The company was created to connect the human brain to the computer, and is expected to work by reading and writing data or using AI aids using electrodes flowing through the brain. In July 2019, we unveiled an interface device consisting of a polymer electrode that can be implanted into the brain and a tiny chip called ‘N1’.
I am trying to implement this BCI practically through the EEG device which can be obtained inexpensively with the development of technology. Through this, I try to work in the form of measuring and responding to the brain waves of viewers in virtual reality. This work can be a task to visualise by measuring the feelings, emotions and concentrations of viewers in the virtual space. In addition, in the field of BCI, they are also experimenting to link virtual reality with EEG. This is a kind of data visualisation process, and effective visualisation will be left to the designer.
Attempts like as above can help answering about what makes us who we are, such as “How do we perceive, learn, and remember?”, “What are the characteristics of emotions, empathy, and consciousness?”. I also think meaningful conversations will be possible between other areas of knowledge over the brain science and art.
What is the ultimate goal of virtual reality? I think it is ‘reality’. Disconnected from the ‘real’ world outside the VR device and feel like we are actually embodied in the simulation.
This requires a complete level of simulation. Additionally, the quality of the environment constructed in virtual reality varies depending on the computing power of the computer. Current consumer computers performance has limitation. But what if we combine it with the cloud computing services I mentioned earlier and quantum computers that are billions of times the performance of current supercomputers?
The journal was published by Google AI Quantum, led by John Martinis in the journal Nature on October 23, 2019. The 54-Qubit Sycamore processor is the first quantum computer to achieve a ‘quantum supremacy’. Quantum supremacy is a turning point where quantum computers can do more than the most powerful existing supercomputers. In a nutshell, the computational problems that traditional supercomputer processors had to perform over 10,000 years can be solved in just 200 seconds. It will also contribute to the development of graphical operations that require high computing power.
‘Beauty’ of Immersive Contents
In the future, the new immersive media will be focused on user’s experience and participation than just seeing. In this respect, what is the ‘beauty’ of ‘Immersive Contents’ mediated through spatial experiences? I think we need to discuss this. This is because the discussion on ‘beauty’ is easy to use in reaching out to the general public, and it becomes popularity that anyone can understand. It will be possible through the visual language, the universal language of mankind. Then, what is it necessary for the public to become viewers as creative observers in the new interactive art of immersive content?
I think first need a process to understand that visual perception is a complex mental process, same as abstract artists understood such as Mondrian, Roscoe and Morris Lewis. They have extensively experimented with ways to mobilise various aspects of the viewer’s attention and perception. Mondrian and Roscoe drew attention to art in a psychologically new way by extracting form and colour. Like this dialogue, we need to understand how the brain responds to works of art and how we deal with unconscious-conscious perceptions, emotions and empathy. But, how is this conversation useful for artists?
Since the beginning of modern scientific experiments in the 15th and 16th centuries, artists have been interested in science. Filippo Brunelleschi, Masaccio, Leonardo da Vinci, Albrecht Dürer, Pieter Bruegel the Elder, Arnold Schönberg and works of abstraction this is an example. Just as Leonardo da Vinci used human anatomy knowledge to describe the human body in a more overwhelming and accurate way. Modern artists can also create new art forms and other ways of expression based on a new understanding of the biology of perception and the biology of emotions and empathy.
From this perspective, the 4th year project will examine the ‘beauty’ of the BCI-VR format that combines the brainwaves and virtual reality of the viewers and the leap to a new medium. ‘beauty’ has long been considered important in art, and nowadays it is possible to visualise changes in the brain and present them in a more objective form. Therefore, I think that ‘beauty’ can be solved through visual perception and awareness.
Can the virtual space of virtual reality lead us to change our perceptions, feelings and empathy? this is part of the project question. If we study emotional reactions, visual reactions, and minds when humans see and feel something, they can make us perceive unpredictable relationships in the relationship between art, emotion, and mind. As Leonardo da Vinci did, I think this process can push the horizons of art one step further. We can better understand ourselves by incorporating a deeper understanding of how we respond to art and how to create it. This process helps the vision to understand the part of the viewer through the knowledge acquired through the creative process, and is the starting point for productive dialogue between brain science and art.
Immediate integration between aesthetics and the science of the new mind will be difficult, but new conversations may begin and sooner or later have a cumulative effect among those who are interested in the many aspects of the science of perception and emotion.
What if we ask about the boundary between virtual and reality in the BCI-VR form above? I think we need to discuss how we perceive the world. What does virtual reality close to reality mean for viewers?
Basically, immersive content is virtual. According to Jean Baudrillard’s argument, virtual is a simulacre as non-existent symbols. And according to Pierre Levy, it is opposed to reality but can be interpreted as a concept for the creation of reality. Virtual is an extraordinary experience. The virtual experience that tangible content provides is based on extraordinary. In other words, even if their experiences are fascinating, exotic and immersive, they are only temporary, illusions that seem to exist. Festivals also same as them. The festivals provide a deviant experience by allowing visitors to experience a variety of viewing and experiential elements that are not seen in everyday life and the local culture. Particularly, the experiential festivals make you forget the bleak daily life of reality by deviating from the rules and existing social relations or stimulating nostalgia about the past.
Garden an Arcadia – The Image of Utopia and the Truth of Virtual Reality
The pleasure of virtual reality is, ‘walking in the forest and look at the dense leaves’, ‘watching the leaves drift in the wind’, ‘listening to the sound of birds and my rustling footstep’, The key things of pleasure is knowing that moment only exists for me entirely. I enjoy the moments that are unique in time and space, the moments that will never come back.
From this point of view, the content of this project was selected with the theme of ‘garden’ focusing on visual perception, emotion and art. The form of garden is a paradise on earth and looks like Arcadia (Utopia). From a long time ago, most people dreamed of an ideal and liveable society, and they developed to achieve it. In particular, the West describes this world as a mythical world called ‘Arcadia’.
In order to embody the imaginary ideal on the real world, many people have been thinking creatively, and as a result, the ‘garden’ that mimics nature appears. The garden is usually considered as a heaven of real world. People experience various things through elements in the garden. However, the experience through the content of the garden is an object that cannot be owned in reality, and it is cognitively satisfied, and the garden can be treated the same as the virtual reality. So, while these experiences are fascinating, different and immersive, they are only temporary illusions that seem to exist
In addition, garden art is a multi-sensory art that allows people to experience emotion, its aesthetic correlation, through movement. Garden art moves include: 1. Practical and explicit movements of natural elements (wind-shaking leaves, water stirred by boats), 2. Visitor movements made when experiencing the garden, 3. Day, season, weather there are relatively slow movements made by changes in the world, 4. There are implicit and metaphorical movements made by ancestors and monuments. For walkers walking through the forest, new ways of perceiving landscapes are created as trees seems to move. The movements listed above result in a movement called ’emotion’ that touches the mind and mind of the viewer. ‘Hirschfeld’ said that garden art is a multi-sensory art that allows people to experience the aesthetic correlation of emotion through movement and to use the five senses.
I wish in this virtual space, we can find the emotion, understanding empathy, and visual perception of the viewer’s participation find a new and unexpected relationship?
There is an example of how gardens have been used by digital or interactive artists. There is Osmose (1995) by Char Davies, a prototype of a virtual garden, and TV Garden (1974) by Nam-June Paik, a media artist. The garden has been used as the material of art. And their works gave fresh inspiration to the direction of my work.
The distance feels very far, but I tried to redefine it as a one worldview by experiencing the essence of technology and nature through immersive content. Part of the attempt is to include virtual objects represented by nature in the virtual natural ecosystem. Because I think there was a need for a new worldview and life-style where was a technological revolution.
What Kind of User Experience Does This Project Aim for? With Human Visual Perception and Emotion
The project was influenced by the Sense & Sensibility project, a third-year project in the interaction design. In addition, using the third year’s source review ‘Data Visualization and Digital Communication: Designing Data’, I decided to do ‘Visual Culture and Immersive Content: Viewers Perception and Emotion, and Direction of Contemporary Visual Culture’.
The third year Sense & Sensibility project, which influenced me a lot, was a project about ‘Consciousness’ using computer vision. In this project I researched about what is meaning of consciousness, intelligence and awareness on the humans and machines. I examined what consciousness and perception are, rationality, understanding, and empathy.
The curiosity about human visual perception and emotions that began here led to the fourth-year project. In the project, “is the study of visual perception and emotion and virtual reality worthwhile as a new visual culture and art form in the future?” was added. Specifically, “by analysing the brain waves of the viewer’s visual perception input, can the viewers be visually satisfied and aesthetically satisfied?” Also, I can question for “how does this mechanism work?”
Through them I can moved on next question “What kind of user experience does this ultimately aim for?” For example, what kind of effects occur inside the virtual reality, and what types of experiences the viewers experience are visual, auditory, and tactile?
Also, what kinds of experience will be added? For example, it may be a task using motion tracking of a hand and generate an effect by detecting a position according to a viewer’s viewing direction.
One of the ultimate goals is to research how the brain responds to the art work, and how viewers deal with the process of unconscious – conscious perception – emotion – empathy.
About tools – Oculus Quest and EEG (electroencephalogram)
The technical processing part is divided into two. One would be a physical tool such as a virtual reality device, an EEG or a high-performance computer. The other is to build a virtual reality environment.
In the VR devices will use ‘Oculus Quest’, a virtual reality device in the form of a wireless device. I will also work with EEG (eletroencephalogram) as a brain waves measurement device. EEG refers to the flow of electricity that occurs when signals are transmitted between the nerves in the nervous system. It varies depending on the state of mind and body and is the most important indicator of brain activity. Neurophysiological measurements of the brain’s electrical activity are recorded through electrodes attached to the scalp.
After investigating of EEG, the price and the number of sensors, I decided to work with a product called Muse2 from Interaxon. Others include NeuroSky, Emotiv, OCZ, and OpenBCI, all of which are accessible to consumers.
The Muse2 has four-EEG sensors that can be measured in the frontal and temporal lobes. The position of the electrodes is in the TP9, AF7, AF8, TP10 according to the international standard arrangement 10-20 system (EEG).
The above electrodes data can be collected by the viewer’s brain waves, which can be visualised into five types. Delta wave in 1 ~ 4Hz zone, Theta wave in 4 ~ 8Hz zone, Alpha wave in 7.5 ~ 13Hz zone, Beta wave in 13 ~ 30Hz zone, Gamma wave in 30 ~ 44Hz zone. Each brain wave type represents a different state.
Alpha Wave – Alpha waves are brain waves that occur at rest and at rest in the band 8-13 Hz. It appears mainly in the relaxed state and increases in amplitude in the relaxed state. For normal people, an increase in alpha waves means meditation, peace, and a calm internal state.
Beta Wave – A band of 13–30 Hz that is produced by awakening, tension, or stress.
Delta Wave – An EEG that occurs in deep sleep in the band 0.5 to 4 Hz.
Gamma Wave – Generates high cognitive activity in the band above 30 Hz. Usually a waveform that manifests itself when enlightenment or insight occurs.
Theta waves – bands of 4–8 Hz, related to creativity, meditation and drowsiness. Appears when you’re sleepy and awake in a light sleep state. Theta waves are often defined as the boundary between perception and dreams.
If viewers observe random object on the environment ‘A’ of virtual reality, viewers brain waves will change. Someone who concentrates somewhere, someone who has realised something, or who is stressed out by the motion sickness of virtual reality. I can work on these emotional reactions by associating them with objects and effects in virtual reality.
Now that you have looked at the hardware for the physical implementation of your work, it’s time to look at the internal content, or software. It is for how to combine the virtual garden and the data measuring the emotions and consciousness of the viewers. In conclusion, everything will ultimately be a single virtual environment using a tool called ‘Unity’. Unity is a game engine that provides development environments for 3D and 2D video games, and an integrated authoring tool for creating interactive content such as 3D animation, architectural visualisation, and virtual reality.
Picturesque and Form of Garden as a Content
Before this, it is necessary to design form of the garden. I plan to create a garden with terms that translate the form of the garden into a ‘picturesque’ style. We are immersed in the various forms of nature and their infinite forms. The landscape created by nature changes at every moment, and at some point, it looks ‘beautiful as a picture’. The philosophical scenery that enchants the soul is created from the “picturesque” scenery that enchants the human eye.
The English garden captures this term “picturesque.” Based on the theory of 18th-century British landscape gardens, picturesque expresses more natural landscapes or borrows from garden landscapes by reflecting the natural hobby of gardening.
There’s another reason why I have taken ‘Picturesque’ aesthetic as a keyword. In this period, when picturesque appeared, the visual media such as panoramas, dioramas, stereoscopes, photographs, and movies were invented. They expand the public’s senses, and the way of viewing was trying to change. The visual system, based on the linear perspective, was transformed into the ‘visuality of movement’, which is summarised as ‘fluid vision’ and ‘perspective relativity’ under the influence of new optical instruments that appeared in the 19th century.
I’m going to use picturesque reductionism, back to aesthetics as BCI-virtual reality. It can be used for perception of a line, a complex scene, and a work of art that evoke emotion. Or, by reductionism the shape, the elements of nature, whether in form, line, colour or light, can be removed and perceived by the viewer. This separate element may stimulate several aspects in a direction that complex images cannot. In this project, it is possible to use reductionism to elicit the viewer’s new perceptual and emotional reactions.
About Software and Future Direction
Some of gardens I will actually planned to do the targets of the conduct field surveys, and then model internal elements or objects through 3D tools such as Maya and Blender.
I will add these objects to Unity’s assets, and then connect the viewer’s brainwaves in the order of EEG-Analytical Programs-Unity. The interconnection is made up of what is called “OSC – Open Sound Control”. If your devices are connected to the same network, you can share data with unique IP addresses and pod settings.
Currently, I have connected the between EEG and Unity to test whether data is normally sent and received, and I have mapped the received data with a test object to change the colour of the object or rotate it.
Additionally, interactions can be built using the viewer’s hand. Currently only controller interaction is possible, but it will be possible to create interactions by tracking the viewer’s hands in the future. For example, an effect that triggers when a viewer’s hand is within a certain distance from an object, or an interface control inside of the virtual reality.
Boundary of Reality and Virtual – Plato Idea and Jean Baudrillard’s Simulacre Theory
The current direction and current status of the project is the same as mentioned above. In the above discussion, it is meaningful to recognise and prepare for the ‘beauty’ and aesthetics of the new visual culture in the future. Also, I think it’s worth it to finally introduce these things to the public. In addition, this section will discuss critical discussions of virtual reality and boundary of reality and virtual.
In critical discussions, we might ask questions about imaginary boundaries simile using Simulacrum theory. Which is the essence and what is the concept of the original and the illusion. I think this discussion can take a step back from virtual space and analyse it critically.
Simulacrum was originally a concept defined by Plato. According to Plato, the inhabited world consists original form of an idea, a replica reality, and a Simulacrum which is duplication of replica.
The cloned is farther away from the idea as it is replicated. For this reason, Plato regarded Simulacrum as having no value at all because it could not be same existence in the same place even one-moment and not being present.
Simulacrum is typically discussed between Jean Baudrillard and Gilles Deleuze. The definition is divided into two. Baudrillard’s Simulacrum is the result of Simulation and refers to artefacts that make objects that do not exist as if they exist. Deleuze’s definition is different from simple cloning because he sees the definition of Simulacrum as not merely a clone, but as an independent entity, an original self-identity that goes beyond the real rather than the real.
According to Simulacrum’s theory, virtual and reality coexist, but they are opposed. ‘Virtual’, which is an image of ‘reality’ that is replicated from the original, is a reproduction of replication. The work of this reproduction is getting far from reality. This study attempts to explain this based on Baudrillard’s Simulacrum theory. Baudrillard said about the image:
– The first stage is a faithful image/copy, where we believe, and it may even be correct, that a sign is a “reflection of a profound reality”
– The second stage is perversion of reality, this is where we come to believe the sign to be an unfaithful copy, which “masks and denatures” reality as an “evil appearance—it is of the order of maleficence”.
-The third stage masks the absence of a profound reality, where the sign pretends to be a faithful copy, but it is a copy with no original.
– The fourth stage is pure simulacrum, in which the simulacrum has no relationship to any reality whatsoever.
Baudrillard said that while modern times are controlled by industrial capitalism, postmodern ages are information ages dominated by models, codes, media, and cybernetics. As a social and cultural phenomenon, the oversaturation and saturation of information from various media and images from media dramatically change the value and meaning of existing objects. The meaning of things is lost, yielding a more real derivative (simulacrum).
Era of Simulation
The virtual reality implementation in my work is a duplicate of the illusion and also a reality more real than reality. Modern society no longer consumes things as use values or exchange values, but as images and symbols. Today, cultural phenomena are appearing in the Simulation process of producing and manipulating new images by synthesising them with TV, computer and media information.
The imaginary images themselves create new realities and the images interact with each other and lead to the simulation process. Images only exist in the circulation of simulation. In the simulation era, it is filled with images of coping with reality, then they swallow everything up. The image is no longer exchanged with reality, it is a simulacrum that is exchanged with itself in an endless circulation with no directions or borders from anywhere. The world full of signs and images without reality is the era of where we live: simulacrum and simulation.
As such, the attributes of the image that cope with reality bring the media attribution that are the representation of the medium symbol in my project. The visual media, which makes us believe that we have experienced even if we do not experience it, makes us feel the reality of modern people who are in a passively mixed situation in the constantly incoming information and images.
It is the world of simulacre that cannot distinguish which one is the original or the duplicate, rather than which one thing becomes the original of them. In fact, are not we experiencing this Simulacre world every day?
As Walter Benjamin said that replicating a work of art equally in ‘The Work of Art in the Age of Mechanical Reproduction’ puts it into a new context, the original meaning of the work is completely different. We need to think like that. Benjamin feared that in the age of replicating artefacts, replicas whose original context had been removed would not only undermine the original value but also transform the meaning. As an extension of this concern, the implications of reality cloning give us some surprises, but we should think about ‘we are not yet ready for the future’.
Conclusion
The motivation for this study is that ‘beauty’, which is a form of art, is a very important issue in visual culture, and ‘beauty’ is also an essential element in the immersive visual culture. In addition, I think that ‘the beauty’ of the tangible visual culture can be achieved through discussions focused on the share of the viewers discussed above and the viewers’ visual perception and emotion analysis.
Through this project I am expected to contribute greatly to the design of user experience in interaction design and the study of data visualisation. On Jun 2020, this project will be produced and exhibited with concrete content, aiming to create an interest in other design fields and future participation in the immersive visual culture. And I think it’s a good opportunity to introduce the future direction of visual culture to people in fields not related to this field.
As I mentioned, the world is changing faster than in the past, and art is flowing in this direction as more people seek interaction.
We are living in the era of surrounded by technology, things can be reproduced in new shapes and sizes, and in high resolution it is easy to get the real sense of reality, the ultimate goal of the simulation experience. In these times, you can enjoy a rich experience for education, entertainment, and self-understanding. In the meantime, there is a risk that the value of authenticity will be reduced by blurring the boundaries of valuable, unique and temporary real experiences and persuasive and simple replicas. It is obvious that this can cause a metaphysical blow to us.
With new technologies readily available, everyday understandings of what is real and what is not real will be different. We have evolved over thousands of years, experiencing only in our physical space. But now we have thrown this bridle and entered a new phase of civilisation.
We can discover that science and art can interact and enrich each other. The two subjects offer their own views on the essential question of the human condition and they attempt to begin a conversation that has never been before in the new form of modern art. If this effort is successful, it will help us to understand more deeply how we respond to art and even how we create it.
Science seeks objectivity and strives to accurately describe the characteristics of things. Assuming that art’s perception is the interpretation of sensory experience, scientific analysis examines in principle how the brain perceives and responds to art. It can also answer how this experience transcends the everyday perceptions of the world around us. The new mind science will continue to deepen our understanding of ourselves by building bridges between brain science, new forms of visual culture, and other fields of knowledge.
Synopsis
This proposal is to implement virtual space for a new form of multi-sensory (visual, auditory, and tactile) experience based on immersive content and brain-computer interfaces based on the theoretical considerations of immersive content, simulacrum, user experience data, etc. with ‘picturesque English garden’. Even after the proposal is completed, I will continue to study ‘human nature, mind, and brain’ and ‘real content’ And the interpretation of the viewer’s sensory experiences will help researches to understand how the viewer perceives and responds to the work. Also, I will see how this experience transcends the everyday perception of the world around us.
1. Background and purpose
This proposal began with the questions of “Can future immersive content, such as virtual reality, be used as a form of art?”, “How can we use reactions such as the mind and the emotion of human nature to provide a different experience in art?” These questions were based on the reality that the existing viewer experience was not a two-dimensional interaction on the screen, but a realistic content using the viewer’s five senses, and the application of virtual reality technology to the art field was essential in the era of individual data and the fourth industrial revolution, and that the subject could increase interest in human nature and the importance of nature.
For a long time, people dreamed of an ideal and good society. And they tried to make progress in the development and technology of the humanities to achieve that. Especially in the West and the East, this world is Utopia, Arcadia, and Parada describing this as an ideal world, a heavenly world, and the efforts to embody it in this world appeared in the form of a “Garden.”
“Garden” with human intent as a cultural landscape involved has been recognised as a valuable cultural element that shows the unique cultural attributes of the mankind, while having global, humanistic attributes beyond time and space, as a cultural device for humans to adapt to nature to enjoy survival, prosperity, and pleasure. The designed garden represents the ideal and value of owner as a cultural product, is located in its social environment, and the process in which the garden is created by the combination of natural and artificial elements in different times and places inspires modern designers and affects their work.
This proposal is to implement virtual space for a new form of multi-sensory (visual, auditory, and tactile) experience based on immersive content and brain-computer interfaces based on the theoretical considerations of immersive content, simulacrum, user experience data, etc. with ‘picturesque garden’.
2. The content of the proposal
This proposal was a link between individual emotional reactions and curiosity in the reflection of the times of experience with the expectation that the link between realistic content and Brain-Computer Interface would come to us in a new form of art in the future, and with “Virtual Garden” and “Multi-sensory Experience” with the potential to develop virtual art. It targets those who have the potential for the development of art and technology, those who seek a new form of art in fine art, and the public who are not related to art, and all audiences interested in spatial art, such as gardens, environment, media art, architecture, and urban design.
3. Proposal task progress
1. Immersive content
1) Definition
‘Virtual Reality’ is a technology that allows users to experience really difficult situations in the real world by extending and sharing sensory information within a computer-generated virtual environment, while ‘Augmented Reality’ is a technology that combines virtual content with real- world and things to provide users with information on the environment. If virtual reality allows users to experience content disconnected from the real world through immersive devices, augmented reality is distinguishable by the fact that it presents content that is fused with reality. Recently, with the emergence of Mixed Reality, where the combination of the virtual and real worlds becomes more natural, technology is being developed to maximise the immersion experience of users.
This immersive experience has evolved into an online multi-user environment, which is collectively referred to as “immersive content,” expands the scope of virtual information on the current visual base to five senses, and allows multiple users to share and communicate the same virtual space without limits on multiple distances within a single user environment per device.
2) Characteristic
The goal of realistic contents is to embody ‘the practically same space as the real world’ through various sensory stimuli such as human senses of five senses and balance, with three characteristics: ‘Immersion,’ ‘Interaction’ and ‘Presence,’ the main contents are as follows:
(1) Immersion
Immersion means the feeling of digging or falling out of an object. Usually, people who use virtual reality forget their consciousness and perception and react only to their goals when they become immersive because most of the senses that people perceive are concentrated in sight. Therefore, visual requirements should be satisfied to enhance immersion, which is valued in realistic content.
(2) Interaction
Interaction means interacting with a device within a virtual space. As the input and output of the user’s information are instantly responded to within the space, the process of interaction is critical. When a viewer takes action within a virtual reality, it can increase the viewer’s sense of immersion by inducing the objects to respond. Interactions can be implemented in conjunction with external sensors and controllers
(3) Presence
Presence means ‘feeling in a certain environment’ Heeter (1992) used the expression “being there” to indicate the feeling that a user is in a virtual environmentv, while Slater and Wilbur (1997) defined presence is believing that the user exists in an environment other than the real space and the feeling as being remote.
3) Forms and types of virtual reality
Virtual reality is a real-life experience of difficult situations in the real world, and types are divided into three levels depending on how the interface or system is implemented, as follows:
First, non-immersive virtual reality is a virtual reality that is implemented with traditional PCs, such as keyboards and mouse monitors, and can be produced at minimal cost. Although popular, I feel a low level of immersion and presence.
Second, semi-immersive virtual reality is a virtual device that is immersive given by multiple screens and large monitors. A typical example is a movie theatre and a game.
Third, Fully-immersive is a virtual reality that uses separate hardware devices to block the external environment and reproduce the highest level of immersion. Various senses can be felt within virtual reality as hardware advances. Examples include ‘HTC Vive’ and ‘Oculus Rift.’ As an era of experience in the future, immersive virtual reality was used in this proposal, because immersive virtual reality would lead to a familiar and high level of user experience to the public.
Fully-immersed virtual reality is implementable in two different ways: Output Devices to reproduce virtual reality and Input Devices to receive input from viewers. In particular, output devices include a head-mounted display (HMD) that allows viewers to experience virtual space by being mounted on the head and an input device (controller) that receives viewer feedback. And the current virtual reality device automatically calculates the location, direction and height of the viewer by borrowing a method called Six degrees of freedom (6DoF), and changes the point of sight as the viewer moves. This has been changed so that the viewer can move beyond the space limit and enjoy what he or she wants.
4) The spatial significance of virtual reality
The viewer experiences a data-driven space designed by the operator in virtual reality. Realistic content is non-realistic and non-material space, and viewers experience immersion, empathy, behavioural identity, and transformation in virtual space. In particular, the release of consumer virtual reality devices by ‘HTC Vive’ and ‘Oculus Rift’ since 2016 has aroused the general public’s desire to experience virtual space and enabled viewer involvement in multiple areas, not the passive story experienced by past two-dimensional monitors. Thus, if the physical space of reality is formal and limited, including the concept of time, then virtual space is not formal, and time can be manipulated and amplified. These fabricated experiences trick the viewer’s brain into creating an illusion of emotion. For these reasons, the designer’s intention is important when creating a virtual space.
5) Future prospects
In ‘Top 10 Strategic Technology Trends for 2019’ released by Gartner, technologies for immersion experience were selected as key future technologies for building next-generation digital business ecosystems, while IDC forecast that the global AR/VR SW market will grow 44 times from about $2.1 billion in 2016 to about $92.6 billion in 2021.
2. Spatial properties of simulacrum
1) Definition
Simulacrum refers to an artefact that has created an object that does not exist. According to Plato, the human world is made up of a prototype (Idea), a replica (reality), and a replica of a replica (Simulacrum). The reality here is that human life itself is a replica, and we saw Simulacrum as the “imitation of imitation,” which is an inferior subordinate concept of “imitation of idea.” In other words, it was God itself that was created by a whole nature and a hostile being, and Simulacrum is considered as a form of chasing God by imitating its essence. Later, the concept of simulacrum began to take notice after the 20th-century post-structuralism’s representative philosopher Gilles Deleuze presented a spatial concept that was distinct from structuralism. The simulacrum that Gilles Deleuze thinks is not just a dichotomous replica of Plato, but an important concept that has self-identity creating a new area of its own without stopping to clone the cloned model, and later developed into an important concept in modern philosophy.
2) Simulation and Virtual Space
From Plato’s ‘Model Simulacrum’ to Gilles Deleuze’s common feature of ‘Simulacrum, where matching the original is not important’ is ‘reappearance,’ and Jean Baudrillard well highlights the importance of original replication before the concept of ‘Simulacrum’ and well explained about ‘virtual replicas that are more practical than original.’ Jean Baudrillard talked about simulation of a new image in his book [Simulacres et Simulation], published in 1981, which defined modern society as being dominated and replaced by simulation, and he claimed that simulation is the movement of Simulacrum which is the simulated condition. In other words, the core concept of Jean Baudrillard was called a real artefact as ‘Simulacra’ that was transformed into a derived entity, not an actual one, and created as if there were objects that did not exist in real life.
Simulacrum exists together as an infinite source, not as a dichotomous reproduction of the simulation. Many people think of Simulation as Imitation; however, it is wrong. There’s a real object for imitation, and if people copy this real one, that’s what imitation is. These imitations belong only to the image of a traditional reproduction system, far from ” Simulation.” Simulation is an image that has nothing to imitate, and unoriginal images replace reality in themselves, and images dominate reality, therefore It’s more realistic than it is.
In other words, ‘Simulation’ is a more realistic and one independent reality. What we thought was real so far comes from simulation, which we said was unrealistic. In ” Simulation,” realism is nothing more than an “effect” or a “fantasy,” and all re-enactments are made through a code of re-enactment, the relationship between structural terms that are exchanged and generated at all times, rather than a reflection of the existing reality.
Previously, frivolous replicas such as unrealistic, imitative, and unrealistic ones are believed to be more dominant than real, according to Jean Baudrillard. Clones are presentable and are an infinite source. This is because there is no longer any distinction between the “reality” of the real-world and the “replications” of imitation, and the boundary between reality and cloning is gone.
The reality is no longer truth; it’s just an operational reproduction. In other words, the reality that we live in can be ‘replications’ rather than ‘reality’ and ‘replications’ can be ‘real’. The further boundary is gone and inside the simulation.
3) Application of spatial properties of simulation
Virtual refers to a futile world that, subjectively, seems real, but objectively does not exist. In other words, Virtual is an earlier stage of practical or realistic embodiment and is a very major dimension of reality in a philosophical sense. Virtual reality seems to represent a contradictory or mysterious world, as the reality often considered the opposite of virtual, assumes material embodiment, detectable appearance. In reality, however, virtuality and presentability are two different ways of reality. Thus, virtual and reality exist together without dichotomous separation. Therefore, virtual does not mean that it is not true or that it is not clear, but that it coexists with reality, which implies that the present is a source of existence. In other words, ‘virtual space’ refers to a place that exists with an experienced reality, not a vague and mysterious world.
According to Jean Baudrillard, the visual phenomena of the 20th century are no longer paintings, photographs, fiction, and unrealistic visions under the perspective. In other words, a world of simulation that the boundaries of reality and hyperréel-hyperreal become blurred, and the meaning of reality and hyperréel-hyperreal disappear is created. Real experience and memory of action become real, and space creates virtual space through hands-on experience. So, virtual space gradually becomes the product of a simulation that is more real than real, and the simulation hits the simulacrum and destroys the boundary. Jean Baudrillard argues that this place where we live is none other than a virtual reality within a confusion of Simulacrum.
3. User Experience – Emotion
1) Definition
The brain that controls the user experience controls all human behaviour, and neurons do information processing activities such as learning, memory, and problem-solving. The structure of the brain can be largely divided into the forebrain, the midbrain, and the hindbrain. The forebrain is a region associated with the cerebral cortex that accounts for most of the brain. The brain is divided into the left brain and the right brain and is also divided into the frontal lobe, the parietal lobe, the temporal lobe and the occipital lobe based on the measuring of human brain electroencephalogram.
2) Brainwave classification
The types of brain waves for measuring emotions are divided into alpha wave, beta wave, delta wave, gamma wave, and brain waves vary with the degree of brain activityxviii, as follows:
4. Garden and culture
1) Overview
The gardens in the world are different from country to country and from region to region and from time to time, and differ according to the profusion and affluence of flowers and leaves that bloom and fall according to the season, but they are all one in all, given that they are gifts with a thousand faces given by nature entering the garden of the house.
These gardens, which are grouped according to different regions and cultures around the world, maintained a consistent appearance that was formatted in a certain pattern, as they had almost consistent basic principles based on the problems of vegetation and irrigation in accordance with the climate and cultural characteristics unique to the culture.
2) English garden and Landscape
By the era of Elizabeth I, Britain had formed a strong monarchy since Henry VIII, and a high level of culture, such as the literature of William Shakespeare, had blossomed. Britain, which has not been able to make clear its successor after the death of Queen Elizabeth I, is once again in turmoil, and a major change in the garden occurred as well.
The French garden style, which was abundant until the Tudor dynasty, has become a target of criticism, and calls for breaking the framework of the garden locked in the formal style are beginning to emerge through literature.
The appearance of landscape gardens began in the late 17th century, the biggest change in English gardens. The background of the landscape garden can be discussed with the basis of understanding Alexander Pope, the poet of Neo-classicalism with the Grand Tour. In 1727 Sir. John Clerk (1676-1755) sang the ideal beauty of enjoying the landscape in the countryside through the poem The Country Sheet, and Alexander Pope (1688-1744) claimed that French formal garden shows low beauty and true beauty is the beauty that resembles nature.
The desire for political freedom and the tiredness felt in the stereotyped French garden led to the desire for a new garden, and this desire made an important turning point for the English garden.
Grand tour, which was popular with the young, wealthy aristocrats at the time, made many Britons interested in Italian gardens, wanted to create landscapes in front of their homes, and created the most natural but artificial British landscape gardens.
In the early 18th century, there was a growing interest in the Italian landscape expressed in landscape paintings by artists such as Claude Lorrain and Gaspard Puossin and attempts to create a new culture by interpreting these landscapes in a classical way. The tiredness of strictly controlled formal gardens and yearning for nature-like spaces began to make landscape paintings that were no more than flat into three-dimensional ones when they needed the creation of new forms of gardens. This result created the Landscape Garden in England.
William Shenstone (1714-1763), a poet who sang the scenery of the countryside, said the garden resembled a dramatic poem, and created a view point where visitors could see the scenery while expressing the beautiful scenery in his garden work. Experience the scenery through the view has become the main way to appreciate landscape gardens, and various landscape sequences have been created using curved walkways.
The Ha-ha technique also attracted scenic views outside of the territory into the garden. The landscape garden, which was born and started to be created by writers, was established as a genre by William Kent (1684-1784) in the 1730s, and Kent, an ordinary artist, began to make landscape gardens using his painting techniques. This means that the landscape painters began to jump directly into the garden building, and although landscape artists had designed the garden through paintings so far, they later played a role in making garden designers appear in society.
Roman sculptures and garden buildings were the first to be used in Chatsworth House garden, but they became the most popular garden in England, such as Rousham House Garden, Stowe Landscape Garden, and Stourhead Garden. The vast landscape garden was also open to the public, and Richmond Park, designed and created by Charles Bridgeman (1680-1783) was a classic example of the transformation into a public park.
Particularly, Stowe Garden is also known for bringing about an innovative change in English landscape gardens. Landscape gardens have been designed to appreciate gardens in one of the best views, but Stowe Garden is designed to stroll through the gardens or take a tour of connected copper lines to enjoy natural, beautiful, and connected scenery. Since then, many gardens have developed into landscape gardens with sequences along these patterns.
1. Research materials
The garden is a multi-sensory art that allows experiencing the emotions of its aesthetic object with the movement. Garden art movements include first, actual and explicit movements of natural elements, second, visitor movements made when experiencing the garden directly, third, relatively slow-paced movements made by changes in daytime, seasonal and weather, fourth, allusive and metaphorical movements produced by ancestors and monuments. Especially for the walkers who walk through the forest, a new way of recognising the landscape is created, such as a tree that looks like moving. And the movements listed above result in a move called “emotion” that touches the mind and heart of the viewer.
From the above perspective, ‘Picturesque English Garden’ was selected as the material of this proposal with a focus on visual perception, emotion and art, and the main reasons for this proposal are as follows:
First, the landscape created by nature changes every moment, and at one point it is described as “beautiful as a picture.” From the “picturesque” landscape that captivates the human eye, a philosophical landscape that attracts the soul is created. It’s to enjoy a time, a unique moment in the garden that never comes again.
Second, in the 18th century, visual media such as panorama, diorama, stereoscope, photography, and Movie were invented, expanding the public’s senses and changing the way views were viewed in the era of Picturesque. In other words, the visual system based on linear perspective has been transformed by the influence of the new optical instrument into a ‘visuality of movement’, which is summarised as ‘dynamic perspective’ and ‘relativity of perspective.’ This visual process has been applied to ‘garden’.
Third, the gardens of the study are as follows.
2. Experimental equipment
The experimental equipment used in this proposal is a high-performance computer, virtual reality hardware, and a brainwave measurement device for virtual reality implementation.
1) Computer
A custom PC with high-performance CPU and the graphic card was used, as it was required to receive a high level of real-time 3D graphics rendering and simultaneously implement interactive feedback from the viewer to implement virtual gardens with a high sense of immersion. Details are shown in the following table.
2) Virtual Reality Hardware
As virtual reality equipment, ‘Oculus Quest’ that was released in summer 2019 by ‘Oculus’ was used. The higher resolution of 1440×1600, compared with 1080×1200 resolution of the previous model ‘Oculus Rift CV1’, gives a cleaner and more immersive experience. Also, the external body tracking sensor that used to be in the ‘Oculus Lift’ disappeared, and it is also equipped with ‘6DoF inside-out tracking built-in cameras’, a technology that allows to calculate and track user locations on the device itself.
3) Brain wave equipment
The brain wave measurement device is recorded through electrodes attached to the scalp using the electroencephalogram (EEG) . The brainwave record uses the international standard brain wave arrangement, “10-20 system.”
In this proposal, I used InteraXon’s Muse2 for brainwave measuring devices, which can receive information directly from the computer and can automatically convert the raw data into five types of brain waves.
1. Experimental process and fabrication
The main issues of this proposal are in the construction of a virtual reality garden environment using immersion-experienced virtual reality devices, and the main contents are as follows.
1) Studio proposal results
The virtual garden implementation test was conducted in about 2 x 3 metres of space in the interaction design fourth grade studio. To implement the virtual garden space, a programme called Unity (2019.2.9f1) was used. The main details of the production status of the virtual garden are as follows. There are two main categories: the implementation of virtual reality gardens and the implementation of the viewer experience using brain waves.
First, a basic level of the virtual garden environment was constructed using the natural environment assets provided by Unity.
Second, after experimenting with interactions such as catching objects in a virtual garden and moving them to other places using the hand controller of a virtual reality device, the two- handed controller of a virtual reality device has confirmed that it can interact with objects in a virtual garden normally.
Third, using ‘OSC – Open Sound Control,’ it succeeded in transmitting and receiving data with brainwave measuring devices and Unity programmes.
Fourth, I successfully mapped sample objects of the virtual garden environment (Scene) and data values in the unit programme to test the rotation and colour variations of internal objects according to my brain wave changes. They also confirmed that virtual reality devices work even when they are worn.
2) Details of future work
First, based on the previous content, the following is to be carried out in the studio proposal; additional garden visits and data collection in the United Kingdom, further data on realistic content and simulacrum, and additional study of user experience. I will collect and utilise pictorial and sketch materials of the Picturesque gardens in the East and the West, especially in the U.K., and use 3D tools such as Maya to create and apply more diverse tree and plant garden elements.
Second, after placing the garden elements in the Unity programme virtual garden work environment, the interaction will be designed in conjunction with the viewer’s response and behaviour by utilising the trigger-type interaction of the Unity programme. For example, a viewer’s hand and arm joints will be tracked to implement interactions such as sounds or effects as they approach a particular garden element.
Third, it will link the viewer’s brainwave data with the garden elements and natural elements in the virtual garden to create interaction.
For example, if the analysis of the viewer’s brainwave data shows that the viewer is bored to the virtual garden’s experience, it would reproduce instructions or sounds to move to a specific location within the virtual garden.
If the viewer focuses on a target in a virtual garden, he or she will present a higher level of user experience by analysing brainwave data, including presenting in front of the viewer a choice interface that enables further interaction with the target.
Fourth, I will collect brainwave data values from the audience who visited during the exhibition in June 2020 and organise the possibility of virtual reality and emotional data as a new form of art.
Fifth, the virtual garden experiment path is as follows. If the exhibition space of the virtual garden is not sufficiently secured, a certain boxed space will be created to allow viewers to experience and interact with the virtual garden in a limited space. If there is enough exhibition space, it will be produced so that viewers can appreciate the virtual garden while walking along a certain track for experiencing a real garden. The exhibition will allow the audience to experience a variety of interactions and new visual experiences as they walk the track and interact with the objects in the garden.
2. Work process
3. Pros, Cons, and limitations
(1) Pros
(2) Cons
(3) Limitations
To answer these two questions; “Can future immersive content, such as virtual reality, be used as a form of art?”, “How can we use reactions such as the mind and the emotion of human nature to provide a different experience in art?”, realistic contents, simulacrum, user experience data, and gardens in the theoretical background are researched and analysed. As for practical studio work, last semester, I tested a virtual garden prototype, which will continue to be followed by further research in the future by the detailed garden element production and layout and further theoretical background.
This proposal has pros, cons and limitations; however, the new value of this proposal is that the viewer’s creative response (sensory experience) of virtual reality, emotion, and empathy is newly incorporated into the art as a form of ‘data’ so that it can be understood and based on the new art forms and other forms of creativity can be created. And as these interpersonal interests and interactions accumulate, I expect that This approach will contribute to advances in user experience, user interface, and data visualisation in the interactive design in the future. Seeing in the future is not a passive process that is transmitted on a two-dimensional screen, but seeing has again become an active process.
Even after the proposal is completed, my research on ‘human nature, mind, and brain’ and ‘real content’ will continue, and the interpretation of the viewer’s sensory experiences will help researches to understand how the viewer perceives and responds to the work. Also, I will be able to see how this experience transcends the everyday perception of the world around us.
After finishing the essays for the first semester and I am judging myself that the theoretical parts have been arranged, I am move on to create a full-fledged virtual space on the Unity. I started by re-learning how to use the Unity Terrain function. And at the Asset Store, I started to make scene by importing the tree assets I had purchased when I was working on the ‘Getting into Unity’ project.
However, the virtual garden that is being made now is different from the method created in the first semester. The work of the first semester garden was built in the form of an application rather than real-time rendering. Then, it was executed as an ‘unknown application’, and the virtual garden was run using the built-in processor and battery. However, current work is connected to the computer directly with a USB cable, bringing the computer’s CPU and graphics card calculation to the Oculus. Therefore, it is possible to stably enjoy virtual garden natural with a high quality level of graphics. The other reason I changed the way of viewing the garden from wireless to wired was Oculus company developed a software called ‘Oculus Link’ that can use Oculus Quest as a PC VR such as Oculus Rift. And, there are more limitations that I personally felt during the first semester.
Oculus Quest’s battery running time and limited number of polygons can used were being limitation of proceeding project. The battery time is roughly 2 hours, but there was no way to solve the battery running time problem in the actual graduation exhibition. In the case of graphic limitation, assets such as low-poly can be viewed without any delay, but even if it contains a normal game asset or a tree with a slightly higher quality, the screen can be cut off.
However, it can be shown a clear difference from the existing VR called wireless. Also, it was possible to see new possibilities of VR. However, since the wireless form is still a first-generation product, I think there are still many areas that need to be solved and developed in many areas.
Before proceeding with the project, I downloaded a project called ‘Book of the Dead’, which was produced by Unity in 2018, and referenced it in my work. It wasn’t a VR-related project, but just looking at the topography of the natural environment created therein seemed to be a great help. For example, how the terrain was arranged, what objects were placed in each section, how the sound was set, and how the views were processed. Also, I will extract sounds and other assets used in this project to them into my project.
After looking at the Unity demo project, I am back to my project. I also started looking for the plants or materals what I needed for my work. Props and small plants were imported from Quixel Megascan, which was used briefly in ‘Rich Sensors project’ last year. All of Quixel Megascan’s assets are provided high-quality assets by photocopying existing terrain and features. And if you use the program called Megascan Bridge, you can quickly export it by installing a plugin inside of Unity.
Having a long time and persistence, I am focused on created the terrain.
After making the terrain sample, I wanted to give the audience a more realistic feeling, so I did various studies on wind and sound. In the case of wind, it was found that the tree was made with a ‘Speed tree’, or by giving a noise value in the ‘Shader’, it could make the leaves look like they were actually moving.
And there are three render pipeline shaders in Unity. These are called ‘HDRP’, ‘LWRP’, and ‘SRP’. Each render pipeline is not compatible with each other.
So, when I was working, I had to pick one render pipe and set only the appropriate shader. In my case, I used LWRP for VR, a setting that is basically provided when creating a Unity project. The reason for using this pipeline is that, in the case of VR projects, two screens each for per eye, high refresh rate, and a resolution close to 4k make the computational rate usage by the graphics card and cpu very high. Using a lot of internal computer resources means high fever. If a computer has to be turned on for a long time during a graduation exhibition, problems can occur. Therefore, LWRP, a light pipeline compared to HDPR, was selected to slightly reduce the amount of computation and project capacity.
I also looked at the setting called post-processing v2. This is to make the scene look a bit more quality by processing the colour and blurring of the scene. However, in the VR project, although it was applied on the scene screen, it was not applied when viewed in VR. Post-processing receives external settings from the main camera. In VR, there are three cameras: left, right, and centre. Probably there are additional settings, but I will investigate and fix them in the future.
I am developing my project step by step. Now it’s time to connect the EEG and the virtual garden, not just the virtual space. In the first semester, only interlocking with OSC and cubes were used to rotate and change colours. What I’m going to do this time is to spawn a pre-specified plants at a specified location in response to an EEG.
—————————————————————–
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class NewBehaviourScript2 : MonoBehaviour
{
public GameObject[] newObject;
public Transform newPosition;
int randomInt;
// Start is called before the first frame update
void Start()
{
}
// Update is called once per frame
void Update()
{
if (Input.GetKeyDown(KeyCode.Q))
{
RandomSpawn();
}
}
int GetRandom(int count)
{
return Random.Range(0, count);
}
void RandomSpawn()
{
randomInt = GetRandom(newObject.Length);
var position = new Vector3(Random.Range(-4, 5), 0.5f, Random.Range(-4, 5));
GameObject clone = (GameObject)Instantiate (newObject[randomInt], position, Quaternion.identity);
Destroy(clone, 5.0f);
}
}
Once I pressed the keyboard, the object was spawned. I can call an object using the ‘Instantiate’ function. However, it must be stored in the form of prefab in advance for you project folder. Also, when spawning an object, the ‘Destory’ function could be made to disappear after a certain period of time.
If this work goes further, if I map the object spawn script that I wrote in advance when the OSC reaches a certain value later, I think that it will become a garden that changes according to the emotional state of the individual that I originally envisioned.
This is the process of my implementation. What I am going to do in the future is to create a empty space with only grass and spawn objects on there.
