Maya cabin hackathon

Since this year my projects Meditation Lab and Silence Suit are part of Hack the Body program initiated by the art-science lab Baltan. They want to combine different programs so they suggested that Hack the Body should work together with people from the Age of Wonderland program.
That meant I could work with Branly again. I met him last year and that was a very impressive experience. Branly works with people using ancient Maya spirituality.
At the same time I could try out the Sensiks cabin. With this cabin you can create multi-sensory experiences. This is very similar to what I want to do in my Hermitage 3.0 project. (This will be a space where I can optimise meditation by changing the environment and influencing the senses.)
I brought my Silence Suit which already has a lot of working sensors. We could use the suit to log biometric and environmental data and see how they are influenced by the actuators in the Sensiks cabin.
The main aim of the hackathon was to explore if ancient Maya culture and rituals can be transferred to a high tech environment. The team members were David, Branly, Masha, later to be joined by Michel.

Day 1: exploring
The first afternoon Branly explained the Tuj/Temazcal. It is used in a purifying rebirth ritual. It is a small dome-like structure that is heated by hot stones and steam. The experience resembles a sauna. The rebirth ritual is multi-sensory too: touch (temperature, rubbing with twigs and salt), smell: different herbs and resins, taste: hot drinks (herbal infusions, cacao, honey). Sound: beating of a drum, like heartbeat. Vision is excluded mostly. The Tuj is dark except for red hot glowing stones. We decided to take this as a starting point for building our experience.

Tuj/Temazcal Wikipedia image

The Tuj is located on a beach or in the woods. A quiet, relaxing space. The ritual isn’t limited to experience in the dome. Preparations start days before. The space around the dome is also part of the ritual. For example the structure has a low door so you have to get on all fours to enter. This immediately takes you back to your childhood.

Sensiks control panel photo by Masha Ru

Sensiks control panel photo by Masha Ru

The Sensiks cabin has lots of different actuators: smell, airflow, light, sound, temperature and VR. Everybody had a test ride. We all felt the cabin was rather clinical. We wanted to connect it to the environment. Make it part of a bigger ritual like the Maya rebirth ritual.

Day 2: concept development
Next day we were joined by other Hack the Body participants and hackers. One of them was Michel with whom I collaborate on the Silence Suit.
The whole group had a very interesting discussion about what an experience actually is and where it is experienced. Is it meaningful to recreate an experience that can never match the real thing? The most interesting would be to create something that can’t be experienced in the real world. We wanted to work on changing our state of mind through bodily experiences.

Another level of conciousness... Photo by Masha Ru

Another level of conciousness… Photo by Masha Ru

Day 3: design and experiments
The Maya team was joined by technology wizard Michel. We decided that we did not want to mimic the actual sensory experiences but try to induce a state of mind, another level of consciousness. We used these keywords as our guideline: womb, unknown, subconscious, abstract and random, rhythm. The next step was to translate these abstract concepts into an experience in the cabin. Actuators that we could use: smoke, heat, sound, red and blue lights.

Michel at work Photo by Masha Ru

Michel at work Photo by Masha Ru

In the womb the developing child experiences the heartbeat and breathing of the mother. In the rebirth ritual they make use of a drum to simulate that heartbeat. We wanted to use our own heartbeat and breathing using life data from the Silence Suit. The Sensiks cabin would provide the feedback through sound and light and influence the user. We did little experiments to try out the effects of hearing your heartbeat and breathing, using smoke, scent, heating the cabin, using airflow, etc. It was promising.

Experimenting with sound Photo by Masha Ru

Experimenting with sound Photo by Masha Ru

Day 4: building and presentation
We wrote a scenario of the ritual which started and ended outside of the cabin. Our aim was to slow heart-rate by manipulating the feedback. Just like the peaceful heart-beat of the mother will quiet the unborn child. This is also a way to connect to the heartbeat of the cosmos.
From this came the idea to limit the experience to 260 heart-beats (there are 260 days in a Maya year). By slowing your heart-rate you can make the experience last longer. Four stages of 65 beats would offer different experiences aimed at first going inward and then returning to the outside again.

The ritual starts outside Photo by Masha Ru

The ritual starts outside Photo by Masha Ru

The main challenge was to get the Sensiks and Silent Suit systems working together and to time the events to the users’ heart-rate. We didn’t even have time to test the final scenario.
One of the jury members agreed to be the guinea-pig. And even though we didn’t manage to manipulate the heart-rate feedback we could hear her heart-beat slowing down as she progressed through the experience. Later she described that she could turn inwards and let go of the world outside the cabin. This was exactly what we were aiming for.

Presenting "260 beats womb reset" Photo by Stellarc

Presenting “260 beats womb reset” Photo by Stellarc

Some conclusions
For me the “260 beats womb reset” experience was a proof of concept. That you can actually change a state of mind through relatively simple means (light, sound, smell and airflow) using physiological data as input. An interesting insight is that it is important to make the experience bigger than the box. To create a larger ritual that is not isolated from the rest of the environment. The user must be lured and triggered to actually use the cabin, it must make sense in the context of life.

It was a great inspiration to work with Branly, David, Masha, Michel, Fred (the inventor of the Sensiks) and all the other participants. Michel did a great job of getting everything to work in time for the presentation and combining the systems. We’ve been able to create a spiritual experience using technology. It will be worthwhile exploring this further. I feel a step closer to realizing my Hermitage 3.0.

Edit >> In addition to this report there is an interview with me by Olga Mink from Baltan Laboratories all about the hackathon. Included there is a very nice video impression of the whole week.

Bewaren

Bewaren

Bewaren

Introducing Silence Suit

first sensors

Meditation stool with soft sensor and heart-rate sensor

For over a year I’ve been working on a meditation wearable. It measures biometric and environmental input. Its goals is to use the measurements to improve your meditation and use the data to generate artistic visualisations. The wearable is part of a bigger project Hermitage 3.0, a high-tech living environment for 21st century hermits (like me). Now that the wearable project is taking shape I’d like to tell a little about to process of creating it.

The sensors
I started with a simple but surprisingly accurate heart-rate sensor. It works with the Arduino platform. It uses an ear-clip and sends out inter beat intervals and beats per minute at every beat. With some additional code in Processing I can calculate heart-rate variability. These are already two important measures that can tell a lot about my state while meditating. Then I added galvanic skin response to measure the sweatiness of my skin, a nice indicator of stress or excitement. I added an analogue temperature sensor that I put on my skin to measure its temperature. Low skin temperature also indicates a state of relaxation. I also made a switch sensor that is attached to my meditation stool. Sitting on it indicates the start a session, getting up marks the end.
All sensors were connected with a wire to my computer but the aim was, of course, to make it wireless so I’d be free to move. But I could already see day to day changes in my measurements.

A little help from my friends
As things were becoming more complex I posted a request for help in a Facebook group. A colleague, Michel offered to help. We first looked at different ways to connect wirelessly. Bluetooth was a problem because it has very short range. Xbee also wasn’t ideal because you need a separate connector. We also made a version where we could write to an SD card on the device. But this of course doesn’t offer live data which was crucial for my plans. We finally settled for WiFi using the Sparkfun Thing Dev ESP8266. We were going to need a lot of analogue pins which the thing dev doesn’t offer. So we used the MCP3008 chip to supply 8 analogue i/o pins.

Overview of all the sensors

Overview of all the sensors

More is more
We could then increase the amount of sensors. We’ve added an accelerometer for neck position, replaced the analogue skin temperature sensor with a nice and accurate digital one. Around that time a wearable from another project was finished. It is a vest with resistive rubber bands that measures expansion of the chest and belly region. Using the incoming analogue values I can accurately calculate breath-rate and upper and lower respiration. Then it was time to add some environmental sensors. They give more context to for example GSR and skin temp readings. We’ve added room temperature and humidity, light intensity and RGB colour and air flow.

Vest with sensors

Vest with sensors

Environmental sensors

Environmental sensors

Seeing is believing
From the start I’ve made simple plots to get a quick insight into the session data. For now they don’t have an artistic purpose but are purely practical. At this point it is still essential to see if all sensors work well together. It’s also nice to get some general insight into how the body behaves during a meditation session.
Data is also stored in a structured text file. It contains minute by minute averages as well as means for the whole session.

Session data plot with legend

Session data plot with legend

I’ve also made a Google form to track my subjective experience of each session. I rate my focus, relaxation and perceived silence on a 7 point likert scale and there is a text field for a remark about my session.

Results from Google form: very relaxed but not so focussed...

Results from Google form: very relaxed but not so focussed…

Suit
I used the vest from the other project to attach the sensors to. But last week costume designer Léanne van Deurzen has made a first sample of the wearable. It was quite a puzzle for her and her interns to figure out the wiring and positioning of every sensor. I really like the look of this first design. It’s fits with the target group: high-tech hermits and it also is very comfortable to wear.

Upper and lower part of the suit

Upper and lower part of the suit

Back with extension where soft sensors to detect sitting will be placed

Back with extension where soft sensors to detect sitting will be placed

The future
The next step will be adding sensors for measuring hand position and pressure and a sound-level sensor.
Then we will have to make the processing board a bit smaller so it can fit in the suit. We can then start integrating the wiring and replacing it by even more flexible ones.
When all the sensors are integrated I can really start looking at the data and look for interesting ways to explore and understand it.
I’m also looking for ways to fund the making of 15 suits. That way I can start experiments with groups and find ways to optimise meditation by changing the environment.

baltan session at Holst centre

Receiving an invitation from Baltan Laboratories to do a workshop with a group of researchers from the Holst Centre was a pleasant surprise for me. It was especially nice since I already knew some of the people there. I’ve received advice and help from Bernard Grundlehner from IMEC on the e-Pressed project. Now I would have the chance to work with a range of researchers, I felt very honoured.

The program would be as follows: in the morning I would show some of the tools from my ‘tools for awareness’ toolbox, in the afternoon we would explore questions around the Hermitage 3.0 concept that I’m working on. The topic for Holst Centre’s monthly High T presentation would be emotion monitoring.

The group
There were 5 researchers from the Holst Centre present: Marc, senior scientist of the integration team; Yvonne, a process development engineer; Marco, a PhD researcher on wearable sensors; Herman, program manager; Juan Diego, development engineer. Also present was Alessandra, a journalist and independent curator.

Toolbox session
For the morning session I chose to present a varied selection of works going back to 2005. I started out with Lovescape. This is a piece of software that allows you to map the things you love and print out the map. It was presented in a museum where people could sit down and work on their map. This is an older piece and didn’t expect to get so much reactions on it from the group. The way in which the work transfers the personal to public was immediately commented upon. This is of course a central issue in my work. Because I’m in complete control of what I share I don’t have a problem with that. But when I asked the group if they would like to create a Lovescape they all declined because of this issue. There were some nice ideas to work with online data to create Lovescapes. The idea of mapping dear things and people can be taken a lot further.

The next work I presented was Collecting silence, a project for which custom hardware has been developed to collect location data, stress level, decibel level and images of silent locations. A website shows the data on a map and a multimedia scrapbook page for every location. Here the discussion centred around data collection and the senses. I use only Galvanic Skin Response as a sensor to detect stress. IMEC in their Body Area Networks uses 4 inputs to detect stress/arousal. Here you see a clear discrepancy between art and science. As an artist you don’t have the means and knowledge to work with sensor technology in that way. Also the portability has to be taken into account. For me the stress level is just indicative of an inner state. But there were some issues with the GRS sensing. Moving also makes you sweat which influences the measurements. Using a protocol can eliminate some of these issues as would combining different sensors. We talked about how to determine the real influence of noise by excluding other senses. Also knowledge and context will influence the way you perceive what you hear and experience. How much does the mind interfere with awareness? It would be nice to set up an experiment to try this out. There is a dead chamber at TNO and as a side step it would be nice to do a measuring there. All were eager to learn about the correlation between silence and stress. I still have to work on an application to visualise that.

The next piece we discussed was Numuseum input and mood_movie. Numuseum input is custom desktop software that asks for my mood, energy level, stress level, state of peacefulness and a comment every two hours. mood_movie shows my mood of the last 28 days in an interactive animation. We talked about how we could make this work mobile and what to do with the large body of data that I collected over the last seven years.

Finally I demonstrated breathing_time a piece that I was still working on at that time. This is a networked wearable with software developed for a performance at the TIK festival. It sends and receives breath flow data from five people at different locations and creates a combined animation with sound in near real time. We talked about the CommonSense platform I used for this work. The researchers were pleased to see that the visuals not only create insight but also make something surprising and artistic from the data.

I think the goal of the workshop – to present a different view on how to use technology in same environments but with a completely different approach, and to be inspired in the interaction – was definitely reached. For me the most valuable insight was realising the difference in viewpoints. As an artist my works are always very personal and individually motivated. I want to discover things about myself and my life. But the scientific point of view is more about how to make this knowledge available for everybody and apply the same method to different or broader questions.

Hermitage 3.0 session
In short the Hermitage 3.0 project is an attempt at making a not only a sustainable, self aware house, but a house that helps its’ inhabitant reach non material goals like relaxation, awareness and wisdom. With the collected data artworks can be created.
As a starting point for the workshop I’d made a list of questions about how to tackle and optimise this project. We didn’t get round to answering all of them but some important issues were looked at.

My first and most pressing questions concerned feasibility. How realistic is the idea of monitoring “everything” to improve quality of life and more specifically mental and inner states? When I asked: “Is this technically possible?” the whole group replied “yes” without hesitation. All the technical knowledge is present to achieve this.

The next issue is of course how to research this. The suggestion was to look at similar undertakings, such as Philips’ HomeLab, now ExperienceLab, to do all kinds of research related to my project. Also projects like MARS-500 and Biosphere spring to mind. Inspiration can also be drawn from architects like Phillipe Rahm and artists like Chris Salter.

The costs for a state of the art hermitage were estimated at around 2 million Euros. So how to go about funding it? There are a few options:
- Find a big company interested in ‘the next experience’ like Sony, Nokia or Google. I would be heading the project and work with the researchers.
- Find a university to team up with for research
- Launch the project as an Open Source project and have people from all over the world to contribute both knowledge and data.
It could also be a combinations of those. Start off as Open Source and then get funding as a Kickstarter project. It could also be part of an EU Framework project where you can collaborate with universities, companies and for example media labs from Europe.

Another question was: What techniques for changing the environment will be most suitable for influencing states like mood, relaxation and alertness? Actually this list could be endless. But parameters like smell, light, sound, temperature (air and surface) and body monitoring could all be researched as separate subjects. In the automotive industry there has been a lot of research into local heating. We joked that I could just rent a car to research the influence of temperature. The project itself would show which parameters have the most impact.

For tracking objects and people in the hermitage there are several solutions like RFID for proximity, beacons with triangulation or video tracking.

The afternoon was very valuable to me for looking at ways of solving the uncountable issues that are involved in this ambitious project. Talking to experts has given me confidence in the relevance and feasibility of the project.

High T
The monthly high T presentation was dedicated to emotion monitoring and visualisation. I presented my installation Eden that uses the emotion recognition software FaceReader to transform real time facial expression to landscapes.

A different take on emotion monitoring was provided by Bernard Grundlehner from IMEC. He presented the VALENCE project that uses alpha brainwaves and brain activity detection in the different hemispheres to determine positive and negative emotions and relaxation. The students from St Lucas art academy in Ghent made a game with the headset output.

After the talk I wanted, of course, to try it myself but unfortunately my head is too small for the headset. Bernard tried an older version but that didn’t work very well. I had a nice opportunity to talk to Bernard again. He explained more in depth how the system worked.

Many thanks go to Baltan Laboratories and Holst Centre for this inspiring day.

Alessandra Saviotti has written a nice article about the Baltan Holst workshops in general and this workshop in particular: http://www.digicult.it/news/contamination-and-experimentation-an-art-science-workshop-series