Virtual View: results experiment 2

The analysis of the second experiment has taken a long time. At first there appeared to be no significant results on any of the variables, except on the heart-rate during the cognitive stress task. So I consulted different people with a degree and research experience and asked for help. I’ve really learned a lot from them. They all have different approaches and ways of working so I’ve picked out all the good tips and insights. My thanks go to Sarah, Malcolm and Marie. The latter is a researcher at Tilburg University, her knowledge of statistics dazzled me. She is the one who recommended a different analyses which has yielded more significant results.

Research questions

My main questions for this experiment were: Which type of stimulus results in the most stress reduction and relaxation? And which stimulus produces the highest heart-coherence? As I want the values on that variable do drive my landscape animation.
To test these questions I used the following dependant variables: BPM, heart coherence,
later I introduced heart-rate variability calculated from inter-beat interval, self reported stress and self reported relaxation. These were measured during the baseline measurement, the cognitive stress tasks and the stimulus sets.
The independent variables are: stimulus set 1 with 12 landscape photographs and synthetic nature sounds, stimulus set 2 with more abstract landscapes styled by me with the synthetic natures sounds in the background. Stimulus set 3 consisted of 12 photographs of kitchen utensils and a soundtrack of someone preparing a salad. The expected direction of the variables will be explained below.


After struggling for some time with the non-significance of the variables in the different sets I discovered that the randomisation of the sets hadn’t been ideal. There were 33 participants who viewed the sets in 6 different orders. On top of that the group size per order was different. Some groups were only 4 participants, others 10. This is something I’ll have to take into account in my next experiment.
I used a repeated measures analyses. On my first, non-significant results, I used my baseline measurement as a covariate. Marie said that wasn’t the way to go. So I used just repeated measures where the baseline is just the first measurement, no covariates. And I did post hoc analysis (Bonferroni) to see the differences between the set results.
This is an overview of the results:
Results overview
Sarah made this clear lay-out of the research results compared to the expected results.
As you can see from the blue results the subjective stress measurements are significant compared to the baseline for all three stress tasks. For the first stress task (note this task isn’t connected to set 1, it is just the first task after the baseline measurement) the difference in heart-rate is significant during the stress task. There is also significant difference in HR during the landscape set. The kitchen utensil set heart-rate is also significant. Even though the heart-coherence is has the right direction non of the changes are significant. There are also no significant differences between the subjective relaxation questionnaires.

On Sarah’s recommendation I also looked at correlation between all the variables. That is very interesting as it reveals relationships between the variables. As the subjective relaxation questionnaire didn’t show any significant results I was curious to see how it correlates with the stress questionnaire. There should be a significant negative correlation between the two. And there is, it is especially strong for the baseline stress measurement and all relaxation measurements. On the other hand there was no correlation between subjective relaxation and heart-rate, a lowering of this value may be considered an indication of relaxation. All in all the relaxation questionnaire doesn’t give convincing results. There was a very strong correlation between heart-rate and heart-rate variability. In fact too strong, as Sarah pointed out they measure the same thing so it has no use including this variable in the results.

First set

As the stress stimulus was strongest the first time (view below) Marie advised me to do an analyses on the first set that was shown after the first stress task, independent of what kind of stimulus set it was. This was the distribution: set 1:    shown 11 times; set 2: 14; set 3: 8. The results from this analyses completely matched the other results. Heart-rate and heart-rate variability are significant (this is of course an average of all three sets shown), heart coherence and self reported relaxation were not. There was no interaction effect between the set shown and either the heart-rate or heart-rate variability which suggest that the order has no effects on the results.


Stimulus overview

I made some manual graphs to see the effects of the stimuli on heart-rate and heart coherence side by side. There is no significant difference between the pictures in the three sets. For me it is still nice to see the difference between the pictures. The graph is done manually in Photoshop.

Stimuli used


After getting advice from Malcolm  about the results he suggested I calculate heart-rate variability from the inter-beat interval values that I’d logged. Heart-rate variability is known to correlate well with stress (negative) and relaxation (positive). So that’s valuable information to add to the results.
I wrote a script in Processing to calculate and visualise the HRV for the whole experiment divided in a 5 second windows. The white line is the baseline measurement, the red lines are the stress inductions and the green lines are the audiovisual stimuli. You can tell from the image that the stress induction has some effect.

HRV results
Looking at my correlation table however there is only a significant negative correlation between the baseline subjective stress measurement and the HRV. Neither the other stress measurements nor the subjective relaxation measurements show any correlation.
It is hard to tell from the image but the photo realistic landscape set has a significant difference from the baseline measurement. The third set is almost significant (p = .054).


The first conclusion should be that the differences between the stimuli in the sets are small. There are significant* differences in average heart-rate between the sets (68,26 (baseline); 66.46* (set 1); 66.75 (set 2); 66.32* (set 3)). But the differences are really small. There is reduction in all the sets. Set 3, the kitchen utensils has the lowest average.  The set isn’t very stimulating which might explain the low heart-rate. This conclusion is also backed by the fact that the results from using only the first set shown are comparable to when working with the individual sets.
Heart coherence, which I want to use for driving the animation and to trigger the interaction with the installation showed that the styled landscapes with sounds had the highest heart coherence average but the results were not significant.  It does not seem a good measure for pure relaxation. Heart coherence is a difficult term but this description gives a good indication of the different aspects of this state: In summary, psychophysiological coherence is a distinctive mode of function driven by sustained, modulated positive emotions. At the psychological level, the term “coherence” is used to denote the high degree of order, harmony, and stability in mental and emotional processes that is experienced during this mode. (From The Coherent Heart p. 12, Mccraty, Rollin Ph et all, Institute of HeartMath). On page 17 of that document it states that: “In healthy individuals as heart rate increases, HRV decreases, and vice versa.” As HRV and coherence are closely linked the same is true for heart coherence. Even though heart coherence is much broader than relaxation is also encompasses activation of the parasympathic nervous system which is also a marker for relaxation. Important in heart coherence is the inclusion of positive emotions. This is what I try to evoke by using landscapes based on generally preferred landscapes.
The Virtual View installation should provide a relaxing distraction for people in care environments. Cognitive states that relate to this goal are soft fascination and a sense of being away as introduced in the Attention restoration theory (ART) by Kaplan and Kaplan. I’m guessing now that heart coherence might correlate with those cognitive states. This is something I will explore in the next experiment.

The stress task was perceived as stressful judging from the subjective reports. These findings are partly backed by the physiological data. Only the heart-rate of the first stress task differs significantly from the baseline. Our goal by introducing a stress task was to create bigger differences in heart-rate. For that to be successful the stress task should really produce stress. Although people reported feeling stressed we can’t measure it three times in a row. So for the next experiment I’ll work with 3 groups who will all get only one stress stimulus and one landscape stimulus.

All in all this experiment doesn’t prove that my styled landscapes with synthetic nature sounds create the most relaxation and heart-coherence but the results neither prove that they don’t. So for the next experiment I’ll continue with the styled landscapes and introduce animation.

Virtual View: results experiment one

In this post I want to give an overview of the results of the first and I will spare you the heavy statistic speak. So don’t expect a scientific article. The data is there and I may write a proper article one day but it isn’t appropriate for this blog.

Together with Hein from the Open University I looked at the data from the first experiment. This is an exploratory experiment so we’re looking for trends and directions to take with us to the next step.
The students did a splendid job organizing the dataset. For each participant there was basic demographic data (gender and age) means and combined means for the perceived relaxation questions, the separate images and images combined in sets. For each set there are means for: beats per minute (BPM), the inter beat interval (IBI) and heart-coherence.
To make sure our self constructed questionnaire was valid I did a scale reliability test. All the sets had good reliability for all 5 questionnaires. This just means that there is an internal consistency between the questions. The questionnaire it self isn’t validated for measuring relaxation. We just asked the three questions.

We did 4 analyses on the four variables: perceived relaxation (measured with the questionnaires), BPM, IBI and heart-coherence.
The stimuli sets were {sound}:
1. Preferred landscape with water element {running water @ 48 Db}
2. Preferred landscape in autumn {repetitive bird calls @ 47 Db}
3. Preferred landscape as abstract painting {melodious birdsong @ 56 Db}
4. Neutral hospital interiors {neutral hospital sounds @ 48 Db}
5. Landscape with deflecting views {running water and melodious birdsong @ 43 Db}

Self-reported relaxation

self-reported relaxation

self-reported relaxation, sets 1 to 5. Green is females, blue is males

The three questions we asked after the baseline measurement and after every stimulus set were: I feel at ease, I feel relaxed, I feel joyful and happy. Reported on a scale of 1 to 10. The three questions were merged into a relaxation scale. The hypotheses was that the overall relaxation scale would be lower for the hospital interior set (d) than for all of the landscape sets.
There was a significant effect for relaxation. As you can see from the graph set number four (hospital interiors) shows a distinct decrease of the sense of relaxation. Although the abstract paintings also score lower, this trend is mainly caused by the dip in relaxation scores on the hospital set, this confirms our hypotheses.

There was also something going on with the interaction between age and relaxation. To gain more insight into what’s happening with the age effect I looked at the data and noticed there are two clear groups: 25 years old and younger and above 39 years. The groups are about the same size (young 15, older 18). There were no participants of the age between 25 and 39 years. To test for the significance of the relaxation for the two groups I ran a test that showed that for the young participants the relaxation effect isn’t significant but for the older participants it is.

relaxation divided by age group

relaxation divided by age group. Blue is older.

For the heart-rate we used two measures based on the same data: beats per minute (BPM) and inter beat interval (IBI). So it doesn’t make a difference which data analyses I discus here. The hypotheses was that the BPM would be higher for the hospital interior set (d) than for all of the landscape sets.
There we no significant differences between the sets. Our hypotheses has to be rejected.

heart-rate for men and women

heart-rate for men (blue) and women (green)

But there is again something going on with age, this time in relation to heart-rate. Looking at the graph below it is clear that the heart-rate in reaction to the landscapes and sounds is at odds for set two and set four. The older and younger people react quite differently.

Beats per minute for two age groups

Beats per minute for two age groups. Younger is blue.

Heart coherence
The hypotheses for heart coherence was that the coherence level would be lower for the hospital interior set (4) than for all of the landscape sets.

Heart-coherence for men and women

Heart-coherence for men (blue) and women

There is a significant trend for the age coherence interaction. Looking at the graph we can see that the coherence for the women is almost the same over the 5 sets but higher then the baseline coherence measurement. The men show a much more varied response and on average a lot lower then the baseline measurement. It is interesting to note that the abstract painting set, number 3 has a very high score for the men.
Looking a bit deeper into this trend there is again a relation to age. For the younger participants there was no significant difference between the sexes where heart-coherence is concerned. The graph of the older participants shows a significant difference between men and women. The older men cause the interaction-effect between gender and heart-coherence.

Difference in heart-coherence between older men and women

Difference in heart-coherence between older men (blue) and women

So although the average heart-coherence for the hospital interior set (4) is at the lower end for both men and women the effect isn’t convincing in view of the other scores of the other sets. The results don’t support the hypotheses.

For an exploratory first experiment the analysis has yielded some interesting results. The main hypotheses that the self-reported relaxation, heart-coherence, BPM would be lower  for the hospital interior set (4) than for all of the landscape sets is partly supported.
The self-reported relaxation and the heart-coherence showed significant results.

The lack of significance for heart-rate may be due to the small group or may suggest that the differences between the sets wasn’t big enough. To influence this I want to reduce the amount of sets in the next experiment and introduce a stress stimulus to create more contrast between the states of the participants.
Judging from the analyses it is clear to me that for next experiment the age should be more homogeneous.
For me the most surprising and promising was the high heart-coherence of the men on the abstract paintings. People were skeptical about using these abstract stimuli as there is not much support in literature that non-realistic images have any effect on viewers. Of course this will require more research but it is an interesting and unexpected result.


Last week I joined the Seeker project, a co-creation project by Angelo Vermeulen exploring life in space(ships). It’s been really inspiring so far as living in space combines knowledge from the fields of architecture, sustainability, farming, water and power supply and Quantified Self. The latter being my addition, of course :)

Together with architecture master students from the TU/e I’m looking into the interior of the ship which will be based on two caravans. As life in a spaceship is crowded and noisy my aim is to make a quick and dirty prototype that will:

  • detect the noise level
  • detect the users’ heart-rate
  • promote relaxation by pulsating light and sounds from nature

Noise level, heart-rate and soundtrack will (hopefully) be send to the base-station so people have a life indication of the status of the ship and the people living in it.

This is the sketch:

Today I’ll have a talk with the technicians for MAD to see what is possible. I’m thinking of using the following sensors:


Noise level:

Playing sound:

The cocoon itself will be the good old paper lamp:

science hack day

Last weekend I took part in the first Dutch Science Hack Day in Eindhoven. I had posted my idea on the forum and was hoping for a nice group of experts to work with. The idea was to create a mood enhancer. When you’re sad it could help you be become happy again. When your happy you could help others who are sad to improve their mood or support them. It will consist of a) mood detection, b) mood changing, c) mood sharing.

On the forum one participant, Siddhesh (PhD student TU/e), had already expressed his interest. After I’d introduced my idea I was joined by Leonid and Huang-Ming both students at industrial design at the TU/e and Ketan also a PhD student at the TU/e. We were later joined by Iwan an interior architect. So we had a nice mixed group from different countries.

I was pleasantly surprised at how swiftly we decided on the use case and technologies to be used. Everybody was very eager to start to work and do so in their field of expertise. We decide to use two hardware sensors (heart-rate and skin conductance) to provide the level of arousal and one on-line software sensor,, that uses portraits to classify moods. The heart-rate sensor was already finished because we could reuse it from another project by Leonid and Huang-Ming but there was still a lot of work to be done.

For output we wanted to do something with light and sound as they are the least obtrusive when you’re working. We wanted to work with a physical object to display the mood and also enhance it and to use Twitter to share moods. We had difficulty to decide if the visualisation should just be personal feedback or should also display a friends’ status. As time was limited we decided on just feedback. The application moved from enhancement to awareness of moods which was enough for just one weekend.

I took on the task of implementing the valance through the API. It would all have to be done in 24 hours so that was pretty challenging. Registering at was easy. The API was pretty straight forward and only later I discovered the it could not just detect smiling or not smiling but a whole set of moods: happy/sad/angry/surprised/neutral value and confidence, based on the expression of the person in the the photo. There’s was also a lot of other info to be gotten from the image using the faces.detect method, the accuracy of the results was surprising, even under less favourable light circumstances. The main hurdle was uploading an image for and keeping it in sync with the rest of the application. In the end we used the local Dropbox folder to store the web cam captures and letting Dropbox sync with the web version, the URL and file name are used in the request.

The others worked on building the Galvanic Skin Response sensor, the lamp object and the integration of the heart-rate sensor and software for the new purpose.

We used Processing as the main language to read the values from the sensors, connect to the web and drive the output. The sensors write their current values to a file separate and one script reads all the sensor input to generate a visual output, change the colour and position of the lamp and change the sound.

The main application shows a changing, interactive landscape of lines and circles. The   amount of arousal the corresponding valence determine:

  • The position and colour of the circle. When you click on a circle the web cam image and heart-rate value is shown, allowing you to trace back how you felt during the day.
  • The position of and colour of the light object
  • The sound being played

Iwan made a nice presentation and we were finished just in time. The presentation went well and the jury picked our design as the best in Overall happy living category! That was just the icing on the cake of great and inspiring weekend.

Science Hack Day Eindhoven 2012 winners compilation from M.A.D. ART on Vimeo.

Being one of the winners we also presented at the Internet of Things event at the High Tech campus in Eindhoven.

MADlab kindly supplied me with an artist residency to cover expenses.


I’m being challenged at the moment. Yesterday (and a little earlier) I noticed that my heart-rate belt was having problems. So I reckoned it was a good idea to replace the battery. After I’d replaced it the watch couldn’t find the belt anymore. So I couldn’t do any measuring of the heart-rate. So today, as soon as the shop opened (which unluckily was not until 13.00 hours) I went to a Runnersworld shop to see if they could help me. The guy was very patient and tried out all kinds of things in a methodical way. But in the end we had to concluded that the belt was broken. I had to buy a new one. It’s my third belt in seven months! That’s really bad considering the watch and belt cost me 400+ euros… But of course I feared most for my project so I spend another 80 euros on a new belt but at least now the project is running again.

Which, I must say, is becoming a bit of a burden. I moved to another house. It’s so noisy! I woke up from the neighbours walking around at 5 am. Everything they do makes a hell of a noise. It’s almost as if they’re in my house. In the new house there were no plates, no knives and no matches. It’s difficult to cook when there’s no fire… The evening shop was open so I could buy matches and I got the knives and plates from the other houses. But it’s a draggggggggggg… It seems that everything that can go wrong goes wrong.

And I just miss Internet so much. I’m here at a friends studio to post to my blog (by the way the last time I worked on that blog my IP address was blocked by the server so I couldn’t work on any of my sites) and now there is a ‘limited connection’ == no connection. And that’s another of the many things that go wrong with this project. I hope this isn’t a foreboding of the new year because then this year will be hell.

I’ve also discovered I’m not too keen on Amsterdam. Or maybe I just don’t like big cities. I wanted to go for a walk last night but I just couldn’t go anywhere. It’s all bricks and concrete. Or it to scary to walk alone at night. I did go out because for a second I thought I was going crazy. I bought myself a beer on way back.
The new house I’m staying in is designed like a little museum. It has all kind of stuff from the ‘Northeners’. It’s cute and it’s more luxurious then the nature theme house I was earlier but it’s very present. It takes up a lot of space and attention and that’s hard when there’s so much work to do.

Well, that’s about it, hopefully next post will be more cheerful.

Power cut

As I was collecting the data this morning (it’s Christmas) there was a power cut. It was in a large part of the estate and may be a large part of town. I rang the alarm number and they said the cut would last till 1 pm… Luckily the batteries of my laptop were charged so I could upload the pictures and GPS log. Cool stuff, batteries :)

Last night was the first night to try out the electrode gel. It worked like a charm! The belt felt very solid around my chest so I didn’t use the medical tape. That was very nice for my skin. The gel itself is neutral and doesn’t give me any skin problems. So for the first time I recorded a whole night.


Right, I’ve just discovered how the gaps in the picture series appear. When there’s not enough light to take a picture the camera makes a long low beep. I thought that was just a warning but now I’ve discovered that it just fails to take a picture. So for yesterday I managed to complete the series without gaps. The only problem there was yesterday was the Suunto watch. It didn’t find the heart-rate belt because the batteries were low. I tried to replace them but couldn’t. I had to go back to the shop to have it fixed. So I only could start measuring at 10 o’clock. That was a bit of a shame.

Kinky looking electrode gel

Kinky looking electrode gel

From the user group I got a reply for my questions on how to continue measuring the heart-rate during the night. I have to use electrode gel. I ordered it Monday and it arrived today. It looks funny. Unfortunately it’s only for two nights in Breda. I hope it works. Tonight I only measured till 0.30… It will be for Amsterdam as well when it works. A problem is that I will have different durations in measurement time. I hadn’t thought about that. I will have to find a programmatic solution for that. As I want to compare the rates at the same time in different places.


Yesterday I continued working on the heart-beat graph. I’ve managed to animate it. The speed of the animations varies with your cursor position. The speed ranges from 1000 milliseconds to 1 millisecond. When the animation goes faster the string of dots seems to be alive, like some snake like creature. Fascinating to watch.

The dots are 15 pixels apart. For my test file with 20 hours of data the heart-beat graph has a width of 107374 pixels! Flash is pretty powerful to caculate this image in less than a second, amazing.

As for the data collection. I’ve slept a night with the belt fastened by broad strips of medical tape. It worked till 7 am, which is an improvement but still not perfect. I’ve posted the question at a user group and hope to get some tips for improving this.

ps. My lowest heart-rate was 34 bpm this night! The lowest I ever measured.