working on numuseum

After a long time I’ve taken up the numuseum website. It’s been nagging me for ages that it’s so outdated and not working properly any more. I’m keeping it simple but will be implementing some new things.

designI want to create a now part (“nu” means now in Dutch) and a museum part. Now always shows the most recent data. I’ll start of with a picture of the sky with time and location data. I will overlay that with personal data like mood and heart rate. The museum part will show the now part history in some interactive way.

I’ve found a cute, free font Jaapokki Regular that I’ll be using for the website.

The menu at the bottom gives access to the archive of net-art pieces, an about and contact page.

I’ve already started coding the sky part. I use a very neat FTP app (AndFTP) to send the sky pictures to the server. A PHP script sorts the pictures (most recent first) and grabs the date-time and locations data (from EXIF headers).

home

sleepGalaxy: final design

Displaying different activities with the right duration and start time

Displaying different activities with the right duration and start time

There were still a couple of variables to visualise once the basics design was ready. I had to work on integrating my pre-sleep activity. In the end I used three activity types: sport, social and screen (computer and television). Of the first two I’d logged duration by recording start and finish time. For screen time I just logged total duration because it was often scattered.
I was looking for a way to display all aspects (type, start, finish and duration) in a way that fitted with the nice, round shapes I’d been using so far. Then I realised the pre-sleep activities were recorded from 18:00h onwards. So the main circle could act as a dial. I could split up the space from 18 till 23:59 using the activity duration. I calculated the starting position of each activity as a degree on the dial and added the minutes the activity lasted. Using the arc shape with a substantial line thickness resulted in nice, bold strokes around my “night” circles. Each activity type has its own colour.

The final night design (rating still in green)

The final night design (rating still in green)

I was happy with the result but then the recovery line just looked plain ugly. I decided to use the same arc shape on the other side of the circle. The more recovery the thicker the stroke in green. The less recovery the thicker the line in red.

Finally there was the subjective rating of the sleep. I think it is important to incorporate how the night felt for me. Emfit uses a star system from 1 to 5 stars. So I played around with stars, ellipses and other shapes but finally settled on simple golden dots. A five star night would have the fifth and biggest dot in the middle of the deep sleep circle, this seemed fitting.

UFO like rating design

UFO like rating design

When the individual nights were finished it was time for the overall poster design. I somehow had got it into my head that this would be easy. But it was quite hard the capture the look and feel I was aiming for. I wanted the poster to be simple so that the individual nights would stand out and make a nice “galaxy”. On the other had I did want a legend and some explanation of what was on display.

Sketch of the poster design

Sketch of the poster design

My first idea was to go for a size of 70 x 100 cm, the nights would have a size of around 10 cm. This was too small for all the details to be visible. My final poster will be 91 x 150 cm. The nights are big enough and they all have enough space on the sheet while it is still possible to compare them. I found the nice, slim font Matchbook for the title, the legend and text. I’ll be sending the pdf to the printer next week.

sleepGalaxy: design & calories

Design

Design

I’ve been working on the overall design step by step, alternating between coding and looking. I want to incorporate my calorie intake after 6 PM. I’m not recording the times I ate and I suspect they influence my whole sleep. So the most logical position is to circle all around the “sleep circles”. There is a lot of difference in daily intake after 6 PM, ranging from zero to 900 calories so far. I wanted to plot every calorie so they would have to change sizes depending on the amount. I also wanted to spread the calories evenly around the entire circle. How to go about that? Fortunately, I’ve found this great tutorial. The code is deprecated and the feed doesn’t seem to work any more but I managed to recycle the code concerning the plotting of the elements in a circle.

calorieViz1

Plotting numbers instead of dots

The code uses translate and rotation, which (for me) are very hard to grasp concepts. So instead of using the dots in the design I used numbers to get insight into how the elements are placed on the screen.
By keeping the size of the calorie circle constant, you can already see relations between the sleep duration, the amount of calories eaten and recovery.

cals2

Evening with a lot of calories

cals1

Evening with less calories

In the design you can also see an eclipse. These are the stress and happiness values for the whole day. I poll them by picking a number between 1 and 7 in the form at the end of the day. The mood is the bright circle. The stress circle covers the brightness depending on the amount of happiness felt during the day. By vertically changing the position, I can create a crescent. This can turn into a smile or a frown. The opacity of the black circle indicates the amount of stress. I’m coding this at the moment.

sleepGalaxy: recovery

As I explained in my previous post I find the recovery measurement very useful. It seems a good representation of how rested I feel. It is calculated using RMSSD. The Emfit knowledge base explains it like this: “… For efficient recovery from training and stress, it is essential that parasympathetic nervous system is active, and our body gets sufficient rest and replenishment. With HRV RMSSD value one can monitor what his/her general baseline value is and see how heavy exercise, stress, etc. factors influence it, and see when the value gets back to baseline, indicating for example capability to take another bout of heavy exercise. RMSSD can be measured in different length time windows and in different positions, e.g. supine, sitting or standing. In our system, RMSSD is naturally measured at night in a 3-minute window during deep sleep, when both heart and respiration rates are even and slow, and number of movement artifacts is minimized…” Here is an example of how recovery is visualised in the Emfit dashboard:

Emfit dashboard

Emfit dashboard

I looked for a way to integrate this measure in a way fitting with my “planet metaphor”. I’ve chosen a kind of pivot idea. It vaguely reminds of the rings around planets.

Using the mouse pointer to enter different values of recovery

Using the mouse pointer to enter different values of recovery

I thought it would be easy to just draw a line straight through the middle of the circles. I wanted it to tilt depending on the height of the score. It was harder then expected. I ended up using two mirroring lines and vectors. Starting point was the excellent book by Daniel Shiffman, The nature of code.

Integrating with circle visualisations.

Integrating with circle visualisations.

Once I got the basics working, I went on to refine the way the line should look projected over the circles. Going up from the lower left corner indicates positive recovery, visualised by the green coloured line. The more opaque the better the recovery. Of course, negative recovery goes the other way around.

Slight recovery

Slight recovery

The is a difference in the starting points from which the recovery is calculated. Sometimes my evening HRV is very high. This results in a meagre recovery or even a negative recovery. I might think of an elegant way to incorporate this in the visual. May be I have to work with an average value. For the moment I’m still trying to avoid numbers.

Almost maximum recovery

Almost maximum recovery

Negative recovery

Negative recovery

sleepGalaxy: kick off

Finally, I’ve started to work on a piece that’s been on my mind for almost two years. Ever since I met the nice people from Emfit at the Quantified Self conference. They kindly gave me their sensor in return for an artwork I would make with it.

Emfit QS

Emfit QS sleep sensor

You put the sensor in your bed, go to sleep and it wirelessly sends all kinds of physiological data to their servers: movement, heart rate, breath rate. All this data together they use to calculate the different sleep stages. From the heart rate they’ve recently started calculating HRV and recovery. This latter value to me is best indicator of my sleep quality and how energetic I feel.
Emfit offers a nice interface to explore the data and view trends.
emfitInterface

In sleepGalaxy I want to explore the relationship between sleep quality and the following variables: exercise, social and work meetings, calorie and alcohol intake, screen time and overall happiness and stress during the day. I’m under the impression that these have the most impact on my sleep, that is, the sleep phases, the ability to stay asleep and recovery.

Google form

Google form

To track the variables I’ve created a Google form that I fill in every night before I go to sleep. I’ve set an alarm on my iPad so I don’t forget.

Excel sheet with some of the Emfit data

Excel sheet with some of the Emfit data

firstNight

First circle visualisation

From all the Emfit data I’ll be using a subset. My first sketches focus on the sleep phases. I’ve spend a couple of hours programming first the basic idea: transforming the sleep phases into concentric circles. Going from awake to light sleep, REM sleep and deep sleep in the centre.

The next step was to make sure the different phases are displayed correctly, representing the amount of time spend in each phase and total time in bed. I’m programming in Processing and I’ve created an class called Night. After reading in the Emfit excel data as a csv file I loop through the rows and create a night object representing every night.
Displaying the circles went fine but the proportions between the circles just didn’t look right. I realised I had a conflict working with minutes in a decimal context. I wrote a little function that converts the minutes of the hours into decimal values and then adds them to the whole hours:
float min2dig(String time){
String[] tmp = split(time,'.');
float t = float(tmp[0])+(float(tmp[1])/60);
return t;
}

Now the basis of the visualisation is ready. The image below displays sleep phases of the four nights in the excel data from above. I look forward to adding more data. To be continued…
firstNights

Virtual View: developing animation

The past month I’ve been working on my landscape animation. By chance I discovered a great book by Daniel Shiffman called The nature of code. The book explains how to convert natural forces into code. I’m working through the book picking the forces and algorithms that suit my needs. So far the noise function in Processing has proven very useful. It allows for creating more natural variation (as opposed to the random function.) I use it in creating the landscape horizons and some forms of animation.

PerlinNoiseHills

Test for creating hills with Perlin noise

In a previous post I described how I calculated the colours used in a woodblock print from Hokusai. Since then I have discovered the colorlib library. A super fast library for creating palettes and gradients from existing pictures. You can sort colours and manipulate the palette using various methods. This means I can change my colours dynamically depending on user input.

Colorlib palette from Hokusai picture. Sorted on the colour green.

Colorlib palette from Hokusai picture. Sorted on the colour green.

Apart from working through the book and creating basic animations I’m working on the look and feel of the landscape.

As I explained earlier this is based on the work of Hokusai. To my delight I discovered that a colleague is one of the few Dutch experts on Japanese woodblock printing, having received training in Japan. On top of that Jacomijn den Engelsen is also an artist whom I’ve admired for years. I met with her yesterday in her studio to learn more about this fascinating technique.

Jacomijn

Jacomijn demonstrating the Japanese woodblock printing technique.

The characteristic look of the pieces comes from the use of water based paint on wet rice paper. For every colour a separate woodblock is used. The typical black outlines are also printed from a separate block.

Screen print from animation. Colorlib gradient used for sky and water.

Screen print from animation. Colorlib gradient used for sky and water.

The prints have a very flat, 2D feel. That is what I like, it is a kind of primitive picture of a landscape. The view people will be seeing won’t be a 3D simulation of nature but an artistic representation, a work of art with healing properties.

I’m not a painter or draughtsman so I was very happy with the tips Jacomijn gave me on how to make the landscape more convincing while still keeping the ‘Japanese flatness’.

Virtual View: designing the first experiment

I had an idea what I wanted to research in my first experiment after reading the different articles. Looking at the end users, frequent visitors to hospitals and the chronically ill, I want the final piece to be first and foremost a pleasant and relaxing experience. It would be nice if there was an actual physical change that can be measured. The piece should have a stress reducing and restorative effect too. This can be both a subjective experience and a quantified measurement in form of heart-rate and heart coherence. And there are of course the landscapes and the sounds that should induce these states.

So how do you convert these goals into an experiment design? You follow a course and you ask people who have a lot more experience with designing psychological experiments!

I started out with way too complex idea. Combining stress induction and testing stimuli effects in one experiment. I’ve had great input from my professor Hein at the Open University, Sarah (PhD in psychology), Ilia (developer of stimulus creation software) and Malcolm (information scientist and psychologist) from Heartlive. Discussing my idea’s with them helped me a lot.

Together with the students I looked at the type of landscapes and sounds that would be most valuable to explore for the Virtual View installation. We’ve decided to test 5 sets with 6 landscape images based on, among other things, the most preferred landscapes as defined by Ulrich. We also explore the mystery aspect of landscapes as outlined in the attention restoration theory by Kaplan and Kaplan. Each set of images has a sound to go with it. We use one contrast set of neutral hospital interiors accompanied with hospital sounds. Another thing we want to explore is non photo realistic landscapes. As the final piece will consists of computer generated graphics with a certain degree of abstraction we want to compare the response to abstract landscape paintings to the photo realistic material.

From the little research that has been done on the effects of (nature) sounds we’ve come to different combinations of running water and birdsong. These are the sets and sounds {in curly braces}:

a. Preferred landscape with water element {running water}
b. Preferred landscape in autumn {repetitive bird calls}
c. Neutral hospital interiors {neutral hospital sounds}
d. Landscape with deflecting views {running water and melodious birdsong}
e. Preferred landscape as abstract painting {melodious birdsong}

While experiencing the stimuli the participants’ heart beat will be measured with the Heartlive sensor. This will give data in the form of beats per minute, inter beat interval and heart coherence. A questionnaire on the perceived relaxation state will give insight into how the different stimuli sets are experienced by the participants and how they effect their sense of relaxation.

We expect combination d) the have the most positive effect compared to the other sets: higher IBI values, lower BPM values and higher coherence and the most self reported relaxation. The neutral hospital interior we expect to score the lowest means on those variables.

The sets and the images in the sets are randomised for each participant. The sounds are attached to one set. The participants will see all the sets (repeated measures). In the end we’ll be able to compare the different means of all the sets.

In the next blog I’ll explain more about building the experiment in EventIDE, the stimulus creation software I mentioned above.

working with data

I’ve been experimenting with the design and data visualisation using the personal data values: mood, stress, energy level and inner peace. Depending on the data value the lines, shapes and tone of each visualisation varies. This will create a different structure for every timeslot in every day.

Inner peace will be a big organic and mysterious shape.

Energy and stress level will be pictured using horizontal an vertical lines respectively. The upper image is average energy and stress level. The lower is low energy and high stress.

Mood will be an arrow head/smiley pointing upwards or downwards. It is the most upper layer, clearly visible on the peace shape.

This is an experiment with combined output for 4 timeslots using real data. It gets a bit busy and the lines in different directions are making me feel a little giddy. So work to be done but it’s a promising start.

hardware progress

I’ve been working with plastic and paper to create different device prototypes. I was very happy with look of the plastic prototype. I want the design to be light:

The part I like most is where I stitched the plastic together with nylon thread:

But when I did 30 minutes of meditation wearing the device I nearly suffered from hyperventilation. It was so stuffy. Something that was less obvious when I worked with the paper prototypes. Also the plastic became steamed very quickly which points to the greenhouse effect it houses.

So I’ve switched back to paper. That has the advantage that it works a lot quicker and that you can glue the parts. Also the data is a lot more stable over time. The only thing that still needs looking into is how to make the edge of the cone comfortable. I’ve used the soft part of Velcro until now but it comes lose. I’m now considering felt. It is a beautiful combination with paper. I must decide quickly now because I still need to make 5 items.

The electronics are done, I need to calibrate a few more wind sensors like this:

But I’m getting the hang of it. It needs a soft subsurface (not shown). Asserting pressure on the sensor board influences the values… We keep learning :)

work in progress

I’m working on the breath detection software and the device design. I’ve concentrated on detecting the breathing in and out and the event between breaths. Especially the distinction between in and out is hard as there is wind flow in both cases. So I use the difference in temperature to detect the in and out. I set two calibration points (by pressing a different key), one between breaths and one after completely inhaling. In both cases I take the wind value and the temperature value. With these two extremes known I can now detect the breath status in a rather robust way. View the screen dump from Processing:

I rather like the space look of the breathing cone. More work should be done on it of course: