3D Scanning and Motion Capture.

Fun with the Xbox 360 Kinect

So we've finished with all the group projects this term, yah! I really enjoyed all the projects we did this term, my favourite was the brain waves feed project, were I put to my music writing and mixing to the test, I found the project to be really fun.

So today we had a lecture on 3-D scanning and motion capture using the Kinect, we were taught by a guy called Musaab Garghouti, he was a pretty cool guy who taught us quite a bit.

The Xbox 360 Kinect us a pretty neat device, it uses 3 lenses which emits a infrared to capture anything in it's viewing vicinity, it also uses a laser grid to calculate depth. 

Specifications- 

• Frame rate of 30 Hz.
• RGB video stream uses 8-bit VGA resolution (640 × 480 pixels) 
• Angular field of view of 57° horizontally and 43° vertically
• monochrome depth sensing video stream is in VGA resolution (640 × 480 pixels) with 11-bit depth, which provides 2,048 levels of sensitivity

Projects with the Kinect

3-D Mapping

We had two projects we had to work with, the first we played with was motion capture. One of us had to stand in front the Kinect in a T shape so the device could capture all of the body, and then we could move about, do a little dance. The Kinect then captured all of the movement, (but the device was pretty limiting though as too fast movement or quick bursts, the Kinect wouldn't pick it up. 

After it had recorded someone's movement, we then put their body on a skeleton mesh then rendered it in 3DS Max. We then put the skeleton on a pre-rendered  skin and had the character act out the body movement of the person who the Kinect recorded. 

3-D Scanning

The second project we looked at was 3-D scanning with the Kinect, this was a pretty fun. The device we used to visualize the scan was  full of bugs and kept crashing so we couldn't do much else with it. There isn't much of an explanation to this as it broke too much and took so long to reboot, that we ran out of time. But the main jist of this device and software is that it will determine the angles of the objects or shapes in front of it using a laser grid. Then will render the image in 3-D and output it onto the tablet in the picture below. We got a few images with it before it gave up completely gave up.

Taken by Natalie Williams

Infrared Technology

Infrared technology has much potential and is a really useful technology. It is mainly used for its motion detection capabilities such as security or game play, two different areas but those two are the main areas infrared technology is used in. 

• Affordable
• Reliable
• Works easily with most other technologies
• No warm up time
• The future potential is massive
• Easy to use and adapt

EEG Final Product

We wanted to use the EEG brain wave detector but iDat only had one and another group was using it so we just created our own data, (Simon said we could) and Ian and Stuart used that data and gave each of the brain wave state a value with an added variable. 

The variables we used were 

• Size- Colour
• Position- Number of trees
• Brain state- Soundtrack

This is what is looked like once the programmed was run.

Screenshot taken by Stuart Westgate

My job in the group was to come up with 4 soundtracks for each of the brain waves states. 

I took a few royalty free songs, samples and beats I found, I recorded a few hits and beats of some objects in my room, because I wanted a very natural sound when it came to the music. Since we are using brain waves and having a very natural visual, I thought it would work quite nicely. I then played around with them, mixed them in Audacity and Pro Tools to make the 4 tracks and sent them to Stuart. 
I quite enjoyed this project more then the others as I actually put my music taste and ability to the test and it came out rather well. The others were happy with my end products as well, a rather hassle free group and project. 

• The first melody was to represent the Delta brain was as it had no beat, and could be used to portray deep sleep or no movement.
• The second melody had a little beat to it and was to represent Theta, it had not a lot of bass to it so it portrayed a relaxed feeling.
• The third melody had bass in it and portray Alpha brain waves so as to be in a mental and physical relaxation.
• And the fourth melody was more upbeat than the rest and portrayed a state of awareness and high brain activity. The Beta brain waves. 

Final Product

Video taken by Ian Marsh

The idea behind the representation of the variables was, even when you are resting or sitting down the neurons and veins are all still active.

As soon as the user changes brain state the soundtracks that represents calm, suddenly change to a higher up beat track, and that variable changes the soundtrack and onscreen pattern change, and repeat until the user has reached a different mind state.

The sudden pattern change also represents the creation and death of neuron connections in the brain. Trillions of new connections and old connections die throughout someones life. The sudden changing and multiple soundtracks represent the change of thoughts, and the sheer capacity of information that can travel through the human brain all at once. 

Research on brains

Brainwaves

The human brain is made up of billions and billions of cells called neurons, which used electricity to communicate with each other (that's what the Matrix is all about, well probably). The combination of millions of neurons sending signals to each other at once produces an enormous amount of electricity  just for an automatic ability like breathing can cause the brain to fire a large amount of electricity around, so what do you think what would happen in your brain if you were exercising.  
We can measure the electrical activity in your brain by using sensitive medical equipment (medical experts call it an EEG), measuring electrical levels over the scalp. 




The combination of your brains electrical activity is called a 'Brainwave pattern', the pattern emits tiny electrochemical impulses of different frequencies. The brain wave patterns are split up into 4 different groups which are known as - 

•  Beta- emitted when we are alert , or we feel tense, afraid or agitated. The frequency ranges from 13-60 pulses per second. 
•  Alpha- When we are in a state of a physical and or mental relaxation, although we are aware of what is going on around is. The frequency is around 7-13 pulses per second.
•  Theta- This is a state of reduced consciousness, say you are drowsy or thinking of sleep. The frequency is around 4-7 pulses per second.
•  Delta- This occurs when you are unconscious  deep sleep or even in a coma. The frequency is 0.1-4 pulses per second. 

"A typical neurone makes a bout ten thousand connections to neighbouring neurones. Given the billions of neurone's, this means there are as many connections in a single cubic centimetre of brain tissue as there are stars in the Milky Way."  

- David Eagleman

Project 3 are GO!

Live Data

Right so yea, we did historical data last project so now we are using live data. We will make a 'make use of a near real-time message passing infrastructure to access the data'...

Sources of Life Data- 

• Biological
• Environmental
• Financial
• Social
• And pretty much anything that changes quickly and is important

This could be examples of weather, the stock exchange, animal movement or behaviour, ocean tides etc. The list goes on and on, this technology is very open. 

The best example of live data I've found so far is this video.

OgilvyOne London connected Heathrow's live air traffic control data and live scores from the Wimbledon Tennis Championship, and mashed it up with players' nationalities to create live, useful, customised messages for departing passengers at Terminal 5.

The Idea and Concept 

The Team

• Stuart- Programming/Concept Design
• Ian- Programming/Concept Design
• Natalie- Research/Concept Design
• Me- Audio/Sound/Concept Design

The idea of the project is to control live data feeds using your senses or other variables. After much debating and discussing we decided to create a visualization with sound and audio to represent growth using movement. Growth represented by tree's, so the more active the brain is, the bigger the tree's will grow and the more amount of them there will be. 

This is how active the brain is in its various modes.

• Beta 15-30Hz > Normal, awake brain patterns
• Alpha 9-14Hz > Relaxed and calm brain patterns
• Theta 4-8Hz > Deep relaxation or problem solving brain patterns
• Delta 1-3Hz > Sleep brains patterns

RSSIV Final Product

Final Product

Here is our final product, even though you only see a small portion of the RSS feed working with our work, there are 10 other visuals and many more different audio's. If you want to see the final product in full then give me a shout. 

Group 2

So me and Natalie have joined forces to create something epic, we have decided to go with temperature and light feeds. We like them.

Team- Me- Visual/Programming
      Natalie- Audio/Visual

My suggestion is that we go with movie and documentary explosion scenes, Natalie agreed. Don't we make an amazing team! I am going to handle the visual, Natalie handle the audio. We thought about this for a long time, some big explosion music would be too easy, same as just some standard dramatic music, that isn't the iDAT way! So we just started making stupid random explosion noises and that's how we came up with our idea but making the audio by ourselves without computer or mixing aid. 
The audio is changed by wind speed, the faster the wind speed the louder the audio will be. And with the visuals, they are determined by the temperature, the hotter the temperature is outside the bigger the explosion will be (we thought this was a nice touch). 

I researched a lot of action movie explosions, went through a lot. And I mean a lot.

Here are my top video's- 

• Hospital's Explosion by The Joker

• Independence Day White House Explosion Scene 
• The Expendables

And of course...

• Gangyam Style Explosion

Those are the movie explosions, I wanted to throw in some documentary war explosions as well for a mix so Natalie found some. I can't upload them, to upload them all would take quite a while and they aren't on Youtube but you can possibly see them in our final product. 

Project Round 2

The name of the game is an RSSIV data feed.

Focus Of This Project- Simon

•Bringing real world data into your work

•Focus on non-linear playback of media

•No coding required (but you can if you want)

Knowledge is Power

The foundation of Knowledge is Data

   For this project we have to create a chunk of media to fit an RSSIV feed that Simon has created. RSSVI's is a processing sketch that acts as a client, pulling data from the Eco-OS server. As a group we can focus on audio or visual, probably do both, a lot more fun that way. We can also use a controller for the feed, or to create one ourselves, this allows some control over the data feed.

Ecoid- Information taken from i-DAT.org

Eco-OS explores ecologies. Eco-OS further develops the sensor model embedded in the Arch-OS system through the manufacture and distribution of networked environmental sensor devices. Intended as an enhancement of the Arch-OS system Eco-OS provides a new networked architecture for internal and external environments. Networked and location aware data gathered from within an environment can be transmitted within the system or to the Eco-OS server for processing.

Eco-OS collects data from an environment through the network of ecoids and provides the public, artists, engineers and scientists with a real time model of the environment. Eco-OS provides a range of networked environmental sensors (ecoids) for rural, urban, work and domestic environments. They extend the concept developed through the Arch-OS and i-500 projects by implementing specific sensors that transmit data to the Operating Systems Core Database. Eco-OS also enables the transmission of data back to the Eco-OS ecoids to support interaction with the environment (such as light shows and the transmission of audio/music in response to the network activity).

Eco-OS Core Database: is an extension of the established Arch-OS Core database. The Eco-OS Core collects the data transmitted to it by the ecoids. The data is parsed up and published through a range of flexible tools (flash, Max MSP, Processing, Java, etc), feeds (xml, rss) and web 2.0 streams, such as Twitter and Facebook, which allow artists, engineers and scientists to develop visualisations, sonifications (music) and interactive projects. Eco-OS can operate in passive mode, simply collecting data from the environment or interactive mode, feeding back recursively through the environment.

Ecoids: are sensor devices (small pods) that can be distributed through an environment (work place, domestic, urban or rural). The sensors allow environmental data to be collected from the immediate vicinity. The sensors can be connected together through the formation of Wireless Sensor Networks (WNS) that enable the coverage of an extensive territory (several kilometres). Each ecoid has a unique id and its location within a network can be triangulated giving its exact location. Consequently locative content can be tailored to a specific geographical area.

Ecoids consist of programmable (Processing, Java, etc) embedded technologies (Arduino, etc) and network technologies (Zigbee/Xbee, GPRS and Bluetooth). Designed to be attached to objects (architecture, trees, rocks, etc), free form (water-based, balloons, free standing) or as mobile sensors. They can be powered or draw power from the environment (solar).

Ecoids can also be used to produce content be receiving instructions from Eco-OS. Distributed performance can then be orchestrated across a large territory through light displays or acoustic renditions.

The Operating Systems project explores data: as an abstract and invisible material our potential to perceive our reality through data marks an evolution in human consciousness, the evolution of human perception through the emergence of senses more finely attuned to data!

Data generates a dynamic mirror image of our world, reflecting, in sharp contrast and high resolution, our biological, ecological and social activities. Reluctantly, we are becoming aware of the data shadows that cloud the periphery of our existence, as if through a glass darkly. The reluctance is, to some extent, the result of the fear we feel when we catch a glimpse of this data/mirror world out of the corner of our eye. Somewhere there is an attic, and in that attic stands a large ugly data portrait of our world. Reified its metaphorical and haptic potential are powerful tools for transformation. Operating Systems proposes a range of tools and initiatives that have the potential to enhance our ability to perceive and orchestrate this mirror world.

Concept to Design to Implement

Inspiration
I had a very interesting drive home with a friend the other night, she was drunk but we couldn't decide if she was sober enough. If we had some sort of hand held breathalyzer on us then we could of tested her, need less to say we didn't and she drove us both home what seemed like quite drunk. But me being quite drunk as well, didn't mind too much, just looking back on it, it was a really bad idea. 

Other uses
Could use this for a drinking game for students


The Code






The Finished Product and Testing



How it works

The code first tells the Arduino board what sensor is attached (gas sensor). The sensor returns a value (in our case we are looking for the gas value of alcohol) when the program detects this value, when the user has blown into the sensor, it runs another mechanism o trigger power to something that is attached. What we originally wanted was to allow a connection with the battery, what we have now decided is a connection with the 5 LED's (3 green 1 amber and 1 red) we had attached. 

So it should now allow electricity from the battery to the LED'S. Yet another program would run, different values would light up different light sequences. Green for legally allowed to drive, amber for warning and red for illegal to drive. 

Here's the finished product: 

Despite a well researched and implemented code, and a fully tested circuit, our home-made breathalyzer was not sensitive enough to detect even an "illegal" amount of alcohol. This was because of the gas sensor we were given was not powerful enough even when I drunk quite a bit and breathed on it for a good 2 minutes. This design would just be practical in any situation or design that we could create, we would need a more sensitive gas sensor. As described by one of the team members: Iain: "you would have to be completely smashed, for it to detect any alcohol on your breath" 

The Idea

The thought process was based on a gas detecting sensor that can detect and echo a value for alcohol.

Once the alcohol value was picked, the team decided to aim at young adults that enjoy going out drinking, or playing "drinking games" (example: drinking a shot every time a character says a specific word when watching a movie).
We also decided to target "punishment" for when someone has been drinking too much. Like the designated driver for example whom can't go over a certain level of alcohol.
The general idea, is to "remind" or "scare" the drinker as a harmless prank for the unsuspecting alcoholic. But it can be used to associate a certain level of drinking with shock.

Iain as the diabolical genius he is managed to find a simple way of shocking someone by reversing the charge, but this may be problematic as the drunker you get the less neurons are fired around your brain meaning you feel less pain. So to up the charge so the user would feel the shock would be too dangerous and very unethical. 

What happens is the gas sensor detects a certain value as well as carbon dioxide (to determine if its a human, or just a strong smell of alcohol) and then turn on the battery which was wired through the glass to send a current through the drink, and to the drinkers tongue to give them a very light shock.

First question: 

Is alcohol conductive? 

We tested it simply using a 9v battery and an LED light. 

Yes. The sugars in the drink allow conductivity. 

Second question

Is 9v enough?

No. There was no shock or even a static sensation from the drink. 

Third question

Health and safety

Unfortunately due to the rules that are part of the course, we are unable to add any more batteries or a higher voltage. 

The amps in electricity are dangerous, and it doesn't take a high amount to cause harm, even though we are working with low numbers, we are not authorized to carry out these experiments, and neither are our tutors. 

Fallback 

As this is a very short project, we are sticking with the alcohol detecting, with a result used to warn the user that they have had enough. Unfortunately this takes the fun out of the idea, as initially it was to be used for pranks or "whose the first to get the shock". Now it is simply "who gets the red light first". 

Testing and Designs

Testing and Designs

The Team

• Natalie Williams -Concept and Impact Research
• Christa Briggs - Concept
• Josh Lane - Tester/ Documenter
• Iain Van Gardingen - Programming

This a test using Tango in a little trough, showing that it conducts electricity. We found out that Tango is the same density and conductivity as alcohol.

This helped us in your project as now we can conduct alcohol sending an electric shock to the user by basically reversing a transformer (so says Iain).  

The Equipment

Designing and Constructing

The Gas Sensor 

The Project

THE PROJECT!

HARDWARE- Hardware

Lee taught us basic Arduino today, a kind of refresher course. He shows his past projects to us, one project working with sound and light sensors to detect bats and their natural habits. This seems like a really interesting idea as ‘the bat’ is a hard creature to study close up, so using technology to detect real time situations is a unique idea.

Our idea is to create an alcohol detector, making the process of there being a sensor for the alcohol and then it lighting up a bulb if there is too much alcohol on the users breath. If there is alcohol detected on the users breath, then the battery port will turn on, turning on the light bulb. more making it flash.

Iain has even found out a way of shocking someone reversing a transformer, not sure how practical this way in the shape of psychical pain but we’ll see.



Equipment we will use...

The MQ-3 is an alcohol gas sensor that is available for about £5 from Sparkfun, Seeed Studio, and others. It’s easy to use and has sparked the imagination of anyone who has dreamed of building their own breathalyzer device for measuring the amount of alcohol in the human body.

http://arduino.cc/blog/2010/09/23/arduino-breathalyzer-calibrating-the-mq-3-alcohol-sensor/

Alcohol Gas Sensor MQ-3

This alcohol sensor is suitable for detecting alcohol concentration on your breath, just like your common breathalyzer. It has a high sensitivity and fast response time. Sensor provides an analog resistive output based on alcohol concentration. The drive circuit is very simple, all it needs is one resistor. A simple interface could be a 0-3.3V ADC.
Features:

• 5V DC or AC circuit
• Requires heater voltage
• Operation Temperature: -10 to 70 degrees C
• Heater consumption: less than 750mW

Dimensions:

• 16.8mm diameter
• 9.3 mm height without the pins

http://dx.com/p/mq-3-alcohol-ethanol-sensor-module-for-arduino-brown-blue-148800


Youtube examples.

This is a kickstarter project looking for funding.
http://www.kickstarter.com/projects/1342192419/pulse-sensor-an-open-source-heart-rate-sensor-that


Arduino – Using a Sharp IR Sensor for Distance Calculation

http://luckylarry.co.uk/arduino-projects/arduino-using-a-sharp-ir-sensor-for-distance-calculation/

Using a Sharp GP2Y0A02 series infrared distance sensor it’ll detect and measure anything within a 20-150cm range and it does this by triangulation from where it emits a beam of IR and from when it receives it.

Synesthesia

Synesthesia

Synesthesia is a neurological condition where one of the subject’s (known as an synesthete) sensory or cognitive pathway is stimulated leading to an automatic, involuntary experience in a second sensory or cognitive pathway. Research has been performed that synesthesia could be hereditary and can run strongly in families but no real conclusions have been made yet as to narrowing a specific gene or element leading to this suggestion

Synesthetes are in some sense, people of the future. Some features of human evolution can be deduced from 'synesthesia-phenomena' directed studies, and considering we only use about 12% of our full brain capacity, it can be suggested that synesthetes can access a larger portion of their brain without even fully understanding or realising it.


Experiences with Synesthesia

Synesthetes often report that they were unaware their experiences were unusual until they realised other people did not have them. A lot of cases have been reported that the person would describe certain examples with total different examples, such as describing music with shapes and colours. E.g “The guitar on that track is really red sounding” or “That song feels very angular”. People could have colours for numbers, letters, days of the week and months, musica notations such as how a guitar sounds or how the drums feels, or intellectuals using places or images to remember complex mathematical equations. It could also be very obscure definments such as using a golden fork to describe loyalty (this was used by the  Russian novelist Vladimir Nabokov to describe his experience with synesthesia).

Testing for Synesthesia.
To test synesthesia, Rama and collaborators designed an experiment where they could measure the vividness of the colours associated with the numbers 2 and 5. They chose those because you can make them look almost identical, although reversed, by choosing a boxy font. Then they made up a picture of mostly fives, with a few twos scattered within there. Then they asked people to pick out the twos. Most ordinary folks could do it within about twenty seconds or so.
But true synesthetes could do it immediately. That’s because to them, the twos popped out as a brightly colored triangle.  This established beyond much doubt that synesthesia was “real,” and more particularly that was a measurable phenomenon with real consequences.
This, in turn, strengthened the hypothesis that the origin of synesthesia was to be found in the structure of the brain. Indeed, it turns out that the region of the brain responsible for processing graphemes lies adjacent to the region responsible for processing colours.


Synesthesia in Art
The phrase synesthesia in art has historically referred to a wide variety of artistic experiments that have explored the cooperation of the senses.

Visual music and art- Starting in the late 1950's, electronic music and electronic visual art have co-existed in the same digital medium. The interaction of art and synesthesia and the computer design has increased tremendously. Nowadays, students of art and music have digital software at their disposal that uses both musical and visual imagery. Given the capability of the Internet to publish and share digital productions, this has led to an enormous avalanche of synesthesia-inspired art on the internet.



Forms of Synesthesia in Art

• Synesthetic art: a cross-sensory perception evocated by the experience of an artwork
• Synesthetic images: images that accumulate striking metaphorical resonance
• Literary synesthesia: a poetic expression or metaphorical articulation of a sensorial correspondence
• Synesthetic metaphor: a metaphor that exploits a similarity between experiences in different sense modalities
• Poetic synesthesia: a semantic metaphoric fusion, to create a virtual image
• Kinetic synesthesia: experiencing dance in multimedia scenographies
• Synesthetic canvas: an electronic screen
• Conceptual synesthesia: elicited from time, graph, grapheme, written word, personality, or thought/memory
• Synesthetic cinema: translating consciousness and perception into sound and moving images
• Tele-synesthesia: a synesthetic experience evoked by a telematic use of new media; the 'travelling' senses