Interface and application programming

Assignment: Write an application that interfaces with an input & / or output device.

For interfacing with a physical device (such as a sensor or button) there is a large number of options in terms of coding and available software. The global lecture listed quite a few ways to interface. However, I still missed some key tools that I used in the past, namely MaxMSP, Flash (using xml socket for interfacing) and, my favourite, Macromedia Director (using serial plug-in, also with Agamanolis' Isis programming language): great for combining the physical world with interactive visual movements.

Although I prefer these tools as they let you design first, and then add the coding (in contrast to e.g. the Python programming language), these might also be(come) a bit obselete nowadays. Thus, I decided to focus on learning a new tool: Processing. Still primarily code based, but in a visual (arts) supportive way.

Processing progress

With my background, learning Processing was easy. I decided not to jump ahead and to learn and progress step-by-step by following the great tutorials on getting started, using text, interactivity, sound and finally electronics. For example, I created a sketch enabling you to play with line visuals starting from an arrow shape.Besides from altering and getting all the tutorial code to work, I played with adding text (triggered by mouse event) and sound (using the Processing sound library and the x,y position for changing volume and rate).
Then I got the interfacing to work:

(1) a physical button triggers a sound fragment and visual change (make sure the button is rightly positioned on the board...)

(2) a distance sensor (showing the distance in real time with added font, using this video tutorial and the work during the Input devices week).

Although now relatively abstract, when combined and further designed this code will come in handy for my final project (e.g. sensed distance to object triggers sound and volume). Look forward to doing more visual coding!

More coding

Thanks to a good video tutorial I found on the web, and adjusting the code for the serial connection and full screen, I was able to make a nice graphical visualisation of the distance sensor using Processing. This time I got rid of the breadboard and connected the sensor directly to the board.

I also succeeded in coding a nice piece of interaction: The closer you come to the sensor, the louder the sound of the audio fragment "Don't stand so close to me" by The Police sounds.


  • For including and adding fonts to your sketch:
    • In Processing IDE, go to Tools> Create Font
    • In Processing code, use: font = loadFont("Helvetica-200.vlw<or any other preferred font and size");
  • For using the serial port in Processing, check these steps:
    • Check whether the port is not busy by another tool (e.g. Arduino)
    • For listing the port names (and their position), use: prinln(Serial.list());
    • Then use its position in the Processing code: string portName = Serial.list()[3<or any other port position depending, starting from 0];
  • When looping sounds: Use mono (not stereo) sounds (Processing has an unresolved issue for loop playing stereo sound fragments)
  • When using sound fragments (such as mp3's): Store these in a 'data' folder in the location of your Processing code

Files (Code)