Week 08. Midterm

1.Idea progression

For this midterm, my partner,Cezar and I decided to start from his idea of ‘Ghost box’. The process of the box is like this.

The box would allow one to communicate with the dead. It kinda looks like a radio—it has a speaker and a frequency knob. On top of it, there’s a transparent space where we display a hologram of the spirit you’re communicating with. When you turn the “frequency” knob, the graphics changes. In most of the time, it’s just an animating oscillator line of sorts but at some frequencies, that line becomes the image of a face or something. creepy stuff! Also, we build the ghost box technology which scans different radio frequencies at a high speed and spits out results from FM radio receiver. 

Also, By controlling the knob little by little, ghost sound will come out of the speaker with images. (As for the sound effect, when you play the image01, then you can make it come out with sound01 together if we adjust the knob1.)

This is his draft giving us starting point.

In addition to this Cesar idea, I suggested that before people get to this device, We can put ultrasonic sensor that can detect object distance to attract or lure people. I thought that with a sound called “Help Me” like Ghost whisperer, people could approach the device out of curiosity and it would be a kind of fun trap. So we decided to use mp3 module or fm radio receiver.

2. preparing the material and set up the board &. testing the ultrasonic sensor and fm radio and mp3 module

In the second meeting, we set up basic materials, such as acrylics, ultrasonics sensor, fm radio receiver, speaker, etc. on the breadboard to make boxes. At first trial, we tried to get the sound out of the ‘speakers’, but with the advice of Tom and Jiwon, we decided to have the sound come out of the iPad with the sound from the p5 sketch.  We also found that the holographic images that Cesar had been prototyping beforehand worked well. And I experimented with arduino to detect distance using mp3 module that I bought, but decided to throw it away because there was a problem with the module and use radio only.

3. making p5 sketch with sounds

In the third meeting, we tested how the ultrasonic sensor and the fm radio receiver worked in the box that Cezar had been making. And I made several different ghost images within several p5 sketches as prototypes, roughly as below. We decided to pick out some of them and make them the final image. These are some of p5 sketches.

let img;
let mic;
function preload() {
  horror02 = loadImage('horror02.jpg');
  //horror01 = loadImage('horror01.jpg');
  //horror04 = loadImage('horror04.jpg');

function setup() {
  mic = new p5.AudioIn()
  //loadImage.resize (width, height);
  // image(horror04, width / 2, height / 2, 0, 0);
  image(horror02, width / 2, height / 2, 0, 0);
  //image(horror01, width / 2, height / 2, 0, 0);

  createCanvas(500, 500);
function draw() {
  micLevel = mic.getLevel();

 //image(horror01,  random(-2, 2)+100 , random(-2, 2)+110, 0, 0);
image(horror02, random(width / 2.7, 120) - 140, constrain(width / 1.1 - micLevel * height * 5, 0, height), 0, 0);
 // image(horror04, random(-2, 2) + 50, random(-2, 2), 0, 0);
var song;
var amp;
var button;

var volhistory = [];

function toggleSong() {
  if (song.isPlaying()) {
  } else {

function preload() {
  song = loadSound('radio.mp3');

function setup() {
  createCanvas(500, 500);
  button = createButton('toggle');
  amp = new p5.Amplitude();

function draw() {
  var vol = amp.getLevel();

  var currentY = map(vol, 0, 1, height, 0);
  translate(0, height / 2 - currentY);
  for (var i = 0; i < volhistory.length; i++) {
    var y = map(volhistory[i], 0, 1, height, 0);
    vertex(i, y);
  if (volhistory.length > width - 50) {
    volhistory.splice(0, 1);

  stroke(255, 0, 0);
  //line(volhistory.length, 0, volhistory.length, height);
 // ellipse(100, 100, 200, vol * 200);
//referred Dan's example.

4.  set up and put the storyline in ghost box  and make the code image, sound, distance sensor with p5 input serial in one sketch.

In the fourth meeting, we decided to make more changes to the visual sketches because they might be too monotonous if we show just images to them.

I suggested adding a storyline to the visual sketches. Whenever I control the knob little by little, it would be more fun if the sound and appropriate image were shown in the story. The story is roughly as follows. people go to the device attracted by the eerie ghost sound ‘help me’. Then, the images appear in order by turning the knob one by one. A total of seven images were used here.

1. The story begins with the giggling man, seeing people trapped. This is giving us more suspense.( I think the image can go with ’01_gigglingman’ sound) 2. The door open squeakly. we can get the impression that people enter the door. ( with 02_squeakingdoor sound) 3. As soon as we open the door, the ghost appears one by one. A female ghost who screams with people (the image is shaking together to maximize the effects of screams, or if people make their own sounds, it’s fun to interact with.) (with 03_screamingwoman sound) 4. Another female ghost appears.(with 04_creepywoman sound) 5. A Cursed doll also show up and laugh at people who entered the door. (with 05_cursed doll sound) 6. That’s when the door closes and the souls of people are trapped here forever. (05_creepy doll laugh) 7. Finally, it ends by showing the haunted house.(with creepy music sound) (with 06_slamdoor sound)

Simply speaking, it’s a story that attracts people by the ghost sound and then gets my soul trapped in a haunted house forever. 

We also confirmed that images and sounds are controlled with p5 serial control in one p5 sketch. Several images were arranged in order by dividing the range of 0-255, which is the value of the potentiometer, into different parts in this way. Then, in between, fm radio frequency images were put in between images. The p5 sketches were confirmed to be equally responsive on the iPad by localhost.

5. final test

In our fifth meeting, whenever we adjusted the potentiometer, we checked that the image worked well in p5 for each sound according to the story line. The distance sensor also worked well on the p5. The distance was set at about 30 – 40 centimeters. 

Final test with p5 sketch with serial communication

I think it was an important time to get a sense of how physical computing is going in general as we go through mid-Term. My difficulty was that the coding part was generally difficult because of my behind schedule. I tried input serial with the p5 image sketch that I made, but it didn’t work out well, so I think I relied on my partner a lot for coding Also at first, I didn’t think I would control images and sounds from one p5 sketch, but it was actually possible! I don’t understand all the parts that write p5 sketches and code with arduino with input serial yet, but I think I will take this mid-term as my starting point to know it.