Processing + EPOC via OSC

Related articles: AffectCircles

How would you like to create interactive art that responds to your thoughts, moods, and facial expressions? Thanks to Mind Your OSCs and oscP5, interpreting the Emotiv EPOC‘s data within a Processing sketch (and by extension, Arduino) could not be easier, even with the consumer (i.e. most affordable) version of the EPOC. This effectively allows anyone to develop a great variety of (open-source, if desired) EPOC applications including physical computing, even if they have only the consumer headset [1].

Here is how it works. The EPOC headset and software read your neuroelectrical signals and interpret them as a set of predefined outputs that reflect your facial expressions, mood (excited, bored, meditative, etc.), and conscious intentions (see the EPOC docu for more info). Mind Your OSCs formats that output as a series of OSC messages, and sends them to a network port. Using oscP5, Processing can listen to that port and read any OSC messages that can then be parsed and their values assigned to variables that determine various interactions.

To begin, you need an Emotiv EPOC and the Mind Your OSCs application which you can download for free from the Emotiv store. If you want to experiment before committing to purchase the EPOC, you can download Emotiv’s SDK Lite for free, which includes an EPOC emulator and scripting tool called EmoComposer (also useful for testing interactions with Processing sketches without having the headset on). You also need to download and install Processing and the oscP5 library.

Connecting Processing to Mind Your OSCs

In the right-hand side of the Mind Your OSCs window, you can see the IP address and port number for data going out of Mind Your OSCs (connection info for data coming into Mind Your OSCs from the EPOC device or an emulator, etc., is displayed on the left-hand side of the window):

OSC Connection

In Processing’s setup() method, you set up a connection to the same port showing in Mind Your OSCs:

void setup() {
  //start oscP5, listening for incoming messages on port 7400
  //make sure this matches the port in Mind Your OSCs
  oscP5 = new OscP5(this, 7400);
}

Interpreting EPOC Events

Each OSC message sent from Mind Your OSCs has three parts:

  1. an Address Pattern in the form of /COG/PUSH (meaning Cognitiv suite → Push action, e.g.) [2]
  2. a Type Tag string that identifies the data type of the Argument (always ‘f’ for Mind Your OSCs, for floating point)
  3. one or more Arguments; for Mind Your OSC’s, always one argument, a floating point number between 0 and 1 that represents the value returned by the EPOC function identified by the Address Pattern

You can use the oscEvent() method to parse selected messages from Mind Your OSCs. Here is an example to select the messages corresponding to the Cognitiv suite’s (move) Left and Right actions, and assign their values to the variables left and right, respectively:

void oscEvent(OscMessage theOscMessage) {
  // check if theOscMessage has an address pattern we are looking for
  if(theOscMessage.checkAddrPattern("/COG/LEFT") == true) {
    // parse theOscMessage and extract the values from the OSC message arguments
    cogLeft = theOscMessage.get(0).floatValue();
  } else if (theOscMessage.checkAddrPattern("/COG/RIGHT") == true) {
    cogRight = theOscMessage.get(0).floatValue();
  }
}

Remember that even though the OSC messages from Mind Your OSCs have only one Argument each, OSC messages in general can have more than one, so they need to be treated like arrays. The statement left = theOscMessage.get(0).floatValue(); means “read the value at index 0 (i.e. the first and only value associated with a message sent from Mind Your OSCs) as a floating-point number, and assign it to the variable left.”

You can use oscP5’s plug() service to automatically forward select messages to your methods without having to parse them with oscEvent() (example).

Example Sketch

Here is an example Processing sketch that moves a circle left or right when the operator wearing the EPOC thinks, “move left,” or, “move right,” respectively (the Left and Right Cognitiv actions must be trained using the Emotiv software, prior to running the sketch).

ProcessingEpocOsc1 Screen Shot

/**
 * ProcessingEpocOsc1
 * by Joshua Madara, hyperRitual.com
 * demonstrates Processing + Emotiv EPOC via OSC
 * uses EPOC's Cognitiv Left and Right to move a circle
 * left or right
 */
 
import oscP5.*;
import netP5.*;

public float cogLeft = 0;
public float cogRight = 0;
int circleX = 240;

OscP5 oscP5;

void setup() {
  size(480, 360);
  frameRate(30);
  smooth();
  
  //start oscP5, listening for incoming messages on port 7400
  //make sure this matches the port in Mind Your OSCs
  oscP5 = new OscP5(this, 7400);
}

void draw() {
  background(0);
  
  // draw graph ticks
  int i;
  for (i = 1; i <= 11; i++) {
    stroke(map(i, 1, 11, 0, 255));
    float tickX = map(i, 1, 11, 60, 420);
    line(tickX, 250, tickX, 269);
    line(tickX, 310, tickX, 329);
  }
  noStroke();
  
  // draw bar graph
  drawBarGraph(cogLeft, 270);
  drawBarGraph(cogRight, 290);
  
  // determine whether to move circle left or right
  if((cogLeft >= 0.5) && (circleX >= 0)) {
    circleX -= 5;
  } else if ((cogRight >= 0.5) && (circleX <= 480)) {
    circleX += 5;
  }
  
  // draw circle
  fill(color(25, 249, 255));
  ellipse(circleX, 150, 90, 90);
}

void drawBarGraph(float cogVal, int barY) {
  if(cogVal >= 0.5) {
    fill(color(22, 255, 113));
  } else {
    fill(color(255, 0, 0));
  }
  float len = map(cogVal, 0.0, 1.0, 0, 360);
  rect(61, barY, len, 20);
}

void oscEvent(OscMessage theOscMessage) {
  // check if theOscMessage has an address pattern we are looking for
  if(theOscMessage.checkAddrPattern("/COG/LEFT") == true) {
    // parse theOscMessage and extract the values from the OSC message arguments
    cogLeft = theOscMessage.get(0).floatValue();
  } else if (theOscMessage.checkAddrPattern("/COG/RIGHT") == true) {
    cogRight = theOscMessage.get(0).floatValue();
  }
}

Spectacle of the Mind

Here is video footage from Spectacle of the Mind, a Global Mind Project performance featuring Stelarc, Domenico De Clario, and Jill Orr, all using EPOCs in multimedia performance art (albeit not with Processing or Arduino, but they are capable of similar things).

Notes & References

  1. You can do the same thing using GlovePIE, but it requires a developer edition of the EPOC, or better.
  2. The various Address Patterns are: /COG/NEUTRAL; /COG/PUSH; /COG/PULL; /COG/LIFT; /COG/DROP; /COG/LEFT; /COG/RIGHT; /COG/ROTATE_LEFT; /COG/ROTATE_RIGHT; /COG/ROTATE_CLOCKWISE; /COG/ROTATE_COUNTER_CLOCKWISE; /COG/ROTATE_FORWARD; /COG/ROTATE_REVERSE; /COG/DISAPPEAR; /AFF/Engaged/Bored; /AFF/Excitement; /AFF/Excitement Long Term; /AFF/Meditation; /AFF/Frustration; /EXP/WINK_LEFT; /EXP/WINK_RIGHT; /EXP/BLINK; /EXP/LEFT_LID; /EXP/RIGHT_LID; /EXP/HORIEYE; /EXP/VERTEYE; /EXP/SMILE; /EXP/CLENCH; /EXP/LAUGH; /EXP/SMIRK_LEFT; /EXP/SMIRK_RIGHT; /EXP/FURROW; /EXP/EYEBROW.

13 Replies to “Processing + EPOC via OSC”

  1. Hello there,

    I am having trouble connecting the Emotiv Xavier Control Panel 3.3.3 to Mind Your OSCs. Whenever I attempt connection, the program displays ‘waiting’ and after some time, the program crashes.

    Any suggestion, please.

    Thank you,

    1. Hello, Justine. I have not done any development with the Emotiv devices in some time. You might do better to inquire on the Emotiv forums. Good luck, and please let me know if you find a solution to your problem.

    2. Hi Justine and Madara ,

      I’m new to use Emotive and I’m trying use Insight EEG. I have the same problem and I’m stuck and unable to start my project. Did u get any solution for the crash of Emotive Xavier control panel 3.3.3. Any help is appreciated.

    3. Hey Justine, I am also currently working on a project with the emotiv EPOC and processing, and I have the exact same problems as you have! Did you find a solution yet?

  2. Thanks for the article, its pretty awesome! I just recintly bought the SDK research verison, and I was wondering if there is a way to access the individual channels from the EPOC device???

    1. Hi, Keith. I have not used the research edition, so I do not know much about how to access the data you write of, but I know of no way to do that with MindYourOSCs. You would likely need to write your own layer between the EPOC and Processing, or modify MYO.

  3. Hey Josh – firstly thanks so much for the article… Proce55ing and all its community builders are a huge part of accessibility in this world of interactivity.

    I’m looking into consumer eeg’s whilst relying on Proce55ing – have been trawling the net for some kind of sign as to whether the extra $400 dollars spent (considering the developer sdk cost) on the Epoch compared to the Neurosky Mindwave is worth it. I know the Epoch accesses more mental states but the general “perceived” reliability of both these products is so wayward that I am not sure if the extra money is worth it if the signals are going to be noticeably abstract.

    Your thoughts on this would be greatly appreciated.

    1. Hi, Peter. Thanks for reaching out. I have not used the Neurosky device, so I am ill-equipped to compare them, but I will say that I do not at all regret the $300 I dropped on the EPOC (consumer ed). I have not used my EPOC in a while except for simple demos of mind-controlled robots, but I have witnessed impressive control with it, and I look forward to getting more out of it while developing some things for Robomancy.com.

      Although there are some things I miss doing for lack of the dev ed of the EPOC, the OSC interface satisfies most of my needs.

Leave a Reply

Your email address will not be published. Required fields are marked *