Posts Tagged ‘BCI’

FBLR Mind-Controlled Robot

By tchnmncr on January 12, 2011 | Categories: Blog, Portfolio | Tags: , , , , , | 10 Comments

Related articles: Processing + EPOC via OSC | EPOC-to-Arduino “Goodbye, World” | Arduino + fischertechnik | BYTR Virtual Servitor

FBLR (“Fibbler”) is a robot that moves forward and backward, and turns left and right, in response to its operator’s thoughts. The signal flow is brain → Emotiv EPOCMind Your OSCsProcessingArduinofischertechnik robot. I wrote a Processing sketch (zipped with data files) that converts the EPOC signals via OSC messages, to Arduino signals that move the robot. The sketch uses the arduino, controlP5, and oscP5 libraries. The Arduino uses Firmata (standard with the Arduino software) and a motor shield from Rugged Circuits (see this article for more info).

FBLR Processing Interface

There are four knobs for adjusting the actuator thresholds. The EPOC’s Cognitiv values are compared to said thresholds to determine whether to enable the robot’s motors and which directions to turn them in:

Cognitiv Action Motor 1 Direction Motor 2 Direction
PUSH 0 0
PULL 1 1
LEFT 1 0
RIGHT 0 1

Setting a knob to 0 (all the way to the left) will enable its associated motors regardless of the Cognitiv value. Four bars display the levels of the incoming Cognitiv values; if a value is greater than or equal to its threshold, the bar displays green, otherwise it displays red.

The slider can be used to change the delay from 0 to 200 milliseconds. The shorter the delay, the more frequently OSC messages are evaluated, but the robot may also “stutter” for trying to retrieve messages faster than they are delivered. There are a variety of ways to handle this; I chose one that was simple and effective but not the most elegant. I found that 100ms makes for smooth operation most of the time. Note also that push, pull, left, and right are evaluated in that order by the draw() loop, so if, e.g., push and left both exceed their threshold values, only moveForward() will fire.

Here is an EmoScript for EmoComposer, to test the interaction without needing the headset. See the Emotiv EPOC SDK guide for more about that.

Occult Applications

Mankind’s oldest fantasy is to move something with the power of thought. // Tan Le, co-founder of Emotiv

Any use of the EPOC seems to be magical in a folk sense, but I am interested in new-media interactions that respond to explicitly magical intentions or commands (e.g. sigil concentration), ecstatic or gnostic states of consciousness, state entity projections, etc. I expect the EPOC to feature in some of the robomancy projects I will be developing in the near future.

FBLR mind-Controlled Robot

Processing + EPOC via OSC

By tchnmncr on January 5, 2011 | Categories: Blog | Tags: , , | 12 Comments

Related articles: AffectCircles

How would you like to create interactive art that responds to your thoughts, moods, and facial expressions? Thanks to Mind Your OSCs and oscP5, interpreting the Emotiv EPOC‘s data within a Processing sketch (and by extension, Arduino) could not be easier, even with the consumer (i.e. most affordable) version of the EPOC. This effectively allows anyone to develop a great variety of (open-source, if desired) EPOC applications including physical computing, even if they have only the consumer headset [1].

Here is how it works. The EPOC headset and software read your neuroelectrical signals and interpret them as a set of predefined outputs that reflect your facial expressions, mood (excited, bored, meditative, etc.), and conscious intentions (see the EPOC docu for more info). Mind Your OSCs formats that output as a series of OSC messages, and sends them to a network port. Using oscP5, Processing can listen to that port and read any OSC messages that can then be parsed and their values assigned to variables that determine various interactions.

To begin, you need an Emotiv EPOC and the Mind Your OSCs application which you can download for free from the Emotiv store. If you want to experiment before committing to purchase the EPOC, you can download Emotiv’s SDK Lite for free, which includes an EPOC emulator and scripting tool called EmoComposer (also useful for testing interactions with Processing sketches without having the headset on). You also need to download and install Processing and the oscP5 library.

Connecting Processing to Mind Your OSCs

In the right-hand side of the Mind Your OSCs window, you can see the IP address and port number for data going out of Mind Your OSCs (connection info for data coming into Mind Your OSCs from the EPOC device or an emulator, etc., is displayed on the left-hand side of the window):

OSC Connection

In Processing’s setup() method, you set up a connection to the same port showing in Mind Your OSCs:

void setup() {
  //start oscP5, listening for incoming messages on port 7400
  //make sure this matches the port in Mind Your OSCs
  oscP5 = new OscP5(this, 7400);
}

Interpreting EPOC Events

Each OSC message sent from Mind Your OSCs has three parts:

  1. an Address Pattern in the form of /COG/PUSH (meaning Cognitiv suite → Push action, e.g.) [2]
  2. a Type Tag string that identifies the data type of the Argument (always ‘f’ for Mind Your OSCs, for floating point)
  3. one or more Arguments; for Mind Your OSC’s, always one argument, a floating point number between 0 and 1 that represents the value returned by the EPOC function identified by the Address Pattern

You can use the oscEvent() method to parse selected messages from Mind Your OSCs. Here is an example to select the messages corresponding to the Cognitiv suite’s (move) Left and Right actions, and assign their values to the variables left and right, respectively:

void oscEvent(OscMessage theOscMessage) {
  // check if theOscMessage has an address pattern we are looking for
  if(theOscMessage.checkAddrPattern("/COG/LEFT") == true) {
    // parse theOscMessage and extract the values from the OSC message arguments
    cogLeft = theOscMessage.get(0).floatValue();
  } else if (theOscMessage.checkAddrPattern("/COG/RIGHT") == true) {
    cogRight = theOscMessage.get(0).floatValue();
  }
}

Remember that even though the OSC messages from Mind Your OSCs have only one Argument each, OSC messages in general can have more than one, so they need to be treated like arrays. The statement left = theOscMessage.get(0).floatValue(); means “read the value at index 0 (i.e. the first and only value associated with a message sent from Mind Your OSCs) as a floating-point number, and assign it to the variable left.”

You can use oscP5’s plug() service to automatically forward select messages to your methods without having to parse them with oscEvent() (example).

Example Sketch

Here is an example Processing sketch that moves a circle left or right when the operator wearing the EPOC thinks, “move left,” or, “move right,” respectively (the Left and Right Cognitiv actions must be trained using the Emotiv software, prior to running the sketch).

ProcessingEpocOsc1 Screen Shot

/**
 * ProcessingEpocOsc1
 * by Joshua Madara, hyperRitual.com
 * demonstrates Processing + Emotiv EPOC via OSC
 * uses EPOC's Cognitiv Left and Right to move a circle
 * left or right
 */
 
import oscP5.*;
import netP5.*;

public float cogLeft = 0;
public float cogRight = 0;
int circleX = 240;

OscP5 oscP5;

void setup() {
  size(480, 360);
  frameRate(30);
  smooth();
  
  //start oscP5, listening for incoming messages on port 7400
  //make sure this matches the port in Mind Your OSCs
  oscP5 = new OscP5(this, 7400);
}

void draw() {
  background(0);
  
  // draw graph ticks
  int i;
  for (i = 1; i <= 11; i++) {
    stroke(map(i, 1, 11, 0, 255));
    float tickX = map(i, 1, 11, 60, 420);
    line(tickX, 250, tickX, 269);
    line(tickX, 310, tickX, 329);
  }
  noStroke();
  
  // draw bar graph
  drawBarGraph(cogLeft, 270);
  drawBarGraph(cogRight, 290);
  
  // determine whether to move circle left or right
  if((cogLeft >= 0.5) && (circleX >= 0)) {
    circleX -= 5;
  } else if ((cogRight >= 0.5) && (circleX <= 480)) {
    circleX += 5;
  }
  
  // draw circle
  fill(color(25, 249, 255));
  ellipse(circleX, 150, 90, 90);
}

void drawBarGraph(float cogVal, int barY) {
  if(cogVal >= 0.5) {
    fill(color(22, 255, 113));
  } else {
    fill(color(255, 0, 0));
  }
  float len = map(cogVal, 0.0, 1.0, 0, 360);
  rect(61, barY, len, 20);
}

void oscEvent(OscMessage theOscMessage) {
  // check if theOscMessage has an address pattern we are looking for
  if(theOscMessage.checkAddrPattern("/COG/LEFT") == true) {
    // parse theOscMessage and extract the values from the OSC message arguments
    cogLeft = theOscMessage.get(0).floatValue();
  } else if (theOscMessage.checkAddrPattern("/COG/RIGHT") == true) {
    cogRight = theOscMessage.get(0).floatValue();
  }
}

Spectacle of the Mind

Here is video footage from Spectacle of the Mind, a Global Mind Project performance featuring Stelarc, Domenico De Clario, and Jill Orr, all using EPOCs in multimedia performance art (albeit not with Processing or Arduino, but they are capable of similar things).

Notes & References

  1. You can do the same thing using GlovePIE, but it requires a developer edition of the EPOC, or better.
  2. The various Address Patterns are: /COG/NEUTRAL; /COG/PUSH; /COG/PULL; /COG/LIFT; /COG/DROP; /COG/LEFT; /COG/RIGHT; /COG/ROTATE_LEFT; /COG/ROTATE_RIGHT; /COG/ROTATE_CLOCKWISE; /COG/ROTATE_COUNTER_CLOCKWISE; /COG/ROTATE_FORWARD; /COG/ROTATE_REVERSE; /COG/DISAPPEAR; /AFF/Engaged/Bored; /AFF/Excitement; /AFF/Excitement Long Term; /AFF/Meditation; /AFF/Frustration; /EXP/WINK_LEFT; /EXP/WINK_RIGHT; /EXP/BLINK; /EXP/LEFT_LID; /EXP/RIGHT_LID; /EXP/HORIEYE; /EXP/VERTEYE; /EXP/SMILE; /EXP/CLENCH; /EXP/LAUGH; /EXP/SMIRK_LEFT; /EXP/SMIRK_RIGHT; /EXP/FURROW; /EXP/EYEBROW.

Emotiv EPOC

By tchnmncr on January 20, 2009 | Categories: Blog | Tags: , , , | 2 Comments

Redirect to new EPOC page.