Project 2: Dodeca – Part 3
For my second project in Physical Computing I continued to work with Mai on our Dodeca! We had some helpful feedback during and outside of class that gave us two specific development goals. The first goal was to add some kind of button input to give more control to the user to trigger events. The second was to develop an application for the dodeca specific to the device.
Button attempt #1: Tap
While we could have added additional button hardware, like a tactile switch, we really wanted to keep the simplicity of the aesthetics of the dodeca. Since we were already taking measurements with the built-in IMU it seemed that “taps” would be a great way to go.
The approach we took to recognize taps was to create a buffer to hold values from the accelerometer and look for short peaks in the buffer like the four taps in the following chart:
To check for a tap I wrote this function (shortened below to only check one axis):
void checkTap(float x){
float adjustTapThresholdX = 0;
// add values to buffer
for (int i = 9; i > 0; i--){
adjustTapThresholdX = adjustTapThresholdX + buttonBufferX[i];
buttonBufferX[i] = buttonBufferX[i-1];
}
adjustTapThresholdX = adjustTapThresholdX/9;
buttonBufferX[0] = x;
// Look for short peak in buffer
// Check that newest reading is not a peak on X
if(buttonBufferX[0] < adjustTapThresholdX + tapThreshold && buttonBufferX[0] > adjustTapThresholdX - tapThreshold){
bool tapTriggered = false;
// Check for peak above high threshol or below low threshold
if(buttonBufferX[1] > adjustTapThresholdX + tapThreshold || buttonBufferX[1] < adjustTapThresholdX - tapThreshold){
for (int i = 2; i < 10; i++){
if(buttonBufferX[i] < adjustTapThresholdX + tapThreshold && buttonBufferX[0] > adjustTapThresholdX - tapThreshold){
tapTriggered = true;
}
if(tapTriggered && millis()-buttonTime > tapTriggerDelay){
tapStatus = 1;
pulseStatus = 1;
tapCount++;
//Serial.print("tap ");
//Serial.println(tapCount);
buttonTime = millis();
} else {
tapStatus = 0;
}
}
}
}
}
Accurately measuring taps turned out to be a bigger challenge than I expected in part because movement and rotation of the dodeca was also a part of its use. While fine tuning tapThreshold for taps I found that a low threshold was desirable to allow a “light” tap, but the low threshold resulted in unwanted triggers to the tap while rotating the dodeca. On the other hand using a higher threshold reduced the unwanted taps but meant using a very “hard” tap to trigger, which is difficult when using one hand, and also had an undesirable result of not recognizing “light” taps.
Button attempt #2: Capacitive Touch
After having some trouble with the reliability of the tap sensor, and still brainstorming around “invisible” buttons, my thoughts moved to capacitive touch. While the Nano 33 IoT doesn’t have specific pins for touch sensing I did find the helpful CapacitiveSensor library that can be used with any Arduino to make a capacitive touch input. This library can use any two pins, configuring one as a “sender” and the other as “receiver”, to sense touch on the “receiver” pin.
Capacitive touch has the benefit of working through materials, such as the PLA enclosure we had printed, so I went ahead and added some copper tape on the inside of the enclosure and soldered it to pin 5 on the Nano. Additionally I added a 1 mega Ohm resistor between pin 8, which I setup as the “sender”, and pin 5, which I configured as the “receiver”.
Separating out the capacitive touch button from the IMU readings made it much easier to register a touch on the enclosure seen as the higher green values the following graph:
Again I ran into an issue when play testing though. While the capacitive touch sensor worked more reliably than taps from the IMU they missed any satisfying sense of physical feedback.
Button attempt #3: Mouse clicks?
While developing the P5.js sketch that pairs with the dodeca it was helpful to allow mouse clicks and movement as input to test the sketch without needing to connect the dodeca. While play testing I found that it was actually enjoyable to use one hand on the physical dodeca, to move the virtual dodeca, and another hand on my computer mouse to control clicks. Personally I think mouse clicks are the way to go but have left capacitive touch and taps enabled as well to give the user options.
Interface
During our last class Tom mentioned a musical touch app developed by Brian Eno and Peter Chilvers called “Bloom”. Inspired by the meditative nature of the app I set out to pull a P5.js sketch together with a soothing interface and calming sounds.
User Samulis over on FreeSound.org has put together some nice sound packs of various orchestral instruments. I found the harp to be especially soothing and pulled some recordings of notes in an A major scale to use in the sketch.
As mentioned above this sketch works with a mouse as well! P5 sketch link here.
Ultimately I learned a lot through this process and have greatly enjoyed collaborating with Mai. This second project has especially shown me the value of play testing while in development because there are many different ways to accomplish the same task and the experience of each can vary wildly.
Arduino code:
#include <Arduino_LSM6DS3.h> // This library reads sensor values from the Intertial Measurement Unit (IMU).
#include "MadgwickAHRS.h" // This library generates Euler angle measures from the IMU.
#include <CapacitiveSensor.h> // This library enables capacitive touch sensing between two pins
// Initialize a Madgwick filter:
Madgwick filter;
// Set sensor's sample rate to 104 Hz to improve accuracy of Euler angle interpretations by Madgwick library
const float sensorRate = 104.00;
// buffer to smooth readings
int xBuffer[3];
int yBuffer[3];
//float accel;
// Tap button
int buttonBufferX[10];
int buttonBufferY[10];
int buttonBufferZ[10];
long buttonTime = millis();
long tapCount = 0;
int tapStatus = 0;
float tapThreshold = 1; // tap only sensed outside of this value
int tapTriggerDelay = 100; // delay before a new tap sensed
// LED indicator info
int ledPin = 17;
int brightness = 1;
int fadeAmount = 4;
int pulseStatus = 0;
// Capacitive Touch info
CapacitiveSensor cs_8_5 = CapacitiveSensor(8,5);
bool touchTrigger = false;
int touchBuffer[10];
void setup() {
// turn off autocalibrate on channel 1 - just as an example
cs_8_5.set_CS_AutocaL_Millis(0xFFFFFFFF);
// configure the serial connection:
Serial.begin(9600);
// Initialize the IMU sensor
if (!IMU.begin()) {
Serial.println("Failed to initialize IMU!");
while (true); // halt program
}
// Start the filter to run at the sample rate
filter.begin(sensorRate);
Serial.println("IMU initialized!");
}
void loop() {
long capTotal = cs_8_5.capacitiveSensor(30);
float adjustTouchThreshold = 0;
// add values to buffers
for (int i = 9; i > 0; i--){
adjustTouchThreshold = adjustTouchThreshold + touchBuffer[i];
touchBuffer[i] = touchBuffer[i-1];
}
adjustTouchThreshold = adjustTouchThreshold/9;
touchBuffer[0] = capTotal;
capTotal = adjustTouchThreshold;
// touch sensor add to tap
if(capTotal > 250 && !touchTrigger){
// tap happening
if(millis()-buttonTime > tapTriggerDelay){
tapStatus = 1;
touchTrigger = true;
pulseStatus = 1;
tapCount++;
//Serial.print("tap ");
//Serial.println(tapCount);
buttonTime = millis();
} else {
tapStatus = 0;
}
}
else if(capTotal < 230 && touchTrigger){
touchTrigger = false;
}
pulseLED();
// Local variables for orientation data from IMU
float aX, aY, aZ;
float gX, gY, gZ;
const char * spacer = ", ";
// Local variables for orientation from Madgwick library
float roll, pitch, heading;
// Try to read the IMU sensor
if (
IMU.accelerationAvailable()
&& IMU.gyroscopeAvailable()
) {
IMU.readAcceleration(aX, aY, aZ);
IMU.readGyroscope(gX, gY, gZ);
// Update the filter, which computes orientation
filter.updateIMU(gX, gY, gZ, aX, aY, aZ);
// Print the heading, pitch and roll
roll = filter.getRoll() + 180; // 180 to -180 y leaves x unchanged
pitch = filter.getPitch() + 90; // 90 to -90 x leaves y unchanged
// adjust the sensor values to be closer to 10bit analog reads
//aX = (aX + 1)*512;
//aY = (aY + 1)*512;
// do some smoothing to reads
pitch = (xBuffer[0] + xBuffer[1] + xBuffer[2] + pitch)/4;
roll = (yBuffer[0] + yBuffer[1] + yBuffer[2] + roll)/4;
// save new value to buffer
xBuffer[2] = xBuffer[1];
xBuffer[1] = xBuffer[0];
xBuffer[0] = pitch;
yBuffer[2] = yBuffer[1];
yBuffer[1] = yBuffer[0];
yBuffer[0] = roll;
// If there is a short peak print "tap"
// accel = aX + aY + aZ;
// Serial.print(aX);
// Serial.print(',');
// Serial.print(aY);
// Serial.print(',');
// Serial.println(aZ);
checkTap(aX, aY, aZ);
// print the X rotation:
Serial.print(pitch);
Serial.print(",");
// print the y rotation:
Serial.print(roll);
Serial.print(",");
// tap status
Serial.println(tapCount);
//Serial.println(capTotal);
}
}
void pulseLED(){
// if a tap has happened trigger an LED pulse
if(pulseStatus == 1){
// Write brightness to LED
analogWrite(ledPin, brightness);
brightness = brightness + fadeAmount;
// Reverse the direction of the fading at the ends of the fade
if(brightness > 240)
{
fadeAmount = -fadeAmount;
}
if(brightness < 0)
{
fadeAmount = -fadeAmount;
pulseStatus = 0;
brightness = 0;
}
//Serial.print("fading LED to ");
//Serial.println(brightness);
delay(2);
}
}
void checkTap(float x, float y, float z){
float adjustTapThresholdX = 0;
float adjustTapThresholdY = 0;
float adjustTapThresholdZ = 0;
// add values to buffers
for (int i = 9; i > 0; i--){
adjustTapThresholdX = adjustTapThresholdX + buttonBufferX[i];
buttonBufferX[i] = buttonBufferX[i-1];
}
for (int i = 9; i > 0; i--){
adjustTapThresholdY = adjustTapThresholdY + buttonBufferY[i];
buttonBufferY[i] = buttonBufferY[i-1];
}
for (int i = 9; i > 0; i--){
adjustTapThresholdZ = adjustTapThresholdZ + buttonBufferZ[i];
buttonBufferZ[i] = buttonBufferZ[i-1];
}
adjustTapThresholdX = adjustTapThresholdX/9;
adjustTapThresholdY = adjustTapThresholdY/9;
adjustTapThresholdZ = adjustTapThresholdZ/9;
// Serial.print(accel);
// Serial.print(',');
// Serial.print(adjustTapThreshold - 1);
// Serial.print(',');
// Serial.println(adjustTapThreshold + 1);
buttonBufferX[0] = x;
buttonBufferY[0] = y;
buttonBufferZ[0] = z;
// Look for short peak in buffer
// Check that newest reading is not a peak on X
if(buttonBufferX[0] < adjustTapThresholdX + tapThreshold && buttonBufferX[0] > adjustTapThresholdX - tapThreshold){
bool tapTriggered = false;
// Check for peak above high threshol or below low threshold
if(buttonBufferX[1] > adjustTapThresholdX + tapThreshold || buttonBufferX[1] < adjustTapThresholdX - tapThreshold){
for (int i = 2; i < 10; i++){
if(buttonBufferX[i] < adjustTapThresholdX + tapThreshold && buttonBufferX[0] > adjustTapThresholdX - tapThreshold){
tapTriggered = true;
}
if(tapTriggered && millis()-buttonTime > tapTriggerDelay){
tapStatus = 1;
pulseStatus = 1;
tapCount++;
//Serial.print("tap ");
//Serial.println(tapCount);
buttonTime = millis();
} else {
tapStatus = 0;
}
}
}
}
// Check that newest reading is not a peak on Y
if(buttonBufferY[0] < adjustTapThresholdY + tapThreshold && buttonBufferY[0] > adjustTapThresholdY - tapThreshold){
bool tapTriggered = false;
// Check for peak above high threshol or below low threshold
if(buttonBufferY[1] > adjustTapThresholdY + tapThreshold || buttonBufferY[1] < adjustTapThresholdY - tapThreshold){
for (int i = 2; i < 10; i++){
if(buttonBufferY[i] < adjustTapThresholdY + tapThreshold && buttonBufferY[0] > adjustTapThresholdY - tapThreshold){
tapTriggered = true;
}
if(tapTriggered && millis()-buttonTime > tapTriggerDelay){
tapStatus = 1;
pulseStatus = 1;
tapCount++;
//Serial.print("tap ");
//Serial.println(tapCount);
buttonTime = millis();
} else {
tapStatus = 0;
}
}
}
}
// Check that newest reading is not a peak on Z
if(buttonBufferZ[0] < adjustTapThresholdZ + tapThreshold && buttonBufferZ[0] > adjustTapThresholdZ - tapThreshold){
bool tapTriggered = false;
// Check for peak above high threshol or below low threshold
if(buttonBufferZ[1] > adjustTapThresholdZ + tapThreshold || buttonBufferZ[1] < adjustTapThresholdZ - tapThreshold){
for (int i = 2; i < 10; i++){
if(buttonBufferZ[i] < adjustTapThresholdZ + tapThreshold && buttonBufferZ[0] > adjustTapThresholdZ - tapThreshold){
tapTriggered = true;
}
if(tapTriggered && millis()-buttonTime > tapTriggerDelay){
tapStatus = 1;
pulseStatus = 1;
tapCount++;
//Serial.print("tap ");
//Serial.println(tapCount);
buttonTime = millis();
} else {
tapStatus = 0;
}
}
}
}
}
