Livecoding as Human Interface¶
Citlali Hernández aka Turbulente
Xavi Domínguez aka er Lunni
Warring
The content of this section is a copy of a working document that is constantly being updated. To see the latest updates visit here: https://hackmd.io/@fablabbcn/HJvhtz0oV
Main Goals for today¶
- Understand basic concepts to prototype an interface
- Navigate the immensity of possibilities, tools, technologies, protocols and languages to design and implement interfaces.
- To learn about the creative process of a transdisciplinary artist.
- Discovering, understanding and practicing livecoding.
- Tinkering Human Interfaces
Interface¶
Designing an Interface Application¶
Key questions¶
- Your interface application in 1 sentence Ex: A user interface that shows the value of a sensor in a website
- What is my budget ?
- How much time do I have?
- Is it a proof of concept ? Is it a prototype ? Is it the final application ?
- What technical knowledge do I already have? What other profiles do I need on my team?
Key Ingredients¶
Basic Concepts¶
Languages¶
Protocols¶
A protocol is a set of rules that governs how data is transmitted and received over a network. Internet protocols are the rules and standards that enable communication between devices over the internet.
Websocket¶
WebSocket is a communication protocol that enables two-way communication between client and server over a single, long-lived connection. It allows for real-time data transfer between the two endpoints without the need for frequent HTTP requests. WebSocket is commonly used in applications that require real-time updates, such as online gaming, chat applications, and stock market tickers.
OSC - Open Sound Control¶
Open Sound Control (OSC) is a protocol for networking sound synthesizers, computers, and other multimedia devices for purposes such as musical performance or show control
Tools, technologies, protocols and languages to design and implement interfaces¶
Projects¶
Coding Robots - Families - CosmoCaixa¶
Ironskulls ( Azul Petroleo)¶
Smart Bike Project¶
Reference: http://fabacademy.xavidominguez.com/final.html Language:Arduino, App Inventor,Javascript, html5, php Data Interfaces: MySQL, TinyDB Networking Communication: Bluetooth JBtek-HC-05 Wireless
K3 - Parlant amb la Júlia¶
Examples¶
SC + Processing ( Serial )¶
Processing + Arduino ( Serial Communication )¶
Input: Arduino with Mois Sensor Output: Processing + Arduino LED Communication: Serial Wired
Processing Code¶
import processing.serial.*;
Serial myPort; // The serial port
PFont f; // STEP 1 Declare PFont variable
float r = 30;
float turn = 0;
float goldenR = (1 - sqrt(5))/2;
float h = 100;
int value = 1024;
boolean noWater = false;
String plantState = "";
void setup() {
//fullScreen();
size(700, 1000);
background(0);
f = createFont("Arial",16,true); // STEP 2 Create Font
//printArray(Serial.list());
// Open the port you are using at the rate you want:
myPort = new Serial(this, Serial.list()[0], 9600);
}
void draw() {
// Read String from serial
if (myPort.available() > 0) {
String inBuffer = myPort.readStringUntil(13);
if (inBuffer != null) {
inBuffer = inBuffer.replace('\n', ' ').trim();
value = int(inBuffer);
}
fill(0);
rect(80, 80, 500, 150);
textFont(f,16); // STEP 3 Specify font to be used
fill(0,100,0); // STEP 4 Specify font color
if(value>900){
plantState = "Plant NEEDS WATERING";
noWater=true;
}else{
noWater = false;
if((600<value)&&(value<900)){
plantState = "Plant is DRY";
}else{
plantState = "Plant is HUMID";
}
}
textAlign(CENTER);
text(plantState,180,100); // STEP 5 Display Text
}
turn += goldenR;
float alphaBlatt = random(100);
float alphaRand = random(100, 150);
translate(width/2, height/2-h);
noFill();
if(noWater){
stroke(120,120, 120, 120);
}else{
stroke(0, random(255), random(150), random(100));
}
bezier(0, 0+(height/2)+h, 0, random(0, 300), 0, random(0, 300), random(-300, 300), random(0, 350));
stroke(0, alphaRand);
rotate(turn);
if(noWater){
fill(120, 120, 120, 0);
}else{
fill(random(200, 255), 0, random(150, 255), alphaBlatt);
}
ellipse(0, 0, 50, random(100, 300));
fill(0);
ellipse(0, 0, 2*r, 2*r);
}
void keyPressed() {
if (key == 's') {
myPort.clear();
myPort.write("1");
}
}
Arduino Code¶
int val = 0;
void setup()
{
Serial.begin(9600);
pinMode(A0, INPUT);
pinMode(9, OUTPUT);
}
void loop()
{
val = Serial.read();
Serial.println(val);
Serial.flush();
int mois = analogRead(A0);
Serial.println(mois);
if(val>=1){
digitalWrite(9, HIGH);
}else{
digitalWrite(9, LOW);
}
delay(2000);
}
A-Frame + NodemCU + WebSockets¶
APP Inventor + Bluetooth HC08 + Sensor¶
APP Inventor¶
Arduino Code¶
#include <SoftwareSerial.h>
SoftwareSerial BT1(4,2); // RX, TX recorder que se cruzan
void setup()
{ Serial.begin(9600);
Serial.println("Enter AT commands:");
BT1.begin(9600);
pinMode(A0, INPUT);
pinMode(9, OUTPUT);
}
void loop()
{ int mois = analogRead(A0);
if (BT1.available())
Serial.write(BT1.read());
if (Serial.available())
{ char c = Serial.read() ;
Serial.print(c);
BT1.write(c);
}
BT1.print(mois);
delay(1000);
}
Grasshopper firefly¶
Acces to the repo with the files
Simple serial writte¶
Using Grasshopper to Control a Pan / Tilt Servo from Andy Payne on Vimeo.
Simple serial read¶
Image processing¶
Kinect processing¶
Grasshopper 3d + Firefly hand tracking from nachetz on Vimeo.
Generative Design¶
Reference: http://www.generative-gestaltung.de/2/ Languange: p5.js
IoT Devices with IFTTT¶
Reference: http://iot.futurelearningunit.com Language: Arduino Device Interfaces: IFTTT Data Interfaces: Google Sheets Networking Communication:WiFI ESP8266
Python¶
Example file: here Language: Python, web application Device Interfaces: Serial Data Interfaces: Pandas, CSV
Node-Red¶
User Interfaces: Node-Red Device Interaces: MQTT
The super freak option¶
User interfaces: blessed-contrib Language: ascii/ansi art and javascript
Livecoding¶
MDEF Task¶
Task
Experiment with the tools shown in class. And choose the ones that best suit your research project.