Controlling Music In Max Using Gadgeteer

This tutorial should give a good step by step overview of programming Gadgeteer with some simple code using Visual Studio and then using Gadgeteer as an interface to Max to control sample playback.
All software examples for the tutorial can be downloaded in one zip file by clicking the following link: SerialGadgeteerMax

INTRODUCTION

I am currently building a musical instrument based on Gadgeteer and will share the development of this over the coming months. I plan to test the existing modules for Gadgeteer and build my own. Gadgeteer will form an interface to Max MSP software.

Background
I plan to use Gadgeteer to interface to Max MSP. This work is an extension of my final project for the MRes. Digital Media course. For this project I created an instrument based on the Arduino 2650 MegaMini – this development board was connected to the following sensors:
4 Push On/Off Buttons, 3 Momentary Buttons, 2 Pressure Sensors, 4 Potentiometers, 1 Infrared Distance Sensor,
The instrument employed a wireless link to Max 5.0 via Xbee radios for two way serial communication. Data from the sensors was transmitted to Max. Feedback from Max was received and indicated on the instrument via leds and text on an LCD display. The idea was that one set of controls would control many different parts of the Max patch. The leds and LCD screen would indicate which parts of the Max patch were activated.

Switch case statements in the Arduino code responded to information recieved over serial. Another configuration of the instrument was also developed using a PING ultrasonic rangefinder and a 3 axis Accelerometer, 2 small pressure sensors and 3 push on/off buttons.

An image of the instruments can be seen here:
http://dm.ncl.ac.uk/benfreeth/files/2011/09/Picture-12.png

and further information on the sensors, Xbee communication and code here:
http://dm.ncl.ac.uk/benfreeth/

Getting Max and Gadgeteer Talking
In order to control music in Max MSP with the sensors and touch screen modules of Gadgeteer the two must be able to talk to each other. The .NETmicroframeworks software running on the Gadgeteer mainboard must talk over serial to the Max MSP software running on a laptop. This first post shows getting Max and Gadgeteer talking to each other, setting up event handlers to detect button presses and releases, sending data to Max via Serial, reading this data into Max and then doing something with it. In this case simple sample triggering.

The steps for this are create the code for Gadgeteer, build the gadgeteer interface, create the code in Max, link the two and test.
The software solution for Visual Studio is included in the zip file at the start of this tutorial.

1.0 – Gadgeteer Code
Open Visual Studio… Click New…Project…

Select a new .NET Gadgeteer Application template…Give this a suitable name…Click Ok.


1.1 Program.Gadgeteer Designer View

Switch to the Program.Gadgeteer designer view by clicking the Program.Gadgeteer tab shown below:

This brings up the Program.Gadgeteer designer view. Shown below:

To begin with only the Fez spider main board is shown. Click on the Toolbox tab on the far left of the screen.

A push button will be used to control triggering of samples in Max MSP so we need to add a push button. Click and drag a button onto the main designer window. Next click in the yellow box on the button module. The possible Sockets that will accept the button module are highlighted in Green as shown below:

Click on the yellow Socket on the button module and drag the patch chord to Socket 4 on the main board. The connection turns blue to indicate that the module is linked to that socket on the mainboard as shown below:

Visual Basic then automatically generates code stored in a file called ‘Program.generated.cs’ this is found in the Solution Explorer on the right side of the screen. Click on the Solution Explorer and then on Program.gadgeteer to reveal the Program.generated.cs file as shown below:

The Program.generated.cs initializes the main board and the button. Here is the code:

This code makes an actual instance, or object, of the FEZspider class and of the Button class. The two objects are declared and allocated memory. The two objects are instantiated. Mainboard and button are the identifiers (name) for the object or instance of each class. The button name is written in Camel notation – the very first letter is lowercase.
The keyword ‘new’ allocates memory for the object.

1.2 Physically Connect The Button
Physically connect the GHI Electronics button module “Button 1.2″ (not “Button 1.3″ – this one does not work correctly yet) using one of the ribbon connectors as shown below:

Physically Connect The Serial Module To Fez Spider main board Socket 8. As Shown Below:

Note: do NOT connect the GHI Electronics Serial Module in the Program.Gadgeteer designer view in Visual Studio. The code for the Serial Module will be set up in the Program.cs code.

1.3 Serial Communication
Ensuring that you are editing the Program.cs code (if not click on the program.cs tab) create a variable for the serial connection:

The ‘GT.Interfaces.Serial’ part of the code represents a ‘serial communication interface port’
The variable has the “identifier” (an arbitary name to identify it) in this case it is ‘mySerial’.

Camel Vs. Pascal Notation
As an aside, Variables in .NETmicroframeworks are written using Camel notation. This makes it easier to differentiate between variables and methods and classes are written using Pascal notation (first letter and second letter are uppercase, there is no gap between the two).

1.4 Event Handler Delegates
The Program Started method is invoked or called by the system after the hardware has booted up and the event dispatcher is running. It is a good place to set initial states in modules, show welcome screens for displays and set up event handlers. Lets set up two event handlers. One for when the button is pressed and one shown for when it is released.
This is shown below:

To create the Event Handlers type button.ButtonPressed +=
then press the tab key twice. The code will auto complete.

and button.ButtonReleased +=
tab tab

These create new instances of the Button.ButtonEventHandler class.
The word ‘new’ allocated memory and initializes the Button event handlers.
The += operators explanation:
= assignment operator
+ increment
The += operator combines addition with assignment i.e. increment and assign.
In .NETmicroframeworks == means equals not = (the assignment operator).

The Debug.Print(“progStarted”); shows that the program has executed this far by printing “progStarted” in the Debug window when the program runs.

1.5 Setting Up The Gadgeteer Socket To Handle Serial Communication
Gadgeteer Sockets that can handle serial communication are labeled on the Fez Spider mainboard with a U. Sockets 4, 8, 9, and 11 can be used. U stands for UART – Universal Asynchronous Receiver Transmitter.
More information on UART can be found here:

http://en.wikipedia.org/wiki/Universal_asynchronous_receiver/transmitter

Create a new instance of the Serial class which defines the Gadgeteer Socket the serial module will be connected to. This is stored in the variable mySerial (which represents the serial port). This initializes an instance of the Serial class. Gadgeteer socket 8 will be used.
This is shown below:


The type of serial communication for Gadgeteer needs to match the type set up in Max.

1.6 Serial Settings In Max MSP
Leaving Visual Studio and looking at Max MSP software.
The serial object is used to read and write from the Serial buffer.
The serial object in Max looks like this:

From the Max serial object help file:
“Arguments are:
1. letter abbreviation for port or port name
– a is the first logical port in your computer, b is the second, etc. (default a);
2. baud rate (default 4800);
3. data bits (default 8);
4. stop bits (default 1);
5. parity (default 0 (no));”

The baud rate will be increased to 9600 in Max.

The order is slightly different – in Gadgeteer baud, then parity, then stop bits. These all need to match to ensure correct communication between the two. The Gadgeteer Serial settings and the settings in the Max serial object now match.

1.7 Open The Serial Port
Returning to Microsoft Visual Studio. Still in the Program Started method type the following to open the Serial Port and begin serial communication:

When the program runs and this statement is executed the code opens the serial port.

1.8 Invoke the Event Handlers!!!
In the following lines of code the Program Started method invokes the event handler methods. One method calls or invokes another method. The word void when used in the following statements means that the called method (here the event handler) will not return a value to the Program Started method.

After the button released event handler method has been called by the program started method the code will print “buttonreleased” to the debug window and send the string “4″ to the serial port on Gadgeteer mainboard Socket 8 if a button release is detected. If not it will do neither.

The button pressed event handler method works in exactly the same way.
Comment out the two statements:

throw new NotImplementedException();

To comment them out add two foward slashes at the beginning of each statement so they read:

// throw new NotImplementedException();

When they are commented out they are ignored by the complier and are shown in the code as green text.

1.9 Attach Gadgeteer to the Computer
Attach the Gadgeteer board via the USBclientDP module (connected to Socket 1 of the main board)  as shown below.

Click on the Solution Explorer tab on the right of the Visual Studio window

Double click Properties.

Click the .NET Micro Framework tab and check that Device: is set to EMX_Gadgeteer, as shown below:

1.9.1 Compile the code
Click the play button next to Debug.

The code will compile and upload to the Gadgeteer board.

1.9.2 The Output window
If the Output window is not already shown press Ctrl Alt O (Control alt and the letter o).

Once the code has compiled and loaded onto the Fez Spider main board the debug window should read: progStarted
As shown below:

This indicates that the code has run without breaking up until just after the event handlers were declared. progStarted is the result of the debug statement included in the code:
Debug.Print (“progStarted”);

1.9.3 Gadgeteer Button
Press the Gadgeteer button. When it is pushed in (pressed) – “buttontoggle” will be printed to the Output window in VisualStudio. When pressed once the led on the button module will light up red. When pressed a second time it will switch on
When the button is released a “buttonreleased” will be printed to the Output window in VisualStudio. These are shown below:


Step 2 – Serial Monitor

The Fez Spider main board of Gadgeteer is now programmed with the code created in Visual Studio.
To check the information that Gadgeteer is sending by serial communication via Socket 8 connect the GHI Serial Module via USB to a USB port on your computer (I am running Windows on my Mac Book Pro as a Virtual Machine on Virtual Box, Visual Studio runs on this virtual machine). This is shown below:

Using the GHI button module, press and release the button while looking at the Serial Module.
When the button is pressed the RX led on the Serial Module flashes green. This is potentially confusing as you would assume that the data is being transmitted. I think of it as having received it from the Fez Spider main board via Socket 8 on its way out of the Serial module via the USB cable…

To check what is being sent out of the Serial Module I used a Serial Terminal program created in Processing that I have been using to program Xbee radios.
This program is included in the zip file at the start of this tutorial.
To run this program open the SerialTerminalProgram folder, then Xbee_terminal5 folder, then the applet folder and double click on:
Xbee_terminal5.jar
This is shown highlighted in blue with a yellow dot below:

The program checks for opens a connection with the Gadgeteer Serial module via USB.
It shows the serial port that Gadgeteer is connected to. This is shown below as /dev/tty.usbserial-A900F4ZL

The terminal program can be used to send data via serial communication to the Gadgeteer and recieve data via serial from the Gadgeteer.
Type something into the Terminal and press return, while watching the GHI Serial Module. The red TX led on the GHI Serial Module will flash indicating that data has been recieved via USB.
Now press the Gadgeteer button. You should see the following appear in the window:
A 3 when the Gadgeteer button is pressed and a 4 when it is released. This is shown below:

These correspond with the buttontoggle and buttonreleased in the Output window of Visual Studio.
This test confirms that when the button is pressed the buttpressed event handler method has been called by the program started method, the code has printed “buttonreleased” to the debug window and sent the string “3″ to the serial port on Gadgeteer mainboard connected to Socket 8, this data has then been sent via usb to the computer where the serial buffer has been read by the terminal program and the string “3″ displayed in the window of the terminal program.

Great! Now that gadgeteer is sending data via serial it can be used to control something with this data. It can be used as a physical interface to Max MSP.

Step 3 Max 5.0

3.1 Introduction
I am currently working with Max 5.0.
Max is a visual programming language for music and multimedia.

More information and a trial version of Max can be downloaded from the cycling 74 website:
http://cycling74.com/products/max/

More info on Max:
http://en.wikipedia.org/wiki/Max_%28software%29

3.2 Starting Max
Make sure that the terminal program used to test the serial connection between Gadgeteer and the computer has been closed. Start Max. Open the file:
Serial_In_PressRelease150212b.maxpat

This is found in the zip file at the start of this tutorial.

The Max patch will look like this:

3.3 Configuring The Serial Port

Under the Serial Port Section of the .maxpat file Click on the Max button object. It is to the left of the comment “Print a list of serial ports to the max window”. Clicking on it will do this. Press Command M. This will bring the Max window to the front. In the Max Window a list of serial ports will be displayed.

From the Terminal program used in the last section we know that the port gadgeteer is connected to is serial port a:usbserial-A900F4ZL.
Double check that in the Serial Object section the serial object is set to:
serial a 9600 8 1 0

This is shown here:

If not press Command E to unlock the Max patch. Double click in the box that says serial a 9600 etc. and type the correct settings.

Returning to the Serial Ports section click on the drop down box and select the serial port that gadgeteer is connected to (on my computer usbserial-A900F4ZL).

Then click the toggle box (Shown to the left of the comment “Switch on serial communication”. The toggle box will display an X as shown below:

This will have opened the serial communication link between Max and Gadgeteer. Check that this is the case by pressing Command M to open the Max Window again.

You should see the following when the Gadgeteer button is pressed:

data 51
data 10

and when released

data 52
data 10

These are shown below:

3.4 The ASCII Look up table
Where have the 3 and 4 gone? They are right there except they are shown as ASCII.
Using an ASCII look up table from: http://www.ascii.cl/
You can see that 3 is represented by 51 in ascii and 4 by 52.
Max interprets the string sent by Gadgeteer as ascii.
The 10 represents a Line Feed. This means move to a new line.
It seems that Gadgeteer is sending out a new line after each string representing a button press or release. This helps to keep the data separate.
Referring back to the Serial Object section of the Max patch. The ‘print status’ and ‘print data’ objects are responsible for showing this information in the Max window.

3.5 Utilising The Data
Using the ‘sel’ object the 51 and 52 sent by Gadgeteer can be detected. The ‘sel 51′ object sends out a bang from it’s left output when a 51 is sent from Gadgeteer to Max via serial. Another way of stating it would be when Max detects the number 51 in the serial buffer (because the string 3 has been sent by Gadgeteer) the ‘sel 51′ object sends a bang out of its left outlet.
This bang can be used to switch on a virtual led in Max in the Gadgeteer Button Led section of the Max patch shown below:

3.6 Triggering A Sample
The button press can be used to trigger a sample.
First load a sample into the buffer. In Sample Buffer1 click on the ‘read’ message. This will open a dialogue box allowing the selection of an audio file from your hard drive.
Load a suitable .aiff file.
The Sample Buffer 1 section in the Max Patch:

Next, make sure the gain sliders are turned up. These are in the Audio Level Sample 1 section of the Max patch. Click and drag the sliders to increase the volume. This is shown below:

Click on the ‘ezdac~’ object. This looks like a speaker and is found in the Audio out section of the Max patch. As shown below:

This will start audio output from Max.

Now when the Gadgeteer button is pressed the sample in Sample Buffer 1 (the sample is actually stored in the buffer object called ‘playsample’) will be played by the ‘groove~’ object and be heard over the speakers on the computer. The bang sent from the ‘sel 51′ object and diverted by the ‘ubutton’ object will trigger the ‘startloop’ and ‘stop’ messages linked to the ‘groove~’ max object.

The ‘groove~’ object and ‘startloop’ ‘stop’ messages in the Play / Stop Sample 1 section of the Max patch:

The ‘ubutton’ object in the Gadgeteer Button Led section of the Max patch is the small red square it outputs bangs linked to the messages 1 and 0. The message 1 triggers the ‘startloop’ message in the Play / Stop Sample 1 section of the Max patch.

The same procedure can be used to set up another sample to be triggered by the Gadgeteer button release – using Sample Buffer 2 etc.

4.0 Conclusion
This tutorial should give a good step by step overview of programming Gadgeteer with some simple code using Visual Studio and then using this as an interface to Max to control sample playback.

Now that this is working I plan to make this connection wireless using Xbee radio technology based on the instruments I built as part of my MRes. Digital Media seen here:
http://dm.ncl.ac.uk/benfreeth/2011/08/25/arduino-mega-skull-kontrol-prototype/

and programming the Xbee’s:
http://dm.ncl.ac.uk/benfreeth/2011/08/26/xbee-radio-programming-the-new-radios/

Feel free to email me comments or questions about this tutorial:
b.c.freeth@ncl.ac.uk

This entry was posted in Tutorials. Bookmark the permalink.