garageCube

[Grovenstien]

Hello!

Um i have a great big multitouch screen that i built and i am currently using Touche to track infrared blobs and create TUIO messages which are sent to a mouse emulator called BBtuio.

Heres the link to my thread and the demo http://nuigroup.com/forums/viewthread/4595/

This is however only a single point interface and i wondered if any one in here had managed to intergrate TUIO with modul8 perhaps a plugin using pyglt or pymt? Any hoo it would be very cool to have multitouch support in modul8.

If anyone has any ideas then please let me know!

Thanks

[ilan]

I am only familiar with TUIO from the post on CDM from today. Modul8 supports MIDI that allows to you send more then two messages to modul8 at a time and it also support OSC that allows you to use modul8 with the multi-touch capabilities of the Lemur and iPhone/iPod Touch apps like TouchOSC.

So theoretically an interface can be written to allow TUIO to communicate with modul8.

[zoophar]

Hello Groovenstein,
nice table !!! isn'it ?
that's a really nice job!!! coongratulations

some time ago, we tried, (onegop & me) to send some tuio midi signals to m8...
we tried all but used Processing.client version...
the aim was to interact "directly" from TUIO to M8... using the fiducial symbols... we were thinking: "what we hold is what wee see"...
the advantage was also that we could use it on two machines:
one with m8, and the other with P5 TUIO and the table...

I saw in you demo that you are catching the mouse...
Our reflection/test was different in the way that:
• Each TUIO.fiducial symbol corresponds to a M8.knob (or M8.keyword).
• The coordinate of this TUIO.ficucial corresponds to the value of the M8.knob.(or M8.keyword)

The big probS were to get clean messages from tuio (because of a crappy table, made in 5 min)..

the refresh rate of the webcam via-tuio was not big enough (15frames/sec)...

The ligth inside the box was so bad and not infrared, so we could only do it with a particular ambient ligth, only from 1:00PM to 4:30PM...

BUT once we had something that could see symbols a bit better than a mole,
we started M8 and mapped the midi messages outcomming from processing
(because the TUIO "core" app version we used had not the midi working) rigth int M8.

it worked,and we could modify the X,Y position of a layer in cooordination with the x,y position of a physical marker on the "crappy-table"... using midi... this part was really simple

later on, i tried something with the fingers... and tougth about something like a mirror of m8's GUI in processing:
using ControlP5's lib to make a clone of the M8.interface.
each slider or knob are binded from P5 to M8...

PRIVATE SCREEN (table): processing/TUIO, put your finger on a slider and modify it's value. this value will be sent to the corresponding slider/knob/keyword in modul8 via midi.

PUBLIC SCREEN: see the gig

we didn't go further because the hardware table to hard to make at home..:
(ligth parameters (inbox / outbox:ambient / outbox:ligthshow), transportation, sustainability, size , tools etc, etc ...)note also that it was on PPC computers... not intel...

But the proof of concept worked, and we had a lot of fun...
i think the harder part is not make m8 see the signals but more the hardware table with tuio...
and you have one that looks so GRRRRRRRRR(eat)!!!!!!

[Grovenstien]

Hi Zoophar!

Thanks for the reply and the praise!

I am interested in knowing more, how did you manage to get the TUIO signals through to control M8? I am a complete idiot when it comes to code so do you have any links or tutorials that could help?

Thanks for your interest.

Cheerio!

[zoophar]

Hello Grovenstein
i'm sorry if the infos are incompleete, i have the memory as big as a redfish... and the tiny-bit code isn't up to date anymore...
but well if you want to spend some time on it:

do you have MidiMonitor? if not, go GET IT !!!

The harder part is to have the TUIO signals going out of the client in MIDI, properly...

in my dreams: The TUIO interface sends infos to the processing client...
In processing(P5), there are knobs and sliders that are disposed like M8's inteface. Each sliders in M8 has it's brother in P5, they talk via midi.
For this, i used 3 librairies:
> TUIO
> CONTROLP5
> PROMIDI
but maybe MIDIBUS
is a simpler librairy to use for MIDI (but requires java sound synthesizer)

P5 is projectioned on the screen of the table, you can "touch" the sliders and interact with them.

when the value of the slider changes, it sends the midi message corresponding to the slider.

there has to be a trick done to say:
When the TUIOpointer is in a place that coresponds to the size/place of the slider,
the Midi value of this slider is equal to the Y of the pointer
(hahem'... substract screen size,scale the Y/X of the slider blah.blah...)

i did something like:

Code: Select all

   if(tcur.getScreenX(width) > x1 && tcur.getScreenY(height)> y1 //x
  && tcur.getScreenX(width) < x1+layerW && tcur.getScreenY(height)< y1+layerH){//y=
    value1 = tcur.getScreenX(width)-10;
 
    midiOut = midiIO.getMidiOut(0,0);
    int n = (int)round(map(mouseX,40, 80, 67, 70));
    //float n = map(mouseY,40, 80, 90, 100);
    note = new Note(1 ,n, 50);
    midiOut.sendNote(note); 
  }

The reality is not as nice as my dream since i don't have a table anymore (i never had what we can really call: table...)

The code is not up to date, the program is really very basic: only one slider mapped...
nothing more or less than a dirty skectch copy-pasted from the examples, but if you want to get an eye on it, i can send it to you...
just give me your e-mail...

Once you see some signals in MidiMonitor, it's so easy to map these signals into M8... fingers in the noise !!!!

See you

[Grovenstien]

Hi Zoophar,

Thanks again for your continued enthusiasm! I see what you are saying but is this not just an elaborate way to gat a single point interface?

Will having 10 faders that match the ten vid channels in M8 be able to be faded at the same time with 10 fingers! like you can with a hardware controler!?

I am now thinking of building a flash app that looks like M8 and trying it that way round.

It seems however you try ad do this it requires a long chain of format conversion to get to midi! Surley it would be better to try and intergrate OSC or TUIO into M8! Ahwell if i knew how to do that i would def give it a go!

Cheers

[zoophar]

Hello Grovenstein,
well not exacly... it's only unfinished (better say just started)

what is my sketch capable of? what are the results:
> on processing:
i can see "all" my fingers (i've had a very small table so "all" means 4 fingers).
all those pointers can interact with the sliders that appears in the window. at the same time (ex: 3 fingers on 3 sliders changes 3 values)
i just created 3 sliders for now

> MIDI in processing: (i'm a lazy potato)
only one slider is capable to send midi
(just have to write it for all sliders)

Once this MIDI signal (the only one for now) is sent to IAC driver or network driver(*), Midi Monitor will see it and show the informations of this midi signal.

> MIDI in Modul8:
activate the midi mappig in the menu bar and select a knob/or button to be controlled by MIDI. while keeping the focus on M8, send the wanted midi signal with TUIO (TUIO sends tuio_signals to P5 who sends the midi to Modul8).
the M8 button you selected is now tagged with the reference of the Midi Note (or ctrl for sliders) and will react to this signal.

i don't have a table anymore... so i can only simulate...

are you still intersted?
we could share some capabilities...
but this might be a bit long and maybe laborious to get it working,
because we will have to do it in "blindside"( me: code and tuto; you: test and rapport)...

(*)IAC driver or network driver
you need 2 projection in order to project the interface on your table and the result of M8 to the public, (2 grafic cards(<< not tested) or 2 computers are needed)

PS: i don't think flash can send midi...
M8 integrates OSC arlready but i don't know how to do it

best regards

[Grovenstien]

Hi Zophar,

Cool so it is multitouch! Just checking.

As for the OSC to midi i think i have solved that problem with OSCulator http://www.osculator.net/wp/ you may well have seen this already. So all we need now is an interface that is multiouch capable and can send OSC messages to OSCulator P5 looks good, but i want a more star trek feel to it! Like OSCtouch http://hexler.net/software/touchosc only bigger! Also check out the Lemur having animated physics driven wobbley input thingy's would be great!

I know all about midi mapping and i am quite capable of running the touch table indipendently from the show! I can either use two Macs networked together or see if i can get my new Mac mini to output the interface to the Table and the show to the MX50 via Svid, Whatever this is not a problem!

You are right about Flash it cannot send Midi however it can send OSC so OSCulator blah blah!

Id love to have the code and colaborate in testing P5 with the setup my mail is alex@v-lux.net

Cheers

[Grovenstien]

Hi Zophar,

To my amasement multitouch capable apps in .swf format work with modul8 2.5.8. This is awsome news as it means i can mix up some of the nui flash demos with M8.

Wo hoo!

[zoophar]

Hello Grovenstein,
Dd you tested what i sent to you and did it help?
i posted this application to download for other people interested on this page

Hope you get what you want

[Grovenstien]

Hi Zoophar,

Thanks for sending me the app. Very nice icon! I haven't really had much time to properly play with it but i can make a few observations as follows:

Launches fine and recieves TUIO messages from Touche(tracker)with no problems.

My final screen res is 1280 x 1024 which is upscaled on a 1024 x 768 projector.

Although touche is calibrated perfectly and on other apps my touch's appear directly underneath my fingers, when using your app the touch points are perfect in the middle but drift off towards the edges! ( i will send some picks.

It does not seem to be outputing midi? or at least when i enter modul8 and try the midi learn function and fiddle with a fader nothing happens? Probably a switch in your app i havent flicked!?

Cheers!

[zoophar]

Hello Grovenstein,
thanks for your comments, it helps me going forward!!!
and thank you for your patience!!!!

>Finger Calibration:
Yes i had some questioning about the calibration of the fingers (in the Read-me.txt) but i didn't expect to have such squint...
For TUIO messages, i used the reactable engine (reacTIVision).
Maybe it has something to do with the nature of the message... (could not test Touché because of Tiger only).

>MidiSignals:
I used the MidiBus Library, It says you have to install mmj in order to Midi with my app.
And in this version, this Midi signal is sent to IAC_driver; so this message is local only.
(because i wanted to focus on the finger calibration, and this is faster for you to test midi in local)

Once this 2 things done, when tickling the fader, do you see something in Midi Monitor?

kind regards