garageCube

[plasticpool]

I would like to be able to pass video from Max/MSP to Modul8 and vice versa without going out and back in to my machine, that is, purely in software. Is this practical/feasible at this time?

[plasticpool]

I would really like to hear an answer to this question from one of the m8 team. Is it possible to write a video buffer (to the VRAM, RAM or hard drive) that other programs can use, practically? Without a huge performance hit? This, to me, is the crucial missing element in all VJ software.

[yves@garagecube]

As far as I know I don't think it is possible to share VRAM buffers between two different processes in OS X.

The final frame in Modul8 is in VRAM so it would require first to copy it to RAM. Then the frame should be transfered to the other application using an inter-process mechanism. There are various ways to do that... Cocoa has a very easy API for this purpose. It allows an application to call a function from another process in order to share functions and datas. This is something I could implement pretty easily. It would allow any application to grab frame or send keywords to Modul8. Could be interesting.

Now I don't know welll Max, so I don't know if it is possible to access this kind of API from it.

Yves.

[plasticpool]

If you give me the name of the API I can find out whether Max can work with it. I would really like to experiment with this. Chaining visual programs together would open up a lot of possibilities, beyond the simple advantage of seamless transitions between different programs in one's sets.

[yves@garagecube]

Well, basically I was referring to the distributed objects available in the Cocoa API.

Currently it could be accessed from Objective-C, Java or Python. So I guess a Max plug-in needs to be written. Now if there is a protocol supported by Max that allows frames to be transferred let me know.

Regarding distributed objects:

http://developer.apple.com/documentatio ... /10000102i

Yves.

[plasticpool]

thanks! i'll bring this up on the Max list -- and read about distributed objects.

[refresh]

As far as I know I don't think it is possible to share VRAM buffers between two different processes in OS X.

The final frame in Modul8 is in VRAM so it would require first to copy it to RAM. Then the frame should be transfered to the other application using an inter-process mechanism. There are various ways to do that... Cocoa has a very easy API for this purpose. It allows an application to call a function from another process in order to share functions and datas. This is something I could implement pretty easily. It would allow any application to grab frame or send keywords to Modul8. Could be interesting.

Now I don't know welll Max, so I don't know if it is possible to access this kind of API from it.

Yves.

One feature I would LOVE to have is the abiity to grab a viewport or area of screen from another app and use that as a layer in M8. this would be such a killer feature.

For instance I am flash developer and I can script very sophisticated dynamic animation, even have it respond to audio and midi, and yet there is no way to use this media in M8. this is a real drag for me. I realize that ya'll are relying on QT for flash support but QT has a very outdated version of flash which makes it just about useless.

on the other hand i don't really want a flash rendering engine in there if it's gonna slow things down, you can see this in Grid software when using their (Max's) flash renderer..

However if you could just capture an area of flash running in the flash player or even better a web browser and spit it into a layer in M8, then that would make me seriously hot and bothered...

But it doesn't stop there.. oh no.. if you could capture a totally arbitrary area of screen and bring it into M8 than suddenly i am in digital nirvana land.. why? because I am now sampling Maya and Processing and yes perhaps even After Effects ram previews.. Indeed that would be revolutionary IMO..

as it stands now i have to spend hours trying to dumb down my nonlinear flash..maya.. processing into linear sequential frames..

still "render logical recording" beats the hell outta keyframing in After Effects

[yves@garagecube]

Well being able to capture an arbitrary area of the screen is an interesting idea.

I have some plan regarding improved web/flash integration in Modul8. However I'm so busy finishing the new engine of the 2.5 that I don't have time to work on that now. But you can expect big improvement in this area for the next major release.

Yves.

[plasticpool]

Here are a few replies on this topic from the Max list/fora. The one that seemed most promising is:

I wouldn't really bother with trying to compile a meta-app or plugins to do what you were talking about (sharing video betwen apps).
Instead, I would use something like OSC, UDP, whatever. Just stream the data to a socket and read that socket in the other app. I know nothing about module8, but if it's a live video app worth its salt, it will have some kind of socket access implementation.

Does M8 have 'some kind of socket access implementation?

In the meantime, I'll keep faking smooth transitions. I'd like to be able to turn off the always-on-top behavior of the fullscreen M8 window. I've found that I can click and drag the non-fullscreen M8 display window to fill my second monitor but I suspect that this might not be as efficient as letting M8 do its auto-discover-resolution wizardry. That would make it easier for me to switch between apps during sets efficiently.

Thanks again! I'm looking forward to 2.5.

[yves@garagecube]

Well, socket is the default API for transferring TPC/IP or UDP packets over a network.

You can access socket through the Python scripting mechanism of Modul8. It is pretty easy to open a socket and send data to another computer using a module for instance.

Now it is not possible (yet) from Python to grab a picture of the Modul8's composition. It would require that I add a function for that.

Even if you use sockets to transfer data locally (using the localhost), I'm not sure what kind of performance you can obtain with non-compressed video.

Also I guess you still have to write some code to open a socket in Max, read back the data and show them in the composition... unless Max has already a socket based protocol for importing video... ?

Yves

[plasticpool]

I figured sockets just meant IP transfer but I wanted to make sure. Yes, Max has objects for sending and receiving streaming video. Sounds like this is worth trying.

[eskatonia]

It would be possible using core image pbuffers to transfer video on the graphics card without reading back into memory...

but only if both programs supported it, you'd need a 'core video send' function in MAX/MSP and a 'core video recieve' function in Modul8....

I'd love to see all the mac vj developers team up and support this but I the MAX/MSP guys are not to quick with adding new core image features.

[xrayconcept]

just an information for this:
check the video capture soft screenography at http://www.verticalmoon.com

[yves@garagecube]

Eskatonia:

Are you sure it is possible to pass a PBuffer from one process to another ?

This is very interesting... Have you any sample code for that ?

Yves.

[eskatonia]

I can't find sample code for this but I believe that this was one of the design goals for Core Image / Core Video.

ie to allow different programs to pass video to each other on the graphics card.

I would suggest asking on Quartz-dev apple developer list.