Custom FX support, chainable FX, sharable FX library
  • jasonbeyers
    super8 member
    Posts: 125
    Joined: Mon Oct 08, 2018 7:16 am

    Custom FX support, chainable FX, sharable FX library

    by jasonbeyers » Sun Oct 11, 2020 2:01 am

    So I've been thinking about this one for the past year and have done some experiments lately.

    I've organized this idea into three parts, in descending priority order:

    1. Official support for user-defined FX (for quads, triangles, circles)

    Today it is *technically* possible to define surface FX within <Madmapper Dir>/FX/quad_triangle_circle, using the existing FX shaders as examples.

    In the past I had used custom FX for lines, and this was a wonderful way to generate patterns that were aware of the line segment being used. However, these custom line FX shaders broke with a recent version of MM, I think with v3.7 - specifically, not all FX parameters got updated correctly when playing a cue and the “restart” logic for custom FX did not behave as expected. So, I’ve avoided line FX since then, as it was never a supported feature anyway.

    Revisiting this, I am working with filter-type shaders more and more in my projects. Using these filter style shaders in ISF or Material form relies on selecting a texture input, and this texture input is exposed with a drop-down menu within MM. This drop-down menu becomes challenging to navigate (see https://forum.garagecube.com/viewtopic.php?f=29&t=35196), and it is also rather time-consuming to try switch among filter shaders at different stages of an internal syphon-routed pipeline within MM.

    Switching between the built-in Surface FX shaders is straightforward and very fast, since the texture input is implicitly set already (whatever is on the surface), allowing for rapid experimentation.

    So, I experimented with building my own surface FX, and have had great results! Using the learnings from converting shadertoy/glslsandbox shaders to materials, and examples of the built-in FX, I am having so much fun with it. There is some huge potential here :)

    I’m already seeing big time savings for my experimental projects, but since custom FX are not officially supported, I’m cautious about diving deeper into this approach & investing more time in converting shaders to that format.

    I haven’t run into any product issues with custom quad/triangle/circle FX so far (unlike with custom line FX), but official support for these would give myself & other customers the confidence to try these out, and open the door to some awesome compositions.

    One thing I have noticed though: MM needs to be restarted before new shader inputs (for custom FX) become visible in the UI.

    2. Chainable surface FX for quad/triangle/circle

    Assuming custom FX are supported someday, I think the next big opportunity would be *chainable* FX.

    VJ tools like TouchDesigner, Notch, and Magic Music Visuals are basically designed around this concept, using nodes to define these FX chains.

    Smode is an example of a VJ tool that is not node-based, but it still supports processing chains on certain objects. To me that is the best of both worlds: a graphical layout to arrange the pieces (like Madmapper or Resolume) with a basic linear processing chain on each piece.

    The built-in “Multi FX” shader in MM hints at this possibility. The limitation here is that “Multi FX” (and custom FX that someone might create) is really just a single shader.

    If users could attach *different* FX shaders to the same MM surface, and arrange them in different orders, this would avoid having complex internal syphon/spout routing chains that folks like myself have been using for years. There will always be a use for internal syphon routing but this could help avoid some of those workflows.

    This would be ideal:
    * A “+” button under the FX shader selection to add additional ones
    * A draggable handle next to each of the active FX shaders to re-arrange them, just for a particular surface
    * A button to bypass one or more of the FX in a multi-FX chain
    * A button that appears in in “Edit Cues” mode to save the FX *order* into a cue

    Proper Multi-FX would mean you could combine commonly-used FX like “Color Controls” with anything else. It would also give folks an option to simulate multi-pass shaders even if MM does not support passes within a single shader.

    I feel that high-performance hardware (including GPU) can handle complex processing chains with other tools like Touchdesigner and it would be better to allow multi FX in Madmapper, and rely on the end user to decide what is too much for their system. If folks are using other tools to do this alongside Madmapper (say, with syphon/spout), they are taking the performance hit anyways.

    For reference, I regularly use 5-6 stages within Madmapper using internal syphon routing, each with a filter-style shader effect, using a 2018 Macbook Pro and a Radeon VII eGPU. Having multiple FX applied to a single surface would accomplish the same thing but with much less effort to set up :)

    3. FX library for quad/triangle/circle

    This 3rd item would be nice but would be less impactful than the first two.

    Allowing custom FX could make the FX dropdown menu a bit cumbersome, depending on how many FX someone tries to make. Name conflicts with built-in ones could be problematic but can be avoided.

    Assuming custom FX is supported someday, it would be great if there was a proper media library for custom FX like there is for Materials, allowing folks to drag-and-drop an FX from the list onto a surface (and potentially into a slot in the processing chain) as an alternative to selecting the FX from the existing drop-down menu.

    It would also be great if people could *publish* their FX creations for others to use, just like for regular materials. Personally, I am sitting on 400+ filter-style shaders I have collected and built over the years, and I would be happy to share the MM-ready variants to everyone if there was a mechanism to do so. The conversion from ISF to Custom FX is very similar to ISF-to-Material since FX seem to behave like Materials.

    Short of a sharable FX library, folks could just post their custom FX creations on these forums, to load into their Madmapper file directories.
    ---

    Sorry for the long read! Out of all the possible future opportunities for Madmapper, custom FX (and chainable FX) is what excites me the most, by far :)

    After custom FX, the next most exciting enhancement would be cue-able audio controls (enable/disable, sensitivity for just that audio control, range, filtering, etc).

    Thanks for your consideration!
  • franz
    madMapper master
    Posts: 1033
    Joined: Fri Feb 18, 2005 7:05 pm
    Location: Paris, France
    Contact:

    Re: Custom FX support, chainable FX, sharable FX library

    by franz » Mon Jan 18, 2021 6:01 pm

    #1 is coming
    #2 is currently being discussed
    #3 will come together with #1

Who is online

Users browsing this forum: Jasper.LIVE and 9 guests