There is a workaround, albeit outside of Madmapper.
If your projector is already mounted, I would consider bringing your video into After Effects and handle all masking there. In After Effects' Preferences, under the Video Preview tab, select "Enable Mercury Transmit." Select your secondary display (the projector) as the video preview device. Handle the keystone with After Effects' Corner Pin effect. Handle any specific mapping with the Mesh Warp Effect. For each mask, create a new Shape Layer by navigating to the Layer Menu > New > Shape Layer. Set the Fill to black, and set the Stroke to none (click the text that reads "Stroke:" (not the rectangle next to it) and select the leftmost thumbnail in the popup window. (The term "None" will appear.) Then use the Pen tool to draw the path outlining your shape. Repeat this step for every mask. After Effects also allows Bezier curves with the Pen tool. Then re-export your video, bring it into MadMapper, and export to miniMAD as you had done before.
These instructions may seem pedantic, but I just wanted to make sure you could get your project done right, even if you don't use After Effects this way! And if you do not have After Effects, and are eligible for the trial period, download the 30-day trial. (And if you are being paid for this project, consider purchasing After Effects once you get paid.)