Page 1 of 2

High CPU

Posted: Sun Oct 02, 2016 7:36 pm
by broken
Running the new beta 3 for a show since it works nicely with HAP and HD content.

Noticed my CPU meter kept creeping up and up and up, got to about 80% which I've never seen on this system before even using PhotoJPEG in the past with the same amount of active layers/blending happening.

Had to load up a syphon program to pass my visuals over while I restarted Modul8 to tame the CPU. Once Modul8 was restarted, cpu would be around 13-18% (running all the same clips, layers, blending, fx as before) and stay low for a while before creeping up again to 70-80% usage.

The visuals didn't seem to suffer, but I didn't want usage to go over 90% and have things crash so I opted to change my syphon source and restart Modul8 a handful of times during the gig. Haven't ever noticed Modul8 eat up this much power before, so thought I'd mention it.

cpu.jpg (241.57 KiB) Viewed 6364 times

Nvidia 780 6GB VRAM
32gb ram

Re: High CPU

Posted: Sun Oct 30, 2016 8:50 pm
by broken
Continuing to see really high CPU in beta 5 with HAP encoded 1920x1080 clips.

Experienced a few crashes with Modul8 I'm assuming is related to CPU when I went for a break and returned to visuals frozen. Everything starts off playing back under 20% cpu across all 4 cores of my i7, which eventually creeps up to 60-80% after a while.

Anything I can send in, or test to help with this issue?

I've had some more system freezes when using PhotoJPEG in beta5 (like in this thread - ) so I'm sort of stuck with using HAP now (it runs better than PhotoJPEG anyhow - other than the hot cpu)

Re: High CPU

Posted: Mon Oct 31, 2016 11:51 am
by Pierre Guilluy
Please send your related crash logs.

What's the exact type of your HAP files (HAP-Q, HAP-Alpha)?

Re: High CPU

Posted: Mon Oct 31, 2016 4:19 pm
by broken
HAP files with quality set to 80 in AME. 30fps.

The only dialog that popped up was Modul8 quit unexpectedly.. Ignore, Report, Reopen..
The Report option is for sending to Apple, and I was at a live show when it happened, so I didn't take a screenshot or do anything other than try to get Modul8 running again.

There wasn't anything about emailing crash logs when Modul8 restarted.

I'll see what I can dig up

Re: High CPU

Posted: Mon Oct 31, 2016 6:25 pm
by Pierre Guilluy
How many HAP clips are you playing at the same time when the CPU goes up to 80%?

Could you be more specific about the HAP format you're using (depending on the format, Modul8 will either use FFmpeg or AVFoundation player)?

Re: High CPU

Posted: Mon Oct 31, 2016 6:52 pm
by broken
I pretty much had 2 layers playing HAP clips (1920x1080) and using Slide Show (layer) to trigger new clips - fading is enabled with 0.3 to 0.5ms.

Both HAP layers were blended with ADD on top of a static JPG layer (artist promo image).

a 4th layer was used for artist logo and was transparent PNG

For the HAP codec, I've been using v12 from the github page as of Sept 5th 2016. Using Adobe Media Encoder and selecting Quicktime/HAP and putting quality to 80.

In Modul8 it shows my HAP clips using FFMPEG in the Media Information panel. Is there another way I should be encoding these clips so that Modul8 will use AVFoundation?

Since typing this message, I've had my project from Friday night open and have seen a CPU rise from 11% (across all 4 cores) to 41% and rising as I am typing this.

Re: High CPU

Posted: Mon Oct 31, 2016 7:05 pm
by broken
Regarding crash logs.. I've only been able to find 2 .crash files for Modul8 on my system.. and they aren't from Friday night when I had 3 crashes.. but instead from Saturday night when a breaker flipped and my desktop powered down.

I found them in /Users/andrew/Library/Logs/DiagnosticReports/

Is there anywhere else I should be looking for logs or crash reports for Modul8? There's a ton of logs I found for Madmapper in /Users/andrew/Library/Logs/GarageCube/ but not a single log for Modul8 I can find.

Re: High CPU

Posted: Mon Oct 31, 2016 7:20 pm
by broken
Some more weirdness.. Have to assume this is an actual bug.

All my clips were encoded to HAP at the same time, same settings.. Imported into Modul8 16 at a time from Finder to each media set.

Clips from Media ID 0 - 44 are set to FFMPEG
Clips from Media ID 45 - 100 are set to AVFoundation

I didn't set any of these, and they were all dragged into the project in the same session, encoded the same..

Something's up

EDIT: Seems untriggered clips are set to FFMPEG.. once they are used, it changes to AVFoundation.. I'm not sure why or whats happening here

CPU burning up to 70% now since I opened it today
Screenshot 2016-10-31 13.23.47.png
Screenshot 2016-10-31 13.23.47.png (63.66 KiB) Viewed 6283 times

Re: High CPU

Posted: Mon Oct 31, 2016 7:51 pm
by broken
Finally crashed again after about an hour of running..
(21.14 KiB) Downloaded 218 times

Re: High CPU

Posted: Wed Nov 02, 2016 10:46 am
by Pierre Guilluy
Your crashlog seems to indicate that you're running out of memory.

It also indicates that 116 threads were running when the application crashed, which is a lot even though most of them were idle.
It could mean there were many movies still loaded when they should have been released...

Could you describe what was Modul8 doing when the crash happened (how many movies loaded, playing, etc)?

By the way, as the FFmpeg / AVFoundation selector seems to confuse users by not being clear / consistent, we may remove it in the next version, or at least replace it with a more meaningful option.

Re: High CPU

Posted: Wed Nov 02, 2016 3:54 pm
by broken
That's odd. I've got 32gb of RAM and usage never goes above 5gb.. Which is weird as I assumed this version of Modul8 was 64bit and able to access all of my available RAM.

My 6GB video card memory as you can see from the screenshots I provided is also not being fully used up.

I continue to get red dots on a bunch of videos as they seem to be released from memory, however I'm streaming direct from disk using a RAID SSD setup, so nothing really should be in RAM should it?

Re: High CPU

Posted: Wed Nov 02, 2016 4:00 pm
by broken
Pierre Guilluy wrote:Could you describe what was Modul8 doing when the crash happened (how many movies loaded, playing, etc)?

I was basically leaving my system switching clips on it's own using the Slide Show (layer) module.. When I had experienced the crashes during a show on the weekend I was switching 2 layers of clips with the Slide Show module, and one layer was changing videos ~10 seconds, the other ~30s before it would change.

In order to speed up the crash process for this testing I changed the Slide Show module settings to switch clips on 2 layers, first layer ~5s and the other ~2s

So 2 layers playing 1920x1080p clips, blended with ADD, switching clips 5 seconds and 2 seconds each.
1 layer of a static background JPG
1 layer of transparent PNG artist logo (zooming in and out via modul8 fx)

There would have been 100 clips loaded of 1920x1080.. HAP encoded using v12 HAP from github 80% quality.

I set up Slide Show (layer) to switch clips, and left the project open while I responded to you. It took approximately 1h to crash with those settings. I had 2-3 crashes on the weekend switching clips at 10seconds and 30seconds as well.. 100 HAP clips being used as well.

Re: High CPU

Posted: Tue Nov 15, 2016 11:15 pm
by broken
On the weekend I was noticing a clear slowdown or choppyness to the output once the overall CPU hit 40-50% - I passed off to CoGe while restarting Modul8 several times and once playback started on the fresh start up it was smooth again.

I wasn't using the SlideShow (layer) module at all this time, rather triggering clips from a APC Mini MIDI controller. Between 25-30% CPU things were fine, once it pushed past 40% I could tell something was lagging a bit.

Didn't want things to crash out so I kept on restarting Modul8 once the CPU rose to 50-60% and got choppy. HAP clips again. about 80 this time.. Couple of HAP-A with transparency.

Sometimes after restarting modlu8 the cpu would stay under 30% for a while, but other times it would almost be 5minutes since I restarted before I felt things were getting a bit laggy and restarted modul8.

In the past I've used PhotoJPG exclusively and never saw the CPU meters rise this high (unfortunately in the latest beta I'm getting full system lockups with PhotoJpeg projects that's hard to nail down) so I just wonder if there's some sort of memory leak happening that's running the cpu hot.

Happy to do further testing if there's anything I can provide that's useful

Re: High CPU

Posted: Thu Dec 01, 2016 6:52 pm
by broken
Guess I'm just talking to myself here, but I may have found part of the issue with high CPU..

As I mentioned in a previous post, I'm using RAID SSD setup, and really shouldn't have had anything loading into ram as I had things configured properly in 2.9 to accomplish this via suggestion from user gmint:
All you need to do is to set the following preferences and you'll get direct from disk every time:

"When imported, a media is stored in memory when its size is less than:" 0 MB
"When opening a new project, maximum preload memory size:" 0 MB

Voila, everything is direct from disk!

In Modul8 v3, there's different options in the Medias tab:

Start releasing medias when Modul8 uses more than % of the full memory:
When opening a new project maximum preload memory size:

I've had the "Start releasing medias when Modul8 uses more than % of the full memory" set to whatever the default was 70% I think?
I've had the "When opening a new project maximum preload memory size" set to 0 MBytes in order to NOT preload media and go direct from disk on my RAID SSD.

When opening this new beta to my default/template set I experienced the same ramping CPU issues I've been reporting, got up to 50% usage across all cores, video was glitchy and not smooth at all.

Since you suggested I was running out of memory by the crash report (even through I have 32gb ram installed and never see usage over 4gb despite Modul8HD being 64bit?) I thought maybe if I lowered the value for percent of full memory used before the media gets released it would bypass any ram / memory issues..

Low and behold setting this value to 1% IMMEDIATELY dropped my CPU usage from 50% across all cores (i7) to 7-10% and video playback became smooth again.

Basically it seems to be releasing every clip from RAM once it's the next one is done playing - all my clips have red dots in the media sets apart from the currently playing clip, and the previous clip.

Anyhow, any ideas about this? Is this the correct configuration in v3 to stream direct from disk? Just a bit confused about why I would have memory errors suddenly, and only with this app. After Effects, Premiere, C4D, Realflow.. anything I can think of that gets used on the same computer has no issues using my full 32gb, or crashing.

Re: High CPU

Posted: Mon Dec 05, 2016 11:14 am
by Pierre Guilluy
Modul8 is a 32bit application because it's built on top of a core macOS framework (Carbon) that weren't ported to 64bit, explaining the 4GB RAM limit.

That being said, the application shouldn't crash under any circumstance.

I guess the problem comes from the dynamic loading/unloading function that's linked to the memory preference you're mentioning, and I'll take a look at that algorithm in order to make sure it works properly.