RAM pre-load vs uncompressed

General discuss about Modul8

Moderator: Garagecube

RAM pre-load vs uncompressed

Postby luxcollective » Tue Feb 19, 2013 1:41 am

I have 16bg of RAM on my 2012 Macbook Pro and want to organize and size my media sets and RAM limits so that every clip in the mediaset loads into RAM... the issue is that Modul8 defaults to the "uncompressed" RAM load option and burns through the available RAM in the first couple of pages leaving the rest of the clips on the hard drive... Is there a way to keep it from doing that? I manually click all the media over to RAM preload and have enough to cover all the clips but the next media set defaults back to Uncompressed preloaded, its very frustrating... I have a huge show in 4 days so hoping someone can lighten me.
luxcollective
Member
 
Posts: 12
Joined: Tue Jun 03, 2008 4:40 am

Re: RAM pre-load vs uncompressed

Postby gmint » Thu Mar 14, 2013 7:19 pm

I've been experimenting with this lately and I've come to determine that if you're pulling from an SSD, "Direct from Disk" seems to be significantly faster than either RAM pre-load or uncompressed. On my rMBP with 16 Gigs of RAM, I'll see a several second delay with certain 100 Meg (or so) movies when set to RAM pre-load, while I see no delay when using direct from disk.

Seems to me that perhaps the caching mechanism in Modul8 is long overdue for an update, especially now that SSD drives are becoming so prevalent for high performance applications like this. As I've pointed out in another thread, it doesn't seem like there's very much insight into how the caching mechanism works in the first place, much less enough to consistently recommend a strategy with any specificity.
User avatar
gmint
ultim8 member
 
Posts: 301
Joined: Wed Aug 04, 2004 4:30 am
Location: Des Moines, IA

Re: RAM pre-load vs uncompressed

Postby gmint » Sat Mar 16, 2013 6:51 am

I just wanted to follow up on this one and say that I'm now more convinced than ever that if your media is on an SSD drive, the best setting for clips is always "Direct from Disk." Now maybe someone will come along and disagree or prove me wrong, but I had been using a mix of both RAM based cache settings under the assumption that they would be preferable (after all, RAM is still faster than an SSD) but on my production machine (consisting of five 1024x768 feeds plus a 720p feed) I get noticeably better frame rates when everything is streamed from the SSD.
User avatar
gmint
ultim8 member
 
Posts: 301
Joined: Wed Aug 04, 2004 4:30 am
Location: Des Moines, IA

Re: RAM pre-load vs uncompressed

Postby vanakaru » Sat Mar 16, 2013 10:44 am

I hope developers would look into this. Problems with RAM preload may be the cause why I could not use M8 for live shows any longer due to stalling when new clip is loading.
Discussed here:
viewtopic.php?f=1&t=8115
vanakaru
master
 
Posts: 669
Joined: Mon Sep 25, 2006 9:24 pm

Re: RAM pre-load vs uncompressed

Postby gmint » Sat Mar 16, 2013 3:34 pm

vanakaru wrote:I hope developers would look into this. Problems with RAM preload may be the cause why I could not use M8 for live shows any longer due to stalling when new clip is loading.
Discussed here:
viewtopic.php?f=1&t=8115


You are correct, there is a delay when clips (especially large ones) are loaded into RAM. However, as I said, with Direct from Disk off an SSD, I'm seeing basically instantaneous media switches. In fact, I went from switching layersets to just switching media on a layer directly.
User avatar
gmint
ultim8 member
 
Posts: 301
Joined: Wed Aug 04, 2004 4:30 am
Location: Des Moines, IA

Re: RAM pre-load vs uncompressed

Postby broken » Wed Apr 30, 2014 6:36 am

Really noticing a discernible difference in speed on a brand new built system when I load "direct from disk" using a Samsung 840 EVO 250gb SSD as opposed to pre-loading to RAM uncompressed (DDR3 1600).

No lag at all going direct from disk. Need an option in preferences on what setting to use for imported media for sure.
User avatar
broken
super8 member
 
Posts: 112
Joined: Sun Dec 23, 2012 11:44 pm
Location: canada

Re: RAM pre-load vs uncompressed

Postby anomad » Tue May 06, 2014 4:10 pm

. yes yes yes!

. we really need the ability to choose imported media setting ! i use some older hardware and would like to force my 'more important' video clips to be in RAM whenever possible.

. is this difficult to implement on the back end? maybe it could be added in v2.8.1 ? :D


-james
(a nomad. )
anomad
master
 
Posts: 411
Joined: Sun Oct 21, 2007 10:07 pm
Location: north cakalacky, usa

Re: RAM pre-load vs uncompressed

Postby gmint » Tue May 06, 2014 4:14 pm

I assume both of you know that you can set this on a per-media basis, correct? If that is the case, I guess I'm not exactly sure what you would be asking for...
User avatar
gmint
ultim8 member
 
Posts: 301
Joined: Wed Aug 04, 2004 4:30 am
Location: Des Moines, IA

Re: RAM pre-load vs uncompressed

Postby broken » Tue May 06, 2014 9:01 pm

gmint wrote:I assume both of you know that you can set this on a per-media basis, correct? If that is the case, I guess I'm not exactly sure what you would be asking for...

unless that preference gets saved into the media file (so when it's loaded again the same option is saved), setting each individual clip to Direct from Disk can be really tedious when you don't have a 'defined' media set at every gig.

for shows i customize/personalize what clips i'm playing, and that invariably leads to dragging selections of 16 clips to the slots, switching banks, and repeating.. every time, every show I would need to then click on each of my individual clips and set this preference, which is then lost the next time i attempt to drag the clip in from Finder..

ideally a setting in the preferences with the 3 options in a dropdown would suffice and save a lot of time. choose which method of playback you want for clips that are imported into the slots.. now it picks RAM pre-load for me, and gives me crappy performance because of it.

i personally never want my clips set to RAM pre-load.. just a waste when I've got a SSD
User avatar
broken
super8 member
 
Posts: 112
Joined: Sun Dec 23, 2012 11:44 pm
Location: canada

Re: RAM pre-load vs uncompressed

Postby gmint » Tue May 06, 2014 9:08 pm

broken wrote:
gmint wrote:I assume both of you know that you can set this on a per-media basis, correct? If that is the case, I guess I'm not exactly sure what you would be asking for...

unless that preference gets saved into the media file (so when it's loaded again the same option is saved), setting each individual clip to Direct from Disk can be really tedious when you don't have a 'defined' media set at every gig.

for shows i customize/personalize what clips i'm playing, and that invariably leads to dragging selections of 16 clips to the slots, switching banks, and repeating.. every time, every show I would need to then click on each of my individual clips and set this preference, which is then lost the next time i attempt to drag the clip in from Finder..

ideally a setting in the preferences with the 3 options in a dropdown would suffice and save a lot of time. choose which method of playback you want for clips that are imported into the slots.. now it picks RAM pre-load for me, and gives me crappy performance because of it.

i personally never want my clips set to RAM pre-load.. just a waste when I've got a SSD


I gotcha. All you need to do is to set the following preferences and you'll get direct from disk every time:

"When imported, a media is stored in memory when its size is less than:" 0 MB
"When opening a new project, maximum preload memory size:" 0 MB

Voila, everything is direct from disk!
User avatar
gmint
ultim8 member
 
Posts: 301
Joined: Wed Aug 04, 2004 4:30 am
Location: Des Moines, IA

Re: RAM pre-load vs uncompressed

Postby anomad » Tue May 06, 2014 9:28 pm

gmint wrote:I gotcha. All you need to do is to set the following preferences and you'll get direct from disk every time:

"When imported, a media is stored in memory when its size is less than:" 0 MB
"When opening a new project, maximum preload memory size:" 0 MB

Voila, everything is direct from disk!


. dang.. i had forgot about that :)

. i guess i was thinking of a way to access the RAM preload uncompressed / RAM preload / Direct to disk as a keyword that could be activated from a module, rather than selecting each one and choosing the load status of the clip…

. however, i hope to get a new macbookPro soon so perhaps everything will be Direct to Disk and i won't need to worry about this ! :D

-james
(a nomad. )
anomad
master
 
Posts: 411
Joined: Sun Oct 21, 2007 10:07 pm
Location: north cakalacky, usa

Re: RAM pre-load vs uncompressed

Postby gmint » Tue May 06, 2014 9:57 pm

Yeah, actually I got a new MacBook Pro a while back and it was for that reason that I did some research into this particular topic. As I said before, I really wish that GarageCube should provide some better insight into this particular area as the whole caching mechanism seems to be really out of date given how prevalent SSDs are becoming...
User avatar
gmint
ultim8 member
 
Posts: 301
Joined: Wed Aug 04, 2004 4:30 am
Location: Des Moines, IA

Re: RAM pre-load vs uncompressed

Postby broken » Wed May 07, 2014 3:54 am

gmint wrote:I gotcha. All you need to do is to set the following preferences and you'll get direct from disk every time:

"When imported, a media is stored in memory when its size is less than:" 0 MB
"When opening a new project, maximum preload memory size:" 0 MB

Voila, everything is direct from disk!


Ahh.. I tried 5mb for the imported option, and 200mb for max pre-load.. but none of my clips are under 10mb so I figured it should work but it didn't.

Going to have to give 0 MB & 0 MB a shot. edit: IT WORKS! (THANK YOU!)

I couldn't find any concrete info on how the options behaved or anything (only this topic really) so really appreciate the heads up :)
User avatar
broken
super8 member
 
Posts: 112
Joined: Sun Dec 23, 2012 11:44 pm
Location: canada

Re: RAM pre-load vs uncompressed

Postby broken » Fri Jul 24, 2015 5:10 pm

Been using the "Direct to Disk" method for the past while, but there seems to be some stutter/delay happening with a fraction of my clips when they loop around.

Playback is instantaneous when clips are clicked (no delay like Ram pre-load) but every single time the loop hits the end and starts over the video freezes for a second.

I can overcome the freezing by clicking on the ram-preload option or by creating a ram disk and loading my media to it instead of my RAID 0 SSD setup.. which seems odd as I've got more than enough disk speed to playback 1 clip looping.

These are 1080p clips, but size doesn't seem to even be an issue. I get the freezing on a 65mb file just as much as I do on a 400mb file..

HAP seems to make this more noticeable, if I convert HAP clips that are freezing to PhotoJPEG there's a slight freeze..

PhotoJPEG takes up more CPU though, so there's drawbacks to using it vs HAP (Using GTX780 with 6GB VRAM - so I'd like to use the GPU as much as possible).

All of these clips playback 100% problem free in CoGe regardless of HAP, PhotoJPEG, small, big doesn't matter.. Wish modul8 could do the same lol
User avatar
broken
super8 member
 
Posts: 112
Joined: Sun Dec 23, 2012 11:44 pm
Location: canada

Re: RAM pre-load vs uncompressed

Postby gmint » Tue Jul 28, 2015 9:41 pm

broken wrote:Been using the "Direct to Disk" method for the past while, but there seems to be some stutter/delay happening with a fraction of my clips when they loop around.

Playback is instantaneous when clips are clicked (no delay like Ram pre-load) but every single time the loop hits the end and starts over the video freezes for a second.

I can overcome the freezing by clicking on the ram-preload option or by creating a ram disk and loading my media to it instead of my RAID 0 SSD setup.. which seems odd as I've got more than enough disk speed to playback 1 clip looping.

These are 1080p clips, but size doesn't seem to even be an issue. I get the freezing on a 65mb file just as much as I do on a 400mb file..

HAP seems to make this more noticeable, if I convert HAP clips that are freezing to PhotoJPEG there's a slight freeze..

PhotoJPEG takes up more CPU though, so there's drawbacks to using it vs HAP (Using GTX780 with 6GB VRAM - so I'd like to use the GPU as much as possible).

All of these clips playback 100% problem free in CoGe regardless of HAP, PhotoJPEG, small, big doesn't matter.. Wish modul8 could do the same lol


Interesting, I've not seen such an issue. Have you tried setting "When imported, a media is stored in memory when its size is less than" and "When opening a new project maximum preload memory size" both to zero? I know that sounds like an odd choice, but in my use I've found that actually loading the content into RAM takes longer than just playing it directly off of an SSD... Might be worth a try in your situation.
User avatar
gmint
ultim8 member
 
Posts: 301
Joined: Wed Aug 04, 2004 4:30 am
Location: Des Moines, IA

Next

Return to Modul8 (General)

Who is online

Users browsing this forum: Bing [Bot], Google Feedfetcher and 1 guest