Board index FlightGear Release candidates 2.8

OpenGL "Out of Memory"?

This is the archive of topics about the 2.8 release candidates.

Re: OpenGL "Out of Memory"?

Postby ot-666 » Wed Aug 08, 2012 8:48 pm

fredb wrote in Thu Aug 02, 2012 11:50 am:
ot-666 wrote in Tue Jul 31, 2012 9:23 pm: but there is no regular 64bit fg version from Jenkins :?

Not true anymore.


Big thanks :D the 64bit version works great.
After flying the Blackbird with all eye candy diald up a notch over papillon81 germany scenery for about 40minutes, FGFS consumed 13,8GB memory :shock:
Looks amazing, but started to slow to 1 gigit frame rate after about 60 minutes and consuming 15.2 GB memory... (did not crash).
Callsign: ot-666
Working on LOWI and other stuff - Custom Scenery Overlay Repo: http://gitorious.org/fgfs-custom-scenery/custom-scenery-overlay/
VMX22 - Osprey... sometimes in 2014
ot-666
 
Posts: 746
Joined: Sun Nov 08, 2009 5:14 pm
Location: Germany, Konstanz
Callsign: ot-666
IRC name: ot666
Version: GIT
OS: win7 64bit

Re: OpenGL "Out of Memory"?

Postby Hooray » Thu Aug 16, 2012 9:01 pm

Hi,

could you please check if the problem disappears, using identical startup settings, just with random buildings completely disabled?
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

Re: OpenGL "Out of Memory"?

Postby ot-666 » Fri Aug 17, 2012 7:56 pm

Almost same conditions, but random buildings complety disabled eats up 8.5 GB after a couple of minutes, but still climbing...
Identical flight with random buildings activated eats up 14.7GB and climbing...
Last edited by Gijs on Sat Aug 18, 2012 11:05 am, edited 1 time in total.
Reason: No useless quoting please
Callsign: ot-666
Working on LOWI and other stuff - Custom Scenery Overlay Repo: http://gitorious.org/fgfs-custom-scenery/custom-scenery-overlay/
VMX22 - Osprey... sometimes in 2014
ot-666
 
Posts: 746
Joined: Sun Nov 08, 2009 5:14 pm
Location: Germany, Konstanz
Callsign: ot-666
IRC name: ot666
Version: GIT
OS: win7 64bit

Re: OpenGL "Out of Memory"?

Postby Hooray » Fri Aug 17, 2012 8:30 pm

Wow, that's surprising - something is definitely wrong here (I am currently running FG on a computer with just 2gb of RAM and don't see this, but many "eye candy" features are disabled), could you try finding out what feature needs to be disabled to stop the memory from gobbling up that severely?

Is this with or without Rembrandt enabled?

To me, that sounds like a serious bug, and I am surprised that it's still there obviously, despite the recent release - such things really shouldn't be in a release at all, we may be in for a rude awakening if other users experience the same issues - because most users will "just" have 3-8 gb of RAM, certainly not 16gb like you do - so FG would just crash...

PS: Are you able to build from source, so that you could test patches? I recently talked to ThorstenB about extending his system monitor such that it also reports memory usage (allocations) per subsystem by overloading the new/delete operators at the SGSubsystem level and writing all stats to the property tree.

Note that a similar issue was reported in the support forum: viewtopic.php?f=17&t=17249
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

Re: OpenGL "Out of Memory"?

Postby Hooray » Tue Aug 21, 2012 9:04 am

Could you please check again, if setting any of these properties to 0 during startup, has an effect or not:

  • --prop:/sim/rendering/building-density=0
  • --prop:/sim/rendering/vegetation-density=0

(just to see if this is related to the issue discussed here: viewtopic.php?f=68&t=17113&p=164821#p164821 )
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

Re: OpenGL "Out of Memory"?

Postby ot-666 » Tue Aug 21, 2012 9:12 am

Hooray wrote in Fri Aug 17, 2012 8:30 pm:Wow, that's surprising - something is definitely wrong here (I am currently running FG on a computer with just 2gb of RAM and don't see this, but many "eye candy" features are disabled), could you try finding out what feature needs to be disabled to stop the memory from gobbling up that severely?
Is this with or without Rembrandt enabled?


Whitout Remabrandt and it seems like that as soon i fly over any big enough custom scenery the memory consumtion is going way up.
It is not possible to fly with the fgfs 32bit version from LOWI to EDNY without a memmory related crash.

Hooray wrote in Fri Aug 17, 2012 8:30 pm:PS: Are you able to build from source, so that you could test patches? I recently talked to ThorstenB about extending his system monitor such that it also reports memory usage (allocations) per subsystem by overloading the new/delete operators at the SGSubsystem level and writing all stats to the property tree.


Sorry no :oops:

I will give the properties a try as soon i am back from work...
Callsign: ot-666
Working on LOWI and other stuff - Custom Scenery Overlay Repo: http://gitorious.org/fgfs-custom-scenery/custom-scenery-overlay/
VMX22 - Osprey... sometimes in 2014
ot-666
 
Posts: 746
Joined: Sun Nov 08, 2009 5:14 pm
Location: Germany, Konstanz
Callsign: ot-666
IRC name: ot666
Version: GIT
OS: win7 64bit

Re: OpenGL "Out of Memory"?

Postby Hooray » Tue Aug 21, 2012 12:50 pm

please check the properties individually. Also, did you even touch the buildings/vegetation settings, or did you see the 13+ gb of RAM usage using just the defaults?

And if they don't make a difference at all (not even if using both), it'd be helpful if you could see if the memory consumption improves by lowering the shader quality level. Can you reproduce this memory growth in locations other than LOWI, with similarly complex scenery - like KSFO or KLSV ? What about using random buildings and vegetation at KSFO? Does it also eat up several gb of memory?

It's possible that this is also related to the custom scenery you are using, so it would be good to exclude as many potential sources as possible - i.e. if disabling random buildings and vegetation doesn't have any noticeable effect, we need to look at other systems, too.

But until we have found the settings that make a real difference, it's hard to find the culprit in the source code.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

Re: OpenGL "Out of Memory"?

Postby Thorsten » Wed Aug 22, 2012 10:47 am

I can't reproduce the huge memory footprint.

For me, Flightgear dies when I dig into swap space. I have 4 GB of physical memory available. With these 4 GB, I've done frequently ~2 hour flights in custom France scenery with random buildings on. This creates a memory footprint of ~3.x GB, but memory gets again freed when scenery is unloaded, so it doesn't ever dig into swap space and doesn't increase over time.

Memory consumption is a problem only if I choose settings to influence peak memory, i.e. 100 km visibility in custom scenery usually is a killer, populating Seattle with random buildings is a killer,...

I have never been able to hit swap memory with trees and buildings off in normal scenery, even 250 km visibility limit (=more than a default Flightgear binary allows) did not lead to excessive memory usage.

So, I can only encourage everyone to follow Hooray's advice:

Monitor your system to find out what makes the difference!

* check if MP mode makes a difference - some people seem to get almost 600 MB more load when online!
* check if AI traffic is an issue for memory usage
* random buildings are known to be an issue - check how much it does on your system
* check the impact of scenery vertex density of default vs. custom (set visibility to some value, use the on-screen statistics to get the vertex count - see how that maps into memory usage)
* check the impact of shaders and other goodies
Thorsten
 
Posts: 10982
Joined: Mon Nov 02, 2009 8:33 am

Re: OpenGL "Out of Memory"?

Postby Hooray » Wed Aug 22, 2012 10:56 am

Just trying to find the lowest common denominator here: Thorsten are you running 32bit? I think ot666 runs 64bit? What about all the other people seeing this?
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

Re: OpenGL "Out of Memory"?

Postby ot-666 » Wed Aug 22, 2012 11:44 pm

I just did a quick test.
Using the ufo with buildings and trees off, 30km view distance, flying over custom france and no clouds.
fgfs hits 1.45 GB and with just 4 MP planes coming in goes to 1.56 GB.

This is no problem with a 64bit fgfs, but a lot of people still use a 32bit version.
Same flight with a 32bit fgfs crashed after around 15 minutes with a OpenGL "Out of Memory" error.

Same flight with view distance to 120km hits around 3.5 GB
Same flight with trees / 1.0 and view distance to 120km hits around 7.8 GB.
Callsign: ot-666
Working on LOWI and other stuff - Custom Scenery Overlay Repo: http://gitorious.org/fgfs-custom-scenery/custom-scenery-overlay/
VMX22 - Osprey... sometimes in 2014
ot-666
 
Posts: 746
Joined: Sun Nov 08, 2009 5:14 pm
Location: Germany, Konstanz
Callsign: ot-666
IRC name: ot666
Version: GIT
OS: win7 64bit

Re: OpenGL "Out of Memory"?

Postby Thorsten » Thu Aug 23, 2012 6:00 am

@Hooray: I'm running a 32bit, yes.

@ot-666: Well, the main question remains unanswered - do we deal with a peak memory usage problem, or do we deal with memory leaks? In other words, is there a certain probability if you are already close to the limit that you accidentially hit it, or does memory consumption really grow over time?

I suspect that it is peak memory, because I do relatively long flights over custom scenery without problems. Obviously the options need to be chosen such that the scene fits into your computer's memory. 120 km visibility range in custom scenery is excessive - this is something I never was able to do. With my 4 GB memory, I can safely do 40 km in custom scenery, that's it.

It's relatively clear that we don't have any problems filling memory by generating objects, trees or loading more terrain vertices. 2 GB only go so far, and seeing 10.000 square kilometers of terrain means a lot of terrain information to be present in memory. Flightgear can, with all options used, occupy a fairly powerful system. With the same options set, it can kill a normal system. It also seems clear that this ability to fill up memory causes problems at some point - I'm just not sure how to communicate this to users.
Thorsten
 
Posts: 10982
Joined: Mon Nov 02, 2009 8:33 am

Re: OpenGL "Out of Memory"?

Postby Hooray » Thu Aug 23, 2012 11:08 am

@ot-666: Could you please check if, after following Thorsten's recommendations, the memory consumption improves to reasonable levels - i.e. when using a similar visibility/terrain range?
Also, is this in any way affected by enabling or disabling the local weather system?

Thorsten wrote in Thu Aug 23, 2012 6:00 am:It also seems clear that this ability to fill up memory causes problems at some point - I'm just not sure how to communicate this to users.


Just a couple of days ago, Stuart and myself talked exactly about that here: viewtopic.php?f=5&t=16083&p=164936#p164833

As you can see, the idea is to get realtime stats from the OS on the free amount of RAM and the amount of swap space used by the fgfs process (easy for Linux) - and write that to the property tree at 1-5 sec intervals, so that Nasal scripts or property rules could use this info to show warnings/errors or even just to scale down some systems (random buildings, local weather, tile cache etc) dynamically.

Once this info is in the property tree, it could also be shown in the rendering dialog - i.e. "Amount of free RAM" vs. "Swap space already used".
In addition, Stuart worked out a way to compute an estimate of memory consumption for the random buildings - so that this could also be linked to a slider, i.e. not just in the form of a "density" setting (0..1), but rather in the form of a "MB/density" such as in 256MB steps for example.
So that people can directly see how much RAM is going to be used by the system.

We also talked about introducing a simple form of "hard quotas" to disable the system and prevent it from allocating new objects once a certain limit is reached.

ot-666 wrote:This is no problem with a 64bit fgfs, but a lot of people still use a 32bit version.

Yes, what is really needed is a good way to track memory consumption per subsystem, like the performance monitor works - such that we get to see exactly where memory is allocated, how many allocations per frame/second/minute etc - so that we can figure out where the real culprit is.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

Re: OpenGL "Out of Memory"?

Postby wrench11062 » Fri Aug 24, 2012 7:29 pm

Hello,

Here are shots of what I am experiencing . And the setup/cfg is set so that I do not recieve the "out of memory". Also, when I do go higher on "I" candy I get locked
to the scenery tile. Have to exit and scale down the graphic choices to be able to go worldwide.(Once flying that is.)

https://lh3.googleusercontent.com/-5Kvy ... driver.PNG
https://lh6.googleusercontent.com/-hheH ... pu-z22.gif
https://lh4.googleusercontent.com/-5cUr ... eading.gif
https://lh5.googleusercontent.com/-1-d1 ... emspec.PNG
https://lh6.googleusercontent.com/-1cKe ... en-017.png
https://lh6.googleusercontent.com/-TSQ1 ... fgrun1.PNG
https://lh3.googleusercontent.com/-A04m ... fgrun2.PNG
https://lh5.googleusercontent.com/-R0YD ... fgrun3.PNG
https://lh6.googleusercontent.com/-tg68 ... fgrun5.PNG

The last picture is just after while in play adjusting the tree level to .04 . Once I make that choice, theirs no more allowances and no going back. Unless you exit and restart.
And then u run the risk of being locked to the tile. BTW, it was a trip from KSFO to KLAX and weather for that was overcast big time, as I live in L.A. Ca. It matched exactly.
The idea of some info displayed while in the menu making choices would be just the ticket for helping/warning a person their-machine+whatever limitations.
I think it would go along way in putting the blame where it belongs also.
FWIW :|
Prime example of what happens when you put a monkey in a capsule full of buttons and switches...Now, wheres that damn banana?...
User avatar
wrench11062
 
Posts: 55
Joined: Sat Mar 24, 2012 1:55 am
Location: Los Angeles Ca. 1 nm east of KLAX 24R
Callsign: Tchaser/CrshLdr
Version: Git 64Bit
OS: Win7 64bit

Re: OpenGL "Out of Memory"?

Postby Thorsten » Sat Aug 25, 2012 9:32 am

Also, is this in any way affected by enabling or disabling the local weather system?


Since weather sets visibility, and memory consumption goes like (vertex_density + tree_density + building_density + other_stuff) * visibility^2, it's always weather that decides in the end if you run out of memory or not. This is trivial as far as it goes. Advanced Weather is perfectly capable to let you have 10 km visibility on the ground and 120 km as you climb to cruise altitude, which is a potential killer in custom scenery or with a lot of trees or buildings.
Thorsten
 
Posts: 10982
Joined: Mon Nov 02, 2009 8:33 am

Re: OpenGL "Out of Memory"?

Postby Hooray » Sat Aug 25, 2012 10:53 am

Yes, that's the "obvious" relationship - but I was thinking more along the lines of less-obvious connections here, such as e.g. Nasal etc.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

PreviousNext

Return to 2.8

Who is online

Users browsing this forum: No registered users and 2 guests