Board index FlightGear Release candidates 2.8

OpenGL "Out of Memory"?

This is the archive of topics about the 2.8 release candidates.

Re: OpenGL "Out of Memory"?

Postby ot-666 » Sat Aug 25, 2012 4:29 pm

Thorsten wrote in Thu Aug 23, 2012 6:00 am:@ot-666: Well, the main question remains unanswered - do we deal with a peak memory usage problem, or do we deal with memory leaks? In other words, is there a certain probability if you are already close to the limit that you accidentially hit it, or does memory consumption really grow over time?

I suspect that it is peak memory, because I do relatively long flights over custom scenery without problems. Obviously the options need to be chosen such that the scene fits into your computer's memory. 120 km visibility range in custom scenery is excessive - this is something I never was able to do. With my 4 GB memory, I can safely do 40 km in custom scenery, that's it.


After testing some more i think it is peak memory, but with a 32bit fgfs you hit the crash limit after just a couple of minutes over custom scenery and only 30km visibilety (with no trees and buildings). Weather/ local weather seems to have only a small memory impact. But local weather always performes much better.

With a more excessive visibilety range then 120km the memory consumtion is quite big and i guess with buildings and trees, fgfs it is just not able to load and generate all the buildings fast enough. It looked like the memory was still growing, but it was still busy loading tiles and generating buildings.

Testing with 120km visibilety:
Over papillon81 custom scenery at EDDF, with a 120km LOD and local weater set to 250km, the memory consumpion settels at 6.156.400 GB staying at one place (76000ft).
After flying a couple of minutes memory goes up to 14.900.000 GB. But after stoping settels around 14.400.000 GB
Image

Oliver
Callsign: ot-666
Working on LOWI and other stuff - Custom Scenery Overlay Repo: http://gitorious.org/fgfs-custom-scenery/custom-scenery-overlay/
VMX22 - Osprey... sometimes in 2014
ot-666
 
Posts: 746
Joined: Sun Nov 08, 2009 5:14 pm
Location: Germany, Konstanz
Callsign: ot-666
IRC name: ot666
Version: GIT
OS: win7 64bit

Re: OpenGL "Out of Memory"?

Postby Hooray » Sat Aug 25, 2012 5:38 pm

I think neither FlightGear nor the weather system(s) were ever designed with such visibilities (>=100 km) in mind - so we need to focus on "real" use cases that are likely to be encountered by our users. If FlightGear just starts heavy swapping under such "extreme" circumstances, things seem to be fine to me ...

That said, it would obviously be good to keep in mind that CPUs and GPUs (and thus, computers) are going to become more and more powerful ... so it would be great if FG scaled a little better than that.

Now, regarding the 32 bit binary: That would be the most interesting use case actually, because nobody else is seeing this here - and on my 32 bit computer, I have even less RAM installed (just 2 gb). So maybe you could come up with a minimal list of settings startup required to trigger this (i.e. a segfault/crash) reliably? That would enable us to run some diagnostics (debugger/profiler, leak checker) to see what's going on.

The funny thing is, at 76k feet, you are hitting many of the limitations that are relevant in the context of orbital flight ;-)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

Re: OpenGL "Out of Memory"?

Postby Thorsten » Sat Aug 25, 2012 6:35 pm

After testing some more i think it is peak memory, but with a 32bit fgfs you hit the crash limit after just a couple of minutes over custom scenery and only 30km visibilety (with no trees and buildings).


I don't hit the crash limit even after spending 2 hours in custom scenery with buildings and 40 km visibility range. The thing about the 32bit seems to be that it can't use swap space. So... the difference here may be that I have 4 GB of real memory whereas you have less. If so, we understand something.

I think neither FlightGear nor the weather system(s) were ever designed with such visibilities (>=100 km) in mind - so we need to focus on "real" use cases that are likely to be encountered by our users.


I assure you that Advanced Weather will generate you a very real 120 km visibility at 36.000 ft unless you specifically tell it not to. This is part of the problem: We have several memory-hungry systems which are developed independently.

* 120 km visibility is no real problem in default scenery. With custom scenery with 10 times higher linear resolution and 100 times the vertex count it becomes a problem very soon. With the new random buildings, it becomes a problem very soon.

* 100 times higher vertex count in custom scenery is no real problem with 20 km visibility and urban shader. With the random buildings or large visibility, the whole game changes.

* random buildings are no problem in 25 km visibility and default scenery. In custom scenery or with large visibility, the rules change quickly.

You can currently sort of max out each of the options without a problem, provided you don't use the two others at all. The attempt to max out all of them is doomed, and even the attempt to set all to half is doomed because of the non-linearity. I guess most people are not aware that there is really a factor 100 or so between memory consumption of default and custom scenery. Default scenery hardly makes a memory impact - custom scenery does so in a significant way so that the visibility you can support in custom scenery is just ~1/10 of what works in default scenery. I guess most people also don't realize that trees and buildings easily triple the memory footprint of custom scenery.

The funny thing is, at 76k feet, you are hitting many of the limitations that are relevant in the context of orbital flight


It can't really be done with the default engine - EarthView is just so way superior in visuals and memory footprint above 100 km altitude that the way forward is to create a more sophisticated version of that rather than doing something to the default engine.
Thorsten
 
Posts: 10959
Joined: Mon Nov 02, 2009 8:33 am

Re: OpenGL "Out of Memory"?

Postby Hooray » Sat Aug 25, 2012 7:02 pm

I wasn't trying to suggest anything else.
That said, even 4gb of RAM on a 32bit system would usually just mean that you can adress 3/4 of it - unless you have PAE enabled in your kernel, or other support for address extension.
Also, I am not sure if I understand why 32 bit system's shouldn't be able to use swap?
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

Re: OpenGL "Out of Memory"?

Postby i4dnf » Sat Aug 25, 2012 7:22 pm

I've done some simple tests today, and I can definitely say that the terrain itself is not the issue, the issues are the trees/random-buildings, and the textures and their usage.[1]
I've done a very bare materials file, one that assigns the same simple texture to all materials and disables everything else (effects, shaders, trees, buildings) on the terrain.
Results are as follows (for 16km, 32km and 70km visibilty, weather was disabled --prop:/environment/metar/data='NIL\n', I used the UFO so that I can see just what kind of memory the terrain is using, values in MB, linux 64bit)

Code: Select all
visibility km   16   32   70
         
LRSB bare mat         
tsync terrain   307   308   310
custom terrain  475   495   502
         
LOWI bare mat         
tsync terrain   388   391   392
custom terrain  725   736   742



As you can see, while there's an increase in memory usage between terrasync terrain and custom terrain, it's not that dramatic as you might expect.

LOWI represents a mixed usecase, where there are a lot of custom objects. The first case is not that bad, just on the order of tens of MB of more memory used. The Custom testcase shows the real issue, ~300MB of memory used just by custom objects, almost as much as just the terrain mesh.

Now for comparison here's the data just for 16km of visibility for LRSB, with the default materials file, 1.5 trees density:

Code: Select all
LRSB default mat   
tsync terrain    800
custom terrain   1304


So just with terrain and models we're using 800MB with the default terrain. With the custom terrain the issue becomes bigger. Most of that increase is solely due to trees/random buildings in the default terrain case, and due to extra textures in the custom terrain case.

Now let's look how much an aircraft hits. You'd be surprised, but our aircraft models are actualy light ;) :

Code: Select all
LRSB bare mat         
tsync terrain                307
custom terrain               475
   
tsync terrain + IAR80        516.8
tsync terrain + Hurricane    447.5
Tsync terrain + b1900d       815.6


See that, just ~210MB for the IAR80, 140MB for the Hurricane and a whopping ~510MB for the b1900d (due to many big textures used for the effects, and due to the fact that those textures are kept uncompressed in RAM).

Notes:[1]As much as it's a neat feature, I consider (and this data confirms), that we shouldn't use the multiple choice of textures in the material files, we're only making a bad issue worse, the more the terrain is detailed, the more it will load extra textures for the same landclass.

side Note: Thorsten, don't take this the wrong way, but you really need to do something about all that math in the vertex shaders. With default terrain, the test materials-file, enabling the atmospheric scattering completely destroyed performance dropping fps in the 20-30 range, while with just the transition shader enabled it stayed capped at 60fps due to vsync being enabled (it was likely higher than that), no matter how much terrain thus vertex data there was to process.

In conclusion, we have two separate issues. One already known, and most likely fixable, is that random-buildings/trees eat a lot of memory.
The second one, we waste a lot of memory with uncompressed textures, and thus with extra textures entries in the materials file, or with detailed scenery models.

How the second one should be solved I don't know... :(

Thorsten: The hard crash limit for a 32bit process would be around 3.2GB of memory used by that process IIRC. That can easily mean 5GB of total memory used. Monitor the fgfs process memory usage and see when it crashes.
i4dnf
Retired
 
Posts: 745
Joined: Wed Sep 09, 2009 7:17 am
Location: LRBS
Callsign: YR-I4D
Version: GIT
OS: Gentoo Linux ~amd64

Re: OpenGL "Out of Memory"?

Postby Hooray » Sat Aug 25, 2012 7:46 pm

Hey, this is VERY interesting.

Textures have ALWAYS been a problem in FG, not just in the scenery - but also in aircraft (all cockpit instruments are made off stacked textures)

Could you please upload the files that you customized somewhere and share them with us, for our own testing?
Especially, ot-666 should give this a try and report back with his own use-cases.
Also note that I wrote a little patch to track FG's total memory consumption and swap-usage, available here: viewtopic.php?f=5&t=16083&p=165168#p164936

i4dnf: I think we should work out a way to combine our work, so that we could come up with a little "benchmark", I will look into what's needed to allow materials.xml to be re-loaded at runtime, so that multiple versions can be tested subsequently.

Then, we could probably use a Nasal script to load various "test cases" (materials.xml and runtime settings) and track their memory consumption.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

Re: OpenGL "Out of Memory"?

Postby i4dnf » Sat Aug 25, 2012 8:17 pm

Here you go:

Code: Select all
<?xml version="1.0"?>
<PropertyList>
  <params>
   <forest>
     <!-- Maximum distance from which a tree is visible -->
     <tree-range-m>8000</tree-range-m>
   </forest>
  </params>
  <material>
   <name>BarrenCover</name>
   <name>Dirt</name>
   <name>Dump</name>
   <name>OpenMining</name>
   <name>Rock</name>
   <name>cover1</name>
   <name>cover2</name>
   <name>signcase</name>
   <name>Scrub</name>
   <name>BidirectionalTaper</name>
   <name>UnidirectionalTaper</name>
   <name>UnidirectionalTaperGreen</name>
   <name>UnidirectionalTaperRed</name>
   <name>pa_shoulder</name>
   <name>pa_shoulder_f</name>
   <name>pc_shoulder</name>
   <name>pc_shoulder_f</name>
   <name>AgroForest</name>
   <name>Airport</name>
   <name>AirportKeep</name>
   <name>Asphalt</name>
   <name>BareTundraCover</name>
   <name>BlackSign</name>
   <name>Bog</name>
   <name>BuiltUpCover</name>
   <name>Burnt</name>
   <name>Canal</name>
   <name>Cemetery</name>
   <name>ComplexCrop</name>
   <name>Construction</name>
   <name>CropGrass</name>
   <name>CropGrassCover</name>
   <name>CropWood</name>
   <name>CropWoodCover</name>
   <name>DeciduousBroadCover</name>
   <name>DeciduousForest</name>
   <name>DeciduousNeedleCover</name>
   <name>Default</name>
   <name>DryCrop</name>
   <name>DryCropPastureCover</name>
   <name>DryLake</name>
   <name>Estuary</name>
   <name>EvergreenBroadCover</name>
   <name>EvergreenForest</name>
   <name>EvergreenNeedleCover</name>
   <name>FloodLand</name>
   <name>FramedSign</name>
   <name>Freeway</name>
   <name>Glacier</name>
   <name>GolfCourse</name>
   <name>Grass</name>
   <name>GrassCover</name>
   <name>Grassland</name>
   <name>Greenspace</name>
   <name>Heath</name>
   <name>HerbTundra</name>
   <name>HerbTundraCover</name>
   <name>HerbWetlandCover</name>
   <name>Industrial</name>
   <name>IntermittentLake</name>
   <name>IntermittentReservoir</name>
   <name>IntermittentStream</name>
   <name>IrrCrop</name>
   <name>IrrCropPastureCover</name>
   <name>Island</name>
   <name>Lagoon</name>
   <name>Lake</name>
   <name>Landmass</name>
   <name>Lava</name>
   <name>Littoral</name>
   <name>Marsh</name>
   <name>MixedCrop</name>
   <name>MixedCropPastureCover</name>
   <name>MixedForest</name>
   <name>MixedForestCover</name>
   <name>MixedTundraCover</name>
   <name>NaturalCrop</name>
   <name>Ocean</name>
   <name>Olives</name>
   <name>Orchard</name>
   <name>PackIce</name>
   <name>PolarIce</name>
   <name>Pond</name>
   <name>Port</name>
   <name>RUNWAY_LIGHTS</name>
   <name>RWY_BLUE_TAXIWAY_LIGHTS</name>
   <name>RWY_GREEN_LIGHTS</name>
   <name>RWY_GREEN_LOW_LIGHTS</name>
   <name>RWY_GREEN_MEDIUM_LIGHTS</name>
   <name>RWY_GREEN_TAXIWAY_LIGHTS</name>
   <name>RWY_LIGHTS</name>
   <name>RWY_RED_LIGHTS</name>
   <name>RWY_RED_LOW_LIGHTS</name>
   <name>RWY_RED_MEDIUM_LIGHTS</name>
   <name>RWY_REIL_LIGHTS</name>
   <name>RWY_SEQUENCED_LIGHTS</name>
   <name>RWY_VASI_LIGHTS</name>
   <name>RWY_WHITE_LIGHTS</name>
   <name>RWY_WHITE_LOW_LIGHTS</name>
   <name>RWY_WHITE_MEDIUM_LIGHTS</name>
   <name>RWY_YELLOW_LIGHTS</name>
   <name>RWY_YELLOW_LOW_LIGHTS</name>
   <name>RWY_YELLOW_MEDIUM_LIGHTS</name>
   <name>RWY_YELLOW_PULSE_LIGHTS</name>
   <name>Railroad</name>
   <name>RainForest</name>
   <name>RedSign</name>
   <name>Reservoir</name>
   <name>Rice</name>
   <name>Road</name>
   <name>Saline</name>
   <name>SaltMarsh</name>
   <name>Sand</name>
   <name>Sand</name>
   <name>SavannaCover</name>
   <name>Sclerophyllous</name>
   <name>Scrub</name>
   <name>ShrubCover</name>
   <name>ShrubGrassCover</name>
   <name>SnowCover</name>
   <name>SomeSort</name>
   <name>SpecialSign</name>
   <name>Stream</name>
   <name>SubUrban</name>
   <name>Town</name>
   <name>Transport</name>
   <name>Unknown</name>
   <name>Urban</name>
   <name>Vineyard</name>
   <name>Watercourse</name>
   <name>WoodedTundraCover</name>
   <name>WoodedWetlandCover</name>
   <name>YellowSign</name>
   <name>dirt_rwy0l</name>
   <name>dirt_rwy0r</name>
   <name>dirt_rwy11</name>
   <name>dirt_rwy1c</name>
   <name>dirt_rwy1l</name>
   <name>dirt_rwy1r</name>
   <name>dirt_rwy2c</name>
   <name>dirt_rwy2l</name>
   <name>dirt_rwy2r</name>
   <name>dirt_rwy3c</name>
   <name>dirt_rwy3l</name>
   <name>dirt_rwy3r</name>
   <name>dirt_rwy4c</name>
   <name>dirt_rwy4r</name>
   <name>dirt_rwy5c</name>
   <name>dirt_rwy5r</name>
   <name>dirt_rwy6c</name>
   <name>dirt_rwy6r</name>
   <name>dirt_rwy7c</name>
   <name>dirt_rwy7r</name>
   <name>dirt_rwy8c</name>
   <name>dirt_rwy8r</name>
   <name>dirt_rwy9c</name>
   <name>dirt_rwy9r</name>
   <name>dirt_rwy</name>
   <name>dirt_rwyC</name>
   <name>dirt_rwyL</name>
   <name>dirt_rwyR</name>
   <name>dirt_rwyaim</name>
   <name>dirt_rwyaim_uk</name>
   <name>dirt_rwycenterline</name>
   <name>dirt_rwyrest</name>
   <name>dirt_rwytaxiway</name>
   <name>dirt_rwythreshold</name>
   <name>dirt_rwytiedown</name>
   <name>dirt_rwytz_one_a</name>
   <name>dirt_rwytz_one_b</name>
   <name>dirt_rwytz_three</name>
   <name>dirt_rwytz_two_a</name>
   <name>dirt_rwytz_two_b</name>
   <name>grass_rwy</name>
   <name>lakebed_taxiway</name>
   <name>lf_broken_white</name>
   <name>lf_checkerboard_white</name>
   <name>lf_dbl_lane_queue</name>
   <name>lf_dbl_lane_queue_border</name>
   <name>lf_dbl_solid_yellow</name>
   <name>lf_dbl_solid_yellow_border</name>
   <name>lf_ils_hold</name>
   <name>lf_ils_hold_border</name>
   <name>lf_other_hold</name>
   <name>lf_other_hold_border</name>
   <name>lf_runway_hold</name>
   <name>lf_runway_hold_border</name>
   <name>lf_safetyzone_centerline</name>
   <name>lf_safetyzone_centerline_border</name>
   <name>lf_sng_broken_yellow</name>
   <name>lf_sng_broken_yellow</name>
   <name>lf_sng_broken_yellow_border</name>
   <name>lf_sng_lane_queue</name>
   <name>lf_sng_lane_queue_border</name>
   <name>lf_sng_solid_white</name>
   <name>lf_sng_solid_yellow</name>
   <name>lf_sng_solid_yellow_border</name>
   <name>pa_0l</name>
   <name>pa_0r</name>
   <name>pa_11</name>
   <name>pa_1c</name>
   <name>pa_1l</name>
   <name>pa_1r</name>
   <name>pa_2c</name>
   <name>pa_2l</name>
   <name>pa_2r</name>
   <name>pa_3c</name>
   <name>pa_3l</name>
   <name>pa_3r</name>
   <name>pa_4c</name>
   <name>pa_4r</name>
   <name>pa_5c</name>
   <name>pa_5r</name>
   <name>pa_6c</name>
   <name>pa_6r</name>
   <name>pa_7c</name>
   <name>pa_7r</name>
   <name>pa_8c</name>
   <name>pa_8r</name>
   <name>pa_9c</name>
   <name>pa_9r</name>
   <name>pa_C</name>
   <name>pa_L</name>
   <name>pa_R</name>
   <name>pa_aim</name>
   <name>pa_centerline</name>
   <name>pa_dspl_arrows</name>
   <name>pa_dspl_thresh</name>
   <name>pa_no_threshold</name>
   <name>pa_rest</name>
   <name>pa_stopway</name>
   <name>pa_taxiway</name>
   <name>pa_threshold</name>
   <name>pa_tiedown</name>
   <name>pa_tz_one_a</name>
   <name>pa_tz_one_b</name>
   <name>pa_tz_three</name>
   <name>pa_tz_two_a</name>
   <name>pa_tz_two_b</name>
   <name>pc_0l</name>
   <name>pc_0r</name>
   <name>pc_11</name>
   <name>pc_1c</name>
   <name>pc_1l</name>
   <name>pc_1r</name>
   <name>pc_2c</name>
   <name>pc_2l</name>
   <name>pc_2r</name>
   <name>pc_3c</name>
   <name>pc_3l</name>
   <name>pc_3r</name>
   <name>pc_4c</name>
   <name>pc_4r</name>
   <name>pc_5c</name>
   <name>pc_5r</name>
   <name>pc_6c</name>
   <name>pc_6r</name>
   <name>pc_7c</name>
   <name>pc_7r</name>
   <name>pc_8c</name>
   <name>pc_8r</name>
   <name>pc_9c</name>
   <name>pc_9r</name>
   <name>pc_C</name>
   <name>pc_L</name>
   <name>pc_R</name>
   <name>pc_aim</name>
   <name>pc_aim_uk</name>
   <name>pc_centerline</name>
   <name>pc_dspl_arrows</name>
   <name>pc_dspl_thresh</name>
   <name>pc_no_threshold</name>
   <name>pc_rest</name>
   <name>pc_stopway</name>
   <name>pc_taxiway</name>
   <name>pc_threshold</name>
   <name>pc_tiedown</name>
   <name>pc_tz_one_a</name>
   <name>pc_tz_one_b</name>
   <name>pc_tz_three</name>
   <name>pc_tz_two_a</name>
   <name>pc_tz_two_b</name>
   <solid>1</solid>
   <texture>Terrain/unknown.png</texture>
<!--    <effect>Effects/transition-base-grass-rock</effect> --> <!--uncomment this to test shader impact-->

<!--<object-mask>Terrain/city1.mask.png</object-mask>
  <wood-coverage>4000.0</wood-coverage>
  <tree-texture>Trees/coniferous-summer.png</tree-texture>
  <tree-varieties>8</tree-varieties>
  <tree-range-m alias="/params/forest/tree-range-m"/>
  <tree-height-m>15.0</tree-height-m>
  <tree-width-m>8.0</tree-width-m>--> <!--Uncomment this to test trees, WARNING: HUGE MEMORY CONSUMPTION-->
 
<!--<object-mask>Terrain/city1.mask.png</object-mask>
  <building-coverage>2500.0</building-coverage>
  <building-small-ratio>0.4</building-small-ratio>
  <building-small-min-floors>3</building-small-min-floors>
  <building-small-max-width-m>30.0</building-small-max-width-m>
  <building-small-min-depth-m>10.0</building-small-min-depth-m>
  <building-small-max-depth-m>30.0</building-small-max-depth-m>
  <building-medium-ratio>0.6</building-medium-ratio>
  <building-large-ratio>0.2</building-large-ratio>
  <wood-coverage>250000.0</wood-coverage>--><!--Uncomment this to test buildings, WARNING: HUGE MEMORY CONSUMPTION-->
  </material>
</PropertyList>
 


I don't think changing materials files at runtime is a good idea, things loaded stay loaded for a while, so you might get skewed results. Better restart FG with a new materials file selection. --materials-file=Materials/custom-materials-file.xml works just fine ;)
i4dnf
Retired
 
Posts: 745
Joined: Wed Sep 09, 2009 7:17 am
Location: LRBS
Callsign: YR-I4D
Version: GIT
OS: Gentoo Linux ~amd64

Re: OpenGL "Out of Memory"?

Postby Hooray » Sat Aug 25, 2012 8:25 pm

good point, we could use a shell script then and sample the memory consumption for different settings by reading /proc/pid/smaps directly.
EDIT: getting a segfault here (at KSFO/28R), because the custom materials.xml doesn't contain any runway glyphs apparently:
OBJECT_SIGN: unsupported glyph '2'
OBJECT_SIGN: unsupported glyph '8'
OBJECT_SIGN: unsupported glyph 'L'
OBJECT_SIGN: unsupported glyph '-'
OBJECT_SIGN: unsupported glyph '1'
OBJECT_SIGN: unsupported glyph '0'
OBJECT_SIGN: unsupported glyph 'R'
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11329
Joined: Tue Mar 25, 2008 8:40 am

Re: OpenGL "Out of Memory"?

Postby i4dnf » Sun Aug 26, 2012 12:15 am

Add this at the end. I've tested at airports without signs so this didn't become apparent.
Code: Select all
<material include="Materials/base/glyphs-yellow.xml">
  <name>YellowSign</name>
  <texture>Signs/yellow.png</texture>
  <emissive>
   <r>0.9</r>
   <g>0.9</g>
   <b>0.9</b>
  </emissive>
  <xsize>4096</xsize>
  <ysize>128</ysize>
</material>

<material include="Materials/base/glyphs-red.xml">
  <name>RedSign</name>
  <texture>Signs/red.png</texture>
  <emissive>
   <r>0.9</r>
   <g>0.9</g>
   <b>0.9</b>
  </emissive>
  <xsize>4096</xsize>
  <ysize>128</ysize>
</material>

<material include="Materials/base/glyphs-black.xml">
  <name>BlackSign</name>
  <texture>Signs/black.png</texture>
  <emissive>
   <r>0.9</r>
   <g>0.9</g>
   <b>0.9</b>
  </emissive>
  <xsize>4096</xsize>
  <ysize>128</ysize>
</material>

<material include="Materials/base/glyphs-framed.xml">
  <name>FramedSign</name>
  <texture>Signs/framed.png</texture>
  <emissive>
   <r>0.9</r>
   <g>0.9</g>
   <b>0.9</b>
  </emissive>
  <xsize>4096</xsize>
  <ysize>128</ysize>
</material>

<material include="Materials/base/glyphs-special.xml">
  <name>SpecialSign</name>
  <texture>Signs/special.png</texture>
  <emissive>
   <r>0.9</r>
   <g>0.9</g>
   <b>0.9</b>
  </emissive>
  <xsize>1024</xsize>
  <ysize>128</ysize>
</material>
i4dnf
Retired
 
Posts: 745
Joined: Wed Sep 09, 2009 7:17 am
Location: LRBS
Callsign: YR-I4D
Version: GIT
OS: Gentoo Linux ~amd64

Re: OpenGL "Out of Memory"?

Postby Thorsten » Mon Aug 27, 2012 9:47 am

Thorsten, don't take this the wrong way, but you really need to do something about all that math in the vertex shaders. With default terrain, the test materials-file, enabling the atmospheric scattering completely destroyed performance dropping fps in the 20-30 range


Personally, I get hit much harder by fragment shader operations, i.e. my system actually speeds up if I shift even more stuff to the vertex shader or drop fragments (I've over time tried various schemes where to compute things) :-( A 20-30 fps range seems pretty good to me though - I'm usually happy to get 15-20 fps.

Having said all this - I would love to speed things up in any way. I've been staring for weeks at the code trying to identify redundant operations or to replace expensive stuff by cheap stuff. My ability to speed the code up is exhausted, and while I agree in theory with your comment, I don't see how to do it in practice. If you find any way to squeeze a significant performance increase out, I'll buy you a few beers...
Thorsten
 
Posts: 10959
Joined: Mon Nov 02, 2009 8:33 am

Re: OpenGL "Out of Memory"?

Postby i4dnf » Tue Aug 28, 2012 8:58 am

Well, 20-30fps would be enough if that would be with a fully populated world, but that happens with nothing else loaded but the bare terrain, one checkered texture, and the UFO.

One quick and dirty optimization you could do, is to try to perform all the light computations only on vertices "facing" the camera. Something like this:

Code: Select all
....
vec4 eyeDir = normalize(gl_ModelViewMatrix * gl_Vertex);

if(dot(eyeDir.xyz,normalize(normal)) < 0.1){

do all the haze layer

} else {

do some simple fallback stuff

}
....



It might or might not improve the situation.
i4dnf
Retired
 
Posts: 745
Joined: Wed Sep 09, 2009 7:17 am
Location: LRBS
Callsign: YR-I4D
Version: GIT
OS: Gentoo Linux ~amd64

Re: OpenGL "Out of Memory"?

Postby Thorsten » Tue Aug 28, 2012 9:36 am

I thought we cull back-facing stuff during the first pass based on the declaration in the effect file? I can have a look if that helps.

I've tried a scheme to not render nominally invisible (100% fogged) vertices at one point, but that was tricky to get free of artefacts.

With default terrain, the test materials-file, enabling the atmospheric scattering completely destroyed performance dropping fps in the 20-30 range, while with just the transition shader enabled it stayed capped at 60fps due to vsync being enabled (it was likely higher than that), no matter how much terrain thus vertex data there was to process.


Not a fair comparison btw. - the atmospheric scattering framework has easily a factor 5 more computations to do than the transition shader, so it is way slower even on a purely algorithmical level regardless of the implementation.

Otherwise - shuffling everything into the fragment shader gives me a rather stable framerate pretty independent of terrain and visibility range. Problem is - that stable framerate is too low to begin with (if I shuffle every operation to the fragment shader, I get about 8-10 fps for real flight conditions, no matter terrain detail and visibility range). Shuffling more stuff to the vertex shader gives me a visibility limit in detailed terrain of ~40 km, if I would do 120 km I would drop to 2-4 fps, but in default terrain I get way higher framerates (for 80-100 km visibility I still end up around 20ish).

The water shader is a good example for fragment-only load - in stormy weather and overcast skies, I end up too low to do carrier landings, although the framerate is very stable. So I do things like fragment masks (and wrote a lightweight water shader which drops 70% of the partial waves - still looks good enough from high up) to get to usable framerates.

Usually, I see the framerate dictated by my fragment pipeline, but that may be peculiar for my system.
Thorsten
 
Posts: 10959
Joined: Mon Nov 02, 2009 8:33 am

Re: OpenGL "Out of Memory"?

Postby stuart » Tue Aug 28, 2012 9:49 am

Hi Guys,

Just a heads up that I'm currently looking at how to reduce the memory footprint of the random buildings. I'm hoping to be able to use an instantiation scheme similar to the random trees.

Thorsten - you mentioned above that "random buildings are no problem in 25 km visibility and default scenery. In custom scenery or with large visibility, the rules change quickly.". Reading this to mean that random buildings become more of a problem with custom scenery, do you have any insight into why that is the case? Is it simply that custom scenery has more Town and Urban land?

-Stuart
G-MWLX
User avatar
stuart
Moderator
 
Posts: 1469
Joined: Wed Nov 29, 2006 9:56 am
Location: Edinburgh
Callsign: G-MWLX

Re: OpenGL "Out of Memory"?

Postby Thorsten » Tue Aug 28, 2012 10:03 am

I don't think the buildings change in any meaningful way in custom scenery. But there's a sumrule - memory spent to store terrain vertex information is no longer available for buildings. Same for the vertex shader load. My point was that a configuration which is tested to run fine in default scenery may not do so in hires scenery.

I suspect buildings may even be worse in default scenery, because there even suburbs usually get classified as urban, so you end up creating more buildings in default scenery than in hires scenery which has a different landclass for the less densely populated suburbs.
Thorsten
 
Posts: 10959
Joined: Mon Nov 02, 2009 8:33 am

Re: OpenGL "Out of Memory"?

Postby Thorsten » Tue Aug 28, 2012 10:07 am

See that, just ~210MB for the IAR80, 140MB for the Hurricane and a whopping ~510MB for the b1900d (due to many big textures used for the effects, and due to the fact that those textures are kept uncompressed in RAM).


Well, that would be why multiplayer is a killer for some, no? If I see an IAR80, a Beech 1900d and a Hurricane on the horizon, my memory consumption just grew by 860 MB. If airplanes create that much of an impact, they can blow any system, because it's pretty much uncontrollable what you get to meet.
Thorsten
 
Posts: 10959
Joined: Mon Nov 02, 2009 8:33 am

PreviousNext

Return to 2.8

Who is online

Users browsing this forum: No registered users and 0 guests