Board index FlightGear Release candidates 3.0

fg3.0rc1 memory usage

This is the archive of topics about the 3.0 release candidates.

Re: fg3.0rc1 memory usage

Postby Thorsten » Sun Feb 09, 2014 7:56 am

Thorsten, you are kidding yourself this time - you are making this more complex than it is - obviously, things are not just linear in complex setups like ALS/Rembrandt - but that's simply not the point: there's more to FG than just the rendering pipeline


I am giving examples from the rendering pipeline, not because I think it's the only thing in FG, but because it's an area which I understand rather well (and because I think on many systems it's the limiting factor for performance). Sorry if that gave the wrong impression. Obviously, once you factor other things in, bottleneck finding and optimization become even more complex and nonlinear, not less :-)

I think we had the same debate when we were talking about GL-level profilers


Yes, we had. I am fairly sure it's pointless to try to understand how one shader performs in isolation, it won't tell you much how it performs inside the full FG environment. The optimal performance of a shader running standalone isn't the optimal performance of a shader running inside FG. But in the end, I want to optimize performance inside FG, nothing else. Nobody is going to congratulate me for getting 50% higher performance in standalone mode when it causes 20% less framerate inside FG...

Also, optimal shader performance often goes hand in hand with increased memory usage - often the choice is between computing things per frame or storing them from frame to frame - the first costs performance, the second memory.

We may just have to disagree on that one. I think I have good reasons for doing things a certain way...

It does remind me on what happend to ubuntu and other distros too. (...)They also started deleting support for older hardware, while it is still usefull, the OS want be running on it or no more graphics drivers...


I frankly think that's not what's happening here. People go through great pains to implement new stuff in optional ways. Replacing the default renderer by ALS would be way easier than implementing ALS in an optional way such that aircraft-defined effects work in both frameworks. Replacing Basic by Advanced Weather would be much easier than to come up with a common, 'neutral' interface to which any weather system can talk to. Writing and maintaining a single terrain shader would be much easier than write and maintain three quality levels which have different demands on the GPU.

Possibly a quarter of my time goes into implementing stuff in such a way that you can run FG with essentially the options of 1.9.1 or that you can scale back performance needs. We don't even delete the legacy 256x256 terrain textures which duplicate hires textures, so if you have low GPU memory, you can go to the global texture scheme, delete the Textures.hires folder and run with very low texture resolution on the terrain.

In pretty much every discussion on the devel list, having a fallback mode whatever new feature is implemented featured prominently.

The only thing that really works differently between 1.9.1 and 3.0 on a legacy system is that 1.9.1 will work right out of the box, whereas for 3.0 you will have to configure some to make it work.

So the whole discussion is really not about whether we should commit to run FG on legacy systems. The whole devel community is committed to that goal. The discussion is really about how straightforward this should be. Hooray evidently argues that all installation should default to legacy systems, the majority of the devel community has the trend to set the defaults to the assumption that you run on a present-day moderately powerful computer.

And what angers me personally is that there's just zero recognition that we go through all the pains of implementing new things optionally and keeping the legacy door open. I never seem to be dealing with people who say 'Hey, it's great that this still runs on my old XYZ system, all I had to do was...' I only seem to run into people who expect it to work out of the box, complain about growing harddisk needs of the installation, don't bother to read up on some config options and then complain that FG is only for rich people, that we'd ban users and similar nonsense.

So yeah, if you want to run present day software on a legacy system, what's wrong if you have to configure some yourself?

The whole development would really be much simpler if we'd do it the commercial way - we define system requirements, make sure the whole results fits into these, and if we add cool features we just raise them. We could get rid of plenty of legacy code, textures and such, get a more streamlined installation process, reduce the number of user-controllable options to something way more intuitive - there'd be plenty of goodies to that. Except we're not doing it.
Thorsten
 
Posts: 11470
Joined: Mon Nov 02, 2009 8:33 am

Re: fg3.0rc1 memory usage

Postby adrian » Sun Feb 09, 2014 10:51 am

Thorsten wrote in Sun Feb 09, 2014 7:56 am:
And what angers me personally is that there's just zero recognition that we go through all the pains of implementing new things optionally and keeping the legacy door open. I never seem to be dealing with people who say 'Hey, it's great that this still runs on my old XYZ system, all I had to do was...' I only seem to run into people who expect it to work out of the box, complain about growing harddisk needs of the installation, don't bother to read up on some config options and then complain that FG is only for rich people, that we'd ban users and similar nonsense.

So yeah, if you want to run present day software on a legacy system, what's wrong if you have to configure some yourself?


We need to appease your anger fast: there is at least one recognition for all the pains, me. I am very grateful that I can run the latest shaders, your local weather and other goodies on my oldie machine with a reasonable framerate. But don't forget it is the samurai coder way to make everything as efficient as possible, and make it run even on the toaster. That's not the case with terrain here, there is lots of room for improvement, right now terrain rendering is very innefficent and quite last century. An upgrade would not only make it run on toasters, but also give us almost unlimited distance and other underlying facilities. But it is a hard endeavor, perhaps one of the hardest. Do you want to take the challenge?
adrian
 
Posts: 362
Joined: Wed Sep 15, 2010 2:15 pm

Re: fg3.0rc1 memory usage

Postby Thorsten » Sun Feb 09, 2014 11:49 am

right now terrain rendering is very innefficent and quite last century. An upgrade would not only make it run on toasters, but also give us almost unlimited distance and other underlying facilities. But it is a hard endeavor, perhaps one of the hardest. Do you want to take the challenge?


When they start doing 28 hour days perhaps? There's only so much work I can realistically do in a 24 hour day.

I'd do the terrain LOD in a fairly simple way - have precomputed terrain in two (three?) resolution levels on disk, change the terrain loading system such that it replaces tiles as you get close.

Inclination on the devel list goes sometimes to different solutions - going completely into deferred rendering, rendering roads and rivers (which are now very vertex-heavy) as decals onto the terrain etc. That's going to relieve the vertex shader just as well, but cost more memory again.

Personally, I'd like to largely dispense with the landclass vector terrain structure and just leave a coarse vector layer underneath to supply coarse landclass information and generate all details smaller than 100 m procedurally - we'd get rid of the artifical straight lines in the terrain, vertex resolution is only determined by the elevation mesh,...

We don't have an agreed-upon plan, and we don't even have shared goals what the solution should be or for whom. It's pointless to modify the terrain loading routines if the scenery team won't generate and store terrain in different resolution levels for instance.

So what this needs is a discussion into which direction we want to go. Which doesn't seem to be happening.

Anyway - any really good, modern and fast (on a modern system) solution for the terrain is going to leave legacy systems out in the cold.
Thorsten
 
Posts: 11470
Joined: Mon Nov 02, 2009 8:33 am

Re: fg3.0rc1 memory usage

Postby adrian » Sun Feb 09, 2014 12:16 pm

Sounds good to me. Having three or four resolution levels for tiles seems like the most straightforward way, and likely the easiest to implement.
I had thought of a different LOD system, which would have only one LOD, but assign different importance to certain points in the mesh. Store this precomputed relevance factor in two bits of memory, and load the points/ triangles based on the distance versus their importance to the mesh. Of course, this is not the hardest part at all, what's really difficult is dealing with texture coordinates once we strip out points from the mesh. Perhaps one of our Terragear gurus could chime in with factoids.
adrian
 
Posts: 362
Joined: Wed Sep 15, 2010 2:15 pm

Re: fg3.0rc1 memory usage

Postby Thorsten » Sun Feb 09, 2014 2:45 pm

There's no non-trivial uv-map for the terrain mesh. It's a simple projection into the local xy plane. You see distortions of the rock texture on steep faces sometimes due to that. So that's a non-issue as far as I know.
Thorsten
 
Posts: 11470
Joined: Mon Nov 02, 2009 8:33 am

Re: fg3.0rc1 memory usage

Postby sa7k » Sun Feb 09, 2014 3:40 pm

I agree with adrian in thanking for the efforts of making FG usable for the users of old pcs. I only have to disable the shaders and can use the sim on my 9 years old pc (5 years old video card though).
Also, you shouldn't mind the kind of people that would monopolize an entire town's internet connection just to download what he could easily get in many ways if he just made a little effort and even asks for apologies for being asked not to act like an entitled jerk.
Even the forum is really usable with my crappy internet connection. Thanks to everyone who contribute.
sa7k
 
Posts: 382
Joined: Fri Mar 16, 2012 2:24 pm
Location: SA7K
Callsign: LV-EPM
IRC name: sa7k
Version: git
OS: debian

Re: fg3.0rc1 memory usage

Postby punkepanda » Sun Feb 09, 2014 11:13 pm

My experience testing FG on various computer with different hardware setup is that FG does not use the GPU's full potential at all.
Testing it on a 4-5 year slow amd CPU with strong GPU AMD/ATI R9-730.

And the GPU has very little effect on framerate compared to X-Plane (as an example).
X-Plane gives me somewhere between 20-30 fps with full extreme terrain resulution, complete water refection rendering, mid setting on trafic, urban etc. Shadow on low overall. And with HDR rendering with FXAA on 1x. This I would say is about high to very-high detail setup.

At this setup X-Plane uses about 1.5 GB of memory.

This makes me wonder if FG even use the VRAM at all? Does it even compress textures?

The fact is that I get bether framerates in FG on my laptop with a Core i5 cpu with GForce 630m card then i do on a brand new ATI R9 card. Which on benchmark, should have alot more power. X-plane is really slow on my laptop. 10 fps if i turn off al clouds and default settings without any special effects and low resulution.

I also tried my old ATI card. 4-5 years old, same as the CPU. The fact is that i get almost the excact same framerate on a old GPU than I do with a brand new one. And which on benchmarking, if I remember correctly should be about 3-4 x faster in most situations.

So FG has bether frame rates on bad GPU and high CPU than vica versa.
X.plane and all other games ive tried to compare gives me more framerates on a bad CPU and a high GPU..

So the conclusion must be that there is something fundamental wrong with the way FG process the virtual world. This makes me wonder if FG with the current rendering framework even has the potential to potentialy become a nice looking simulator with flyable framerates. In my oppionion it has a atleat 5 times the memory consumption you would expect, from the visual result perceived by the eye. Same with framerates!

I do not expect to get a good answer from Thorsten on this, as he has already come to the conclusion that i dont even know what HDR rendering is and that i don't know what I am talking about at all. The truth is that I dont kow how to code OpenGL. But I dont think coding knowledge has anything todo with the facts that I just presentet.

So anybody with insights to this phenomena and why there is?
punkepanda
 
Posts: 237
Joined: Mon Nov 04, 2013 9:11 pm
Callsign: LostFlight
Version: 2.12
OS: Arch Linux

Re: fg3.0rc1 memory usage

Postby Hooray » Mon Feb 10, 2014 12:20 am

So FG has bether frame rates on bad GPU and high CPU than vica versa.
X.plane and all other games ive tried to compare gives me more framerates on a bad CPU and a high GPU..


You cannot really generalize this, it really depends on quite a few factors, such as startup and runtime settings, but many people have found that FG is indeed heavily CPU-bound, and that CPU upgrades often meant better performance in comparison to just GPU framerates. This is in line with observations made by a handful of core developers, see the devel list for details. You should also find plenty of details in the wiki or using the forum search here:

viewtopic.php?f=37&t=21805#p197956
viewtopic.php?f=18&t=19738#p181564
viewtopic.php?f=18&t=17448

You will find certain folks around here disagreeing with the "CPU-limited"-conclusion quite bit (despite core developers saying the opposite and having worked to remedy this for years) - but always ask those folks first what hardware they're using - "someone" around here keeps claiming that FG would not be "typically CPU bound" but completely ignores the fact that probably 99% of our users cannot spend $3000 US on dedicated gaming hardware - otherwise, there would probably be more people claiming that FG isn't CPU bound :D
Finally, even if you were to completey ignore the rendering side of things, there are algorithmic issues involved here - i.e. certain things would even be problematic in a purely "headless" mode - no matter if it's CPU or RAM resources that are burnt, and that's the kind of stuff that needs to be solved first - or people doing just eye-candy, won't get tired to discuss things that they don't fully understand - and which -quite frankly- they have no chance to understand unless they're given certain tools.

On average hardware with just the default settings, you will FG typically being CPU-limited - you only need to look at your frame-spacing (latency) how time is spent in the C++ subsystems to update things. Extremely fast hardware tends to mask such culprit, as has been previously highlighted by quite a few core developers, who said that they never ran into Nasal GC issues, due to their powerful hardware.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

Re: fg3.0rc1 memory usage

Postby Thorsten » Mon Feb 10, 2014 6:49 am

On average hardware with just the default settings, you will FG typically being CPU-limited - you only need to look at your frame-spacing (latency) how time is spent in the C++ subsystems to update things.


Sorry, but that's just nonsense. Time spent in the GPU influences frame spacing just as well. Frame spacing is determined by whatever the bottleneck is.

You will find certain folks around here disagreeing with the "CPU-limited"-conclusion quite bit (despite core developers saying the opposite and having worked to remedy this for years)


You will find that certain folks, unlike certain other folks and you in particular, posted a lot of observational data and test cases to back their conclusions.

But let's ask straight - how much framerate do you get in wireframe mode with all shaders off? Because then the rendering pipeline is trivial, but all the Nasal, FDM work, tile manager, weather and what not still runs.
Thorsten
 
Posts: 11470
Joined: Mon Nov 02, 2009 8:33 am

Re: fg3.0rc1 memory usage

Postby Hooray » Mon Feb 10, 2014 6:54 am

you are always trying to make this a black/white issue, which it really isn't. This is not about you being right or wrong, simple as that.
On SLOW hardware, you will typically see frame-spacing spikes despite ALL eye-candy being disabled. You are likely to completely miss such problems on fast hardware obviously.
Most but people here are not driving a Porsche.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

Re: fg3.0rc1 memory usage

Postby Thorsten » Mon Feb 10, 2014 6:57 am

But let's ask straight - how much framerate do you get in wireframe mode with all shaders off?


I mean it. Just back what you say with data please. Whichever way it points. That's what is going to convince me. If I'm wrong, if my understanding is incomplete, that's fine. But demonstrate it to me. Don't just make claims based on what other people say.
Thorsten
 
Posts: 11470
Joined: Mon Nov 02, 2009 8:33 am

Re: fg3.0rc1 memory usage

Postby Thorsten » Mon Feb 10, 2014 7:05 am

On SLOW hardware, you will typically see frame-spacing spikes despite ALL eye-candy being disabled


You'll see that on fast hardware just as well. Loading and mipmapping a heavy texture into GPU memory is a prime example - first view of the F-16 from the outside will freeze the system for a second or more. It happens when a model that's hires-textured comes into view. The UK regional textures use four different 2048x2048 texture sheets for agriculture - that sure brings my system to a short stop.

We could all use compressed and pre-mipmapped dds of course, that'd solve it. It's proprietary unfortunately.


Edit: Well, I realize it obviously matters what your expectations and needs are for a judgement where a limitation is.

Some folks need FG to run with 60 fps like a clockwork for simulation purposes, eye candy irrelevant, and a single frame duration of 200 ms every few minutes is inacceptable. If that is the requirement, FG is indeed CPU limited and HLA is the answer.

I'm sort of guessing that most users think of limitations in terms of 'I have now 8 fps, I would like to have 20 at least', like decent graphical settings and can live with a spike in frame duration every few minutes. Typically people in the forum post 'Low framerate' and not 'High frame latency'. If your needs are like this, FG is GPU limited and HLA is not the answer, improvements to the rendering pipeline are.


Edit++:

or people doing just eye-candy, won't get tired to discuss things that they don't fully understand - and which -quite frankly- they have no chance to understand unless they're given certain tools.


Yeah, I really don't have a clue... Guess I've demonstrated that over and over in the past. You know - just demonstrate that you understand better. That's really all I ask. Snide remarks can't replace a good, old-fashioned proof.
Thorsten
 
Posts: 11470
Joined: Mon Nov 02, 2009 8:33 am

Re: fg3.0rc1 memory usage

Postby punkepanda » Mon Feb 10, 2014 3:39 pm

Oki doki.. I've done a little more testing to this and im really surprised over the result. And I think that the fact that I now present would further prove that something is not right.

I took the UFO for a ride, turned off all effects and setting to lowest option on both machines, basic weather clear sky and used the excact same location and heading on the test.
And uffourse I did wait for some minutes for everything to be done loading and rendering at all altitudes and settings :)

Framerate at startup on ground:
Low GPU <-> High CPU = 55 fps
Low CPU <-> High GPU = 16 fps

Framerate at 10.000 ft:
Low GPU <-> High CPU = 22 fps
Low CPU <-> High GPU = 39 fps

Framerate at 10.000 ft with ALS and full on all settings:
Low GPU <-> High CPU = 7 fps
Low CPU <-> High GPU = 35 fps

So.. The conclusion most be that in fact the GPU does its work when a lot of terrain is visible and the other offects or on.

But the big question is: Whats going on at ground level? Why is CPU much more important than the GPU and vica versa at 10.000 ft? I understand that it is more to render at high altitude, but at lower altitude why has the CPU so much effect on framerate and why is the high GPU banging its head aginst the ground scenery?

Atleast this shows that there is room for a lot more visual effects on high GPU machines. And I would guess that so high memory consumption is primary because of uncompressed terain and that it does not use the VRAM (even tough some more testing on GPU level to be sure of this)
punkepanda
 
Posts: 237
Joined: Mon Nov 04, 2013 9:11 pm
Callsign: LostFlight
Version: 2.12
OS: Arch Linux

Previous

Return to 3.0

Who is online

Users browsing this forum: No registered users and 1 guest