Board index FlightGear Support Graphics

How to tell if you are CPU or GPU limited (split)

Graphics issues like: bad framerates, weird colors, OpenGL errors etc. Bad graphics ar usually the result of bad graphics cards or drivers.
Forum rules
In order to help you, we need to know a lot of information. Make sure to include answers to at least the following questions in your initial post.

- what OS (Windows Xp/Vista, Mac etc.) are you running?
- what FlightGear version do you use?
- what graphics card do you have?
- does the problem occur with any aircraft, at any airport?
- is there any output printed to the console (black window)?
- copy&paste your commandline (tick the "Show commandline box on the last page of FGRun or the "Others" section on the Mac launcher).
- please upload a screenshot of the problem.

If you experience FlightGear crashes, please report a bug using the issue tracker (can be also used for feature requests).
To run FlightGear on old computers with bad OpenGL support, please take a look at this wiki article. If you are seeing corrupted/broken textures, please see this article.

Note: If you did not get a reponse, even after 7 days, you may want to check out the FlightGear mailing lists to ask your question there.

How to tell if you are CPU or GPU limited (split)

Postby Thorsten » Mon Sep 21, 2015 5:44 pm

I think the whole issue is very much mystified - unnecessarily so.

First, please note that Mathias described at one point on the mailing list what he's actually interested in - which is running a minimal graphics setup on modestly weak hardware at 60 fps with a hard requirement that frame duration never ever drops below that limit. Given how loading sometimes peaks and drags frame duration, I can readily see how that set of requirement is CPU limited - however I doubt that this is the normal use case.

The normal use case instead seems to be that a user has a computer a few years old, sees ALS screenshots and wants to have the same and sees low framerate. I'm guessing many users would readily trade 60 fps and minimal graphics into 20-30 fps with good graphics and would not have a hard requirement of frame duration.

In any case, the test for limitation is simple:

* switch ALS off, open the detailed shader quality management, switch all to minimum
-> this changes only shaders, nothing on the C++ side, so if framerate goes up, you've been limited by the GPU performance, if framerate remains low, it's CPU speed
-> other tell-tales for GPU limitation are increased framerate when looking at the sky or in fact any change of framerate with view angle

* if you're GPU limited, i.e. see a strong response of framerate to shader quality, switch higher quality back on, use to ufo to go to 3000 ft and compare the view 90 degrees down to the view towards the horizon
-> this changes the number of vertices in the view frustrum only, it is much lower looking down than towards the horizon - however the number of fragments to be processed for the ground is higher looking down. If you see higher framerate looking down than looking towards the horizon, you're vertex-shader limited, if it goes the other way you're fragment shader limited
-> other tell-tales for vertex shader limitation are strong framerate loss with larger visibility or random vegetation and (at least for ALS) no strong response to shader quality level (for technical reasons, it's much easier to leave stuff out of fragment shaders than vertex shaders)

* if you're CPU limited, the next test would possibly be a Nasal-heavy vs. a Nasal-light scenario

If we'd run these tests as a standard for low performance, we'd immediately have a better feeling for where the bottleneck really is. I think Mathias is really working on his use case, and promising that his work will make FG much faster will likely be a disappointment for all the users who want to run the high quality graphics settings which are GPU limited.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Best Version of FG.

Postby Hooray » Mon Sep 21, 2015 5:57 pm

If we'd run these tests as a standard for low performance, we'd immediately have a better feeling for where the bottleneck really is.

that's pretty coarse-grained and doesn't address any of the issues discussed by Mathias, Tim or Fred - but it'd be simple enough to come up with a Nasal module, e.g. "benchmark.nas" and run this as a test and set a few properties that can be displayed in the about dialog, or some dedicated "troubleshooting" dialog.

I am just not convinced, it's all that useful to begin with ...

Otherwise, we are really just talking about recording two separate fgtapes (flight recorder tapes, using either the ufo or ogel) , saving those in $FG_ROOT and running 20 lines of Nasal code to tell/evaluate the difference in frame rate/spacing between both "situations", which could be easily bound to a new item in the menubar
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Best Version of FG.

Postby Thorsten » Mon Sep 21, 2015 6:13 pm

that's pretty coarse-grained and doesn't address any of the issues discussed by Mathias, Tim or Fred


Yes it is - and it's designed to tell whether it is any of the issues discussed by Mathias, Tim or Fred what the particular user is seeing.

I don't mean to diagnose FG details, I mean to understand what the problem a particular user reports is related to. If framerate depends strongly on view angle or shader quality level, there's no point to look at Nasal for a culprit, because Nasal runs independent of view angle or shader quality level.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Best Version of FG.

Postby Hooray » Mon Sep 21, 2015 6:26 pm

I guess we have to give it a try then to see if it's any useful ... creating a fgtape for both situations with the ogel should be straightforward. I assume that fgtapes can be loaded using fgcommands (never checked that). So the next step would be putting the recorded fgtape files into $FG_ROOT and loading those via the menubar. Once that works, we really only need 20-30 lines of Nasal code to load both flights and watch what frame spacing/rate are doing, and then echo something to the console/GUI.

However, fgtapes don't sample any rendering/weather related properties, so we would need to look at README.flightrecorder to see if those properties can be added, so that this stuff isn't overridden by end-users - aka, a fgtape used like this would need to sample/override the corresponding settings
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Best Version of FG.

Postby Thorsten » Tue Sep 22, 2015 6:20 am

I know you're fond of automatic solutions, but I propose that a user simply fires up the ufo and does the tests and reports numbers. Far easier than posting a tape which somebody has to inspect. That would be a second-level diagnostic.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Best Version of FG.

Postby Hooray » Tue Sep 22, 2015 9:28 am

Thorsten wrote in Tue Sep 22, 2015 6:20 am:Far easier than posting a tape which somebody has to inspect. That would be a second-level diagnostic.


You misunderstood: The point was that we create such a tape, add it somewhere under $FG_ROOT and provide a menubar option to load/run and benchmark both scenarios - I don't think any end-user will find this less user-friendly than having to do the whole thing manually.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: How to tell if you are CPU or GPU limited (split)

Postby Philosopher » Tue Sep 22, 2015 10:15 pm

By tape, Hooray, do you mean a profile of rendering options (i.e., properties to set for a scenario)?
Philosopher
 
Posts: 1593
Joined: Sun Aug 12, 2012 7:29 pm

Re: How to tell if you are CPU or GPU limited (split)

Postby hamzaalloush » Tue Sep 22, 2015 11:21 pm

Philosopher wrote in Tue Sep 22, 2015 10:15 pm:By tape, Hooray, do you mean a profile of rendering options (i.e., properties to set for a scenario)?


I'm wondering about this as well, i have interest in such a case, where a set scenario can be replicated and reported back for "benchmarking" purposes.

I actually do this now, for evaluating scenery loading on FPS, for example:

I automated an ILS approach for my airport of choice (OEJN) with the F-16, and recorded fdm path to file using the generic protocol at a rate of 25 hz:

Code: Select all
--vc=230 --nav1=159:109.300 --prop:/autopilot/locks/altitude=gs1-hold --prop:/autopilot/locks/speed=speed-with-throttle --prop:/autopilot/settings/target-speed-kt=250 --prop:/sim/menubar/visibility=false --prop:/sim/ai-traffic/enabled=false --generic=file,out,25,flight.out,playback


I then proceeded to set the "graphics" level at 1(on the scale of 0-5), with ALS turned on, and everything else(random building, vegetation, etc) turned off, i then playback the approach using the UFO, while disabling PUI elements, and log FPS high, FPS low, to a file, while touching no controls at all(including camera).

Code: Select all
--prop:/sim/menubar/visibility=false --prop:/sim/ai-traffic/enabled=false --visibility=20000 --disable-clouds --disable-clouds3d --prop:/sim/rendering/draw-mask/clouds=0 --generic=file,in,25,flight.out,playback,repeat --fdm=external


I actually get very repeatable results, within 1 FPS, until i get GPU heatsoak for which i wait ten minutes and redo the test.
hamzaalloush
 
Posts: 631
Joined: Sat Oct 26, 2013 10:31 am
OS: Windows 10

Re: How to tell if you are CPU or GPU limited (split)

Postby Thorsten » Wed Sep 23, 2015 6:08 am

You misunderstood: The point was that we create such a tape, add it somewhere under $FG_ROOT and provide a menubar option to load/run and benchmark both scenarios - I don't think any end-user will find this less user-friendly than having to do the whole thing manually.


Okay, got it.

I don't see the real problem in looking at the horizon vs. the ground manually, if a user can't do that we probably won't have fruitful interaction anyway, but if you can automate it, why not.

I'm wondering about this as well, i have interest in such a case, where a set scenario can be replicated and reported back for "benchmarking" purposes.


It has been discussed a few times... Usually, I want something tailored for the case I'm interested in anyway, for instance for scenery rendering without trees, desert is my favourite case, for vegetation I choose tropical islands for the contrast between island in view and open sea, for special-purpose shaders I need to go to a location where the shader is, places like KSFO might have significant impact from things like AI traffic which we actually want out of the equation,...

So the idea in principle is good, however making sure that a) your scenario tests what you're interested in and b) the users all have the same thing installed and the same options on is not quite that trivial.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: How to tell if you are CPU or GPU limited (split)

Postby Hooray » Wed Sep 23, 2015 6:23 am

Philosopher wrote in Tue Sep 22, 2015 10:15 pm:By tape, Hooray, do you mean a profile of rendering options (i.e., properties to set for a scenario)?



fgtape = $FG_ROOT/Docs/README.flightrecorder: http://sourceforge.net/p/flightgear/fgd ... htrecorder

(what used to be the old "instant replay" system, but ended up getting generalized by ThorstenB to become a generic "Property Recorder" subsystem.

The system could be useful for doing both 1) creating repeatable benchmark situations that override end-user configuration, but also for 2) sharing end-user created flights for reproducing certain issues.

The case we've been discussing here is to use the flight recorder subsystem (fgtape) to create/save a "benchmark flight" into $FG_ROOT, so that end-users can use that to "play" the flight, while a few lines of Nasal would sample a handful of properties to determine the impact on frame rate/spacing.

The alternative would have been to use Nasal for creating the situation/flight in a scripted fashion, but that is probably not such a good idea, especially because Nasal may very well add to the CPU overhead anyway

Right now, fgtapes are all about aircraft-specific properties: http://sourceforge.net/p/flightgear/fgd ... trecorder/

So we would need to come up with a new XML profile covering rendering related properties.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: How to tell if you are CPU or GPU limited (split)

Postby Hooray » Wed Sep 23, 2015 9:44 am

I'm wondering about this as well, i have interest in such a case, where a set scenario can be replicated and reported back for "benchmarking" purposes.


I just looked at this, and it seems it's indeed relatively straightforward, with two minor issues so far:

With fgaddon being now stripped down to just the ufo and the c172p, it is no longer safe to assume that other aircraft are available/installed.

While it would make sense to use the ufo for the test, the ufo is not even using a proper FDM, i.e. its FDM is a pseudo FDM scripted in Nasal - which ends up not being "supported" by the flight recorder subsystem currently, because that assumes that the FDM engine in use is either jsbsim or yasim.

Like I said, fixing that should just involve including an XML file from the -set.xml file with a few relevant properties, but the next issue is that the scripted FDM probably needs to be taught how to "replay" data (I haven't actually tested/reviewed that code). In general, it should work if the autopilot/route-manager work with the ufo though - because those are using the same /controls properties.

So next I looked at using a similarly simple aircraft like the ogeL, becaue it is using a real FDM (JSBSim), but that is no longer in $FG_ROOT due to the fgAddon migration. With the package manager/aircraft center working again, it would be possible to automatically install the required aircraft (if it isn't available) to run a benchmark.

Or, alternatively, provide a different -set.xml file for the ufo, using the ogeL FDM and get that committed to $FG_ROOT, so that we end up with a simple aircraft that is using an actual FDM, that is supported by the flight recorder.

The next item would be reviewing the rendering.xml dialog to determine the most relevant settings that such a "benchmark" (fgtape) should override, and create a corresponding fgtape config file that we put into $FG_ROOT/Aircraft/Generic/flightrecorder/rendering.xml

That way, any "benchmarks" could include this file, so that the user settings can be reliably overridden when it comes to rendering.

The fgcommands for saving/loading fgtapes can be seen here: http://sourceforge.net/p/flightgear/fgd ... ar.xml#l15
  • flight-recorder-save
  • flight-recorder-load

fgcommands can be invoked from Nasal using the fgcommand() API: http://wiki.flightgear.org/Nasal_librar ... mand.28.29

Any "fgtapes" that we come up with, would be ideally put into $FG_ROOT, e.g. a location like $FG_ROOT/Tests or even $FG_ROOT/Benchmarks

That would ensure that we can easily add a menu item to menubar.xml to load/run a tape and measure/log the impact on frame rate/spacing.

I guess, Thorsten's heuristics could be the foundation for a number of different benchmarks in the time to come, which should greatly help us simplify the troubleshooting process on the forum, because we could ask end-users to run certain tests from a library in $FG_ROOT and report back with the results.

Equally, a few of us would be wasting less time debating whether a certain end-user is actually CPU-limited or CPU-limited :lol:
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: How to tell if you are CPU or GPU limited (split)

Postby bugman » Wed Sep 23, 2015 9:51 am

Hooray wrote in Wed Sep 23, 2015 9:44 am:While it would make sense to use the ufo for the test, the ufo is not even using a proper FDM, i.e. its FDM is a pseudo FDM scripted in Nasal - which ends up not being "supported" by the flight recorder subsystem currently, because that assumes that the FDM engine in use is either jsbsim or yasim.


The UIUC FDM is also supported. Maybe the resurrection of the original Magic Carpet FDM might be useful for benchmarking and bottleneck hunting? We can turn this on by changing one compilation flag from 'OFF' to 'ON'. Would this work?

Regards,

Edward
bugman
Moderator
 
Posts: 1808
Joined: Thu Mar 19, 2015 10:01 am
Version: next

Re: How to tell if you are CPU or GPU limited (split)

Postby Hooray » Wed Sep 23, 2015 9:59 am

To be honest, I completely forgot about that one - and I also missed it when looking into $FG_AIRCRAFT/ufo.
But I agree that it would make sense to have a simple aircraft, like the ufo, with a proper FDM - otherwise, all the Nasal-based FDMs are unlikely to work particularly well with other features/subsystems of the simulator, including the instant replay/flight recorder subsystem, but also things like the autopilot/route-manager, which could be just as useful for coming up with semi-automated tests that can be executed for benchmarking/regression testing purposes.

My suggestion would be to proceed with this, e.g. by:
  • editing $FG_ROOT/gui/menubar.xml to add a new "troubleshooting" item (including a matching entry in $FG_ROOT/Translations)
  • adding a dedicated $FG_ROOT/gui/troubleshooting.xml GUI dialog with a como box/dropdown menu that lists all files in $FG_ROOT/Tests
  • create two fgtape files, as per Thorsten's heuristics, and add those to $FG_ROOT/Tests
  • provide a separaten rendering.xml file for $FG_AIRCRAFT/Generic/flightrecorder, which covers rendering related settings - to ensure that the fgtape overrides any end-user settings

That way, we would end up with a new GUI dialog that shows a library of existing "tests", along with a button to load/execute those (which are really just conventional "fgtape" files, which include support for rendering related settings).

The next step would then be to add a few lines of Nasal to determine the impact on frame rate/spacing.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: How to tell if you are CPU or GPU limited (split)

Postby hamzaalloush » Wed Sep 23, 2015 1:05 pm

I used the generic "playback" protocol with a good degree of consistency, i used the --fdm=external command, wouldn't this prevent any FDM from actually executing, since it just follows a recorded path?

i did four tests, no scenery as a baseline, and then scenery at graphics levels 0 through 2,

for the sake of inspecting consistency, i plotted the four data series of FPS against Time(a benchmark consisting of exactly 70 seconds), stacked on top of each other, apparently the less limited i was in rendering capability, the more some bottle-necks were pronounced:

Image

and here is the actual average FPS for each case, calculated by averaging /sim/frame-rate, and /sim/frame-rate-worst (which i assume is the worst possible frame rate, given the highest latency frame) for the full run:

Image

The test were done on an Acer notebook, with a modest GT540, on Ubuntu with the latest proprietary stable driver, i could have done the test on a more capable system, but that wouldn't have been representative of most systems.
hamzaalloush
 
Posts: 631
Joined: Sat Oct 26, 2013 10:31 am
OS: Windows 10

Re: How to tell if you are CPU or GPU limited (split)

Postby Hooray » Wed Sep 23, 2015 3:19 pm

apparently the less limited i was in rendering capability, the more some bottle-necks were pronounced:


that's really cool, if we could add the corresponding hooks to do create these kind of statistics/graphs more easily, we could grow a really useful library of regression tests, with frame rate/frame spacing really just being the start of a thorough benchmark suite.

Even though I guess, that it would make sense to use the minimal startup profile (no scenery) as the baseline for all benchmarks, so that the impact of different subsystems (think AI/MP, FDM etc) can be measured against a really "lean" startup profile.

Which is why I believe that the baseline should be without scenery/terrain, but also without shaders/effects.

Ideally, we would grow a handful of "tests" and then try those with different variables, i.e. aircraft/location.
Given the SIGAR patches we have, we can also add actual RAM/SWAP utilization into the mix, too.

To be really useful, such a benchmark should ideally wait for 30-60 seconds after booting FG, to ensure that all subsystems are up and running.

My suggestion would be to set up a table of startup settings on the wiki (.fgfsrc), so that people can more easily reproduce "scenery 0...x". Alternatively, we could add a directory to $FG_ROOT, e.g. $FG_ROOT/Profiles and patch FG to load .fgfsrc files from there. That would help ensure that people are actually running the same startup settings.

In its current form, this is already pretty cool and informative, especially if we could create a table for different airports/aircraft to show the impact of those settings, and how they scale with things like visibility/shader quality.

But I would hope that we can add more metrics, and expose those to the property tree - e.g. stuff like RAM utilization (for which we have working code), but also other stuff like Nasal GC stats, but also listener/callback overhead.

I am convinced that this could be really instrumental to understand certain issues, without people necessarily having to be core developers, or requiring them to be able to patch/build fg from source and use profilers/debuggers correctly.

It would be also cool to see if we can speed up simulator time to execute those tests faster.

If you haven't already, please feel free to upload such screen shots to the wiki.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Next

Return to Graphics

Who is online

Users browsing this forum: No registered users and 3 guests