Board index FlightGear Development New features

FlightGear Benchmark

Discussion and requests for new features. Please note that FlightGear developers are volunteers and may or may not be able to consider these requests.

FlightGear Benchmark

Postby Hooray » Thu Apr 22, 2010 12:55 pm

HHS wrote:It would really help if we had a benchmark.

Comparing FGFS 2.0.0 on different computer systems, OS, graphic cards etc.


Several people mentioned that they would like to have some form of "benchmark" to run FlightGear on various different platforms to see how it performs.

I think the idea is not that bad, and that this might actually help troubleshoot some issues.
Also, I do think that such a benchmark could probably be implemented directly in FlightGear, just by using Nasal scripting and some custom XML files.

This would be pretty much related to the idea of "feature scaling" which was discussed in the other thread.

One would only need a way to create a default situation (i.e. like a custom preferences.xml file) and a way to dynamically toggle FlightGear features on/off and tweak them at runtime.

This should be pretty straightforward to do, at least for those features (configuration properties) that are already using listeners or that are read every frame. This applies to most of the recent graphics additions (i.e. shaders), because these can be dynamically enabled, disabled and configured.

So a FlightGear benchmark would then only have to be run with common default settings (e.g. window resolution, color depth, startup airport, aircraft and environment settings) while a Nasal script could then be used to dynamically tune these settings.

Reading internal counters (namely the framerate counter for the time being) would then give us an instrument to see how significant certain settings are.

So, is there any interest in a FlightGear/Nasal based benchmark?
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FlightGear Benchmark

Postby grtux » Thu Apr 22, 2010 2:30 pm

Hooray wrote:
...................................................;;;
Reading internal counters (namely the framerate counter for the time being) would then give us an instrument to see how significant certain settings are.

So, is there any interest in a FlightGear/Nasal based benchmark?



Yes why not a benchmark, though, each individual could get huge differences with similar equipments, or worth, when a supposed to be less powerful equipment gives similar fps performance, for instance one of my other computer with AGP graphics card ..... :) .

But i am feeling , from my recent experience , we need, advice from Graphics system and architecture experts.
It is said such NNNNN graphics card is better than that other one GGGGG graphics card,

WHY ?

I experienced that frequencies are not representatives, the shaders capabilities are there with modern GPU, the memory capacity, could be or not, representative, according to our desktop configuration and so on...............

With a benchmark, we are not certain to define the best configuration, since there is others individuals equipments/operating system/software/parameters which could influence the result.


A valuable Benchmark is "everything else in equal condition" which won't be the case. :(
Cheers

Gérard
g.robin
LFMO
User avatar
grtux
 
Posts: 431
Joined: Thu Dec 14, 2006 5:19 pm
Location: Provence France

Re: V2.0.0 Performance issues (split from Development/Feature Po

Postby Hooray » Thu Apr 22, 2010 4:04 pm

In the beginning, the easiest way to have something like a benchmark in FlightGear would be to simply use static "situations" that are loaded from XML files, these would then override all local custom settings so that users can reliably compare their frame rates when running such "situations" on different machines.

The only problem is that FlightGear always makes the assumption that it is running some form of aircraft/vehicle, so any sort of "benchmark" needs to be provided as an aircraft. Also, one needs to override the global preferences.xml file because there is no way to use a different one.

For example, running the following "benchmark aircraft" (which is not really doing anything yet, just disabling all eye candy and going to a place without any scenery in wireframe mode), gives me about 550-600+ fps.

That in itself might already be useful to show how capable a machine is when it does not have to render much.

If there is any interest to look further into this, we could create more situations (different airports, aircraft and rendering settings), these could then be added to the "Benchmark" aircraft folder (in the form of more *-set.xml files) so that everybody could run these to see how their machine performs.

Once you take a look at the attached file, you'll that this is pretty simple stuff - and that the scheme itself is flexible enough to be easily extensible so that one could create more complex benchmarks.

And at some point, one could also really use Nasal to dynamically benchmark FlightGear, so that features can be adjusted accordingly, one could even use scripted demo flights to stress test FlightGear (e.g. using the f14 and the autopilot to control the aircraft from Nasal).

FlightGear "bare bones" benchmark (no fdm, no eye candy, no nothing... )
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FlightGear Benchmark

Postby grtux » Thu Apr 22, 2010 5:24 pm

Hooray,

But you don't control, these individuals equipments/operating system/software/parameters content ( like i said before ) , but if you ask to every tested equipment to use exactly the same "environment" ,
You must take care FlightGear is not "Stand Alone". We are not testing some Black Box Game system.

The differences could come from elsewhere ......

I'll give you a stupid example,
within Linux with KDE, if i declare the specific window to display FlightGear without border i get some more fps,
==> Exactly the same FlightGear condition 66 fps with border and 71 fps without border. :) :) .

cheers

Gérard
g.robin
LFMO
User avatar
grtux
 
Posts: 431
Joined: Thu Dec 14, 2006 5:19 pm
Location: Provence France

Re: FlightGear Benchmark

Postby Hooray » Thu Apr 22, 2010 6:14 pm

I am aware of that, the benchmark was not my idea - but having a number of benchmarks available could probably provide useful metrics to get FlightGear to run. For example, even the very simple file that I posted can already be used for troubleshooting: if a user is not able to run this with more than 100 fps, he is unlikely to be able to run FlightGear with default settings.

I realize that there are many things that may contribute to a system not working properly, but just having a way to test individual features would be useful.

Just imagine we would create a bunch of additional "benchmarks" like this, each of those testing individual features of FlightGear (shaders, effects, particles, shadows, AI aircraft and so on), all of these could be useful to allow users to see if their system (and configuration) is able to run FlightGear or if it needs to be modified (software/hardware configuration).
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU


Return to New features

Who is online

Users browsing this forum: No registered users and 3 guests