Monitor system performance

[Necolatis]

I have looked at the wiki (http://wiki.flightgear.org/Howto:Use_the_system_monitor) for the monitor, and have some additional questions.

1 - Does flight stand for FDM?

2 - Does total/ms stand for milliseconds used during last second?

This is an example for the aircraft I am working on:

[Philosopher]

Yes to both, AFAICT. You might get some extra performance if you disable ai traffic, I notice that it usually slows my system down by a lot – in your case it's about comparable to Nasal/events, so not insanely huge. What's the deal with the FDM though?

[Necolatis]

Ok, thank you.

What's the deal with the FDM though?

I wish I knew..

[Gijs]

Did you find the cause of the performance hit from the FDM? The version in Git shows around 110 total/ms.

I seem to have something similar on the 744 now. Altough about half as bad (ca. 400 total/ms) as in your screenshot, it's spending much more time on flight compared to the P51D for example. The P51D also has a big/complex FDM, so it looks like it's not (just) the size/complexity of our FDMs. Maybe a certain JSBSim function that both our aircraft use is troublesome?

EDIT; I've reverted the FDM to the version that was shipped with 2.4, back in 2012. Surprisingly the system monitor value is pretty much the same Makes me wonder if the monitor groups other things under "flight"...

EDIT2: Apparently not. When I launch the 744 with the UFO fdm, it's down to just 4 total/ms.

[Hooray]

the monitor will only put the FDMShell() object under "flight, $FG_SRC/FDM/fdm_shell.cxx

to see what/where exactly it is taking to so long, you can put SGTimeStamp statements in a handful of places and log them to a property, which you can then monitor using the property browser (not I/O bound that way)

[Gijs]

I'd definitely.need some more detailed instructions on that I'm very interested in finding the source of it, as even a near empty FDM on the 744 seems to be heavier than any other complete aircraft.

[hvengel]

As a general rule JSBSim should be fairly light since, for the most part, it looking forces up in tables and summing the results. Not particularly compute intensive.

Although the P-51D has a big/complex FDM and it does a number of unusual things like accounting for cooling drag, exhaust thrust, compressibility, mach drag, how Reynolds numbers affect drag and lift... these all take place in fairly simple look up driven JSBSim functions that add very little additional overhead to the FDM even if some of the look up tables are unusually large and have more dimensions that the typical case which is one of the things that makes the FDM appear to be big and complex.

If these other aircraft are really using significantly more CPU time doing FDM functions then these have to be somewhere outside of the core FDM functionality or some of the FDM functions have to have been customized in a way that makes them compute intensive. Perhaps some of the FDM is being driven by Nasal functions and the compute time being used by these functions is being included in the FDM number? For the P-51D I use very little Nasal and this is, for the most part, outside of the FDM although some FDM switch functions look at properties that are driven by Nasal code. It might also be possible that some of the FDM functions are related in such a way as to cause some type of recursion (IE. circular references).

I had a quick look at the performance monitor using the current dev version of the P-51D. The FDM is 99% the same as the last release. Even at an FDM rate of 240Hz it is only using around 72ms/sec which means it is using about 300 micro seconds per FDM iteration. This is less than one tenth what the OP is seeing for his aircraft. The FDM overhead is not even close to being an issue in this case where as for the OP's aircraft it is a huge issue.

Where I see the biggest hit is when the guns are being fired as this generates a large number of sub-models and the ai-model submodule number spikes way up during long bursts of gun fire. This goes to around 370 before leveling off because the custom gun animation throttling code starts kicking in about 2 seconds into the burst on my machine which limits the number of active sub-models. Without the throttling code this would go much higher. The throttling code is part of the FDM and starts progressively limiting eye candy gun related animations (IE. not needed to have an OK gun animation) as the frame rate drops below 30FPS. These are a very simple set of JSBSim switches so doing this type of thing is fairly easy. The gun animations do not change the FDM number.

I notice that the OP is seeing an FDM iteration rate of about 210Hz so it appears that he is using a higher FDM rate than the default 120Hz. This would account for some of the higher than expected FDM number. My tests indicate that going from 120Hz to 240Hz increases the flight number in the system performance monitor by about 50%. If Gijs was using the default FDM rate then I would expect the number for the 744 to get up to around 600 with a 240Hz rate.

[Necolatis]

I fixed the FDM problem by only changing properties the fdm relies on if they are changed.

For example, I have a loop, which checks if the droptank is attached, it would write to pointmass every update. Now it only writes if the pointmass has changed. Same for many other properties. I suspect some writes to certain properties will make jsbsim rewrite some internal tables it relies on, and that would degrade performance, but I do not know.

So every time I write to any property inside fdm/jsbsim I now first check that it is actually different, otherwise I do not write it. That fixed my issue.

[hvengel]

One other thing to test/consider is that the P-51D is a fairly basic single engine piston aircraft and at least one of these other FDMs is a multi engine jet. Could the increased FDM over head be jet engine or multi engine related? Do single engine JSBSim jets show significantly more FDM over head than the P-51D? Do single engine FDMs have significantly less FDM over head than similar complexity multi-engine aircraft with the same basic type of engines?

[hvengel]

...So every time I write to any property inside fdm/jsbsim I now first check that it is actually different, otherwise I do not write it. That fixed my issue.

Nasal generated updates to properties that are being used in the FDM?

[Gijs]

Thanks for the ideas guys. I don't believe I write to fdm/jsbsim from Nasal (I have various JSBSim systems to prevent just that), but I'll double check that.

It would explain why a near empty FDM file (I removed the propulsion, systems, aerodynamics and flight controls) still takes up 100-200 ms.

[Necolatis]

Nasal generated updates to properties that are being used in the FDM?

Yes

[Necolatis]

The size of the jsbsim model, does not seem to matter much, from Jon S. on the jsbsim mailing list:

I have a flight model that has lots of tables – hundreds – and where the aero database is about a quarter million lines long, and it easily runs in real-time at 300 Hz

[Hooray]

properties in the main tree (Nasal etc) are only updated at frame-rate, while the FDM runs at a higher rate, so that is something to keep in mind.
Understanding this means that you can 1) slow down things terribly, but also 2) run Nasal at FDM rate.

[hvengel]

The size of the jsbsim model, does not seem to matter much...

That was what I was trying to say when I pointed out the the P-51D FDM is "large and complex" (but by implication this is large and complex by FG standards) but that I didn't think that added much to the FDM overhead. In fact compared to someone running JSBSim on an FDM where there is complete aero data, like in the one in the quote from Jon S., the P-51D FDM is actually very small at less than 3,000 lines or about 1/9 the size. In addition, since these are table driven each of the table look ups will have the same execution overhead no matter how big any given table ( O(1) for the geeks out there ). But I would expect the number of tables to have close to a linear impact of how fast the FDM can iterate. So it is the number of table look ups and not the amount of data in the tables that matters.

I would even go so far as to say that having a "flight" number in the system performance monitor of more than say 100 or 150 ms/s with a 120Hz FDM on fast hardware is an indication of a serious issue if the FDM is a JSBSim FDM.