Board index FlightGear Development Canvas

Canvas performance

Canvas is FlightGear's new fully scriptable 2D drawing system that will allow you to easily create new instruments, HUDs and even GUI dialogs and custom GUI widgets, without having to write C++ code and without having to rebuild FlightGear.

Canvas performance

Postby Hooray » Sun Oct 13, 2019 5:32 pm

Split off from the topic Fundamental FDM design problems.


swampthing wrote in Wed Oct 09, 2019 12:36 pm:... from my experience more JSBsim systems and less nasal = beter FPS. I don't think for some things you can move away from nasal all together. It comes down to what your plane requires. I'm not aware of any way to make canvas displays with JSBsim code for instance. I'm not against nasal but I can see where people may want to do full JSBsim sytems and I think for the most part if you can get a good grasp of it, you can with JSBsim.


Actually, that's a valid point: Originally, the whole Canvas idea started out as a property-driven 2D drawing system, but admittedly, what we ended up with is a system that is meanwhile tightly coupled to Nasal unfortunately. Indeed, there are some things where you definitely need to use Nasal to set up/initialize things. But under the hood, 99% still is pure property I/O, which is also why the property tree is becoming a bottleneck.

In general, Nasal is not the problem here - but the way the Canvas system is designed, and the way both, Nasal and Canvas, are integrated - it's a single-threaded setup, i.e. we are inevitably adding framerate-limited scripted code that runs at <= 60 hz to the main loop, to update rendering related state. This is a bit problematic, but it's not a real problem to fix.

It would be a problem to fix up existing Canvas-code (think NavDisplay, PFDs, EICAS etc), but with a few minor tweaks, we could come up with a dedicated Canvas mode where scripts updating a canvas property tree, are running out of the main loop. This would mean that they could not access any of the mainloop-APIs, but apart from that, it's actually a no-brainer, i.e. a straigthforward thing to do.

Out of the box, OSG comes with support for creating and updating textures asynchronously, we just aren't using this currently - for obvious reasons, coding such a Canvas texture, would be a different thing. But the hooks required to make this happen, are fairly straigthforward.

One starting point would be changing the assumption that all canvas texture PROPERTIES live in the global property tree, instead each Canvas texture would get its own SGPropertyNode, which isn't accessible from anywhere else.

At that point, you have a Canvas/OD_Gauge context that can be updated by changing said PRIVATE property tree. As long as this property tree is only ever updated from single place (thread), multi-threading things becomes possible, because you only need to serialize access whenever you want fetch/display the updated texture. But apart from that, the update/redraw mechanism could be running in a background thread.

From a Canvas perspective, one obvious issue is dealing with Canvas textures that fetch data/imagery from other textures, because that, too, would require synchronization.

But other than that, you would end up with a Canvas system whose textures can be asynchronously updated by a background thread, scripts doing so would look a bit different, because they would lack access to 90% of the common FG APIs (think geodinfo and friends), because those cannot be considered to be thread-safe.

As you can probably tell, this is something that we once discussed behind the scenes - and it would nicely align with the original idea of "remote properties", i.e. sync'ing and replicating properties between property trees from different threads/processes, the main thing needed to do this is a subscribe/publish mechanism that works over sockets (or some other IPC): http://wiki.flightgear.org/Remote_Properties

This is something where Richard's Emesary work could become highly useful, because the cost of adapting the Canvas system to optionally support an out-of-mainloop mode would be marginal - further, bugman's ongoing work on unit-testing and unit-testing Nasal in particular, should come in very handy, because it would become much easier to start up dedicated FGNasalSys instances (our in-sim Nasal interpreter) that may not run inside the main loop, i.e. lacking most standard FG APIs.

Now, when it comes to using Canvas without Nasal, that's actually a valid use-case, and I find it important to keep that use-case in mind, because over time, we've seen more and more attempts at coming up with frameworks, that basically shield back-end code from changes to front-end code (and vice versa), this is why it is important to primarily work through the property tree, and not rely on dedicated Nasal bindings (cppbind).

It would be a good thing to keep this in mind, because doing so means that multi-instance setups supporting Canvas would become much easier, i.e. there is no problem using Nasal at all, as long as it happens through well-defined interfaces that basically hide the scripting aspect.

Furthermore, a number of core devs have been thinking about using the Canvas system for scenery-related runtime-drawing, which would also require Canvas to become thread-safe, i.e. using a dedicated/private property tree instance to isolate all access to the property tree that is used to update/redraw such textures, which would mean that anything involving OSM2City, photo-scenery, but even random buildings, could be enormously boosted by making the Canvas system available accordingly:

Most recently, James posted this a couple of days ago:
https://sourceforge.net/p/flightgear/ma ... /36781190/
James Turner wrote:We actually have quite a lot of code examples of the ‘painting’ part - since the map-widget, Canvas-map, and launcher-map all do this kind of thing to render some limited GIS data (eg, taxiways, runways or urban areas) into different 2D surfaces - often even, a texture. One could even imagine a system where Canvas itself is used to do the drawing, although then we have to efficiently get the source data available to Canvas/Nasal, and deal with threading issues since this work has to happen asynchronously in an OSG pager thread.


http://wiki.flightgear.org/Canvas_Scenery_Overlays
http://wiki.flightgear.org/Photoscenery

Speaking in general, once each Canvas texture uses its own, private, property-tree instance, it would be fairly straightforward to also add private instances of a Nasal interpreter and a "property rules" system to each canvas - these are existing features in simgear/flightgear, once you add SGSocket-based connectivity and Richard's Emesary system, you have a 2D drawing system that can run asynchronously, out of the main loop, and you don't even have to use Nasal, but can use property rules (or JSBSim systems) to create/update OpenGL textures.

Again, like I said, it's not difficult to add a SGPropertyNode instance to each Canvas texture, but it paves the way for a future where a Canvas texture and the logic creating/updating it, can live (run) outside the main loop, with the main loop only fetching the final image, or possibly streaming it from another process (for which we already have working code, thanks to ThomasS).

Torsten's autopilot/property rules code is highly functional and it could be easily extended to support event handling constructs, so that we can do away with Nasal based timers and listeners to update a Canvas. Richard's Emesary system is a very neat mechanism to establish the coding protocols to do message-based programming, and once you have these systems hooked up to a single private property tree instance, you can literally have your cake and eat it. Most of this is really code we already have, it's just not yet integrated. But once that is tackled by someone, you also end up with a Canvas system that can boost development of anything involving autogen-scenery like osm2city, think random buildings and osm2city integrated via the Canvas system.

These steps would also help move towoards a CompositeViewer based FlightGear architecture:
http://wiki.flightgear.org/CompositeViewer_Support
http://wiki.flightgear.org/Canvas_Troub ... internally
http://wiki.flightgear.org/Compositor

As usual, we have a ton of great code all over the place, just lacking the manpower (and overall plan) to integrate things properly.

But as can be seen, many areas in FlightGear could enormously benefit by adapting the Canvas system to OPTIONALLY support per-texture property trees that are conceptually private, so each canvas texture can have its own scripting interpreter instance, property rules and timers/events (listeners).

The key efforts here are really bugman's work on unit testing and Richard's work on Emesary - using their groundwork, sooner or later, someone will be brave enough to come up with a new canvas mode that can run asynchronous Nasal scripts to create and update a texture , while those scripts will look very different, and probably even intimidate people new to message-based coding and multi-threading, they will come with the benefit of not cluttering the main loop, i.e. providing stable frame rates and frame spacing.
Last edited by Johan G on Tue Nov 12, 2019 2:34 pm, edited 1 time in total.
Reason: Split off from the topic "Fundamental FDM design problems."
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Fundamental FDM design problems.

Postby Thorsten » Sun Oct 13, 2019 6:35 pm

But under the hood, 99% still is pure property I/O, which is also why the property tree is becoming a bottleneck.


Well,, actually we tested that it's not, because changing the property I/O method did not affect the framerate in a bottleneck situation, it was the canvas re-drawing which seemed to be the culprit.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Fundamental FDM design problems.

Postby Octal450 » Sun Oct 13, 2019 6:44 pm

Thorsten wrote in Sun Oct 13, 2019 6:35 pm:Well,, actually we tested that it's not, because changing the property I/O method did not affect the framerate in a bottleneck situation, it was the canvas re-drawing which seemed to be the culprit.


Correct. Bad property I/O (via getprop/setprop) can definitely contribute, but the drawing is the bigger thing.

VERY OFFTOPIC tho

Kind Regards,
Josh
Skillset: JSBsim Flight Dynamics, Systems, Canvas, Autoflight/Control, Instrumentation, Animations
Aircraft: A320-family, MD-11, MD-80, Contribs in a few others

Octal450's GitHub|Launcher Catalog
|Airbus Dev Discord|Octal450 Hangar Dev Discord
User avatar
Octal450
 
Posts: 5583
Joined: Tue Oct 06, 2015 1:51 pm
Location: Huntsville, AL
Callsign: WTF411
Version: next
OS: Windows 11

Re: Fundamental FDM design problems.

Postby Hooray » Sun Oct 13, 2019 7:18 pm

Thorsten wrote in Sun Oct 13, 2019 6:35 pm:
But under the hood, 99% still is pure property I/O, which is also why the property tree is becoming a bottleneck.


Well,, actually we tested that it's not, because changing the property I/O method did not affect the framerate in a bottleneck situation, it was the canvas re-drawing which seemed to be the culprit.



That number was unrelated to any performance findings, i.e. just a ballpark number to make the point that most Canvas functionality is indeed still available through the property tree (which the OP as asking about by considering to use JSBSim systems instead of Nasal to access the Canvas), i.e. does not require Nasal scripting at all, which also means that there is a quite a bit of room left to optimize things by not going through the property tree.

As I have previously said, determining the overhead/performance footprint can be easily done by disabling a whole canvas texture or updating/redrawing of individual elements/groups.

I have previously posted instructions, and patches, illustrating how to patch fg/sg respectively, by adding dedicated draw masks to the Canvas system.
I suppose that most issues surrounding redraw() calls are due to ShivaVG (canvas path handling), which contains non-OSG code ?

More recently, I reached out to Richard in response to his Nasal GC work, suggesting to hook up the OSGStatsHandler infrastructure to the Canvas system and Nasal (GC) respectivel, to enable non-developers to come up with the corresponding numbers to tell if, where and why things are becoming slow.

Hooray wrote:
Richard wrote:During development I was collecting the following information; the numbers in brackets are the count of elements that reference an item.

Code: Select all
bottleneck                  | starting
wait finished 0 usec        | wait for other nasal threads
--> freedead (47240) : 2194ms | free() items
--> garbageCollect:         | GC item below
7226ms m>                     |  - strings
8185ms p(1)>                  |  - vec
9060ms p(2)>                  |  - hash
10206ms p(3)>                 |  - code
10231ms p(4)>                 |  - func
10255ms p(5)>                 |  - ccode
10259ms p(6)>                 |  - ghost
10280ms                  | reap
bottleneck finished: 10280  | completed
Nasal/geo.nas:90 - active line that trigged the GC


Maybe the total count of 'active' nasal elements is useful; but that's really all I used.



I think those would be very useful to expose - for starters, SG_LOG() should do (SG_DEBUG and SG_NASAL) - however, exposing such stats in the form of properties would probably be even more useful in the long term, because we could easily visualize these things in-sim, maybe even by hooking up such properties to the FG/OSG::StatsHandler (the stuff showing OSG stats) - it should be fairly straightforward to come up with an adapter class to visualize arbitrary numerical properties there, using a configurable title/label.
That way, anything caused by the GC should show up there immediately, without people having to understand the performance monitor.

As a matter of fact, I believe it was Andy's original suggestion to re-work the mark/sweep collector to use different pools for different "generations" of objects, he said that this would be the most straightforward optimization that he could think of, without re-implementing the whole GC - he basically said that naPool would be the data structure of choice, because an array of naPools could easily emulate different generations of GC objects.

By making your debugging tools available in the form of properties, you would undoubtely provide the groundwork for any future Nasal/GC improvements, given how long it has taken for the GC to be touched at all (and given the number of failed attempts), I believe that you are in an exceptional position to get those changes committed.

Even though, I would suggest to also add corresponding info to the help/about dialog (or nasal console) to display what Nasal GC mode is currently active, so that end users can provide more actionable bug reports.

Coming up with an adapter template class that could be used to add your Nasal GC properties (stats) to the OSG on-screen stats should undoubtely have a massive effect on people's understanding if/how and when the Nasal GC affects FGFS performance, and maybe more importantly, when it doesn't at all ;-)

Equally, the hard-coded defaults you tweaked manually could become property-configurable, too - i.e. defaulting to your current values, but customizable via properties - so that people only need to edit defaults.xml or override such defaults if needed - the point being, when I instrumented fgfs a few years ago, the Nasal data structures were roughly in the 16MB ballpark, i.e. those GC defaults were way too low.



PS: Not arguing at all, I haven't executed/built FG in months, so I am basically out of the loop - not just in terms of development matters, but also in terms of having arguments on the forum ;-)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Fundamental FDM design problems.

Postby Thorsten » Sun Oct 13, 2019 7:23 pm

As I have previously said, determining the overhead/performance footprint can be easily done by disabling a whole canvas texture or updating/redrawing of individual elements/groups.


We did a test with the Shuttle ADI, as far as I remember you were involved in designing the test and agreed with the findings, so I'm surprised you've changed your opinion again.

But maybe my recollection deceives me :?:
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Fundamental FDM design problems.

Postby Hooray » Sat Nov 09, 2019 4:09 pm

Sorry for the late response, but I don't think I disagree at all, there's probably a misunderstanding here.
I do remember the ADI ball stuff, and I came up with a down-stripped test-case, and Stuart ended up getting in touch to help profile/optimize the underlying code - I believe there is even a dedicated wiki article using the ADI ball for profiling purposes, or rather the stripped-down test case using OpenVG ?

The approach remains the same, like you say, it is pretty conclusive to disable updates/rendering and see how that impacts overall fgfs performance.
So I don't believe we disagree ...


What would really make sense though is adding a boolean property/flag to each canvas, so that it can optionally be registered to show up in the on-screen OSG stats, which will make it much easier for non-coders to tell what is going on, i.e. using the FGStatsHandler API - because these are just cameras (off-screen, but still)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU


Return to Canvas

Who is online

Users browsing this forum: No registered users and 2 guests