Board index FlightGear Development Canvas

Pitfalls of canvas HUDs

Canvas is FlightGear's new fully scriptable 2D drawing system that will allow you to easily create new instruments, HUDs and even GUI dialogs and custom GUI widgets, without having to write C++ code and without having to rebuild FlightGear.

Re: Pitfalls of canvas HUDs

Postby Necolatis » Sat Dec 17, 2016 11:21 am

Yes, I only fixed the horizon line (and pitch lines moves with it, plus the rotation point), was a pain though cause it was svg, it much easier to do such stuff when you program it in my opinion.

In the Viggen the FPV and pitch lines match fairly good (last time I checked). Did trigonometry to get them aligned. Markers around MP/AI aircraft match also, until aircraft pitch approaches +-90 degs. I guess need matrix view projection if want that. It also adjusts for vertical view position changes, although you cannot change it, but you might notice it when the view buffet.
"Airplane travel is nature's way of making you look like your passport photo."
— Al Gore


Hangar: https://sites.google.com/site/fghangar/
User avatar
Necolatis
 
Posts: 1870
Joined: Mon Oct 29, 2012 12:40 am
Location: EKOD
Callsign: Leto
IRC name: Neco
Version: 2018.2.2
OS: Windows 10 Pro

Re: Pitfalls of canvas HUDs

Postby Thorsten » Sat Dec 17, 2016 11:47 am

Once you have a calibrated viewpoint, the 2d parallax correction is trivial to apply.

* store the default view point for the HUD somewhere (I did that under HUD.view)

* move the HUD image like

Code: Select all
var current_x = getprop("/sim/current-view/x-offset-m");
var current_y = getprop("/sim/current-view/y-offset-m");

var dx = me.view[0] - current_x;
var dy = me.view[1] - current_y;

me.HUDImage.setTranslation(dx * factor_x, dy * factor_y);


* figure out factor_x and factor_y (make them properties and just try what is needed to match up with the FG native HUD - takes a minute at most)

So in my view there's zero need to encode this as flags and properties - getting the calibrated view point is hard, applying parallax from there is a piece of cake.
Thorsten
 
Posts: 9895
Joined: Mon Nov 02, 2009 8:33 am

Re: Pitfalls of canvas HUDs

Postby Hooray » Sat Dec 17, 2016 6:31 pm

Thorsten wrote in Sat Dec 17, 2016 11:47 am:So in my view there's zero need to encode this as flags and properties - getting the calibrated view point is hard, applying parallax from there is a piece of cake.


Actually, the "hard part" should already be part of the existing C++ code implementing the legacy HUD, right ?
So porting that code to integrate with the Canvas properties at the placement/element level would not necessarily be rocket science.

Otherwise you do have a point though - when it comes to Nasal/Canvas vs. C++ and OSG we've been approaching a "parity situation", i.e. it's often possible to accomplish something and implement certain functionality using both approaches. However, the real issue is actually two-fold, which is kinda backed up by your introductory posting here:

You have meanwhile an enormous, and basically unprecedented, track record when it comes to FlightGear matters, especially in the rendering department, i.e. you have an unprecedented track record in rendering matters despite not exactly writing/committing C++ code on a daily basis; which is to say that you are able to implement sophisticated features without having to touch any C++ code - which is mainly because of your expertise when it comes to Nasal, effects and shaders - i.e. you usually know exactly how to integrate these features to make effective use of these, and other (think property rules), technology stacks.

However, let's also keep in mind that you are also in a fairly exceptional and probably unique position, in that your background regarding maths/physics puts you in an excellent position to work out even the most sophisticated intrinsics of stuff that would otherwise take people months to figure out.

The other point I was hoping to make when I suggested to tackel this at the sc::element/sc::placement level is that of establishing, and maintaining, compatibility with other important features, use-cases and long-term goals (think performance, optimization).

As you should meanwhile know, I am far from opposed to Nasal space situations - however, the way Nasal is integrated with FlightGear, and exposed to aircraft developers has been hugely complicating matters for people wanting to re-architect certain parts of FlightGear - no matter if that meant implementing better threading support, refining support for multi-instance setups (FSWeekend, LinuxTag, dual-pilot/shared-control).

This is one of the reasons why an increasing number of core developers, and more recently you yourself, have been encouraging contributors to port/reimplement certain Nasal-functionality in the form of property rules, especially when it comes to rendering related contributions, i.e. stuff that may possibly run once per frame.

In general, Nasal has obviously been a huge technology-enabler - but in reality we really need to make sure to restrict the way we're using Nasal, either by moving away from Nasal, or by encapsulating its use, which means that we need to work out a way to formalize and standardize on generic interfaces and building blocks, which can provide a sane migration path if/once Nasal turns out to become a bottleneck.

The way we have been using Nasal is hugely problematic due to a variety of reasons, the lack of code-reuse and standard building blocks being two of the more obvious ones.
Had we provided bindings to SGSubsystem/SGSubsystemMgr, we would not have to "pseudo subsystems" implemented on top of timers and listeners these days.

When it comes to Canvas, it's extensive use of Nasal is basically importing all of Nasal's challenges, which is why it doesn't come as a surprise that most Canvas-based contributions are basically facing pretty much the same challenges that most Nasal code is facing.

De facto, there isn't a single good reason why rendering related Canvas code should have any runtime Nasal dependencies that are potentially triggered per frame - at runtime, we could do away with listeners and timers entirely, at which point merely the initialization/setup overhead would be caused by Nasal, but everything else would be using native C++ code and OSG data structures behind the scenes.

Right now, we have tons of Canvas-based features that implicitly require Nasal, i.e. tons of potential context switches - most of them happening per frame, or using a pseudo-scheduling mechanism build on top of split-frame loops, i.e. spread across multiple frames.

Ultimately, what this is really all about is animating a Canvas without requiring Nasal to do so at runtime - all that is needed is to encode the animation semantics at the property level, and at that point property rules and/or state machines could take over, i.e. native C++ code - no matter if that means translating an element or transforming a nested groups, we really don't want to depend on Nasal handling this, because we're violating the MVC principle, i.e. we end up having MVC related state that does not even live in the property tree (as a matter of fact, try to properly sync/replicate any Canvas MFD across multiple fgfs instances, and you will see for yourself that we're creating tons of features that no longer scale, i.e. that are mutually incompatible with use-cases that end-users do care about, e.g. multi-instance FSWeekend/LinuxTag or shared-control setups).

This is one of the key reasons why the nature, and excessive degree, of Nasal use by fgdata/fgaddon contributors is complicating matters for core developers wanting to re-architect FlightGear (think reset/re-init, saving session state, replay, incremental booting, modularizing fg, better threading support, getting rid of legacy OpenGL code).

Thus, from a technical standpoint, the "right" thing to do would be to provide standard interfaces so that people can work out building blocks, which we can later on profile/optimize or re-implement using more appropriate solutions (think C++, effects or GLSL) - what we're currently doing isn't scaling anymore, and it is mainly because of the way we're using Nasal to work around all sorts of limitations, of which the core group is unaware - despite those workarounds no longer being compatible with use-cases like multi-instance setups (think home cockpit builders).

Now, you may disagree with many of these points because you are able to work out all sorts of workarounds to make things happen still - but your recent comments regarding a distributed AW system are basically proving that these use-cases do matter and that we as a community have failed to provide building blocks and interfaces to allow Nasal space contributions to be developed in a fashion that is well-aligned with ongoing core developments.

As someone who is pretty familiar with the ND/MapStructure code and the Canvas system in general, I can only reiterate that we need to discourage certain ongoing coding habits and actively work towards using a development model where new features are prototyped, provided and implemented in the form of sub-classes inheriting from sc::element, sc::placement and sc::projection - once we start doing that, we are approaching a situation where we can treat MFDs, and MFD components (think pages, buttons, switches, page elements) as dedicated Canvas::elements that also support versioning and proper serialization for replication/synchronization purposes.

Otherwise we will continue to see potentially useful components, features/code (think PFD, EICAS, ND, Avidyne Entegra R9, shuttle avionics) that are hardly compatible with other use-cases/aircraft, and that are rarely reusable from a coding standpoint, short of using copy & paste and a bunch of timers and listeners (which still is the predominant practice when it comes to Nasal based features).

Besides, let's keep in mind that people have been actively considering to phase out/replace Nasal - which is why it does make sense to restrict its use, or at the very least encapsulate it - allowing Canvas features to be directly registered as sc::element sub-classes implemented in Nasal space will allow us to have our cake and eat it, at the mere cost of exposing code that is already available in C++ space, while also benefiting other use-cases - no matter if that means getting rid of the legacy HUD, legacy GUI (PUI), legacy 2D panels code or just the splash screen stuff.

Just because someone holding a PhD in maths or physics is able to work out the intrinsics of properly aligning/projecting a HUD using Nasal space workarounds, doesn't mean that other potential contributors will be able to do the same - which is also kinda confirmed by the other posting here: Canvas arcs
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11163
Joined: Tue Mar 25, 2008 8:40 am

Re: Pitfalls of canvas HUDs

Postby Thorsten » Sun Dec 18, 2016 7:17 am

Actually, the "hard part" should already be part of the existing C++ code implementing the legacy HUD, right ?


Actually I don't think so.

The native (or legacy) HUD projects onto a virtual plane that's always fixed relative to the view and is not tied to any 'physical' object in the mesh.

As a result, the transformation properties are always the same - they're hard-coded and never need to be figured out. Use the F-14b and move the view upward, and you'll quickly see that the HUD appearing in the proper place is but an illusion - the HUD will move out of the airplane with you if you shift the viewpoint accordingly.

In contrast, the transformation properties of a canvas HUD assigned to a part of the mesh depend on mesh location and camera viewpoint and the fact that the mesh ends or may be behind you naturally introduces clipping.

So the fact that the native HUD forces you to a certain projection geometry makes the problem appear easy, but doesn't solve it, because it can't deal with the real geometry of the situation. The default HUD method would fail to get geometry right if you would really assign it to a mesh part.

Otherwise you do have a point though - when it comes to Nasal/Canvas vs. C++ and OSG we've been approaching a "parity situation", i.e. it's often possible to accomplish something and implement certain functionality using both approaches


In my view the answer is really simple:

* use C++ if it's performance critical (like large property I/O over props.nas which is really not optimal)
* use property rules where you have a relatively fixed 'per frame' pipeline to work through
* leave in Nasal for flexibility when it's not per frame, when there's lots of conditionals or performance impact is negligible

Here you're talking about one overall translation in a HUD where you might run dozens of other transformations of the same kind, moreover transformations aren't the performance-critical part and you need flexibility - it's a clear-cut case for leaving it in Nasal as far as my criteria go.
Thorsten
 
Posts: 9895
Joined: Mon Nov 02, 2009 8:33 am

Re: Pitfalls of canvas HUDs

Postby Hooray » Sun Dec 18, 2016 7:20 pm

it really isn't as clear-cut at all once you begin look at other use-cases, i.e. the stuff I mentioned (distributed setups, where you need to be able to selectively replicate/sync parts of the M/V/C to another fgfs instance) - equally, Nasal space "workarounds" don't scale when it comes to supporting scenarios like porting the 2D panels code, or other efforts trying to get rid of legacy OpenGL code running in the main loop.

Thus, the questions is really more about providing, extending and refining building blocks that are not specific to any particular use-case, to help facilitate other ongoing efforts that may benefit from such changes (sooner or later).

We don't typically write reusable Nasal code/modules at all - and settling for Nasal-space workarounds at this level is complicating matters further unfortunately - which has nothing to do with them not working properly - it's a long-term thing after all. Especially in the light of people considering to phase out Nasal, or even just move it to a separate thread.

As a matter of fact, most Canvases could in fact have their own private property tree and a private Nasal instance directly hooked up to that tree, instead of using the current approach - as long as we're working with the assumption that all stuff only ever runs in the main loop, we are not exactly doing Nasal a huge service ....
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11163
Joined: Tue Mar 25, 2008 8:40 am

Re: Pitfalls of canvas HUDs

Postby Thorsten » Mon Dec 19, 2016 6:30 am

it really isn't as clear-cut at all once you begin look at other use-cases, i.e. the stuff I mentioned (distributed setups, where you need to be able to selectively replicate/sync parts of the M/V/C to another fgfs instance)


Unless you propose to make the rest of canvas also xml-configurable it is, because the parallax trafo is something you use a dozen times in other places in a HUD anyway - so providing an xml building block for one element out of a dozen is just confusing design while it doesn't help you a bit getting around any problems Nasal might create.

Given the basic design choice of controlling canvas via Nasal, hard-coding one particular item makes no sense. You can question this design choice - though we do have an xml-controlled hard-coded HUD, and its limitation in terms of flexibility are pretty obvious.
Thorsten
 
Posts: 9895
Joined: Mon Nov 02, 2009 8:33 am

Re: Pitfalls of canvas HUDs

Postby Hooray » Tue Dec 20, 2016 4:52 pm

Sorry, I obviously missed to make my point - which was never about making anything XML-configurable at all
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11163
Joined: Tue Mar 25, 2008 8:40 am

Re: Pitfalls of canvas HUDs

Postby Thorsten » Tue Dec 20, 2016 6:18 pm

Sorry, I fear you might have...
Thorsten
 
Posts: 9895
Joined: Mon Nov 02, 2009 8:33 am

Re: Pitfalls of canvas HUDs

Postby Hooray » Fri Dec 30, 2016 2:41 pm

To be honest, I still don't understand how anything that I said here could be understood to be about making things XML configurable - and no, I don't mean to resurrect this thread - it just seems that we were talking about hugely different things or that some of the terminology I was using didn't quite hit home.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11163
Joined: Tue Mar 25, 2008 8:40 am

Previous

Return to Canvas

Who is online

Users browsing this forum: No registered users and 1 guest