Board index FlightGear Development Canvas

Pitfalls of canvas HUDs

Canvas is FlightGear's new fully scriptable 2D drawing system that will allow you to easily create new instruments, HUDs and even GUI dialogs and custom GUI widgets, without having to write C++ code and without having to rebuild FlightGear.

Pitfalls of canvas HUDs

Postby Thorsten » Thu Dec 15, 2016 8:29 am

I've learned a lot about HUDs in the last two days (some of it pretty painfully) and I'd like to share some info.

The issue is that canvas-based HUDs are different from the FG-native HUD in that they're not assigned to a view but to a surface in the 3d model. And that means the cues they show depend on the head position in the cockpit (if you move the view, the native HUD moves with it, the canvas HUD does not).

That in turn implies that the canvas HUD is likely to be not properly aligned with the outside world unless you carefully engineer both the HUD graphics and the view position. And if that is the case, the HUD provides wrong cues which will actually get you killed if you fly by them.

Here's an example with native HUD superimposed on canvas HUD:

Image

Note how the boresight symbols are displaced, so the canvas HUD doesn't show the aircraft axis where it really points to. Also note how the 10 degree line is displaced.

Now, if I shift the default view point by a few cm, suddenly boresight is aligned properly - but still the 10 degree lines don't match up.

Image

(If anyone wonders whether the native HUD is wrong, I've tested this countless times to provide correct cues to use, so I'm pretty convinced it is right - especially since it adapts to view changes).

So the issue is that it's not just a problem of the right viewpoint, but also of the HUD layout - seen from the right position, the spacing of the lines of a pitch ladder needs to be such that the 10 deg line actually is 10 deg below the horizon on the outside world. It can well happen that the boresight is always pointing at the correct pitch line, but the location of the pitch line 10 degree lower is not actually 10 degrees lower (there's both the spacing parameter and a translation parameter in the game) - the HUD isn't properly aligned unless both conditions are true.

How to do this? The tentative procedure I've come up with is as follows:

* Usually we start with a picture/screenshot of the HUD layout we want to model. Use that to draw the boresight.

* Find the view position in the cockpit for which the boresight is aligned between canvas and FG-native HUD - that's the proper head position. Define this as the pilot's view (the HUD will not be fully aligned and usable in any other view).

* Now, testing everything in this view, add the pitch ladder with a spacing betwen lines chosen such that the lines agree with the native HUD - this may require you to change ladder spacing a few times which is a pain if you do it in SVG, so consider using a function to draw the ladder via direct canvas commands (or try scaling the SVG ladder).

* Then adjust the velocity vector positioning code such that it matches (typically it's an expression like y = A * AoA + B to position it correctly in the vertical, same with beta in the horizontal)

* The same A-parameter needs to be used to translate the pitch ladder, as it tells you how many degrees of some angle correspond to how many canvas pixels. The B-parameter of the ladder can (and will be) different - use it to adjust the zero line.

* As a result, you should have a pitch ladder which aligns properly with the native HUD everywhere

* Now determine the rotation center of the ladder as a function of pitch to get the rotation aligned

* Take a deep breath and do something you really like because you'll be in danger of screaming

* Then add the rest of the HUD layout for which alignment is not critical
Thorsten
 
Posts: 11310
Joined: Mon Nov 02, 2009 8:33 am

Re: Pitfalls of canvas HUDs

Postby zakalawe » Thu Dec 15, 2016 5:51 pm

Just to say, I'm planning to port the C++ HUD to use Canvas for its rendering soon (next few months), but I plan to keep XML compatibility (so aircraft don't see any change) and will keep the placement logic relative to the view, for the same reason. That said, we do have examples (the F-16 i think) of places a standard HUD into the 3D model surface, right? Not sure if the 787 also does this for its HUD.

For a HUD placed into the 3D model I can only agree the 'correctly' viewing the bore-sight and ladder is going to depend exactly on the view/eye position relative to the HUD. For the normal HUD (eg in the UFO) we of course align the boresight with camera's Z-axis.
zakalawe
 
Posts: 1152
Joined: Sat Jul 19, 2008 4:48 pm
Location: Edinburgh, Scotland
Callsign: G-ZKLW
Version: next
OS: Mac

Re: Pitfalls of canvas HUDs

Postby Johan G » Thu Dec 15, 2016 6:18 pm

If possible it would be a good thing if the HUD image/canvas would follow head movements and be clipped to within the HUD display area (which usually is smaller than the combiner frame surrounding it).

As of now it seem most HUDs appear as a transparent display rather than a projected and collimated image. In real life HUD displays are collimated in order for a point on the HUD image to appear fixed to a point at infinite distance. Instances when this would be particularly desirable are when using features causing head movement due to turbulence or high-G loads or if head tracking is used.

As things are now the only way to achieve that with the Canvas HUD is for each aircraft developer to implement ways to make the HUD image appear collimated.

See also these old post (and some of the posts following them): Re: F-16 upgraded to 'production' status and Re: How do I make a 2D PFD with pages
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator
 
Posts: 5546
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

Re: Pitfalls of canvas HUDs

Postby PINTO » Thu Dec 15, 2016 6:44 pm

Johan G wrote in Thu Dec 15, 2016 6:18 pm:As things are now the only way to achieve that with the Canvas HUD is for each aircraft developer to implement ways to make the HUD image appear collimated.


MiG-21bis gunsight currently compenstates for parralax - it wasn't too difficult. 4 lines of code and about 15 minutes tweaking the parralax constants. I decided to not make it perfectly collimated due to the fact that the real one wasn't. But it's very close.

e: here's a gif. It's a lot smoother now than it appears in the gif. Should also mention it's only in the z and y axis, never did x (front/back) due to the view rarely moving that way, if ever. But that's one line of code and another couple minutes tweaking to add in.

Image
Actively developing the MiG-21bis (github repo) (forum thread) (dev discord) (fg wiki)

http://opredflag.com is an active flightgear dogfighting community (using a system that isn’t bombable)
User avatar
PINTO
 
Posts: 947
Joined: Wed Oct 21, 2015 6:28 pm
Callsign: pinto
Version: 2016.3.0
OS: Win10

Re: Pitfalls of canvas HUDs

Postby Thorsten » Fri Dec 16, 2016 7:09 am

If possible it would be a good thing if the HUD image/canvas would follow head movements and be clipped to within the HUD display area


That is easy to do for lateral movements once you have it properly calibrated for the default POV, yes. I'm not quite sure how the effect would look like for forward/backward head movements. I guess if the projection is really to infinite distance, the HUD image would remain fixed in apparent size whereas the HUD frame apparent size shrinks as you move your head back, so the image no longer fits the screen when seen from the rear of the flightdeck...
Thorsten
 
Posts: 11310
Joined: Mon Nov 02, 2009 8:33 am

Re: Pitfalls of canvas HUDs

Postby zakalawe » Fri Dec 16, 2016 11:23 am

PINTO wrote in Thu Dec 15, 2016 6:44 pm:MiG-21bis gunsight currently compenstates for parralax - it wasn't too difficult. 4 lines of code and about 15 minutes tweaking the parralax constants. I decided to not make it perfectly collimated due to the fact that the real one wasn't. But it's very close.


That's a very nice feature indeed, as an optional thing for the standard HUD. Will it need per-aircraft setup of the constants? (I guess so....)
zakalawe
 
Posts: 1152
Joined: Sat Jul 19, 2008 4:48 pm
Location: Edinburgh, Scotland
Callsign: G-ZKLW
Version: next
OS: Mac

Re: Pitfalls of canvas HUDs

Postby Thorsten » Fri Dec 16, 2016 12:30 pm

Short of the clipping, it's how the default HUD works... as I said, it stays relative to the current view.
Thorsten
 
Posts: 11310
Joined: Mon Nov 02, 2009 8:33 am

Re: Pitfalls of canvas HUDs

Postby PINTO » Fri Dec 16, 2016 4:19 pm

zakalawe wrote in Fri Dec 16, 2016 11:23 am:Will it need per-aircraft setup of the constants? (I guess so....)


Yeah, the constants are determined by the size of the HUD canvas object and the size of the canvas texture.
Actively developing the MiG-21bis (github repo) (forum thread) (dev discord) (fg wiki)

http://opredflag.com is an active flightgear dogfighting community (using a system that isn’t bombable)
User avatar
PINTO
 
Posts: 947
Joined: Wed Oct 21, 2015 6:28 pm
Callsign: pinto
Version: 2016.3.0
OS: Win10

Re: Pitfalls of canvas HUDs

Postby Hooray » Sat Dec 17, 2016 7:40 am

Actually, the cleanest solution for the problem at hand would be coming up with a new dedicated Canvas placement mode just for HUDs, which may then automatically handle these things, according to the behavior of the legacy HUD - alternatively, we could extend Canvas::Element to set a corresponding attribute/property to handle certain Canvas elements/groups accordingly (i.e. per element/node).


I'm planning to port the C++ HUD to use Canvas for its rendering soon (next few months), but I plan to keep XML compatibility (so aircraft don't see any change) and will keep the placement logic relative to the view, for the same reason.

Please just be sure to coordinate such ideas/work with TheTom (or just look up his comments regarding C++ code using the Canvas system) - in general, hard-coding these things at the C++ level hasn't served us too well - while it is a good idea from a code unification standpoint to use a single OSG-based back-end, beginning to write C++ code that ends up using the Canvas system is very much akin to hard-coding effects/shaders at the C++ level, which also hasn't worked out too well for Rembrandt ...

We need to keep in mind that it would only take a simple Nasal parser to map the existing HUD logic to Canvas properties, and then modify/create new elements according to what would be useful for the HUD use-case, e.g. by coming up with a dedicated placement for HUDs that automatically handles projections properly.


Once we begin hard-coding systems that end up using the Canvas system, we're sacrificing all its flexibility and violating some major design principles - if you/we are concerned about increasing use of Nasal, and such work being future-proof, the most correct way would be exposing the Canvas::Element interface via cppbind, so that such elements can be implemented by sub-classing Canvas::Element in Nasal space, at which point future porting/updates in C++ space would become much easier.

Equally, the corresponding building blocks needed/useful for the HUD would sooner or later also come in handy for getting rid of the 2D panels back-end using legacy OpenGL code - which is why I think it is crucial to keep the whole picture in mind and only extend the Canvas system at the element/placement and projection level, so that a corresponding parser can be provided, and maintained, by fgdata contributors.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11430
Joined: Tue Mar 25, 2008 8:40 am

Re: Pitfalls of canvas HUDs

Postby Thorsten » Sat Dec 17, 2016 7:44 am

Actually, the cleanest solution for the problem at hand would be coming up with a new dedicated Canvas placement mode just for HUDs, which may then automatically handle these things


Well, it can't without assumptions on what a given view represents (aka, is the currently selected view supposed to be aligned with the HUD or not?)
Thorsten
 
Posts: 11310
Joined: Mon Nov 02, 2009 8:33 am

Re: Pitfalls of canvas HUDs

Postby Hooray » Sat Dec 17, 2016 7:46 am

right, but that can be encoded in the form of flags, attributes or properties - if necessary, a corresponding "placement" could be fully aware of the view manager properties, too.
Imagine having a boolean view-aligned property (per element), and the view number (for instance)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11430
Joined: Tue Mar 25, 2008 8:40 am

Re: Pitfalls of canvas HUDs

Postby Thorsten » Sat Dec 17, 2016 7:55 am

Does the canvas code know *where* in model coordinate space the mesh the HUD is drawn on is located? Because that (and the orientation of that plane) is what you need in addition to the camera pos to work out the parallax movement math.
Thorsten
 
Posts: 11310
Joined: Mon Nov 02, 2009 8:33 am

Re: Pitfalls of canvas HUDs

Postby Hooray » Sat Dec 17, 2016 7:57 am

To be honest, I don't know for sure - but if it doesn't yet, it can definitely be added - after all, we also support scenery placements - so it's a matter of exposing such data in the of placement specific properties that can then be used by the canvas manager.

And given all the talks, and feature requests, about supporting effects and shaders for Canvas textures, too - it seems that making things available makes sense sooner or later.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11430
Joined: Tue Mar 25, 2008 8:40 am

Re: Pitfalls of canvas HUDs

Postby erik » Sat Dec 17, 2016 10:45 am

I just tested the Canvas HUD of the F-16 made by Richard and it accounts for position (you can move the view up and down by pressing the left mouse button while pushing or pulling the mouse).

Erik
erik
 
Posts: 1522
Joined: Thu Nov 01, 2007 1:41 pm

Re: Pitfalls of canvas HUDs

Postby Thorsten » Sat Dec 17, 2016 10:53 am

Sorry, no - it ain't doing that (just pulled latest). Press 'h' once to see where boresight and flightpath vector are in the native HUD, move view around and compare differences between native and canvas HUD (it does seem to adjust the horizon line, but that's all).
Thorsten
 
Posts: 11310
Joined: Mon Nov 02, 2009 8:33 am

Next

Return to Canvas

Who is online

Users browsing this forum: No registered users and 1 guest