Board index FlightGear Development Canvas

Canvas remote drawing

Canvas is FlightGear's new fully scriptable 2D drawing system that will allow you to easily create new instruments, HUDs and even GUI dialogs and custom GUI widgets, without having to write C++ code and without having to rebuild FlightGear.

Re: Canvas remote drawing

Postby ThomasS » Wed Nov 02, 2016 6:32 am

The reason I commented the code was because I wasn't aware of the streaming functionality, especially related to serving an image (and I still have no clear understanding). I just wanted to avoid any unpredictable effect.
ThomasS
 
Posts: 81
Joined: Sun Sep 11, 2016 1:21 pm
Location: West of EDDK
Version: 2018.2.1
OS: Linux,MacOS,Windows

Re: Canvas remote drawing

Postby Hooray » Wed Nov 02, 2016 7:38 pm

That's a HTTP specific feature - it will basically maintain an open connection to the browser and continue to send data whenever the server has new image data - that is why I previously suggested to look up HTTP image streaming.

Anyway, the underlying C++ code would remain the same no matter the streaming protocol in use - i.e. your adapted version of Torsten's original code would remain "as is", it's merely the transport/streaming handler that would be different.

The key mechanism (google search query) is callled "multipart/x-mixed-replace;boundary" (you will also see this being used in the streaming component of the code you kept disabled for now).

To learn why, and how, this works, see:

https://en.wikipedia.org/wiki/Motion_JP ... _over_HTTP
http://blog.dubbelboer.com/2012/01/08/x ... place.html
https://en.code-bude.net/tag/how-does-mjpeg-work/
http://www.howtocreate.co.uk/php/serverpushdemo.php

Let us know if you have any more specific questions.

In general, you will want to review your patch (and the original files created by Torsten) for the "stream" variable/keyword, and you should see where and how that is being used, make sure to also look for "isStream()"
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11191
Joined: Tue Mar 25, 2008 8:40 am

Re: Canvas remote drawing

Postby Hooray » Sun Sep 03, 2017 2:18 pm

This is probably something that we should consider revisiting to get this committed, especially in the light of the recent work on the new Canvas::View element for rendering views to a Canvas, as well as the original work on coming up with dedicated Canvas::ModelViewer and Canvas::PDF elements, otherwise, there will be no sane way for external/out-of-process Canvas displays, like James' "Remote Canvas" to deal with in-process Canvas::Elements that cannot be easily replicated, because there is no simple way to render an arbitrary FligthGear view and/or synthetic terrain externally, short of using something like fgviewer.


Subject: Remote canvas utility
zakalawe wrote:I’ve spent some of my time since FSWeekend figuring out how I might drive some ‘real’ CDU hardware from the in-aircraft CDU options, and went for a slightly ambitious approach: I’ve written an analogue of ‘FGPanel’ for the canvas. Analogue is actually not quite correct - unlike FGpanel this connects at the view level (canvas properties), it doesn’t run the canvas ‘model’ locally, since that would be very tough (Nasal + whole property tree needed). This distinction was unimportant for the canvas up until now, but it’s going to become important I expect.

Image
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11191
Joined: Tue Mar 25, 2008 8:40 am

Re: Canvas remote drawing

Postby ThomasS » Tue Mar 13, 2018 2:53 pm

I just realized James' work on FGQCanvas which seems to be exactly what I was looking for when I opened this thread some time ago. No need to interfere with existing FlightGear core code. FGQCanvas simply attaches to any running FlightGear process and renders a canvas image on the device it is running. Really smart and consistent with existing HLA considerations as far as I can judge this.

But I don't really understand the possible implications also mentioned in the Wiki page of FGQCanvas. To me it appears FGQCanvas relies on a specific way canvas drawing is working today, but this might change in the future. So FGQCanvas might either become useless in the near future or canvas developer have to keep the current implementation preventing desirable improvements to canvas drawing.

Is that correct? Is there a possible conflict?
ThomasS
 
Posts: 81
Joined: Sun Sep 11, 2016 1:21 pm
Location: West of EDDK
Version: 2018.2.1
OS: Linux,MacOS,Windows

Re: Canvas remote drawing

Postby Hooray » Tue Mar 13, 2018 5:44 pm

actually, it's a separate effort, and a separate code-base, it's also a re-implementation of the canvas system using Qt, that also ended up introducing regressions (=bugs).
In other words, features that may work in fgfs may not work at all in FGQCanvas (see James' comments on the devel list)

There were already some issues discussed on the devel list, when James changed the Canvas implementation in fgfs to make it better suit FGQCanvas - which he ended up having to revert due to regressions for some aircraft developers (I believe it was the shuttle)

If in doubt, please ask your questions on the devel list.

I can however say clearly that FGQCanvas will not work with new canvas elements that we've already discussed and implemented on the forum (e.g. the cam-view element created by Icecode) - and that also applies to anything involving Canvas + effects/shaders (think night vision/FLIR etc) - ironically, FGQCanvas would indeed need a streaming facility (e.g. your exact patch) to be able to stream a cam-view from fgfs to another process.

For instance, imagine what is needed to replicate the following canvas in another process (on another computer):
http://wiki.flightgear.org/Howto:Canvas ... ra_Element
Image

A number of aircraft/avionics developers, including Stuart, mentioned already that they're hoping to use this new canvas feature (element) as soon as possible.
This will be the moment when FGQCanvas will stop working for such features, unless something along the lines like your patch is adopted.

Here's James own statement about FGPanel vs glass instruments at the time:

FGPanel: standalone panel rendering
zakalawe wrote:
Hooray wrote:Instead, it would probably make sense to make some more subsystems optional so that they can be disabled on demand, then the same fgfs binary could be used for starting up in an "instrument only" mode that only renders 2D instruments with properties being read from a master instance.

That's more or less exactly what it will be - and yes, it will be part of the main FG codebase, not a fork. As you say, the hope is to make fggc and some related things obsolete, since the same XML+Nasal can be used in the main sim or stand-alone.


Later on, this has been referred to as "FGCanvas" (using OSG, not Qt):

https://www.mail-archive.com/flightgear ... 37441.html
James Turner wrote:Creating a 'fgcanvas' client analogous to 'fgpanel' should be doable too, since
the canvas simply renders to a single osg-Camera. Unlike fgpanel this will
require OSG instead of raw GL of course, but that's the price we pay for
unifying the rendering backend.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11191
Joined: Tue Mar 25, 2008 8:40 am

Re: Canvas remote drawing

Postby dutchman » Wed Mar 14, 2018 1:52 pm

A solution that vanished from the FG radar but that I am still using was FG on android created by Saul Rodriguez.
Mind you this was created in 2014 so before the Phi API. It relies on a telnet connection for data-exchange.
Check out https://sites.google.com/site/flightgearandroid/home. There is also a link to the Github repo's.

Cheers
A propeller is just a big fan in front of the plane used to keep the pilot cool.
When it stops, you can watch him start sweating.
dutchman
 
Posts: 52
Joined: Wed Feb 06, 2013 11:10 pm
Version: 2018.2.2
OS: WIN7-64

Re: Canvas remote drawing

Postby Hooray » Wed Mar 14, 2018 2:06 pm

Ultimately, what works best for you, depends highly on your concrete use-case/requirements.
fgpanel, Phi, FGQCanvas etc all have different pros & cons

However, until there is a real IPC mechanism like HLA/RTI in place, out-of-process front-ends will have serious limitations - so that conventional master/slave setups using a subset of the subsystems is most likely to work for all people interested using the latest Canvas functionality - obviously, there's a price tag that comes with that, i.e. FGQCanvas or Phi can be much cheaper in comparison.

Anyway, the streaming-based solution that ThomasS came up with is certainly useful, and even more likely to be useful once additional Canvas elements are supported.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11191
Joined: Tue Mar 25, 2008 8:40 am

Re: Canvas remote drawing

Postby ThomasS » Sun Apr 29, 2018 10:02 am

I agree to what you said: There could be several solutions for a remote display of canvas and the solution to be used just depends on the concrete use case. What I'm going to do now is to revisit my patch from 2016, get the streaming feature working and create a merge request for flightgear core. Hopefully I succeed in convincing the core developers.
ThomasS
 
Posts: 81
Joined: Sun Sep 11, 2016 1:21 pm
Location: West of EDDK
Version: 2018.2.1
OS: Linux,MacOS,Windows

Previous

Return to Canvas

Who is online

Users browsing this forum: No registered users and 3 guests