Board index FlightGear Development Canvas

Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and other

Canvas is FlightGear's new fully scriptable 2D drawing system that will allow you to easily create new instruments, HUDs and even GUI dialogs and custom GUI widgets, without having to write C++ code and without having to rebuild FlightGear.

Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and other

Postby slawekmikula » Mon Oct 02, 2017 4:55 pm

As Hooray suggested some information about Canvas MapLayer should be better discussed on forum rather on mailing list.

Regarding mailling list talk: https://sourceforge.net/p/flightgear/ma ... /36059733/ and some information in G1000 canvas topic (viewtopic.php?f=71&t=32764) there is a new implementation for Tile Maps rendered directly in MapStructure framework.

It's now rather easy to create additional tile providers. One such example is Stamen maps.
Exemplary patch to implement this is located here: https://gist.github.com/slawekmikula/7e ... 6ffeecbd86
There is commented out three different map styles (terrain + overlay, terrain, toner). One can apply this patch to get new tile rendering.

Screen:
Image

One issue with this is to somewhat cluttering GUI option panel because for stamen there should be three checkbox'es (terrain, terrain+labels, toner) and these layers rival with OSM. They are mutually exclusive.

Some work has been done to convert PUI dialogs to pure canvas gui and maybe there should be some additional configuration for "base" tile maps. Anyway great work for all contributors !
slawekmikula
 
Posts: 128
Joined: Sun Feb 19, 2017 10:31 am

Re: Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and oth

Postby Hooray » Mon Oct 02, 2017 6:28 pm

Thanks for reposting this in public, I think there's several answers here.

First, to address the issue of mutually-exclusive "layers", I would suggest not to use a checkbox but rather a combo/select box.
To see how this is done, refer to $FG_ROOT/Docs/README.gui and $FG_ROOT/gui/dialogs for existing examples.

That should be straightforward to do.
Like I said previously, I don't think it's a good idea to "hard-code" these map providers/web services at the lcontroller level - instead, my suggestion would be to introduce a new PropertyList-XML based configuration file that contains a list of supported APIs, and then only parse that to populate said combo box. That way, it would also be straightforward to provide a UI to add/edit and maintain/update entries.

Regarding the PUI/Canvas "unification", there is the pui2canvas parser - but for something like the map-canvas.xml dialog, my suggestion would be to create a new Canvas based dialog file from scratch and simply port map-canvas.xml to use a native Canvas window, and do away with PUI.

For the relevant Nasal/Canvas bits, see:

http://wiki.flightgear.org/Howto:Creati ... ialog_file
http://wiki.flightgear.org/Canvas_Snipp ... o_a_Canvas
http://wiki.flightgear.org/Howto:Creati ... GUI_Widget

Note that this kind of groundwork would also be immensely useful for any MFD efforts (think G1000), because such a Canvas based MapStructure dialog could be used as the foundation for prototyping/developing these MFDs, especially in conjunction with Richard's MFD framework.

Hooray wrote:Regarding fgcamera, I would suggest to postpone this and first get in touch with Torsten and Marius_A - because Torsten stated in the fgcamera topic that he was going to port Marius work to become an addon. So, I would suggest not to duplicate any efforts there.

Regarding Stuart's suggestion on supporting other web services for fetching tiled maps, I would actually suggest to proceed differently and merely generalize Stuart's groundwork and then provide a simple in-sim UI dialog for creating, adding and editing/managaging profiles at runtime - this could be easily done using a Canvas GUI dialog with roughly 10 input fields, so that arbitrary APIs can be added - and saved in the form of an XML file that can be added to the base package.

That way, it would also be easier to update such APIs should the need arise (think changing domain names etc).

Like you say, this should not be very difficult - I could post a handful of related snippets that would merely need to be integrated. As a matter of fact, there is already: http://wiki.flightgear.org/Canvas_Snippets

And that's basically the foundation for more complex wizard-like in-sim configuration dialogs, e.g. see:
Image


This stuff is entirely based on these "snippets" - but the dialog is already a simple "wizard" that also supports validation of entries.

Regarding in-sim UI matters, you can also get in touch via the Canvas sub-forum - I am the developer of the original "pui2canvas" parser which translates a subset of PUI/XML to Nasal/Canvas:

http://wiki.flightgear.org/Pui2canvas
Image


Like Stuart said already, it would be straightforward to turn existing dialogs into Canvas dialogs this way.
Primarily, we'd need to implement 2-3 additional widgets in Nasal/Canvas:


http://wiki.flightgear.org/Canvas_Widget_Matrix
http://wiki.flightgear.org/Howto:Creati ... GUI_Widget
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and oth

Postby Hooray » Tue Oct 03, 2017 9:26 am

Slawek, just to clarity the statements on the devel list vs. forum/list use and hopefully to put things a little into perspective:

https://sourceforge.net/p/flightgear/ma ... /36061041/
Stuart wrote:Re: Forum vs. Mailing list. As per the official policy document (*)
core development
is discussed here, and the core GUI is part of that. As illustrated
by James' comment
regarding the UI, it's important that this happens here so that it can
be coordinated
with other development activities.


When I suggested to use the Development/Canvas sub-forum for some of your Canvas related questions, I was never "advocating the forums almost exclusively" trying to make new contributors "avoid the mailing list".

Indeed, once you search the archives you will find me suggesting regularly that people check out the devel list for certain core development related topics.

However, the point was that you specifically asked about using/extending the Canvas and MapStructure systems, as well as about porting the map-canvas.xml dialog.

I have been involved in the design, creation and maintenance of all of these 3 components, so I am familiar with them.

This work is not about "core development" at all - this is purely fgdata/base package development. There are no C++ changes needed for this sort of work.

None of this touches the PLIB PUI or Qt5 C++ code we have in FlightGear.

And all Canvas fgdata/fgaddon development talks happen on the forum - as a matter of fact, once you look at any of the recent topics, you will see that they were almost exclusively discussed on the forum.

Indeed, you will also see that even Stuart himself (despite being a core developer) used the forum to discuss his Canvas related work, despite it involving C++ changes (in this case the WebMercator projection).

So, there's that too.

Again, this is not indended to compete or conflict with other ongoing core development - however, if your interest is extending the Canvas/MapStructure systems and seeing the map-canvas.xml dialog ported to become a native Canvas dialog, the forum is the right place (as is the wiki).

And Stuart himself suggested that previously, when he said this:

https://sourceforge.net/p/flightgear/ma ... /36059733/
Stuart wrote:Implementing the terrain maps you list above would be a very easy first task.
Plus if you're interested in improving the GUI, that would also be very welcome.
That's slightly more complicated, but there are lots of resources available in
the wiki and on the forum.



This is because none of this requires any C++ changes, so it is not about "core development" at all.

If in doubt, I can highly recommend to check out all communications channels and see what works best for you from a signal/noise ratio (or even time-to-action).

Just like James said, there are contributors who stopped using the forums for a number of reasons - but there are also long-term contributors (including core developers) who stopped using the devel list for pretty much the same reasons.

Finally, there is a long-standing history of /some/ core developers actively discouraging any developments related to adding more Nasal/Canvas code to FlightGear for things that they'd like to see implemented in C++. Unfortunately, this particularly affects the development of a new GUI scheme.

To sum things up, if you are interested in anything involving C++ ("core development") you are better off with the devel list, if your interests lie elsewhere (Nasal/Canvas, MapStructure), you are probably better off with the forum.

Besides, may I politely suggest not to underestimate the benefits of interacting with a forum of people who can "write the most" and in the process provide concrete answers to questions within a few hours, compared to interacting with people on a devel list who rarely, if ever, document any of their work in the form of README files, updated manuals, release announcements, changelogs or even just newsletter additions.

The forum and the wiki are solving a real problem, and filling a niche, that is largely caused by people not documenting their work and not providing a sufficient degree of support - so it is for a reason that the forum and the wiki work so well, despite the very disparity between words and actions that James was touching on ;-)

Anyway, please do make up your own mind - and do use what works best for you, obviously !
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and oth

Postby slawekmikula » Tue Oct 03, 2017 9:37 am

Hooray wrote in Tue Oct 03, 2017 9:26 am:Anyway, please do make up your own mind - and do use what works best for you, obviously !


Ok, everyone has right in their statements. Please, I'm fine and choose what work best. This topic will be about mapstructure canvas work regarding tile providers. I might add another thread here (canvas) for gui components (or find better topic to discuss it there). Devel list will be for core data (simgear/flightgear).

PS. Request to everyone. Please stick in this thread only to "canvas thing". It would be much more productive especially for me :)
slawekmikula
 
Posts: 128
Joined: Sun Feb 19, 2017 10:31 am

Re: Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and oth

Postby Hooray » Tue Oct 03, 2017 10:23 am

Okay, agreed - sorry for posting my response here instead of sending it in private per PM.
I am glad seeing that I didn't misunderstand that you were indeed asking about extending the Canvas/MapStructure systems (i.e. the fgdata components) and not the C++ code.

Like I said previously, to support multiple "tile providers", my suggestion would be to use a PropertyList/XML file and put all provider-specific data into that, at that point it would be up to the UI (be that PUI/XML, Qt5, Phi or Canvas) to use said file and populate a combo/select box with those providers and use them.

I can help you implement this idea for either PUI/XML or Canvas, and I can also help you implement the MapStructure changes to come up with an lcontroller file that can use different back-ends.

Neither of this will require any C++/core changes, so I am confident that we can keep the discussion here. Equally, I can also provide pointers to clean up the UI dialog according to your suggestions - like Stuart said already, this would probably involve porting the PUI/XML map-canvas.xml dialog to become a native Canvas GUI dialog, which would also not be that difficult to do.

The links I posted previously cover all the relevant bits to come up with such a dialog and instantiate a MapStructure map in the form of a widget.
All using just fgdata level code, so this has zero to do with Qt5/core development - it can be done by pretty much any fgdata developer familiar with Nasal/Canvas basics.

If you should get stuck, or need help/patches or code to do this, please do feel free to get in touch, and I will be posting the necessary code to get/keep you going.

As you can see below, I have previously done exactly that:

http://wiki.flightgear.org/Canvas_MapStructure
Image


Like Stuart said, a native Canvas dialog would not suffer from any of the constraints that you discussed with James, it would be possible to come up with arbitrary behavior - without being specific to Qt5-only builds, i.e. a Canvas solution would also work for people not using a Qt5-enabled build (Qt5 being required by core developer consensus to remain an optional dependency, which makes it unfortunately impossible to use for non-optional Canvas functionality).

PS: Full disclosure: I am making this offer as someone who has not only "written the most" about the Nasal/Canvas and the Canvas/MapStructure APIs in the forum and the wiki, but also as someone who's done the original porting work of the NavDisplay to use the MapStructure MVC framework - thus, it is up to you to decide if this background is helpful or just relevant, or if you are better off with the devel list in this case.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and oth

Postby Hooray » Tue Oct 03, 2017 3:28 pm

Okay, in the true spirit of "writing the most" :lol: I have now documented how to extend map-canvas.xml to add a dropdown menu using an additional combo/list widget populated with items for different map providers, along the lines of Stuart's suggestion.

Like I said previously, I believe it would be appropriate to store tilemap provider specific data in a corresponding PropertyList/XML file, and have drafted a corresponding format here: http://wiki.flightgear.org/Howto:Suppor ... ile_Format

This is entirely based on Stuart's original work (those 3 new layers) and their primary differences - I believe that it would be straightforward to provide a procedurally generated Canvas based UI to help edit/maintain the corresponding tilemap profiles.

http://wiki.flightgear.org/Howto:Suppor ... _providers
Image
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and oth

Postby Hooray » Tue Oct 03, 2017 5:13 pm

Okay, and here's a first stab at a simple procedurally generated UI to edit/maintain such tilemap configs easily:

http://wiki.flightgear.org/Howto:Suppor ... I_Frontend
Image

This is basically based on a slightly customized version of rleibner's GCA configuration dialog.
But it can be trivially told to load an XML file so that it can be edited/saved in $FG_HOME, so that people can easily add new tilemap providers locally.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and oth

Postby Hooray » Sun Oct 08, 2017 6:08 pm

slawekmikula wrote in Mon Oct 02, 2017 4:55 pm:Some work has been done to convert PUI dialogs to pure canvas gui and maybe there should be some additional configuration for "base" tile maps. Anyway great work for all contributors !



Like I mentioned previously, the pui2canvas parser is a fairly simple piece of Nasal code that started out being under 300 LOC, meanwhile it is roughly twice that - but due to it being based on the Nasal/Canvas technology stacks, it is indeed fairly straightforward to also make it load/parse and process dialogs with tons of embedded Nasal code, including bindings, Nasal open/close sections, but also embedded Canvas widgets with their own Nasal code. The screenshot below shows what the parser is currently producing when you ask it to process map-canvas.xml for you - as can be seen, it isn't currently picking up initial widget state (the checkboxes are not marked, despite the map showing a handful of layers). But that is roughly 10 lines of Nasal code that are missing. Apart from that, the GUI dialog is indeed fully functional - which means that the map can be manipulated, all without going through PUI.

Indeed, the PUI subsystem could be entirely disabled via the corresponding fgcommands.

In other words, the same approach would work for other dialogs making heavy use of Nasal/Canvas features (think airports.xml), i.e. stuff that would be tons of work to "port" to a different framework/toolkit - for instance referring to the joystick configuration dialog, the checklist feature or the tutorial system - but also any other dialogs existing outside fgdata/fgaddon, i.e. in external hangars, especially those using lots of custom Nasal added over the years, that nobody is volunteering to "port" or update.

http://wiki.flightgear.org/Howto:Proces ... ing_Canvas
Image

Which is to say that I would personally not expend too much time manually porting existing dialogs from PUI to Canvas (or any other front-end).
It's simply something that I am personally not interested in working on - I'd rather add 2-3 additional Canvas widgets and extend the parser to let it process our dialogs.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and oth

Postby Hooray » Tue Oct 17, 2017 8:43 pm

Regarding your devel list posting, I'd suggest to coordinate this with Stuart - but here's some preliminary feedback from a Nasal and MapStructure/Canvas standpoint (to be taken with a grain of salt obviously):



  • The MapStructure framework is designed to support an arbitrary number of independent instances of a map/layer, which includes custom styling, and different back-ends, which is why controllers are usually front-end specific (think GUI dialog, or MFD avionics). For that reason, doing any variable lookups in the global property tree for stuff that should be specific to the instance of the layer/map is problematic. If in doubt, I'd suggest to create 3 independent maps via the Canvas Snippets article and make sure that your new layer/map can be independently controlled/styled, without affecting the other layers (use different positions, map orientations etc). This is in reference to doing a getprop("/sim/gui/maps/current-tile")
  • it may make sense to also provide a helper function to get a vector all supported/known TILE layers, so that a dialog can be procedurally populated with the corresponding entries.
  • instead of using the props.getNode().getValue API, you can also use props.getValues() to turn a whole XML dialog (property tree) into a hash/map
  • The XML file looks good to me, but imagine what happens if/when you are making changes to the format you have come up with - things may break (silently). Thus, it would be a good idea to add <filetype> tag and <version> tag, so that you can increment the version number and change the underlying Nasal code to check that the code matches the version number (backwards compatibility)
  • From a troubleshooting standpoint, you may want to provide a helper function to reload the underlying XML file from disk, e.g. via io.read_properties() - you can also use props.removeAllChildren() to remove old nodes first - this could make it easier for people to add, develop and test new layers interactively.
  • I'd recommend addding a comment to defaults.xml what you are including there, and why - possibly linking back to the wiki: Canvas/MapStructure (to be fair, in theory, the same properties could be used by Torsten's Phi UI)
  • thinking about it, I would put your XML file into a sub-directory under $FG_ROOT - prefrences.xm/defaults.xml has become increasingly huge, and it would be a good idea to split up the file and introduce smaller files - e.g. sound.xml, rendering.xml etc - with your tilemap stuff in its own file, which would also help clean up the commit logs of the future ;-)


Again, I've really only skimmed the commit: https://sourceforge.net/u/slawekmikula/ ... 4bf7c4c1e/
So may have missed things, and didn't even run/test the dialog. But it's looking pretty good actually - so please don't let this discourage you, it's intended to be constructive, I assure you ;-)

However, in general, my suggestion would be to refer to the "Canvas Snippets" article to learn how to set up independent GUI dialogs with their own dialog-specific MapStructure maps, which should quickly show any implicit/accidental assumptions you are making when adding new layers, because such things would all of a sudden affect all shown maps. That is one of the reasons why we are supporting different controllers for the aircraft source - the same map back-end can be driven by the main aircraft, but also other AI nodes in the tree - and all of these must be truly independent: http://wiki.flightgear.org/Canvas_Snipp ... o_a_Canvas
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and oth

Postby slawekmikula » Wed Oct 18, 2017 8:58 am

Hooray wrote in Tue Oct 17, 2017 8:43 pm:
  • The MapStructure framework is designed to support an arbitrary number of independent instances of a map/layer, which includes custom styling, and different back-ends, which is why controllers are usually front-end specific (think GUI dialog, or MFD avionics). For that reason, doing any variable lookups in the global property tree for stuff that should be specific to the instance of the layer/map is problematic. If in doubt, I'd suggest to create 3 independent maps via the Canvas Snippets article and make sure that your new layer/map can be independently controlled/styled, without affecting the other layers (use different positions, map orientations etc). This is in reference to doing a getprop("/sim/gui/maps/current-tile")
  • it may make sense to also provide a helper function to get a vector all supported/known TILE layers, so that a dialog can be procedurally populated with the corresponding entries.
  • instead of using the props.getNode().getValue API, you can also use props.getValues() to turn a whole XML dialog (property tree) into a hash/map
  • The XML file looks good to me, but imagine what happens if/when you are making changes to the format you have come up with - things may break (silently). Thus, it would be a good idea to add <filetype> tag and <version> tag, so that you can increment the version number and change the underlying Nasal code to check that the code matches the version number (backwards compatibility)
  • From a troubleshooting standpoint, you may want to provide a helper function to reload the underlying XML file from disk, e.g. via io.read_properties() - you can also use props.removeAllChildren() to remove old nodes first - this could make it easier for people to add, develop and test new layers interactively.
  • I'd recommend addding a comment to defaults.xml what you are including there, and why - possibly linking back to the wiki: Canvas/MapStructure (to be fair, in theory, the same properties could be used by Torsten's Phi UI)
  • thinking about it, I would put your XML file into a sub-directory under $FG_ROOT - prefrences.xm/defaults.xml has become increasingly huge, and it would be a good idea to split up the file and introduce smaller files - e.g. sound.xml, rendering.xml etc - with your tilemap stuff in its own file, which would also help clean up the commit logs of the future ;-)



Thanks, for multiple instances of course - i forgot about it. As fot the rest i'll sync work with Stuart review.
slawekmikula
 
Posts: 128
Joined: Sun Feb 19, 2017 10:31 am

Re: Canvas MapLayer for OSM, OpenAIP, VFR Sectionals and oth

Postby slawekmikula » Mon Oct 23, 2017 12:11 am

Hooray wrote in Tue Oct 17, 2017 8:43 pm:[*] The MapStructure framework is designed to support an arbitrary number of independent instances of a map/layer, which includes custom styling, and different back-ends, which is why controllers are usually front-end specific (think GUI dialog, or MFD avionics). For that reason, doing any variable lookups in the global property tree for stuff that should be specific to the instance of the layer/map is problematic. If in doubt, I'd suggest to create 3 independent maps via the Canvas Snippets article and make sure that your new layer/map can be independently controlled/styled, without affecting the other layers (use different positions, map orientations etc). This is in reference to doing a getprop("/sim/gui/maps/current-tile")

I've pushed changes to my fork, which enables multiple instancing of TILE.lcontroller

Hooray wrote in Tue Oct 17, 2017 8:43 pm:[*] I'd recommend addding a comment to defaults.xml what you are including there, and why - possibly linking back to the wiki: Canvas/MapStructure (to be fair, in theory, the same properties could be used by Torsten's Phi UI)

Added comment.

Other issues, I think i'll wait for other reviews. Thanks !
slawekmikula
 
Posts: 128
Joined: Sun Feb 19, 2017 10:31 am


Return to Canvas

Who is online

Users browsing this forum: No registered users and 5 guests