Board index FlightGear Development Nasal

MapStructure Layer controller

Nasal is the scripting language of FlightGear.

MapStructure Layer controller

Postby Necolatis » Mon Feb 11, 2019 2:39 am

I am modifying one of the $FGDATA layer controllers.
So I have put that into my aircraft folder somewhere and renamed it to MMAP.lcontroller

How do I say to MapStructure it should go find my MMAP.lcontroller file so I can add a layer with it?

Hope the question makes sense.
"Airplane travel is nature's way of making you look like your passport photo."
— Al Gore
User avatar
Necolatis
 
Posts: 2233
Joined: Mon Oct 29, 2012 1:40 am
Location: EKOD
Callsign: Leto
IRC name: Neco
Version: 2020.3.19
OS: Windows 10

Re: MapStructure Layer controller

Postby Hooray » Mon Feb 11, 2019 7:33 pm

yeah, the question does make sense.
The issue is, we didn't want MapStructure to be used that way ;-)
Seriously, we wanted to provide a framework to grow a library of reusable and style-able layers that live inside $FG_ROOT/Nasal/canvas/map

At the time, we were primarily addressing issues that arose due to heavy "copy/paste-coding", where people would gather useful bits from different places (aircraft) and then copy/paste, and adapt things as needed, with tons of code duplication and basically no reuse.

That problem arose primarily because Melchior stopped being involved in the project, so that Nasal no longer had an active maintainer around, which is why we are still seeing so much copy/paste coding going on in the Nasal department.

The MapStructure framework was designed to be a MVC framework to grow such a library of reusable components, that people could re-use and style/customize as needed, without having to duplicate the underlying logic at all.

Our hope was that aircraft developers would get involved in creating/customizing layers, and asking for feature requests, so that we could incrementally grow/improve this "charting library" for all sorts of different purposes.

Obviously, we would then get those new/improved layers committed to fgdata, so that they could be used in the future.
That was our hope at least.

Unfortunately, what really happened is that people wanted to be able to use MapStructure without requiring a certain fgfs version, so they still ended up using copy/paste to duplicate the needed/modified layers, so provide support for backward compatibility.

We really didn't foresee that development unfortunately. We were kinda hoping to "force" people into contributing their layers back into fgdata, so that their work could be easily reused by other aircraft/developers.

At the time, this was largely inspired by awesome avionics that were tightly integrated with the underlying cockpit/aircraft (think extra500, or the various Boeing/Airbus NDs), so that people could not easily reuse such code.

The MapStructure framework accomplished all the coding goals, but failed at the contribution level - because very few people felt comfortable writing custom/new layers at the time, and the few ones who did feel comfortable (and who succeeded), were so good at the Nasal internals stuff, that they simply circumvented our design, and explicitly loaded their copy/paste-layers into the MapStructure/canvas namespace manually, even though we deliberately failed to provide a corresponding API ;-)

I believe that artix and omega95 belong to the camp of people who have used the MapStructure framework and who deliberately worked around our design, so that none of their layers would need to be committed to fgdata - which meant that their aircraft would not be restricted by the FlightGear release schedule.

Other than that, the Extra500 folks might have adopted some ideas/techniques, but I believe they ignored the framework altogether, and simply looked at the code to borrow certain ideas.

It really isn't difficult to provide such an API, and it does exist already (again, artix came up with a pretty elegant mechanism). But the issue really is that this is not how the framework was designed to be used.

Personally, my recommendation would be to directly add your work to $FG_ROOT/Nasal/canvas/map (locally) and once you are satisfied, open a merge request and get this merged, and then you can simply raise the required fgfs version - obviously, I do realize that this may not be convenient from a compatiblity standpoint, but it's really the right thing to do from a software maintenance standpoint.

It would be trivial to provide a mechanism for "external" or aircraft-specific MapStructure resources, but I would suggest to also use a dedicated canvas/map sub-folder then, to encourage people to "eventually" commit their work back upstream.


Again, I do realize it's not ideal - but that's basically the current situation. It's all pre-dating rominet's addon system, which would provide for another elegant method to ship custom Nasal resources, possibly in conjunction with Richard's Emesary framework, but those two things were developed several years later.

For the time being, I would suggest to look at the MapStructure work done by artix and omega95.
It should be straightforward to generalize this, and append their loader logic to MapStructure.nas, and provide support for an aircraft specific default location for such layers, so that you can have your cake and eat it.

Right now, it's the equivalent of a hard-coded foreach loop with a pre-defined vector of filenames, certainly not ideal - but our intention was a different one, like I said, we really weren't aiming for a "plugin" system, but wanted people to contribute their layers to fgdata directly.

Stuart's FG1000 work (see the wiki/commit logs) also implement a number of custom/new layers, so he might have the latest experience dealing with this, and maybe he has a concrete suggestion on how to deal with this in the mid-term.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: MapStructure Layer controller

Postby Necolatis » Tue Feb 12, 2019 11:09 am

Alright.

I have already been editing a bit in Artix files when helping out in the IDG Airbus project. Not that I understood the big picture on how it all hang together.

What your saying kinda makes sense. But it also sounds a bit out of touch with how people develop planes for Flightgear. For multiple reasons:

1. Being locked into the 3-5 month release cycle is not nice. Its very motivating to work on a plane and get feed back 3 days later on a new feature you made. Its a very immediate way of coding, and the feedback is good and it comes fast, which makes the feature better. And that way of coding is inspiring and motivates me to making the next feature.
2. Why would anyone being interested in 35-50 layers that is custom to the JA37 plane display? I mean I can understand boeing/airbus and to some degree military MFDs, or of the shelf displays. But alot of us is coding very plane specific things that simply is not reusable. Why should that stuff hang out in FGDATA.
3. What is wrong with having files like .lcontroller in the aircraft folders? If the need is there .lcontroller is a standard format, people can just copy it if they really like it. I fail to see the bad thing in that, as long as people dont start making custom mapstructure so they cannot be reused.
4. The idea; "to make something for your plug-in (aircraft) you must edit the framework (Flightgear)". That is very unusual practice, in fact normally developers of frameworks go out of their way to avoid such things.
5. So what if I make some 10-15 layers for my aircraft, quickly coded but not up to FGDATA standards. Good enough for me though. Will FG project refuse me access to put them into FGDATA? So I cannot even code in my own style anymore on my own aircraft, cause stuff I make for my own displays have to be commited to FGDATA. It just makes everything less accessible, and the process cumbersome.
6. So I code some layer, put them into FGDATA. People start using a couple of them on other aircraft. I suddenly need to change them. But I cannot, I made them for my aircraft but I cannot let them grow/refactor in certain ways cause they are now part of FG and cannot be drastically changed if people use them. Okay, so I put in FGDATA new layers that does almost the same but in a different way. FGDATA map folder grows fast and become bloated.

No hotfixes, no small change or anything like that inbetween release cycles. Unless you get lucky and the devs make a point release.

The whole reason most people start tinkering with FG aircraft in my experience is because it so accessible. Just open up a text/3D-editor and start editing a plane.

An example:
I just finished making lat/lon grids on JA37 display. Longitude grid lines have small arrows on them, and some strategic placed gridline numbers. This is very specific to the JA37, not really reusable in other planes. Why should that reside in $FGDATA?

Imagine if we could not have a engines/ folder inside aircraft. But we had to get a engine commited to fgdata. People would argue forever on which values a certain engine should have. And it would stifle aircraft development.
"Airplane travel is nature's way of making you look like your passport photo."
— Al Gore
User avatar
Necolatis
 
Posts: 2233
Joined: Mon Oct 29, 2012 1:40 am
Location: EKOD
Callsign: Leto
IRC name: Neco
Version: 2020.3.19
OS: Windows 10

Re: MapStructure Layer controller

Postby jsb » Tue Feb 12, 2019 6:55 pm

I really do not want to offend anybody, but the MapStructure framework as is today is quite a mess at first glance. Even with the documentation on the wiki - once you found it between all the more or less outdated stuff - it is far from easy to understand and use. Now, that could mean insufficient documentation or me being too stupid to understand it or bad design or bad implementation - at least some parts of the implementation are evil nasal hacks you have to be a nasal expert to read and understand - a nightmare to maintain/extend.
Not repeating what was already said above, my expectation from an aircraft developers point of view would be that things (code) in FGDATA is simply available (maybe loaded by one command, if we want to make some modules optional). As ACdev I want to provide configuration for some framework, I do not want to copy things from FGDATA to my aircraft. (Yes, callback functions could be the way to extend a generic framework and make it aircraft specific).
Of course this means quite some work for the maintainers of FGDATA in terms of versioning and API stability - so far I did not find a general solution in FlightGear for this problem but the question about compatibility / changing API etc comes every now and then.
jsb
 
Posts: 285
Joined: Sat Oct 25, 2014 9:17 pm
Location: Hamburg, Germany
Callsign: D-JSB
Version: next
OS: Win7/Linux

Re: MapStructure Layer controller

Postby Hooray » Tue Feb 12, 2019 10:27 pm

Yeah, I see - I can relate to your line of reasoning, so no offense taken at all.
I am not sure I would agree with the notion that the MapStructure code is "messy" though - while it is somewhat sophisticated, it's certainly not messy compared to most other Canvas code floating around (e.g. see the NavDisplay hell).

The thing is, none of the code involved here was written from scratch, rather it was based on contributions by others, and some of us would have loved to reuse such features elsewhere (think other aircraft, cockpit or UI dialogs).

Originally, I believe it was Gijs who came up with a 744-specific ND, and then this was incrementally refactored to satisfy additional use-cases.
I believe the original code didn't support multiple instances, and it contained hard-coded assumptions about the underlying aircraft (e.g. properties/FDM dependencies).

Also, others wanted to use certain layers for non-Boeing aircraft, or Boeing variants using different symbology, so that styling needed to be supported. Even others wanted to use different I/O properties. This is basically how "MapStructure" evolved to what you see now - while it is a fairly lean MVC abstraction, it does indeed use fairly sophisticated concepts and somewhat esoteric coding patterns - but that is primarily because the people who wrote most of it, was intimately familiar with Nasal internals (Philosopher even wrote the nasal internals document that you can find in $FG_ROOT/Docs).

Ultimately, it's obviously a workaround - but the compromise works fairly well. We've been promoting a framework-centric development model for such Canvas contributions for several years now - but we're not really deluding ourselves here, the point never was to come up with a perfect framework, but to isolate/encapsulate certain key functionality, so that it can be more easily debugged, instrumented, profiled, optimized/improved and updated over time.

This is indeed something that the MapStructure framework did accomplish. In a perfect world, we would not be using this approach at all, but simply expose simgear::canvas::Element to scripting space, so that all Canvas based avionics can sub-class the corresponding high-level elements

Finally, it really is your call whether you want to use the MapStructure stuff or not - some people deliberately decide not to use it (e.g. E500), while others do a pain/gain analysis and end up adopting it despite its deficiencies (e.g. see the FG1000 created by Stuart).

I certainly don't think it's perfect, but it's definitely accomplished many of the goals we originally had, it just seems to be a little too sophisticated to encourage non-coders to get involved in it unfortunately.

FlightGear is in flux, so many contributions would look very different if they were to be re-done today - e.g. looking at stuff like the addon framework or Richard's Emesary framework (or even both in conjunction).

There are quite a few lessons to be learnt here, and like I alluded to previously, I believe the most correct approach would be exposing sc::Element to scripting space so that custom/new elements can be inherited from others and implemented using the existing interface, this kind of setup would also nicely deal with other challenges, such as multi-instance setups (multiplayer, fsweekend/linuxtag, dual-pilot etc): http://wiki.flightgear.org/Canvas_Devel ... FlightGear


In my opinion, the underlying code is basically irrelevant, the main feat is encapsulation of such functionality, for troubleshooting/debugging and profiling/optimization purposes, but also so that people are able to version/update the API as needed.

This is particularly important because more often than not, we may only end up "prototyping" certain elements in scripting space, that we may sooner or later want to move back into the core - and that is exactly where the sc::Element based approach is enormously compelling, because the Nasal prototype could be ripped out and replaced with a native C++ version, without any of the front-end stuff using the interface of the Nasal code requiring any updates. In the long-term this is also beneficial, because there is less Nasal code added without clear interfacing boundaries - right now, what is happening is hugely problematic.


Again, if you can think of a better system/setup, or even just improvements to the current system, feel free to elaborate so that we can have a discussion.

Personally, I think the sc::Element route is most promising, unless a real IPC mechanism is intergrated (or a messaging variant like Emesary).
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: MapStructure Layer controller

Postby Necolatis » Fri Feb 22, 2019 5:10 pm

Yes, my suggestion is simply one of 2 options:

Let mapstructure look in aircraft specific folders for controller/symbol files.
..or be able to load them on demand from aircraft folders.
"Airplane travel is nature's way of making you look like your passport photo."
— Al Gore
User avatar
Necolatis
 
Posts: 2233
Joined: Mon Oct 29, 2012 1:40 am
Location: EKOD
Callsign: Leto
IRC name: Neco
Version: 2020.3.19
OS: Windows 10

Re: MapStructure Layer controller

Postby Hooray » Fri Feb 22, 2019 5:55 pm

both would be possible to implement, like I said, the loading routines are in MapStructure.nas - look for a vector of hard-coded layers/controllers, that is what would need to be adapted/generalized, e.g. to look in $FG_AIRCRAFT (available in the property tree) - or simply by refactoring that routine to come up with a "loadLayer()" API.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU


Return to Nasal

Who is online

Users browsing this forum: No registered users and 4 guests