Board index FlightGear Development New features

FGPython an propose for Python as an nasal alternative

Discussion and requests for new features. Please note that FlightGear developers are volunteers and may or may not be able to consider these requests.

Re: FGPython an propose for Python as an nasal alternative

Postby bugman » Wed Dec 30, 2015 5:17 pm

I have specifically stayed out of it, because as I said, too much information will smother the fire. The people experimenting with the idea will quickly learn the limitations and advantages - it serves no purpose for me to lay that out for them. I see that there are some false expectations, as well as potential that is not yet visible. This will resolve itself very quickly. I think this is a great idea as this is an idea I had back in the days when Nasal was introduced. Anyway, I suggest less talk and more coding and experimenting!

Regards,
Edward
bugman
Moderator
 
Posts: 1668
Joined: Thu Mar 19, 2015 9:01 am
Version: next

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Wed Dec 30, 2015 5:26 pm

I suggest less talk and more coding and experimenting!

As far as I can tell, the people interested in this (so far), don't have the background/experience to pull this off in any scalable way, i.e. beyond a FGNasalSys-equivalent using Python, which is why I suggested to use HLA (or any other form of IPC) to ensure that Python does not have to be running in the main loop.

You have offered to help with the Python side of this (which is great by the way), but what is missing are people familiar with FG/SG internals (which may include Nasal/FGNasalSys familiarity) to implement your advice/feedback.

Just allowing "hello world" to be executed in Python would be very easy to do with very minor additions to the existing build system/code base.
And if people don't understand how to do this, they are almost certainly not in a good position to debate the merits of Nasal vs. Python or using IPC, like HLA, for integration purposes.

Given that Curt seems to be supportive of the whole thing, we/you seem to have at least one (key) core developer willing to tinker with this, and possibly help review such a patch to get this committed as a build time option, which is more support/endorsement than the average FG feature will get in years :D So people wanting Python support in FG are not in such a bad position after all.

I think you disagreed on the notion that embedded Python would be a gaping security issue allowing arbitrary python modules to be installed without "permission", never meant to say that - like I said, I am more concerned about the mid-term evolution/adoption of Python module use in FG, and not of turning FG/MP into a Python based botnet spreading viruses :D

From where I am standing, I am making my support dependent on people being able to patch/build SG/FG from source, who should ideally already be familiar with a programming language like C/C++ or Java to have any reaosonable chance to pull this off - familiarity with git would be a definite plus, but Python knowledge not really required to embed an interpreter.

But as far as I can tell, "it" will happen anyway based on following the ongoing HLA updates/work posted by Stuart, James and Mathias, with an increasing use of Python via HLA, which will inevitably mean that more and more SG/FG APIs will become available as services via RTI sooner or later.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: FGPython an propose for Python as an nasal alternative

Postby bugman » Wed Dec 30, 2015 5:44 pm

Security through obscurity is false security. If you can bundle some evil Python code with an aircraft, you could also bundle some evil Nasal code to do the same thing. If you want to protect against this, then you should consider a solution that works with both embedded interpreters equally. This is no reason to trash embedded Python.

As for HLA, this can be added later. The first thing is to embed the interpreter. The second is to access the property tree. The third might be to wait for a HLA compatible solution for the property tree to be added to FG, then hooking the Python interpreter into that. I also don't understand the HLA push here, because HLA is a slow technology (just like most multi-processor solutions). I have extensive experience with clustering, and I am well aware of the disadvantages of HLA. However certain built-in modules could provide HLA access to certain functionality. This can all be done down the line - planning for it now is really too much.

Finally, this would all have to be done in C. From the main program, you interact with embedded Python only using C code, via the embedded Python C interface. Therefore the person implementing this needs to either know C, or is willing to learn C.

Regards,
Edward
bugman
Moderator
 
Posts: 1668
Joined: Thu Mar 19, 2015 9:01 am
Version: next

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Wed Dec 30, 2015 6:20 pm

security through obscurity: that was never the point, but frankly, FlightGear (and its plethora of custom niche solutions) are not exactly attractive/relevant to the average script kiddie.

While there are undoubtedly many bugs in FlightGear, including security issues, it takes quite an effort to exploit those, and you would still not be dealing with server software running 24/7 - heck, not even fgms seems of much interest to the security community.

So the whole security issue is two-fold, because Python will undoubtedly be much better maintained than Nasal (or the rest of the whole SG/FG code base in its entirety), but Python support in FG would also have much more relevance than any of the existing FG solutions, just look at the number of Python CVEs.

Python's popularity, especially compared to Nasal, will bring some new issues to the table, without that necessarily having to imply security issues.

Besides, like I said, you don't have to use low-level C to embed a Python interpreter - it is doable using higher-level wrappers like boost/python.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: FGPython an propose for Python as an nasal alternative

Postby Thorsten » Wed Dec 30, 2015 6:21 pm

Thorsten: just taking a small step backwards to see a bit higher view. We have someone (maybe a few people) that are interested in doing some exploration. They think there could be some interesting things to be found. You are demanding that all the things that will be found in the expedition be enumerated in detail ahead of time and their value measured before authorizing the expedition.


No, not really.

I don't have any Python experiences. So I do what the funding agencies usually do when you ask for a hundred million bucks to go to Pluto - I ask Why are you interested?

I don't think it's unreasonable to ask - and I would genuinely like to know (and be convinced).

So far, the reasons given are:

* Because we get better support for heavy-duty numerics.
-> Doesn't really sound like a good idea to do that in scripting.

* Because I like the syntax better.
-> Okay - may be, but the same case could be made for every scripting language in existance.

* Because it can overcome multithreading problems.
-> Demonstrably wrong.

* Because it's going to attract more developers.
-> I doubt that - why did this e.g. not work with GLSL?

* Because it is a technology enabler.
-> What exactly does it enable? Why can't anyone give a single real-use example?

I mean, seriously - there are people who have lots of experience with Python and there isn't any reason for doing it more compelling that has been brought forward? You'd never get a Pluto mission funded on anything that shaky!

I'm just asking questions and pointing out inconsistencies - if anyone really wants to explore, I can't prevent him. But I think it's not grossly off the mark to ask what end result is hoped for - if we get a ton of additional issues to deal with and the net result is that a handful people feels more comfy coding in one syntax rather than another then I am not convinced.

You people really shouldn't take my request for a convincing case personally - I want to know whether it exists, nothing more, nothing less.
Thorsten
 
Posts: 10724
Joined: Mon Nov 02, 2009 8:33 am

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Wed Dec 30, 2015 6:32 pm

You do have a point, even though it may be difficult to prove :D

You could compare Python support in FG to providing additional bindings/modules to scripting space in Nasal - there may not be an immediate use-case or need, but it opens up new possibilities - just like having access to a cloud interface, terrain sampling via geodinfo() allowed you to build a weather system mostly in scripting space.

And the real point is not that many of these things should not be done in scripting space, but that Python comes with "batteries included", i.e. you can actually access native code using said modules, and they will usually be fast enough.

I don't think we can necessarily come up with a good example, because it depends hugely on the use-case/modules you have in mind - you will almost certainly say that certain things should not even be done in scripting space, but that they should be implemented in native code - whereas Python will just enable you to do it, without much cost in terms of performance, because stuff like lapack/libblas etc happens to be fast enough, and is maintained externally by hugely successful projects.

Referring to the OpenCL example I brought up, that would also be trivial to support just by adopting Python in FG and mapping the property tree to an opencl interface, without any modifications at the FG/C++ level at all.

I think that's really what it all boils down: Python is mainly a technology enabler, you don't need to have a concrete use-case in mind to tell that having support for it would be very empowering potentially, much more so than having Nasal support, where the same degree of power would require a conscious and orchestrated effort from core developers to provide functionality -and even just providing Nasal bindings for existing SG/FG C++ code is a bottleneck, despite most core developers not wanting to see even more Nasal use in fgdata space.

Which brings up back to Curt's point: back in ~2006 when a handful of core developers were opposed to the osg port being committed, nobody was foreseeing its impact on FG as a technology enabler, i.e. referring to effects/shaders, deferred rendering (Rembrandt), CameraGroup/multi-window support, Advanced Weather, ALS, EarthView or even the Canvas system.

Back then Mathias was the only one stating explicitly "OSG is a huge technology enabler", and almost everybody else was only seeing the regressions introduced by the port.


Likewise, for Python there are existing HLA bindings available (not so for Nasal), which is to say that Python is becoming increasingly attractive for people wanting to use Python (or even Java)


And with osm2city, atcpie etc there are more and more FG related sidekicks that would be relevant to future FG/Python support.

Regarding GLSL: like you once said yourself, to use GLSL properly, people need to have a strong background in math, but also 3D rendering concepts - so the barrier to entry is higher than it may seem
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: FGPython an propose for Python as an nasal alternative

Postby Thorsten » Wed Dec 30, 2015 6:42 pm

back in ~2006 when a handful of core developers were opposed to the osg port being committed, nobody was foreseeing its impact on FG as a technology enabler,


I wasn't around, but I wonder whether seriously nobody pointed out that we could have e.g. programmable effects. I mean, Tim Moore must have known, he designed the effect system after all, which is pretty ingenious.

Which is to say, there must have been a vision.

It's one thing to say 'We don't know what exactly we will find on Pluto.' It's a completely different thing to even lack a vision of what might be found.
Thorsten
 
Posts: 10724
Joined: Mon Nov 02, 2009 8:33 am

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Wed Dec 30, 2015 7:02 pm

Believe it or not, as far as I remember, Tim also wasn't yet around at that time :)

You are right though, Mathias did make statements along those lines - but they merely appeared hypothetical at the same to many developers given the number of regressions introduced by the "patches", which brings us back to the whole "osm2city" movement we've been seeing in the last couple years (all implemented in Python), and as we all know, a number of developers have been wanting to "absorb" said functionality back into FG, with "project 3000", "osm2city" and other Python-based projects, there is quite a compelling number of features that could greatly benefit from Python support in FG.

For instance, remember the whole "procedural buildings via osm floorplans" discussion that we've had on and off ?
That, too - could be greatly facilitated without any core developers having to provide C++ code to implement wrappers for Nasal.

Subject: Random Buildings
stuart wrote:Torsten has rather hit the nail on the head:
it would be somewhat far-fetched to ask Stuart to make a faster Nasal interface than the existing one just because someone might eventually work on it.


Yes, it would be possible but it probably represents 10 hours of speculative development that I don't have time for right now. Given that people don't seem to making significant use of the tools already available by modifying materials.xml (at least not in a way that is making itself as far as git), I just don't see sufficient interest. That may change once these changes are available, as it'll give people a lot more scope for creating cities.

-Stuart


Fast forward a few years later, we are seeing exactly what I predicted ~4 years ago (e.g. see the Instant City, just add water thread from 2010): Not Nasal, or C++ code, being used for the implementation of such features, but a separate programming language (Python), with everyhing running outside fgfs, and no transition path in sight to change that anytime soon.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: FGPython an propose for Python as an nasal alternative

Postby curt » Wed Dec 30, 2015 7:38 pm

Thorsten,

One thing you didn't list in your questions is performance of nasal vs. performance of python. A subpart of this question is the relative performance of the garbage collectors. This is a potential win for python (maybe?), but useful benchmarking requires actual examples.

So please include performance in your list of questions, and it's something I'm legitimately curious about.

Apparently it was ok to spend a couple hundred million (or whatever it cost) to get some really neat high res pictures of pluto. It generated a lot of great press for Nasa. I bet we also learned a bunch of sciency stuff too even though the media presentation to the public seemed to be mostly focused on the cool pictures and we never really heard anything about the boring science stuff.

I get the sense that you don't want to budge until you are convinced. That is fair, but you have structured the questions in a way that makes it nearly impossible to be convinced. Any of the answers or suggestions that have been presented are interpreted in the most diminished way possible. I'm not convinced either, but I'm curious and interested.

Personally I see some additional benefits that could spill out into some of my other personal projects (i.e. my open-source UAV autopilot/flight control system.) Python integration there (which implies integration with the property system which I use heavily in my autopilot) would allow me to do some really interesting things on the higher level mission planning side ... currently that is done in low level C/C++ code in a pretty rigid / opaque way. Allowing mission planning and high level task 'intelligence' to be coded in python by 'end users' would be a great technology enabler for my autopilot project in the hands of students or hobbyists. This is outside of FlightGear, but it factors in to some of my personal interest. (And for what it's worth, I'm currently building my autopilots around a beaglebone/linux that already has a python environment onboard ... I would just link to the required pieces.) For what it's worth, due to project/research partnerships and connections, there is potential for at least part of this to find it's way into NASA and DLR projects.

If I run down a 'hypothetical' thought path ... and all of this thread is hypothetical, right? I can imagine a number of college or advanced high school level or grad student research projects that could benefit from python integration and tools like numpy and scipy. I currently work in the aerospace engineering department at the university of minnesota (since this past July.) I see a number of projects that leverage matlab and simulink ... things like extended kalman filters and sensor fusion for attitude estimation; things like advanced state space flight control systems; fault tolerant and adaptive control systems, and that's just my own small peephole view from within my own department. There are many ways to do something, but imagine if as student could write a complex kalman filter in python embedded in FlightGear and test it in real time flight comparing the flightgear 'truth' against the filter estimate. (Or replace kalman filter with some other fairly advanced research project.) Sure this could all be done in nasal or many other ways, but what if the ultimate goal was to take the work and package it and use it in some larger more important context? Python code ports to new systems because python is nearly universally available. Nasal really is not .... not without a lot of work to bring the whole nasal system over to a new realm. I'm sure none of this makes an immediate and specific compelling use case, and probably no one in my department would rush down this path if it was available to them because they have a big investment in their current way of doing things ...

So all that to say that I think you are a bit overly harsh in your questions and a bit overly dismissive of the answers or the potential benefits that could emerge from the answers ... and I know you disagree with that so you don't have to restate your disagreement, I'm just stating how it feels from my side. :-)

Thanks,

Curt.
curt
Administrator
 
Posts: 1173
Joined: Thu Jan 01, 1970 12:00 am
Location: Minneapolis, MN

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Wed Dec 30, 2015 7:54 pm

See the GC article on the wiki, Tim and others have already commented on the legacy (reference-counting) memory management scheme in Python.
These days, there's an incremental (generational) collector available, with the added benefit of that being exposed to scripting space, which means you can customize/trigger and even disable GC entirely.

So it's not just about plain performance, but the underlying approach/algorithm that differs - and turning the existing Nasal mark/sweep GC into a generational one was also by Andy:

http://sourceforge.net/p/flightgear/mai ... /29300807/
Andy Ross wrote:But you're right: allocating objects into generations and only
mark/reaping from the most recent one on most iterations is a
straightforward optimization and will definitely make things faster.


But like Stuart mentioned previously, the GC scheme used is practically irrelevant once the interpreter no longer runs within the main loop, no matter if that means worker thread or a separate process that communicates with FG via IPC/RPC to fetch/update properties and run fgcommands/FG specific extension functions.

Regarding the UAV use-case, that's valid, but nothing that could not be accomplished using a standalone Nasal interpreter based on what we have in SimGear, plus the existing bindings/extension functions for property/timer and listener support.

python is nearly universally available. Nasal really is not .... not without a lot of work to bring the whole nasal system over to a new realm.

that is generally true, but please don't forget that Nasal is part of SimGear, so is supported in a standalone fashion and can be embedded in any C app using 120 lines of code, see Andy's repository at: https://github.com/andyross/nasal/blob/ ... asal-bin.c

that's really all the boilerplate that is needed to embed Nasal in another C program, or use it in a standalone fashion (REPL).

That is not to say that Python would obviously be the "better" option, because Nasal is frankly never going to be as relevant as Python is, and has been for years.

However, a UAV project that is already using various SG/FG components could easily benefit much more from Nasal support than it might from Python support, simply because there is so much existing stuff, that could be easily reused - no matter if you need Nasal standalone or embedded, it's really just ~100 lines of code to copy & paste and you are good to go (we have on fact done that to do standalone regression testing) - indeed, Andy's repo contains a bunch of standalone Nasal regression tests: https://github.com/andyross/nasal/tree/master/misc

Personally, I do see the potential of Python, but would strongly suggest people already using SG/FG components to consider using Nasal in their projects, too

Icecode GL's original FGradar prototype was another separate, but SG/FG based project using Nasal in such a fashion:
https://github.com/hamzaalloush/fgradar-clone

And it's using the existing SGSubsystem*Mgr/Group based design, so that it could reuse a heavily stripped-down version $FG_SRC/Scripting, similar to Torsten's FGPanel approach:

https://github.com/hamzaalloush/fgradar ... /scripting
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: FGPython an propose for Python as an nasal alternative

Postby abassign » Wed Dec 30, 2015 9:57 pm

@Thorsten

No it won't - because the 10 ms of Nasal will run while the GPU does its frame because - wow - they actually work in parallel :-) So your whole example is dead from the start - the framerate won't be 33 fps but 50 fps.


A tip: before speaking read well what others write, and not read only yourself!

I think you have understood the problem of the "collapse of the performance" due to the presence of synchronous processes to the mail loop. The fact that distribute the CPU load related to NASAL processes, across multiple cores (not more than one for the truth ...) does not change the problem ... simply moves the problem without solving it. Then there is the strangeness that the option for the use of a second CPU can only be configured manually by editing the configuration file ... There will certainly be a valid reason, but this is a reason that escapes me. However I do not think that the majority of users (70-80% in my forum (230 subscribers)) of FGFS modify this parameter and then your statement is false anyway for 70-80% of users!

For the rest I agree with you, it should be done several tests to find out which is the best way to completely replace the code NASAL with Python.

@Hooray

Which is why you need to use some form of IPC/RPC to ensure that Python can run outside the main loop, or it will be subject to the same issues that Nasal is seeing.


Which is why I think that it is important to discuss the pros & cons before spending any time coding. Disregarding the HLA/IPC option is not just plain wrong, but also disrespectful towards those who spent years coming up with the infrastructure, code and plans to make this happen.


For some time, I'm thinking about the quality of the project HLA, I love being able to clear separation of the two environments (main-loop and aircraft) on this I am completely agree with you! At the same time I'm thinking if you can integrate the folder containing the data of the aircraft with the python libraries used (the same method used by Java jar files) so each author of planes or other objects (such as missiles, car etc ...) they can take what they want as their only problem is to connect properly with the loop-mail address. This approach would be much more sustainable and would stimulate the possibility of developing also objects also outside of the FGFs, without having to interfere with the development of the main-loop.

There is a second reason is that Python runs in a loop to a separate main-loop FGFs, that's clear! Therefore, analyzing what I find there must be something that unites the two loops.

In Java I often had this type of problem, particularly when applied to the management of SAP systems for electronic purchasing in the Internet (B2B and B2C). In this case there are two possibilities, or you use some kind of software bus (interface framework that I understand thatHooray called IPC/RPC) or an external protocol (in this case the situation is similar to the protocols HLA/RTI).

At this point I said: "Let's see what happens if I have to move the needle of the vario" In XML there is a code that allows to associate a value of the vertical velocity with an angle associated with a graphic object (which is this case the needle) like I can do this process by HLA/RTI ?

Python if I think I have to use some function that replaces the XML equivalent, this could be an advantage as it would allow the programmer to work in a unique linguistic and semantic context, not only but also with the opportunity to make a more precise debugging . This means that we must build in Python what XML is already very good, I do not think it is complicated if well designed, but how it is can communicate to the main-loop via HLA/RTI efficiently? I have to remember that an aircraft may have dozens of these processes! In this case I always imagined that you have to send a stream of binary data according to a protocol very simple and light, perhaps built dynamically during operation of the handshake between the two loops. It sounds complicated, but it really is not.

I figured with the same way it is possible that the process Python sends the main-loop via a specific protocol also 3D files, this way is the loop of Python that takes care of load objects into the main-loop FGFS!

It sounds complicated, but it does not seem more than many other things, but lets give big advantages:

1. The process of management of the aircraft is to be paid by the loop-python and interacts to the world Fgfs through an appropriate framework that actually replaces the XML.

2. Is simple can make groups of aircraft controlled by different users as it is no longer necessary to have the local definition file of the plane! I think at this point in a flight training will be similar to many separate processes Python that may reside on your PC or on other PCs.

3. The current structure of FGFs remains unchanged (NASAL, XML, etc ...) to the great joy of the Conservative Party!

4. The framework, which I consider to be rather compact, can be written for many different languages, such as Python, Java, C ++, ADA, Ruby etc ...

5. The Framework takes charge of defining the method of interconnection between the main-loop FGFs, so that it is transparent to the programmer. Not only, but you may also choose the best connection strategy.

Image
abassign
 
Posts: 777
Joined: Mon Feb 27, 2012 5:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2018.3
OS: Linux Mint 19. x

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Wed Dec 30, 2015 10:15 pm

Just briefly, referring to the first paragraph where you are talking to Thorsten: You are wrong about pretty much everything you said there, i.e. the threading directives you are referring to are purely OSG related, and only help with certain well-defined parts of rendering. Apart from that, only a handful of threads are used by FG. Nasal is the only FG component intended to be threadsafe, it's the fg interface that is not threadsafe. You can right now run Nasal threads, and get a ~15 khz update rate easily.

Don't get me wrong, I do appreciate what you are trying to do here, and I do appreciate that it must have taken you a while to come up with the drawing, but you are still missing quite a bit of background info.

So my offer is this: tell me what your programming background is, and I will do my best to explain to you, in layman's terms, how the fg main loop, subsystems and Nasal in particular are working, so that you are on the same page - does that sound good ?

Just tell me a programming language that you are familiar with (if it's too obscure, like COBOL, please name a few different ones, so that I can pick examples that I am familiar with)

I am just saying this because your whole notion of "processes" is already where you are mistaken, FG's main loop is really just a huge ugly nested for-loop that calls the ->update(double dt) loop of each SGSubsystem/Mgr instance (think fdm, audio, nasal etc) - but there are still a handful of pseudo-subsystems that don't even use this design, while others are working behind the scenes in background threads (mainly in simgear), or are interleaved with the FDM.

So I think you could contribute valuable feedback here, but you may need to know a few more facts first - which is why I believe that it would be a good idea to fill you in on a few more details

Otherwise, I do find your posting way too abstract, and lacking in details, and the whole notion of "replacing Nasal" based on what you wrote is hugely problematic - in fact, there are 2-3 core developers who have considered doing exactly that, and they preferred discussing this in private with others, with the only viable option being a modified Nasal parser to rewrite Nasal code in Python (i.e. you gotta deal with existing legacy code, just like you cannot replace PUI with Qt5 unless there's a 1:1 mapping to ensure that there are no regressions introduced !)


(no offense intended or taken)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: FGPython an propose for Python as an nasal alternative

Postby abassign » Wed Dec 30, 2015 10:47 pm

@Curt

what you described is the problem of so many others. I tried to define a slightly different approach that allows to separate completely the main-loop Fgfs with the loop managed by Python (or another language). This way you can have something more solid, an execution environment completely asynchronously to the main-loop, controlled by events that the main-loop send. Will then loop Python to respond to these events in the most appropriate according to the written program.
I think it's important to come off from the couple XML-NASAL, which proved difficult to manage for large projects, such as SU15 or F15 etc ... With the structure that I propose to allow programmer to do what he wants, but at the same time to respect the current environment and at the same time give a real meaning to the hard work HLA/RTI, which makes sense only if made usable by anyone.
De facto connection framework replaces XML calls to functions written in a certain language. Are there not many XML structures that are transcribed through the connection framework and I feel that the current XML interfaces with C++ equivalent structures featured in the main-program.
Certainly ask those who know the core of Fgfs a series of calls to replace the XML interface can seem like an insane .. But in reality we are only asking to move the load of certain activities at the edge.
I would then come to favor the development of aircraft or objects that can respect different models of intellectual property and should not be rewritten every time changes the structure of FGFS.
A good-Connection Framework simple and light can remain very stable over time and grow in functions that do not affect the previous versions.

Let's see how the Conservative Party, currently in power, responds to my proposal, I'm curious;) and I also hope that it can avoid continuing its offensive manner of analyzing the problem.
abassign
 
Posts: 777
Joined: Mon Feb 27, 2012 5:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2018.3
OS: Linux Mint 19. x

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Wed Dec 30, 2015 11:09 pm

abassign wrote in Wed Dec 30, 2015 10:47 pm:@Curt

what you described is the problem of so many others. I tried to define a slightly different approach that allows to separate completely the main-loop Fgfs with the loop managed by Python (or another language). This way you can have something more solid, an execution environment completely asynchronously to the main-loop, controlled by events that the main-loop send. Will then loop Python to respond to these events in the most appropriate according to the written program.
I think it's important to come off from the couple XML-NASAL, which proved difficult to manage for large projects, such as SU15 or F15 etc ... With the structure that I propose to allow programmer to do what he wants, but at the same time to respect the current environment and at the same time give a real meaning to the hard work HLA/RTI, which makes sense only if made usable by anyone.
De facto connection framework replaces XML calls to functions written in a certain language. Are there not many XML structures that are transcribed through the connection framework and I feel that the current XML interfaces with C++ equivalent structures featured in the main-program.
Certainly ask those who know the core of Fgfs a series of calls to replace the XML interface can seem like an insane .. But in reality we are only asking to move the load of certain activities at the edge.
I would then come to favor the development of aircraft or objects that can respect different models of intellectual property and should not be rewritten every time changes the structure of FGFS.
A good-Connection Framework simple and light can remain very stable over time and grow in functions that do not affect the previous versions.

Let's see how the Conservative Party, currently in power, responds to my proposal, I'm curious;) and I also hope that it can avoid continuing its offensive manner of analyzing the problem.


Don't be mistaken here: Curt is the "leader" of the conversative party, so to speak: He explicitly suggested to think in terms of an optional feature, with optional dependencies - i.e. a custom build mode that could be enabled during build time. I briefly mentioned how this could be done. Curt explained also that Nasal had to stay for the time to come.

What you keep doing here, probably without realizing it, is that you are disqualifying yourself from being taken seriously due to the nature of your postings, either because they're ill-informed or too abstract, lacking in detail and missing any relevant context.

Thorsten asked for facts and for evidence - I also stated that those would be hard to bring up in this context.

However, in terms of "facts", the only thing that truly counts is the existing code base, but also existing functionality.

Don't expect to revamp either the source code, or any functionality, without also taking the other into account.

Obviously, all contributors have a certain "vision" for FlightGear, i.e. something that "motivates" them to contribute (to varying degrees), however that vision must somehow align with the facts, i.e. existing features and existing code.

Some people have a remarkable track record of working with FlightGear doing unprecedented things - however, FlightGear has become a massive project, and there is not a single person intimately familiar with all parts/aspects of it and its evolving ecosystem, and sometimes people's involvement dates back to a long time ago, so that their statements may have become inaccurate over time - but at the end of the day, there are "veterans" involved in the project for over a decade, i.e. people who literally spent 10+ years contributing to the projects in various areas, while many have only touched a few parts, others have embarked on large-scale architectural changes (think Mathias Frohlich, James Turner, Timothy Moore) - so their opinions and statements are based on facts, i.e. on working with the code and features that you want to change.

There is a general consensus among all active core developers that the want to split up the simulator using HLA, and that is not because HLA is such a great thing, but because using HLA allows to address a handful of issues at the same time, i.e. allowing things like the fdm/autopilot or scripting to run in their own threads/processes, split off rendering from simulation loop, but also distribute FlightGear in general - so that most subsystems can become standalone processes that only use IPC/RPC to communicate with other parts of the simulation. At the end of the day, HLA may not be the only option to accomplish that (FG already has support for various IPC mechanisms, many custom ones), but HLA is an important standard, especially when looking at the simulation industry, so it will also allow interoperability with other simulators, while allowing a plethora of long-standing FG issues to be tackled at the same time, such as modernizing the multiplayer system, factoring out the AI traffic component, unifying AI traffic and MP - but also providing support for multi-instance setups, like those at FSWeekend/LinuxTag.

All this is makes the HLA pathway extremely compelling, because there is a single IPC mechanism that can help with all of this and more, without having to juggle a handful of different IPC mechanisms or technology stacks (think CORBA).

So even if you don't agree with everything that's been said here, you should give people the benefit of the doubt, given their involvement in the project for so many years.

None of that is to say that Python support could not also be a huge technology enabler for the FlightGear project, but you are currently doing a disservice to the idea by the way you are posting, and by your way of tossing with abstract ideas without looking at the facts, and ongoing trends/developments.

HLA may not seem relevant/applicable to you, or even bugman, but that is mainly because of your background and the use-cases you have in mind, once you look at the bigger picture, you will inevitably have to reinvent an IPC mechanism like CORBA/HLA to accomplish what you want do.

For the time being, I think you are doing a disservice to your desire to have Python support in FlightGear, while also disqualifying yourself from receiving the very help that you obviously need so urgently (I honestly doubt that even bugman is going to put up with you now ...)

At the end of the day, what I think, say or do doesn't matter at all - so you could just as well take your fancy ideas and implement them (well, try to), but I doubt it will take you far given what I have read so far.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11317
Joined: Tue Mar 25, 2008 8:40 am

Re: FGPython an propose for Python as an nasal alternative

Postby abassign » Wed Dec 30, 2015 11:21 pm

@Hooray

Meanwhile, I thank you for that answer seems constructive, and if I put you in the Conservative Party is only to better describe your way of thinking about this issue, I think it's an honor to try to defend the job done and warn about the pitfalls thatIt can present a journey into the unknown.

Just tell me a programming language that you are familiar with (if it's too obscure, like COBOL, please name a few different ones, so that I can pick examples that I am familiar with)


I think Python, although in the future I would not see badly other languages. However, the connection from the framework of FGFS can be written in its natural language C++ (although I think that following the XML parser is not difficult to do), then there is the layer of connection, I would start immediately on the model HLA/RTI, in since, especially at the beginning of this phase, who uses Python, I do not think he wants maximum performance, but the ease of interfacing to its own procedures.
The part of the Python "connection framework" will be written by those who program in Python if the protocol set out in connection framework is clear (I think of it, as should be the mirror current XML). As you may have figured out is the "connection framework" that takes care of interfacing with HLA-RTI or another way of communication enabled by this module, I think this is a must very mportant that allows you to bring all the use of the HLA-RTI .

I am just saying this because your whole notion of "processes" is already where you are mistaken, FG's main loop is really just a huge ugly nested for-loop that calls the ->update(double dt) loop of each SGSubsystem/Mgr instance (think fdm, audio, nasal etc) - but there are still a handful of pseudo-subsystems that don't even use this design, while others are working behind the scenes in background threads (mainly in simgear), or are interleaved with the FDM.


I know that, just look under the hood of the engine that gets scared! So I think the usefulness of an intermediate environment, which I have called connection-framework.

We can start doing some testing with the UFO that requires very little XML, so as to achieve a minimal "connection-framework". In order to understand if the way is right.

There will be other issues to deal with, such as how to launch the application that, if you work in HLA-RTI is a separate task. And what method will be used to send files 3D and 2D, waves etc ... I think that theoretically should be a stream that reconstructs a kind of "virtual folder" which coincides with the folder of the aircraft.
abassign
 
Posts: 777
Joined: Mon Feb 27, 2012 5:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2018.3
OS: Linux Mint 19. x

PreviousNext

Return to New features

Who is online

Users browsing this forum: No registered users and 3 guests