Board index FlightGear Development New features

FGPython an propose for Python as an nasal alternative

Discussion and requests for new features. Please note that FlightGear developers are volunteers and may or may not be able to consider these requests.

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Mon Dec 28, 2015 8:42 pm

you need to make sure that you are not comparing apples and oranges here.
But regardless of the actual numbers, Python does support different GC implementations/schemes.
The on used by Nasal is a conventional "mark & sweep" collector implemented using traditional algorithms and data structures, it is a single file and just a handful of functions that would need to be updated to turn that into a generational (incremental) scheme where younger generations are collected more frequently than those who have already survived several GC passes. For details, please refer to the GC article on the wiki, and see Andy's quotes there.

However, a scripting interpreter embedded via HLA/RTI would not have any direct impact on frame rate/spacing, because it would be running in a separate thread/process, and only do certain IPC/RPC with the fgfs main loop (rendering).

Security is a valid concern, and one that has been raised by Melchior and others previously.
However, web sockets can also be accessed via Nasal.

Like Thorsten said, stuff like matrix operations could be also easily provided to Nasal - the thing to keep in mind here is that numpy is not just python code, but that it does use lower-level C/C++ (or even assembly code), e.g. highly-optimized stuff like libblas - which is to say that the raw power is not so much coming from Python but the fact that these are well-maintained bindings exposing existing libs to Python.

Obviously, it would be pointless to recreate such a module in C/C++ from scratch, but the same libs (BLAS & LAPACK) that numpy etc are using, could also be provided by a thin Nasal wrapper, e.g. using cppbind.

So by mentioning OpenCV, numpy etc you are really just proving the point that you don't want a different language, but that you want more/additional bindings for existing libraries.

That is a point well taken, but also one that must not be taken lightly, not just from a security standpoint (which is relatively safe to ignore given FG's primary use-case), but also from a deployment standpoint - i.e. python development being easily much more active than FG core development, and the plethora of 3rd party modules that people may want to use with FG, would cause addtional issues.

Nasal is relatively difficult to abuse, you need to have fairly good understanding of its internals in order to break FG and the way it is embedded in a sanbox'ed fashion - with Python, a sandbox environment could also be implemented, but as can be seen here, people will definitely want to use certain modules like numpy, opencv etc - and they want to be able to use tons of other libraries, sooner or later there will be a dependency hell resulting from it - imagine having to install different python/lib versions for a particular aircraft/feature (like bombable, advanced weather, fgcamera etc).

Python supported as a federate via HLA/RTI would not have much of an impact on frame rate/spacing at all, so the GC scheme used is kinda irrelevant - but that would also apply to Nasal once/if it is moved to a separate thread via HLA/RTI.

Again, integrating Python in a SGSubsystem fashion would not be much work, but it would be also subject to the same issues that Nasal is currently seeing/causing (including the GC issue).

Thus, it would make more sense to pursue Stuart's and James' approach and move Nasal to a HLA federate, possibly in conjunction with an improved/generational GC scheme (gc.c is tiny and straightforward code).

From a computational standpoint, numpy, cython etc may be great - but for FG it would make more sense to use OpenCL for such features - which is admittedly also supported by Python via pyopencl, while many of such libs come for free, there still is a certain cost to actually use them, and if that comes not in terms of runtime footprint, it will be an increased management/security footprint.

In general, it would make sense to formalize the whole decision-making process when it comes to introducing/accepting new/optional dependencies, because that is indeed a fairly common irritation/theme, e.g. look at the OSG port, boost, Qt5,C++11 or now Python.

It would be great if this could be raised during one of the upcoming "hangouts" so that a corresponding "policy" document could be drafted, supported/endorsed by all active core developers, and published on the website/wiki.

I am not opposed to Python "per se", I am just opposed to not thinking it through - apart from that, I think that OSG/Python/Qt5 etc could be great technology enablers in the time to come.

The C++ integration code exists already, in the form of working Nasal bindings, which are straightforward to adapt for Python use - so that is a no-brainer in comparison to the can of worms this is opening in the years to come, if not done properly.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby wkitty42 » Mon Dec 28, 2015 8:52 pm

curt wrote in Mon Dec 28, 2015 8:13 pm:Hmmm, www2 brings up a good point that's worth considering before going too far down the python path. If we can easily pull in any and all python library functionality, like web servers (for example), then we have some huge security concerns to factor in. A rogue aircraft could potentially turn your computer into a spam bot, or worse. One of the benefits of nasal is that it is carefully sandboxed for what it can or can't do ... would we have the same level of control over security with full blown python integration? I don't know?

this was a recent concern that jumped out at me when i looked at an add-on for fgfs... it has the ability to log into a website and could easily be adjusted to log into other sites and possibly other services (eg: irc)... it is done with a mixture of xml and nasal... it may be for some sort of tracking activities but i can easily see how it could be exploited for nefarious purposes with some modification...
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9148
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Mon Dec 28, 2015 9:03 pm

I have contributed to all Nasal related IPC/RPC tutorials you can find on the wiki (some of them I single-handedly wrote from scratch), I can assure you that it is pretty difficult to abuse these features, because there is a fairly massive setup overhead involed when it comes to doing remote network access from Nasal - usually, it will involve a fairly weird combination of different I/O means like the telnet/props system, but also the generic protocol and multiplayer - it is only a certain combination of these features, that makes these systems possible - in other words, there are much lower hanging fruits (read: attack vectors) than FlightGear's limited Nasal IPC/RPC support.

The code you are referring to is probably related to airliner/multiplayer functionality like ACARS, which takes a ton of configuration to get working correctly, explicit configuration.

Nasal's networking support is rather limited, and mapped to SimGear APIs - breaking out of the "sandbox" (think circumventing IORules) is not too difficult, but takes quite a bit of expertise/familiarity with SG/FG internals. I think there are only 2-3 people involved here who know how to do this. And some of these bugs were recently fixed by Rebecca Palmer.

In general, Nasal cannot be used to spawn a daemon/server thread - whenever it comes to "servers", those are hard-coded in C++, and need to be explicitly enabled via CLI args - only some subsystems can be dynamically added/removed currently, and most cannot be instantiated independently at all.

Nasal's TCP/IP support is extremely basic, and directly mapped to HTTP code in SimGear (which is also custom and not using libCurl ironically).

I tend to agree with sanhozay's idea on identifying some of the more common use-cases so that custom DSLs can be provided, possibly augmented through C/C++ level extension functions mapped to native SG code.

Philosopher has quite a bit of experience here, not just with Nasal but also with creating DSLs.

Regarding the IRC comment, I don't consider that possible using plain Nasal, you would literally have to re-implement the IRC protocol on top of the generic protocol system and would still have to configure the whole thing explicitly. Nasal itself does not have sufficient privileges/APIs to pull that off
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby japreja » Mon Dec 28, 2015 9:04 pm

Don't replace nasal, just have python as another option for scripting. Python can be implemented in such a way that if the user does not have python installed it will not be used. I would personally prefer python as opposed to using nasal, yet another language I would need to learn along with any of its quarks. I think FlightGear would benefit mainly by attracting the wealth of developers already using python, many of whom are also proficient in c/c++.

As for having problems with external modules, just select a few basic ones and make it known that modules are not supported from the start so there is no confusion.

Truthfully, anyone that can compile FlightGear can just follow the instructions on how to implement python from the Python Documentation and have a working test copy to play with (it's probably a tad more difficult than that but....). It doesn't look like it would take much to implement. I am not a c programmer or I would just make a build and upload a test version. If anyone does this please let me know, I will test what I can :D

I have written some python scripts to read things like bright star data and place models with it, I will gladly release them as GPL code instead of MIT for FlightGear. Also my TIGER scripts which I am willing to expand and enhance.
japreja
 
Posts: 334
Joined: Fri May 08, 2015 12:05 am
Location: MT, USA
OS: Windows 10 Pro 64bit

Re: FGPython an propose for Python as an nasal alternative

Postby curt » Mon Dec 28, 2015 9:10 pm

Historically, when all the FlightGear code and data lived under the same roof and under the care of the same developers, I believe it was fair and valid to assume a cooperative development environment (versus an adversarial environment.) But as more and more content developers create material for FlightGear, naturally the 'data' portion of FlightGear is beginning to spread out over a variety of hosts, services, formats, and less well known authors. Generally this is a positive thing, but it also means we can't naively assume all aircraft or scenery models are developed cooperatively. I'm not aware of any existing adversarial behavior (other than perhaps some childish scene model defacement) but we want to be careful to avoid enabling someone with less than positive motives from advertising an exciting new aircraft with a zillion unbelievable new features ... and then people download it only to find it spawns malicious code and harms their system.

I do like the idea of a stand-alone script interpreter communicating via HLA or some similar IPC, but we still have to consider fragments of script code embedded in models to create or enhance fancy animations. Not all script fragments make sense to run externally.

Best regards,

Curt.
curt
Administrator
 
Posts: 1168
Joined: Thu Jan 01, 1970 1:00 am
Location: Minneapolis, MN

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Mon Dec 28, 2015 9:12 pm

I think FlightGear would benefit mainly by attracting the wealth of developers already using python, many of whom are also proficient in c/c++.


That is one of the more commonly expressed motives for adopting more mainstream ("standard") solutions.
To be fair, you will find that contributors like Stuart, Curt and others have repeatedly stated that the similarity of Nasal to C/C++ helps lower the barrier to entry for those new to C/C++-like languages - basically, someone new to programming, could learn enough about C/C++, and even GLSL (shaders) to "grow into" a core developer over time.

Most people really don't want "Python" (the language/syntax) per se, but "Python, the plethora of 3rd party modules".

I am fairly familiar with FG, and regularly build from source, but I would still be kinda pi**ed downloading an aircraft/feature only to find out that I am missing certain plugins/modules to make use of it, and I am not even talking about different versions (backward compatibility) yet).

Admittedly, Nasal sucks from a maintenance standpoint in comparison to Lua/Python or Ruby - but don't underestimate the power of having the degree of "control" the current scheme offers.

The very instant you start to restrict python modules, people will patch up SG/FG accordingly to support "their" module - at which point, you should not underestimate what OpenCV, numpy etc are pulling in in terms of dependencies - you may end up with a stock fgfs installation that is 500 mb in size, + 2gb of python dependencies under $FG_HOME/.pip (the python package manager)

It doesn't look like it would take much to implement.

agreed, the "naive" SGSubsystem-based approach is detailed at: http://wiki.flightgear.org/Howto:Create_new_subsystems
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby japreja » Mon Dec 28, 2015 9:22 pm

curt wrote in Mon Dec 28, 2015 8:13 pm:Hmmm, www2 brings up a good point that's worth considering before going too far down the python path. If we can easily pull in any and all python library functionality, like web servers (for example), then we have some huge security concerns to factor in. A rogue aircraft could potentially turn your computer into a spam bot, or worse. One of the benefits of nasal is that it is carefully sandboxed for what it can or can't do ... would we have the same level of control over security with full blown python integration? I don't know?


I believe this is why many modern projects implement an API key requirement. If any malicious activity occurs the API key assigned to that developer could be revoked. I think there is also a way to block certain modules from being used by using the python API to embed the server module functionality to be a do nothing function if I recall correctly, I would have to re-read the docs.
japreja
 
Posts: 334
Joined: Fri May 08, 2015 12:05 am
Location: MT, USA
OS: Windows 10 Pro 64bit

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Mon Dec 28, 2015 9:34 pm

See:

http://stackoverflow.com/questions/8618 ... g-language
https://wiki.python.org/moin/SandboxedPython

(google terms: "embedding python sandbox")

The point being, this is obviously possible, but anybody wanting to use certain modules would disable such restrictions quickly (similar to how some addons require a modified IORules file)

Note that Lua is much better at embedded sandbox'ed environment than Python is.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby www2 » Mon Dec 28, 2015 10:17 pm

Hooray
That is true.
But current the stock python 3 around 51MB (base on the windows version of blender 2.76b) and not more than 15MB for numpy
And for security that have a white list for modules and an blacklist for default functions like eval.
Or if we can rout the open function through flight gear and lock python os packet we can close the major security concern

edit i have no problem for waiting for HLA
www2
 
Posts: 319
Joined: Thu Apr 16, 2009 2:58 pm
OS: Ubuntu

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Mon Dec 28, 2015 10:50 pm

I am not at all oppposed to Python, and I would in fact help prototyping a corresponding subsystem (even absent HLA), but I do believe that it would make sense to discuss this first, i.e. on the devel list (or even the issue tracker), especially with regard to dependency treatment and other issues resulting from supporting much more functionality than Nasal comes with "out of the box".

I do believe that using HLA/RTI is technically the "right" approach, especially because of GC/frame rate and frame spacing impact for any code running in the main loop:

Subject: the real cost of the Nasal Garbage Collector
stuart wrote:Regarding Nasal as a scripting language and AW, I'm hopeful that the work I'm doing on HLA will allow us to run Nasal in a separate thread from the FDM and display, so Nasal GC no-longer can impact frame-rates. It would also allow for writing a weather simulation completely external to the FlightGear instance, which could be quite neat.

-Stuart



For the use-cases mentioned (numpy etc), I am also convinced that supporting OpenCL would be more future proof - just like shaders are the right approach to write certain graphics related code, having OpenCL kernels is superior to using Python/numpy code for stuff that may otherwise even run in a distributed fashion using OpenCL.

Equally, OpenCL support would not need to be specific to Nasal/Python, i.e. could even be wrapped/supported by fgcommands or property rules (think autopilot filters).
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby Thorsten » Tue Dec 29, 2015 7:54 am

Suppose I want to compute the area of a disc. The radius is about 10 km, it's on Earth, so it's not really flat, but I can measure the radius only to about 100 meters, give or take. Inserting into A = pi R^2 gives about 314.15926 km^2. Now, what to do next to improve the accuracy?

a) compute pi to a million digits and insert that into the formula?
b) use spherical geometry to compute the correction by Earth's curvature?
c) go out into the field and measure the darn radius to higher accuracy?

In fact, the only thing that actually helps with anything is c).

The point of that being?

Unless you plan on doing extensive matrix operations (at least tens of thousands of multiplications per frame - at which point I'd question your sound judgement in systems design) the execution speed of doing it is a non-issue - it's like computing pi to the next million digits, you can do it, it's theoretically better, but the actual improvement you get that way in the end is so marginal you're hard pressed to even quantify it.

Now, OpenCL runs stuff on the graphics card. I happen to know a few folks who ported fluid dynamics codes to OpenCL because you get a factor ~100 more performance that way. Despite that, they had pretty mixed feelings, because the results they got changed with every update of the graphics card driver. Um... yeah. It's not trivial to run accurate numerical code on floating point precision in the first place, and graphics cards sometimes, you know, do funny things. In rendering, it means the color value of some pixels might come out odd and often we can just shrug it off. In numerics - it means you're screwed.

Also, for many folks who like to have nice visuals, the graphics card is the performance bottleneck. Off-loading yet more tasks to the GPU is the last thing I'd do to improve performance (utilizing the extra CPU cores we currently don't really need makes infinitely more sense to me then).

So if scripting is a major performance hog, off-load it to a different thread. I don't think we have many situation where it is, but that's what HLA is going to do. For the discussion whether to have a second scripting language, performance considerations are very much a red herring. In all situations I have known Nasal code to approach getting performance-critical, the performance was actually spent in C++ routines called by Nasal (geodinfo() being the prime suspect).

I think FlightGear would benefit mainly by attracting the wealth of developers already using python, many of whom are also proficient in c/c++.


You mean people regularly like a certain language and code on all projects which supports 'their' language? Rather than perhaps being attracted to a project and taking the week to learn what's necessary to code for the project?

I mean, FG rendering runs on GLSL shaders which is the only rendering standard when you happen to be under Linux and don't get DirectX - where is the plethora of developers familiar with GLSL being attracted?

It's just not been historically true that this has ever happened - people contribute to FG because they like the project and then learn the tricks of the trade they need.

I'd like to see much less wishful thinking and more case-history and evidence based reasoning. If X didn't work out the last n times, why should it be true now?

Edit: Also, see video tape vs. DVD vs. BlueRay. When the DVD arrived, it wiped video tapes out of the shelves in stores pretty efficiently. Because it offered lots of genuinely new features - different language tracks, chapter selection, sharp still images,... BlueRay is out since a few years, but whenever I go into the video section of a store, I mainly see DVDs on sale and perhaps 20% of the space is taken by BlueRay. Arguably it does a few things better - but it's not the same eye opener as the advent of the DVD was. Largely it's just confusing, has a stupid DRM scheme and is a more expensive and involved way to get the same movie experience (from a normal viewing angle - you do see differences from 20 cm distance from the screen).

Nasal coming to FG was an enabler - suddenly all sorts of scripting was possible for people. Another scripting language - even if it is genuinely better - is just being the BlueRay to the DVD - more of the same essential experience, but adding confusion.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: FGPython an propose for Python as an nasal alternative

Postby jaxsin » Tue Dec 29, 2015 12:18 pm

Hooray wrote in Mon Dec 28, 2015 9:12 pm:

Most people really don't want "Python" (the language/syntax) per se, but "Python, the plethora of 3rd party modules".


Couldn't agree more on this front. For all the plethora of 3rd party modules for python, it's core syntax is just... bleeping hell. thank god for tabstop/retab

This sort of interest me in the sense that eventually I would love to become more involved in FG and move into more of the programming side once I understand the process. Python does not excite me, but I lack the knowledge of the POV to the original post
jaxsin
 
Posts: 395
Joined: Mon Dec 28, 2015 4:54 pm

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Tue Dec 29, 2015 12:53 pm

Now, OpenCL runs stuff on the graphics card.
[...]
Off-loading yet more tasks to the GPU is the last thing I'd do to improve performance.


That's an incomplete misconception: OpenCL is about "kernels" (like your GLSL kernels), that can be executed on CPUs, just as well as on GPUs, but also on dedicated FPGA hardware.

Like you rightly said, performance of OpenCL is directly affected by the quality of the driver - in fact, with nvidia, OpenCL gets translated to CUDA behind the scenes, and OpenCL is not a priority for nvidia obviously (competing technology stack).

(utilizing the extra CPU cores we currently don't really need makes infinitely more sense to me then)


right, the point of this being that algorithms implemented in OpenCL can also be executed on an idle CPU core, so don't need to utilize GPU hardware at all.
You can get massive concurrency using OpenCL kernels, even beyond the typical "embarrasingly-parallel" stuff that lends itself to be executed on GPU hardware.

Thus, this is overlapping with your point on multi-threading, with the exception that OpenCL code is explicitly designed to be executed concurrently, so that you only need to worry about the "glue" (interfacing) code.

Otherwise, I do agree that OSG itself does provide some fairly advanced support for parallelism at the rendering level - however, most of this is not currently used by FG, and some of it could be better exposed to SGSubsystems (C++ code), but also to "fgdata subsystems" (think property rules and/or Nasal).

In fact ,it would probably make sense to allow property rules to be registered as fgcommands, and map that to osg visitors to delegate certain things to non-Nasal, native, C++ code instead of using timers/listeners for animating/updating stuff.

I do agree that "more scripting" would bring tons of challenges/complexity with it, but I also agree that we would see "more" contributors (those familiar with Python) using Python to implement features - I just doubt that the we'll be able appreciate the outcome given the resulting management overhead (read:chaos).

To be fair, here's some X-Plane related background info detailing why they decided NOT to use OpenCL a few years ago (~4), but also explaining why offloading certain tasks to the GPU /might/ still make sense (see the comments): http://developer.x-plane.com/2011/05/why-not-gpgpu/


In all situations I have known Nasal code to approach getting performance-critical, the performance was actually spent in C++ routines called by Nasal (geodinfo() being the prime suspect).

That is also generally true (absent some naive code/algorithms), and also even applies in the case of the garbage collector, which is written in C. And geodinfo() demonstrates that algorithmic improvements at the lower level have the most siginificant impact anyway
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby abassign » Tue Dec 29, 2015 9:38 pm

The HLA/RTI approach does not seem the best way to use the python in FGFS. The reason is due to the need to establish a two-way connection that loads the CPU significantly. I have the impression that the HLA/RTI connections using a single task and then become a real bottleneck.
I understand that many people regret even the mere idea of creating na alternative to NASAL, but I need a more radical approach that enables Python to have the same level of activity which currently runs with NASAL.

There are various reasons that make me absolutely certain of the correctness of what I say:

1. NASAL is a dead language like Latin and greek, not followed in any other context than in FGFS and this makes them difficult to understand, but even more difficult to maintain by the very few developers that have a more technical knowledge. The use of a language done for a certain program is a practice in sharp drop in the world of professional SW. It is now considered a nonsense and a practice to be avoided. Currently interpreted languages to be used in complex applications are relatively few in environment GPL (Python, Java, Ruby and a few others). Then there are the languages for the Web (PHP. Java Script, Perl, HTML5, XML, etc ...), but those are for completely different needs.
2. The choice of Python (although I do not really like this language, because I find it not easy to be self-documenting) is certainly "natural", as it is a known language, with an excellent system debugging and code development compact and with excellent code interpreters that have excellent technical to pre-compile the code.
3. Python, unlike other scripting languages, has an almost endless amount of libraries, that help solve problems in many fields of technology. The libraries allow to develop very quickly, otherwise achievable solutions in short times and with a planning effort relatively contained.
4. Python enables true multitasking with the possibility of spreading processes on multiple CPUs, this finally makes it possible to overcome one of the major bottlenecks of FGFS!

Use Python for Fgfs replacing NASAL not think it's a simple task, because it requires to resolve several steps.

NASAL is a language inserted directly into the main code, and if I remember correctly, his interpreter is run separately for each program NASAL active. It seems to me that NASAL is a synchronous process linked to the execution of the main program. It is also true there are triggers, but these are activated by the same code that performs NASAL, and then in fact the system behaves in a synchronous manner. The synchronicity of NASAL is a property that generates the annoying consequence of self-limit the frame-rate in the case of a widespread use (This is a typical problem of synchronous processes that depend on non-synchronous events, such as pressing of a button or hover over an active area of the screen).
I try to explain, let's say you have a GPU high performance that generates a complete image in 20 ms, we make the assumption that there are many processes that consume a total NASAL 10 ms, at the end of the frame-rate will be 1000 / (20 + 10) = 33 fps. But it is necessary to read the status of a process every 1/33 of a second? Or it is sufficient, for example, do it every 1/10 of a second? If the process was asynchronous I can "wake up" the procedure every 1/10 of a second, and then the frame-rate will increase to 43 fps! Of course that is only theory, but I feel that's what happens very often in the process of FGFs.

Python will instead be run asynchronously, because the interpreter is outside the main process of FGFS (This is an advantage because the processes Python can take advantage of a different CPU), but also a disadvantage, because it complicates things a bit. I think the solution is to create a Python object that makes an interface that allows to effectively handle the asynchronous access.

There are other problems posed by Python, such as library management. Python libraries are managed in an unstructured way and then the Python code must declare a coherent path that must be defined "in advance". If Python does not find the required library it is immediately throws an error that stops the process.
I was thinking it might be useful to make a preprocessor that can change some things before run the code, obviously this preprocessor should be performed only once and allow compliance with a set of rules that allow Python environment can be hosted in FGFS .

I thought that it is important NASAL can be executed with Python and the preprocessor will then decide whether the job it will be executed by Python or NASAL. This way everyone can choose the language they prefer, and there will not be problems with the aircraft models earlier.

I'd like someone who has more knowledge of the main code of FGFs can give me a hand to try to build a prototype. I think the best thing is to try to see if the work can be done with a genuine benefit.
Developer of the program https://wiki.flightgear.org/Julia_photoscenery_generator
FDM developer of the G91R1B aircraft https://wiki.flightgear.org/FIAT_G91R1B
JSBSim collaborator
abassign
 
Posts: 947
Joined: Mon Feb 27, 2012 6:09 pm
Location: Italy (living 5 Km from airport LIME)
Callsign: I-BASSY
Version: 2020.4
OS: Ubuntu 20.10

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Tue Dec 29, 2015 10:00 pm

  • Nobody has any doubts that Python would quickly be adopted by people, and probably much faster than Nasal - that is going to be a no-brainer
  • the point was that certain challenges will come with adopting Python
  • Quite a number of remaining core developers are opposed to Nasal, so there is no resistance to provide alternatives - HLA will greatly simplify doing so
  • Nasal, the language, is not difficult to understand - it is close enough to JavaScript, C and C++ to be a no-brainer
  • The FG specific APIs are going to be identical anyway, regardless of the language
  • As has been pointed out, this is not about the language (syntax) but about the plethora of 3rd party modules supported by a vibrant community like Python
  • adding/supporting those is going to cause interesting challenges
  • Nasal the interpreter is plain ANSI C (well, mostly), using linked lists, a hash table and is otherwise also pretty straightforward
  • like Curt mentioned, today, Python would be an obvious choice (Lua and even better one...), but FlightGear did not start today, it is a project with a history and a huge number of working features - just like adopting OSG, GLSL, boost, OpenCL, ShivaVG or HLA would not have been possible 15+ years ago - so FG comes with certain baggage/legacy code
  • it is exactly the huge number of available Python libraries that makes Python adoption problematic, unless it happens externally (e.g. via IPC/RPC, like HLA) - anybody contemplating to use a conventional SGSubsystem for doing so, will be in for a huge surprise - because Python's very popularity is only going to magnify known FG related issues, such as its single-threaded main loop, and the impact on frame rate/spacing that has.
  • I suggest that you review the HLA articles/resources that you can find on the wiki to learn more about it, even regardless of the FlightGear context
  • and while you are at it, you may want to review the whole thread - to ensure that we're on the same page

The use of a language done for a certain program is a practice in sharp drop in the world of professional SW.

you are mistaken, and wikipedia can prove it: https://en.wikipedia.org/wiki/Domain-specific_language

Python enables true multitasking with the possibility of spreading processes on multiple CPUs, this finally makes it possible to overcome one of the major bottlenecks of FGFS!


Sorry, you have no clue what you are talking about: please look up Nasal implementation, and then google the terms "Python GIL" (=global intepreter lock), and then refer to Andy's comments in the devel list archives.


Use Python for Fgfs replacing NASAL not think it's a simple task, because it requires to resolve several steps.


spend 2 weeks reading, and please come back making a more informed suggestion, ideally taking into account what has been previously said in this context.

Sorry that I won't spend much time responding to the remaining part of your posting, but you are -currently- just ill-informed, so you are better off doing a little homework first, or phrasing your suggestions as questions.

I'd like someone who has more knowledge of the main code of FGFs can give me a hand to try to build a prototype. I think the best thing is to try to see if the work can be done with a genuine benefit.


first of all, you should read up on FlightGear internals, you could start by reviewing the links posted in this thread, and reading about FlightGear's main loop, how subsystems are structured, how Nasal is integrated, and how the frame rate/frame spacing are affected by the main loop, and how HLA is supposed to help with that.

(that is, assuming you want to be taken seriously)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

PreviousNext

Return to New features

Who is online

Users browsing this forum: No registered users and 1 guest