Board index FlightGear Development New features

FGPython an propose for Python as an nasal alternative

Discussion and requests for new features. Please note that FlightGear developers are volunteers and may or may not be able to consider these requests.

Re: FGPython an propose for Python as an nasal alternative

Postby bugman » Thu Dec 31, 2015 2:14 pm

Thorsten wrote in Thu Dec 31, 2015 1:42 pm:Two questions, Edward? Is it really that much better for prototyping than Nasal (and why)? The same argument has been made frequently for Nasal.


First question, yes Edward is my name ;) Second, Python is an object oriented language (OOP). This is simply a closer match to C++. The Python design is better for C++ prototyping, and is very often used as such.

Also, suppose I had written AW in Python and had used a large number of modules. How optional is a subsystem really once people need it? De-facto, it'd probably require a lot of people to install all my libs.


Are you worried that the feature might become too awesome, just like AW ;)

It reminds me on the wording of making Qt5 an optional dependency - de-facto it's not really as there's no alternative provided and very few people will write their own or have a two-monitor setup and use Phi.


If it does become such an awesome addition, I would guess the core developers would shift it from optional to builtin.

So yeah, suppose that goes through - I will be able to run a gutted FG without GUI and without the need to install a random collection of Python libs - but won't I de-facto be forced to have both if I really want to use it?


Windows and Mac DMG users would have it built in (i.e. no libs). Linux users building themselves would just need the python-devel package installed, and then compile it in. A new lib would be accepted in the FG core, from how I understand things, if it brings in large value.

Regards,
Edward
bugman
Moderator
 
Posts: 1808
Joined: Thu Mar 19, 2015 10:01 am
Version: next

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 2:35 pm

japreja wrote in Thu Dec 31, 2015 5:20 am:Rather than FlightGear embedding another language, FlightGear would probably benefit much better with a type of loadable module support. Nasal could become one of those modules along with Python and maybe Perl. Much in the same manner that Apache and Apache Modules work. I think that would greatly broaden the horizon.

Scripting would be just one type of module, the GUI could be another type which might make it easier to support QT or even Java, not to mention that the FDM code could become a separate module, one for each supported FDM. So in just discussing Python support we may be limiting ourselves and/or FlightGear .

Should we be discussing Module support instead?
Would that greatly reduce the core FlightGear code?
Has module support been discussed before?


That's not a new idea, and that's exactly where HLA shines.

"Binary plugins" (in the form of DSOs/DLLs) were brought up a number of times - the general consensus was that 1) things would inevitably become platform-specific again, and 2) they won't easily scale across different computers - thus, scripting is considered more superior than having native code "plugins" (or modules), for some background, see:

http://wiki.flightgear.org/Nasal_for_C% ... on_plugins
http://wiki.flightgear.org/FlightGear_Plugins


However, the whole idea behind distributed setups using IPC mechanisms like CORBA or HLA is to address such issues, i.e. you can basically implement arbitrary modules in a standalone fashion, that really only communicate with other parts of the simulation using a message bus framework (e.g. RTI): http://wiki.flightgear.org/High-Level_Architecture

In other words, people are already working towards this, not just with a focus on scripting, but also on splitting out other non-rendering systems, so that the viewer can be run separately: http://wiki.flightgear.org/FGViewer
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby Thorsten » Thu Dec 31, 2015 3:38 pm

Are you worried that the feature might become too awesome, just like AW


No - I'm worried that we end up with a situation in which developers on the latest Linux distros happily write code utilizing the newest of the newest and people on systems 2-3 years behind (like myself for reasons elaborated elsewhere) have to acquire random stuff to make a particular aircraft / feature work despite being able to compile and having a running FG.

Basically I want FG to work out of the box once it's compiled, and talk about a huge collection of libraries to be utilized worries me.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 5:12 pm

If it does become such an awesome addition, I would guess the core developers would shift it from optional to builtin.

That's funnily in line with James' comments on his Qt5 work being "strictly optional", despite him expecting it be shipped with all officials builds and used "by default".

Thorsten wrote in Thu Dec 31, 2015 3:38 pm:Basically I want FG to work out of the box once it's compiled, and talk about a huge collection of libraries to be utilized worries me.



I think that's a valid concern, and a point previously made by other contributors, i.e. see Melchior's comments quoted a few days ago.
HLA and Python support are definitely going to to be huge technology enablers, but there is the very real possibility for FlightGear to become increasingly "distributed" (as in "disparate") in areas where FlightGear the project may not yet be ready for it.

We started seeing that in the way increasing Nasal adoption has helped contributors use/develop FlightGear in areas that were neither foreseen, nor necessarily endorsed or even supported by core developers or the project as a whole.

Developments like the package manager/catalog system and the fgdata/fgaddon migration/split make it increasingly feasible to split up the project into entities over which "we" no longer have much control, including the possibility to see interesting, desirable, features developed in a "dead-end" fashion that isn't really likely to make it back into the project ever.

Personally, I don't have any doubts that Python support would be great, and that we would be in much better shape with Python support had it been added instead of Nasal, but these days, we simply have to live with certain facts, and porting/rewriting tons of legacy Nasal code is simply not feasible anytime soon (not even if done in an automated fashion using a Nasal to Python converted), which only leaves us with providing support for additional scripting languages, at which point the most likely thing to happen is that desirable features/functionality may only be available in language X, and never be made accessible in language Y.

We are already seeing tons of useful SG/FG APIs that are not exposed to Nasal space, and a future where aircraft/addons may end up using a handful of different scripting languages like Nasal, Python and Lua, there would be quite a bit of potential chaos coming out of all this. The only viable path to deal with that scenario is an IPC stack like CORBA or HLA/RTI that can deal with these challenges.

Compared to Python or Lua, Nasal sucks for a a number of reasons - but let's not underestimate the degree of stability and longevity Nasal provides for the same reasons that make it not very compelling for people wanting to use Python or Lua.

I don't think that it would take a whole week to prototype Python support in the form of a dedicated SGSubsystem that runs inside the main loop, I could probably come up with such code in a single weekend of spare time hacking, making the property tree and other APIs functional is obviously taking more time - but apart from that, the steps are straightforward actually, the links I posted should be able to get people going quickly, i.e. those seriously interested in Python support.

In all fairness, I am not prioritizing anything related to this myself, but a number of core developers have previously expressed a strong desire to add Python support, i.e. beyond Curt's encouragement, there are people who really would like to provide a Python alternative, and potentially even a replacement.

I don't agree with all the reasons behind that thinking, but I am willing to help people come up with the boilerplate/integration code to end up with a Python interpreter running within the main loop, it's not rocket science after all - and having options is generally a good thing for the project. Over time, this may at least have the benefit that scripting in FG stops being the chaotic mess that it has become over time.
Last edited by Hooray on Thu Dec 31, 2015 5:17 pm, edited 1 time in total.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby www2 » Thu Dec 31, 2015 5:14 pm

First i work on an python interface prototype that use Telnet/prop protocol for access the prop tree.
Second we can what blender do using a custom build python and add to download and build script.
This make the build proses little longer.
Be site that we can set a requirement that the python code must work on e.g the 3.4.* branch of python and no Deprecation Warning's.

edit:
I see nasal use for small embedded scrips that use for e.g. landing lights for kai tak or LOWI where python shine in cockpit development, aircraft systems, interface to external service for e.g. weather or i wand farmin add a connection to a external media player using websockets or http post request.
Last edited by www2 on Thu Dec 31, 2015 5:41 pm, edited 2 times in total.
www2
 
Posts: 319
Joined: Thu Apr 16, 2009 2:58 pm
OS: Ubuntu

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 5:19 pm

you could ship python as part of the build tree, i.e. 3rd party libs - but most people will probably want to use what is commonly provided by their distro, no matter if that means 3.0 or even 2.6 ... building python from source should usually not be necessary, it being much more widespread than even OSG or OpenGL devel packages on most systems

https://wiki.python.org/moin/Python2orPython3
http://www.i-programmer.info/news/98-la ... sited.html

BTW: there already is a python module to access the property tree ...
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby www2 » Thu Dec 31, 2015 5:40 pm

@Hooray i know and i use ubuntu 14.04 that have python 2.7 and python 3.4 on board.
I say that we need to use python 3.*, the end of life of python 2 is in 2020.
The download_and_compile.sh is most use for development beside this we can do python compiling as an option if the user have an newer version of python.
www2
 
Posts: 319
Joined: Thu Apr 16, 2009 2:58 pm
OS: Ubuntu

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 5:45 pm

If you are able to build SG/FG from source and know C++, I am willing to help you set up a sgsubsystem stub where you can integrate a Python interpreter and tell you how to proceed to hook this up to different simulator features like GUI dialogs, bindings, scenery models etc. - and in fact, I may turn the whole thing into a tutorial for the wiki while we're at it.

Note that the "official" recommendation is to settle for a safe subset of Python2/Python3: https://docs.python.org/2/howto/pyporting.html

My suggestion would be that this would probably be the best compromise for Python in FlightGear, this subset would then be documented on the wiki, and augmented as necessary.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby www2 » Thu Dec 31, 2015 6:30 pm

Hooray
My opinion is that we go python 3 than python 2
I can build from source using download_and_compile.sh only i have lack of knowledge about C++.
Beside this i have no problem to helping porting most important nasal library's to python (canvas,prop,geo).
www2
 
Posts: 319
Joined: Thu Apr 16, 2009 2:58 pm
OS: Ubuntu

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 6:34 pm

www2 wrote in Thu Dec 31, 2015 6:30 pm:Beside this i have no problem to helping porting most important nasal library's to python (canvas,prop,geo).


geo.nas, props.nas etc pre-date the Canvas system, which introduced the cppbind system - i.e. a boost/bind-based system expose native C++ code to Nasal directly (and vice versa).
Thus, when it comes to Python, it would make more sense to use this approach directly, e.g. using SIP: https://www.riverbankcomputing.com/software/sip/intro

That way, there's no unnecessary scripting space overhead, i.e. you could directly map an existing C++ class (e.g. SGGeod) to a Python class and make that available in a corresponding namespace like "geo"
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby www2 » Thu Dec 31, 2015 7:08 pm

Oke i have view props.nas and big part is addition to the library.
www2
 
Posts: 319
Joined: Thu Apr 16, 2009 2:58 pm
OS: Ubuntu

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 7:42 pm

Like I said, I may be willing to come up with the code snippets and patches to help you guys prototype the whole thing, i.e. run a working python interpreter in the main loop, but we will need a list of people interested in this, as well as a list of their skills/background to tell who can handle certain things.

Getting Python itself integrated is a piece of cake, making it fully functional (parity with APIs available to Nasal) is another story, and I am not going to handle that - I am only interested in some parts of this, so I am willing to help you come up with a prototype of this, i.e. a Python interpreter linked into fg as a build time option able to run Python code and call a few FG APIs (think fgcommand).

Apart from that, you will need someone able to apply the pointers given so far, which is more about C++ code and understanding the way Nasal is embedded, than having to know Python ...
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby curt » Thu Dec 31, 2015 9:13 pm

Out of mild curiosity, I worked my way through some of the examples in the official python/C integration documentation here:

https://docs.python.org/3.5/extending/embedding.html

The examples pretty much just work. From there I've experimented just a little bit more and learned a tiny bit.

At least with the examples, there is a single universal python interpreter and it's state is carried along, so independent little code fragments can read and write to the same variable space and see each other's changes. I figured out how to run python code fragments from C++ (which is the first trivial example at the above link.) I figured out how to import a small python module from within my C code, and reference it's variables and call it's functions. So now I can read/write a variable from a python module using native python code, I can read/write the same variable with python fragments, and finally I can get a PyObject * pointer to the module/variable to manipulate it directly from within C (without calling python code fragments or calling python module functions.)

Interesting stuff, fun to see it actually working.

The more interesting challenge would be to figure out an efficient way with minimal overhead to allow native python access to the FlightGear property tree. It's all very doable ... but maybe trickier to do with minimal overhead.
curt
Administrator
 
Posts: 1168
Joined: Thu Jan 01, 1970 1:00 am
Location: Minneapolis, MN

Re: FGPython an propose for Python as an nasal alternative

Postby Hooray » Thu Dec 31, 2015 9:29 pm

see f_setprop()/f_getprop() in $FG_SRC/Scripting/NasalSys.cxx - that's the most optimized getters/setters we have, everything else is using either props.nas or cppbind to wrap SGPropertyNode* pointers - with props.nas having the largest overhead, because it's going through Nasal space.

For Python, SIP or SWIG would be the easiest option to hook the SGPropertyNode class up to the python environment, at that point you only need to pass a pointer to the root node, which is part of the globals class - i.e. there's some getProps() helper somewhere in there.

Which sums up the way, the existing property interface for Nasal is working, too.

So the code is there, and it's not specific to Nasal, the same APIs would work for Python - it's just the marshaling/typing that would obviously have to use Python, which is why I suggested to use Boost::Python, but even without it, it's not very difficult.

It's obviously a different thing if you want to run Python outside the main loop, because then you really need to use some of IPC, even if it's just the existing props/telnet interface
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: FGPython an propose for Python as an nasal alternative

Postby curt » Thu Dec 31, 2015 9:31 pm

Replying to myself ... I don't want to scare anyone ... I am more interested in interfacing the property system and python for my personal open-source uav autopilot project. For that I am the sole author and control all the parts exclusively, I am also the only customer and user. :-) So I probably will do far more radical changes for my own project. I am thinking that the python system would host the property tree data, and the imported python modules, or code fragments called from C, could reference the property tree elements directly by name, maybe like. (python) alt_meters = props.sensors.gps.altitude-m (I'm thinking as I type here.) On the C side, SGPropertyNode * would be replaced by PyObject * (or a thin wrapper around PyObject *) to access values in the property tree that is now owned and maintained by the python interpreter. For FlightGear we have to be a whole lot more careful and conservative because it is orders of magnitude more complex than my hobby autopilot project.
curt
Administrator
 
Posts: 1168
Joined: Thu Jan 01, 1970 1:00 am
Location: Minneapolis, MN

PreviousNext

Return to New features

Who is online

Users browsing this forum: No registered users and 6 guests