Wow, there's really lots to catch up with here, you have again been highly productive!
And it seems your code is now also available in CVS - congratulations! (also to WooT!!)
Thorsten wrote:To be honest, it has left the AI subsystem a while ago
- most of the layer placement and environment management is now Nasal (the current count is 1500 lines of code...), and the CPU demanding low level cloud transformations run as shader effects. Still, all the cloud models could still be used in standard AI scenario definitions...
Using shaders to handle the cloud transformations was indeed a nice and clever move - this possibility should have occurred to us much earlier!!
If it's in Nasal now it would probably be better to move it to plain C++ somewhere in the future. That aside, it's pretty impressive.
While I do of course agree that certain things would probably be better done in C++ space, I also think that it's important to keep in mind how all this became possible in the first place: It is due to the flexibility of the Nasal and AI systems that all this could be done to this extent without touching any C++ at all.
So, it would be a pity to directly move everything into C++ space, because it is this very flexibility that also makes everything so configurable now. Porting everything to C++ space would also mean that we would have to sacrifice some of the flexibility and customizability that is provided by the use of Nasal/XML, so that changes/updates to this system would then need to be committed to CVS and FlightGear would need to be rebuilt in order to make changes take effect.
That would probably slow down development quite a bit, especially given that code is not easily discussed, reviewed and committed/maintained in FlightGear CVS on a regular basis.
That is something that most users and contributors here have noticed, which is probably also the reason for websites such as unitedfreeworld and others that provide a place for people to make their contributions available without necessarily requiring access to CVS.
Having on the other hand a system that does not necessarily depend on being maintained in CVS is a good thing because it also means less work for the developers.
Personally, I think it would be better to look into which parts of the script would benefit most from being run in C++ space, and then see how these could be added there while retaining the full flexibility of the Nasal based approach for the time being.
Several ideas to do this were already suggested a couple of weeks ago.
In the beginning, having dedicated Nasal helper functions that run in C++ space would be the most obvious way of doing this, without sacrificing any flexibility.
Porting the features over to C++ should definitely remain an important consideration, but not a top priority during the prototyping phase. I mentioned this already, but I think splitting the script into separate modules might make sense. That would help organize everything and provide a way to group functions logically.
For example, some stuff should probably be best run in C++ space, other code could be run in a dedicated Nasal worker thread, while yet other code is simply "library" code with helper routines that does not need to be updated as often as some of the other modules, and which could thus be considered "stable".
To be honest, I have already looked into the latest script, and even started re-organizing some chunks of code - however I didn't want to interfere with any of the latest developments (which would actually be another argument in support of modularizing the script, so that people could agree to work only on certain parts/components).
I will provide instructions and examples how it is done - essentially, it boils down to writing a few Nasal calls of high-level cloud placement routies (If anyone is a GUI wizard, maybe we can even get a GUI for that, but I simply don't see how to do it...).
If you can provide some details about the workflow that you envision, we might be able to come up with a working prototype for a GUI.
By the way - if anyone knows how to make a menu option appear from Nasal, please let me know - I just tried directly setting the properties in /sim/menubar/default/ , but that did not seem to create anything, visible, although I can see the properties in the tree...
I just looked into this, it seems the menubar properties are not using listeners but are instead read in once during initialization, all of this is to be found in
menubar.cxx ( see FGMenuBar::make_menubar(void) ).
Which means that even if your properties are 100% valid, they will not be used because the corresponding code is only executed once during initialization (before you set these properties). Have you tried running reinit for the GUI (similar to your environment workaround)?
If that does not work, there are basically two ways to proceed here: 1) use listeners to allow menubar entries to be changed dynamically, or 2) use a new dedicated fgcommand to re-build the menubar on demand (seems much easier to implement to me).
As a crude workaround, you could just try to add the corresponding properties to the property tree BEFORE the GUI is built, that way your menu items will be present without needing a reinit.
The freeze for a certain period of time is (sort of) normal - the setup calls are all done in a single frame - which may take forever to execute. In tests, I had times ~minute freeze to set up 500 barrier clouds in a tricky situation. I'll soonish work on moving these calls to worker threads outside the simulation frames, I believe this will solve the freeze problem.
When using threads, keep in mind what
I wrote a couple of weeks ago about NOT using any of the FlightGear APIs in a Nasal worker thread, you will not want to access any of the functions that modify state which is maintained in the fgfs main thread, because this requires explicit synchronization/locking or fgfs may simply segfault/crash due to race conditions (i.e. multiple threads invalidating their state without sync).
While making these APIs thread safe is not really complicated, it is simply not yet done so far - so that only those APIs that do not modify state are implicitly thread safe. This would probably be another requirement to keep in mind for the C++ changes required.
When I was playing with threads in FlightGear/Nasal a year or two ago, I had to recompile the binary to add some locking because we were looking into the update rates that can be realistically achieved with threads in Nasal.
Guys, my main problem right now is that I hit a definite performance ceiling with layered clouds - I need to be able to load 4 tiles (to guarantee a decent visibility of clouds) and be above 20 fps. My problem at this point is not that I would run out of ideas how to improve a single cloud
So the performance problem is not due to use of Nasal now, but instead due to the amount of clouds that you have to render (i.e. the system is the bottleneck and not the approach)?
Can you load and display 4 weather tiles without updating them (i.e. just shown statically)?
If it is just due to the sheer volume/amount of clouds that you have to render, then we are back at looking into the data structures and algorithms that are used to determine which clouds are to be drawn. I assume currently, you simply update all objects in a loop?
I mentioned this already some time ago, but normally you'd do certain optimizations to prioritize the drawing of certain objects or even disable drawing altogether if they are simply not visible but this would normally require use of corresponding data structures (like octrees:
http://en.wikipedia.org/wiki/Octree ) in the first place (e.g.
occlusion culling for hidden surface removal:
viewtopic.php?f=5&t=7358&p=70903#p70903 ).
This would basically mean to use spatial data structures in order to determine which objects are to be drawn (lots of fancy 3d maths), instead of drawing everything regardless of its visibility status.
So that is a very real option to optimize the performance of the whole thing. On the other hand, I am not sure if one should really look into doing such fancy things in Nasal space. While it is certainly possible, all the infrastructure for this is already provided by FlightGear/SimGear or OpenSceneGraph and it would seem sort of redundant to start coding spatial data structures in Nasal.
From our point of view, it would be far better (and easier) to simply tell FlightGear "update all objects" and it would internally decide what to update in C++ space.
An easier thing to do would be to start looking into using LOD animations for cloud objects, so that distant weather objects (clouds) use lower levels of detail, or where multiple separate objects are replaced by one simple object (i.e. cloudlets far away simply become fog/particles).