Board index FlightGear Development Weather

A local weather system (v1.4 available)

Everything related to weather simulation, visuals should be discussed in the shader subforum.

A local weather system (v1.4 available)

Postby Thorsten » Tue Mar 16, 2010 9:33 am

While I was originally interested in finding interesting ways to model different cloud types, I came to realize that the arrangement of clouds is as much part of a realistic visual experience of the sky than the individual cloud - so the project evolved into modelling local weather phenomena. I have outlined the project in the Wiki if anyone is interested in details, and the work presented as the Cirrus Sky and as the Thunderstorm will end up as part of the local weather package.

In short, I'm trying to create an alternative weather system (that for the moment bypasses the standard Flightgear weather system) in which weather phenomena are tied to a specific location and connected with each other in a specific way. This for example means that clouds drift with the wind, or that particular cloud types herald a thunderstorm and so on.

What is meant by this in practice is perhaps most easily illustrated best pictorially:

Here is morning Cumulus development over TNCM created by my convective cloud system algorithm. The landmass is a better convective seed than the ocean, so more and stronger convenction is observed over St. Marteens than over the sea. The standard Flightgear weather system creates a cloud layer without such a connection with terrain.

Image

Later in the afternoon, the convection is significantly stronger, and Cumulus development is much increased. Eventually, that may develop into thunderstorms. Currently, the convection algorithm isn't dynamical yet, so the pics represent the algorithm being run at two different times, not a continuous transition - eventually it's supposed to be dynamical though.

Image

Here is a different example created from the barrier algorithm - condensation clouds when air is moved over a barrier (near Clermont-Ferrand) - (I'd call them 'Stauwolken' in German, but I've no idea what the proper English term is). The clouds appear specifically in the windward side of the ridge wherever the airstream is forced above the condensation altitude.

Image

At present, these are more or less teasers - the 'interface' for these (and other) algorithms is rather ugly (you have to edit the Nasal code...) - but I'm working to make it more accessible, at which point there will be another experimental release.

Since it is quite an ambitious project, I'm looking for help with various tasks if anyone is interested:

* creation of more cloud textures - if anyone is good with an image manipulation program, the more distinct cloud textures of various types the better

* development of more and novel cloud models - while I have viable solutions for some clouds, a decent solution for Stratus and other thick layered clouds eludes me at present - if anyone has a good idea, please let me know. In particular, if a rotating cloud model can be replaced by a viable static solution, that's a big thing in terms of numbers of clouds which can be placed without loss of framerate. I can have 1000 distinct static cloud models without a fuss, but if I have to transform the models, 200 is already a stretch.

* algorithm testing and tuning - cloud placement algorithms don't create realistic cloud patterns out of the box, but need to be called with a suitable parameter range to be determined by try and error - which is fun, but takes a lot of time... Also, I guess some things could be solved in a more elegant way in Nasal - I just learned it a few weeks ago, and probably don't code the best solutions.

* research in weather patterns - I know quite well what the typical weather patterns in North America/Europe are, but I'm less sure about tropical weather, so any help what patterns to build would be appreciated

* shaders - I have the feeling fog and haze are much better handled by shaders than by textures - unfortunately I know next to nothing about that - so if anyone who knows can spare the time for a few long mails for a tutorial and a question/answer session, that would be very useful - or if anyone just takes care of these challenges...

* C++ implementation - all features I want to write run with Nasal as far as I can forsee, although I may need a few dirty workarounds. I guess it's too early to really start hard-coding things at this point, but eventually it should be done for thing which need to run fast, and I guess it would be very useful to think already now what structures should be eventually there and where the C++ Flightgear code is going. Again, if anyone can spare the time for a few longer emails and questions and answers, that would be of some help.

* METAR/Multiplayer interfaces - there were some discussions about METAR in other threads already, some ideas about Multiplayer weather - I haven't really thought too much about either, but it would be good to continue the discussions and settle on a viable scheme to merge local weather with METAR or allow to share it over MP

If anyone wants to join in, please let me know what you're particularly interested in by PN or here, and I'll try to coordinate everything and provide a suitable infrastructure for discussions (the Wiki outline, to-do and done lists...).


Edit:

Package download v1.4 here.

The package needs to be unpacked in $FGROOT/ and writes content into Nasal/local_weather, Models/Weather/, Environment/ and gui/dialogs/.

Important note: The package is designed for a current GIT binary.

The documentation is also available online as README.

Just for comparison with the pics above - this is where we are now:

Image

Image
Last edited by Thorsten on Tue Jan 03, 2012 12:24 pm, edited 30 times in total.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: A local weather system

Postby Thorsten » Wed Mar 17, 2010 12:28 pm

To add a few more teasers while hoping for some more resonance... - here is a first attempt at modelling an incoming coldfront - with Altocumulus leading the front, then followed by the main Stratocumulus layers and occasional Thunderheads coming up.

Image

Some stunning view from just above the Stratocumulus layer...

Image

The main problem here are shadows - obviously the shading is wrong, the clouds would cast dramatic shadows onto the sea and onto each other at this angle with the sun - but some things are rather difficult to control I guess, modelling self- and mutual shadowing among 500 clouds possibly is one of those.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: A local weather system

Postby Hooray » Wed Mar 17, 2010 1:55 pm

Hi, you have really been extremely productive in a fairly short amount of time :-)
These are indeed very impressive pictures and it looks like you are making lots of promising progress with your project!

I think most people here agree that all this is very impressive, but probably very few people feel really qualified to respond to your requests, after all this is not the developers mailing list.

But you have now progressed so far, that your questions are ideally asked and discussed with fellow developers (you are now not just a user anymore ;-)). These forums are unfortunately not the ideal place for this. You might want to try the IRC channel in order to talk to some developers or directly subscribe to the developers mailing list.

At present, these are more or less teasers - the 'interface' for these (and other) algorithms is rather ugly (you have to edit the Nasal code...) - but I'm working to make it more accessible, at which point there will be another experimental release.


In FlightGear/Nasal, you can register so called "listeners" using a setlistener() call, so that Nasal routines (functions) can be called once a given property is modified (written to).
These properties can be simply made up by you. So, instead of hardcoding something like "var range_nm=100;", you could also have this value updated dynamically by invoking a callback whenever a property is written to, this could then look like this:

Code: Select all
var range_nm=100; # default value
var range_property="/local-weather-system/range-nm";
var update_range = func() {
 range_nm=getprop(range_property);
 print("range value updated:",range_nm);
};
setlistener(range_property, update_range);


Of course this way of doing it, isn't all that scalable when it comes to tracking a whole number of configuration properties, this is when you will probably want to wrap up all related variables inside some form of helper object (hash) and use a helper method. You can also directly use cmdarg() to get a handle to the modified property.

Code: Select all
# use a hash to store related variables
var configuration = {}; # create a hash
configuration.range_nm = 100;
configuration.wind_dir_deg=320;
configuration.wind_speed_kts=25;



Such listeners could then be used to allow Nasal scripts to become more configurable at runtime so that users can dynamically modify the behavior of a Nasal script or function.

You may want to take a look into the base package folder ($FG_ROOT/gui/dialogs) where you can find a number of XML dialogs that may be used to interface with the simulator. Basically, all FlightGear dialogs are based on this type of markup language (and these XML dialog files may also contain embedded Nasal code themselves).

Nasal in XML dialog files also has the advantage that related resources can be shipped together, i.e. are self-contained.

but eventually it should be done for thing which need to run fast, and I guess it would be very useful to think already now what structures should be eventually there and where the C++ Flightgear code is going.


"Premature optimization is the root of all evil" ;-)
In other words, in the beginning you could take a look at where exactly most of the time is spent when your script is running.

Nasal provides the systime() call for getting accurate timing results, so you could use subsequent calls to systime() in order to benchmark your script and individual functions by recording "time stamps" before and after a call, the timing statistics could then in turn be printed to the console or even be published in the property tree to show hotspots and provide information for optimization potential.

Also, I don't know if this is really anywhere documented (or even being so far used in FlightGear?) already, but there is the possibility to invoke Nasal code that is to be run separately from the main loop, inside a Nasal thread.

So this provides the opportunity to run performance critical code (computations) outside the main loop, in some form of Nasal "worker thread". If this is done in a conventional producer/consumer fashion, the whole thing is relatively straightforward to do.

However, for the time being, you have to ensure that not any of the FlightGear internal APIs are used inside other Nasal threads, because these are by default not yet synchronized with the fgfs main thread (can be added with explicit locking).

So, Nasal variables would need to be used for communications between multiple Nasal threads, access to FG APIs would need to be done in the main thread.

algorithm testing and tuning - cloud placement algorithms don't create realistic cloud patterns out of the box, but need to be called with a suitable parameter range to be determined by try and error - which is fun, but takes a lot of time...


I see, this is probably another thing that would obviously benefit from some form of runtime configurability, so that the Nasal script doesn't need to be edited and FlightGear restarted.
Another thing that might be worth investigating is to manually load the Nasal script itself, so that it can be reloaded at runtime. The io.nas module has a helper function named "load_nasal" that provides a way to load scripts at runtime, this could probably be used to reload a script on demand, one could try to first set the module name to "nil" and the reload the script. I will check if this works and post some code later on.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: A local weather system

Postby zakalawe » Wed Mar 17, 2010 5:06 pm

I'm happy to answer questions about the C++ code, via email or here, and I'm happy to work to get specific features into the core code, or especially into the Nasal interface, which is currently quite basic, but very easy to extend. As other people have mentioned, 90% of this should be fine in Nasal, but please do ask if you think something could or should be done in the C++ side - I should be able to tell you with five minutes if a given problem is a suitable candidate for compiled code instead of script. The existing C++ code has a huge library of mathematical and geospatial helpers, most of which are not well exposed to Nasal, and that's something that can certainly be improved.
zakalawe
 
Posts: 1259
Joined: Sat Jul 19, 2008 5:48 pm
Location: Edinburgh, Scotland
Callsign: G-ZKLW
Version: next
OS: Mac

Re: A local weather system

Postby WooT » Wed Mar 17, 2010 5:32 pm

oh ! This looks really amazing !

This reminds me of having thought about implementing convection for thermal soaring in FG. I have been reading a lot of documents, made some drafts, but the code needed to create such a thing is really above my skills...

Its really great to see you are at it !

Do you plan to include lift/sink in your weather system ? This would be just amazing !
I would be glad to share documents and ex-glider pilots thoughts if you like.

Congratulations on the screenshots, this looks stunning ! Can't wait to see how this turns out.

I'd be glad to contribute for more cloud textures ! I have some clouds photographies that just wait for an use.
WooT
 
Posts: 92
Joined: Tue Mar 17, 2009 5:09 pm

Re: A local weather system

Postby Hooray » Wed Mar 17, 2010 6:11 pm

Also, I guess some things could be solved in a more elegant way in Nasal - I just learned it a few weeks ago, and probably don't code the best solutions.


You seem to be doing a pretty good job so far, given that you must have learned quite a lot in a rather short amount of time.
So, I really would not worry too much about this, while it is true that there are some constructs in Nasal that are preferable over others (i.e. due to performance implications), there are enough people here who can help review and improve Nasal scripts.

If you want the community to help you review your code, you should probably make sure to comment your source code well to clearly illustrate your intentions and point out things that seem wrong or incomplete, but also to describe your algorithms.

You could for example post your script here using the forums or even better using a pastebin website like http://www.codepad.org (just select "plain text") at http://codepad.org/mkproj you can even create a new "project" page for your pastes, which gives you a subdomain like http://myweather.codepad.org

Doing it like this makes it easy for people to comment on and fork your script to add improvements.

Of course you could also directly use a github/gitorious website if you think that would be useful.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: A local weather system

Postby WooT » Wed Mar 17, 2010 11:32 pm

I have been reading your pages on the wiki, you made a really interesting essay. Your description of cloud modelling is really informative too, it summarizes very well the questions and answers when facing the problem.

I think I understood how you plan to separate airspace in tiles to describe what type of weather can occur in the area where the plane is.

Regarding the tiles where one can expect high convection and cumulus clouds, I have read the draft of the approach that you are planning to use on the 'cirrus clouds' thread ( viewtopic.php?f=5&t=7107&st=0&sk=t&sd=a&hilit=cirrus#p66673 ).

This looks pretty interesting, and I wanted to point you to some excellent document that is available here : ftp://luerkens.homedns.org/CumulusX/EN/Help.pdf

This is the help file for a FSX cumulus addon, but starting page 10 of the PDF, the author describes with much detail how he approached the problem.

I think that most of your algorithm meets what is described there, but it seems their approach is a bit more complete, specially about the conditions needed for such a 'glider's sky' to appear, and about the strength / density of thermals distribution.
Their approach is interesting in the way that it would work perfectly across multiplayer, given that the generation of the phenomenons is based on shared initial conditions, and make use of pseudo random generator, that will give the same results on any machine.

This document contains quite a pile of dense information, but I think it is really worth reading !

As for the thermals themselves, I have tried to improve how FG modelizes them, you can see it in the CVS AIThermal.cxx.
I describe it here : viewtopic.php?f=6&t=3377&st=0&sk=t&sd=a&hilit=ridge+lift#p31882
This is very draft and incomplete, but it does take life cycle, leaning , and thermal structure in consideration. It is just waiting for more improvement, and , most of all, the kind of system you are planning to make to place them realistically, and place a convincing cloud model above them.

I am not sure how I could contribute to you project, given that my coding skills are quite limited, but I am really willing to help in any way that I can. What you started here with much efficiency is something I really want to see in FG someday, but when looking at the amount of work this would imply to succeed, I had given up. Seeing that you and others are taking the challenge, I see another chance and would be really happy to give my little stone to the building, if any !
WooT
 
Posts: 92
Joined: Tue Mar 17, 2009 5:09 pm

Re: A local weather system

Postby WooT » Thu Mar 18, 2010 9:17 am

regarding multiplayer, I have had the following thoughts :

I don't think there is a need to have much information transmitted over the network, provided that some conditions are met :

Everyone should use the same metar data at a given time.

Atmospheric objects should live in the global space, not relative to the user's aircrafts positions.

Make sure that any random generated number is the same for everyone. This can be achieved by using pseudo random generation, using a common seed.
An interesting seed is based on simulator time, and could be described this way :

yyyymmddhhmmss , where yyyy is the year, mm , monthes, and so on to seconds, or any unit that comfortably contains computational cycles, but still allows for smooth rendering.
Using that seed , and making sure you generate numbers on a timer, and that you count every generation , everyone should get the same sequence of numbers for anything that has to look like random.
Each day will produce a different pattern, and the patterns would only repeat once 10000 years :)

This would be nice in a static environment, but unfortunately, clouds and atmosphere phenomenons move with the wind...

A solution could be to first determine the longest cycle of life for all the elements that may move, then use a "time machine" : when a user logs in, determine at what time to start a dummy, accelerated simulation of the atmosphere, and run it fast until the current time, without displaying anything. The time zero would be the instant when the 'oldest possible' moving instance was born before the user logs.
One would have to fetch the old metar reports in the dummy period, so that instances move "like they did for other users" during the dummy simulation.
All this using pseudo random with common time based seeds should lead to similar conditions over multiplayer, I think.
Last edited by WooT on Thu Mar 18, 2010 9:45 am, edited 1 time in total.
WooT
 
Posts: 92
Joined: Tue Mar 17, 2009 5:09 pm

Re: A local weather system

Postby Hooray » Thu Mar 18, 2010 9:43 am

METAR/Multiplayer interfaces - there were some discussions about METAR in other threads already, some ideas about Multiplayer weather - I haven't really thought too much about either, but it would be good to continue the discussions and settle on a viable scheme to merge local weather with METAR or allow to share it over MP


If you have questions about the FlightGear METAR system, you could try to get in touch with the user "mfranz", I think he is the developer of the METAR system and is also the maintainer of the Nasal system and has basically created or at least reviewed all scripts that you can find in the Nasal directory, so he is certainly the most knowledgeable person to talk about the METAR and Nasal systems.

I agree, for the time being I would also tend to ignore the multiplayer aspect and just concentrate on the local scenario. The current multiplayer system has too many limitations to be a good foundation for shared weather scenarios.

If a static seed (i.e. position, time, METAR strings for surrounding tiles) is indeed used by all clients, there would not need to be any explicit synchronization. Instead, all clients would work locally work with the same seed and algorithms.

This was also the powerful idea behind ridge lift, because it is locally computed by all clients and does not need to be communicated to other clients to have it take effect - all clients will see the same effects, as long as the "seed" (sampled terrain tiles in this instance) is identical among all clients.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: A local weather system

Postby Thorsten » Thu Mar 18, 2010 10:30 am

Wow - that's a lot of response :D

I'm picking two topics out of that for now:

Thermals

I aim to put in thermals as well as their lifecycle - essentially under the assumption that the convection seeds are what and where they are, and that from those, thermals are generated with some periodicity modulo randomness, make a cloud which drifts off, eventually (dependent on wind) looses contact with the seed and first the thermal, then the cloud decay. Also, some clouds may overdevelop and form a Cb if the thermal is very strong.

Conceptually I know how to code this, and since it can run in a slow loop with less than 1 Hz CPU power is not an issue. Technically, I don't know a good way how to make a thermal lift anything from Nasal. There is /environment/wind-from-down-fps - but I can't set that. My (very dirty) workaround would be to place an AI thermal initially on the North Pole, define regions where my thermals are supposed to be and change the AI thermal location to my current location whenever I am in such a region. Obviously, that is rather crude and misses the sophistication of WooT's thermal model, and I hope there is a clean solution available, or can be made available. The ridge-lift thread is rather good reading, and I'm glad to find that many things have already been treated with great sophistication - so we can use a lot of these ideas.

If there is a property (or more if you think of differential lift acting on the aircraft) I can write from Nasal to generate lift by setting it from a script, then I'd propose to do the following:

I let my convection algorithm determine thermal hotspot location and strength (i.e. (lat,lon) and a number [0..1] characterizing strength). We get the cloud base from temperature and dew point inside the weather tile. The cap cloud is initialized above that spot, displaced by an angle of the ratio windspeed/(max. lift) in the thermal. So, I suggest that WooT supplies an algorithm which, given (lat, lon, strength, evolution_time) as input, produces

a) a position (lat,lon) of the thermal cap cloud as a function of time and b)
b) the lifetime of the cloud in different phases

These are needed to manage the cloud model. And then there needs to be a second function

lift (mylat, mylon, myalt, lat, lon, strength, evolution_time)

which I use to set the thermal lift whenever the airplane is in an effect volume below the cloud (i.e. inside the thermal).

Tile geometry

Atmospheric objects should live in the global space, not relative to the user's aircrafts positions.


I've been thinking about this for quite a while. Basically, a local tile is much easier to manage, because you can simply define a 30x30 km square (or any other suitable size) and forget about spherical geometry. For a global tile scheme, that doesn't work because that would not cover earth. But in a (lat,lon) tile scheme, you'd have to struggle with the fact that tiles up north are much smaller than tiles on the equator - so spacing clouds using constant angular distances will give artifacts. I don't know of an easy way to compensate - in fact, I don't know if my scheme can cope with spherical geometry at all.

So, for the moment I'm trying for local tiles. But - with a server managing tile parameters, one could think of a scheme in which the server checks if a tile is already created (because another player is already in the vicinity) - and if that is the case, any other player in the region gets the weather tiles which have been spawned by the presence of the first player. So the server would not generate weather all across the globe, but only on request locally where an aircraft is, but take care that everyone else gets the same tile setup info.

Make sure that any random generated number is the same for everyone. This can be achieved by using pseudo random generation, using a common seed.


Doesn't that make the assumption that no other Nasal script calls the random number generator in flight? I don't know if aircraft scripts would usually do that, but I can imagine that some at least do. So if there are other calls from loops running parallel to the weather management, even synchronizing the seed wouldn't guarantee the same outcome...

I'll respond to other topics later or by PN!
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

OFFTOPIC (reloading Nasal scripts at runtime)

Postby Hooray » Thu Mar 18, 2010 11:12 am

Hooray wrote:Another thing that might be worth investigating is to manually load the Nasal script itself, so that it can be reloaded at runtime. The io.nas module has a helper function named "load_nasal" that provides a way to load scripts at runtime, this could probably be used to reload a script on demand, one could try to first set the module name to "nil" and the reload the script. I will check if this works and post some code later on.


Okay, just for the record: I was wrong! :lol:

While doing it like this does work to some extend, it stops working once you have code that is repeatedly calling itself using callbacks (i.e. settimer, setlistener stuff). You can try for yourself.

What I did is this, I put this in a file named "reload.nas" in $FG_ROOT/Nasal:
Code: Select all
var control_property = "/reload-nasal-script";

var reload_script = func(filename,namespace) {
 
  # fixme: suspend running scripts??
  # reset the namespace
  if (contains(globals, namespace)) {
        globals[namespace] = {}; # nil??
      }

  io.load_nasal(filename,namespace);
};

var reload_foo = func {
  if (!getprop(control_property)) return;
  var filename = "/Nasal/foo.txt"; # nasal script without .nas extension, to avoid automatic loading during startup
  reload_script(getprop("/sim/fg-root") ~ filename,"foo");
  setprop(control_property,0);
 
}

_setlistener(control_property,reload_foo);


Then I added another text file (no .nas extension) to $FG_ROOT/Nasal/foo.txt:
Code: Select all
print ("foo script loaded!");

var delay=1.0;
var msg="Hello World 1!"; # modify this and save the script

var info = func {print(msg); settimer(info,delay);}

info();


When you now start the simulator and set /reload-nasal-script to 1, it will in fact (re)load the nasal file correctly, but all functions in that script running then, will keep running - i.e. you will have multiple versions of these functions running, once you keep reloading the file.

You can try it for yourself, just by keeping "foo.txt" open in a text editor, then slightly changing, saving and reloading it.

So I guess the method is pretty hackish (think about threaded scripts!), there would need to be some way to make the Nasal VM purge or reload and replace the bytecode for a script, that could be pretty useful.
Having such a feature would make it even possible to "hot swap" code at runtime.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: A local weather system

Postby WooT » Thu Mar 18, 2010 12:00 pm

I , unfortunately, do not know about nasal *at all*...

On the other hand, I have been tinkering with c++ for ridge and thermals, and from this experience I know we can easily write to wind-from-down. I even created a separate property , it is in environment/thermal-lift-fps. environment manager sums it to ridge-lift-fps to give wind-from-down fps.

I am thinking of a slightly different approach to what you proposed :

Your convection algorithm decides the position of the cloud cap ( given the cloud cap position, my thermal model takes care of thermal foot and everything in between ), its altitude, the thermal strength, and other variables to describe it if we feel the need ( I am thinking diameter for instance, life cycle length... )

*Your nasal AI system creates an instance of my c++ thermal, which begins to live, and your system manages the thermal movement with the wind

*My thermal writes to thermal-lift-fps, which is summed to ridge-lift by environment manager.

*My thermal writes to new properties that we can add to the tree : cloud existence, cloud height, cloud diameter, even we could think of cloud texture depending of life phase ( fluffy hairy at start, then coliflower when well formed, then destructured at end...), so that your system can draw the cloud.

*At the end of its life, the c++ thermal warns your system, and either it self destructs, or the nasal kills it.


This makes the assumption that nasal can create instances of the AI c++ thermals, and/or kill them, and that it can read/write to properties for a specific thermal instance in the tree / or to environment/thermal-lift-fps

If this is not possible, then another scenario could be :

*Your nasal system creates a thermal, determines where its cloud cap has to be, and its characteristics.

*Your system takes care of moving it with the wind.

*A nasal translation of my c++ thermal model takes care of determining the lift, and writes to environment/thermal-lift-fps, which in turn is handled by environment manager, just like for ridge and old thermals, to write to wind-from-down.

*This same nasal objects takes care of determining the cloud shape and aspect, and passes it to your system.

*At the end of the life cycle for this thermal, it sends a signal to your system so that it is killed.

This version makes the assumption that it is possible to get the thermal height ( cloud cap altitude minus ground level altitude ) from nasal. I don't know if this is possible.


Both versions share a common issue : what happens if two or more thermals overlap ? so far there is only one thermal-lift-fps to play with. Maybe the second scenario would allow to sum overlapping influences and only then send it to c++. On the other hand, each c++ thermal instances can have their own thermal-lift[i], send this to your system, which in turn writes to thermal-lift-fps.

I'm thinking out loud here, I have no clue what would be the most efficient in cpu work, or if any of the above is possible, due to my total ignorance of how nasal works, and even more how it can work together with the c++ parts....


So if there are other calls from loops running parallel to the weather management, even synchronizing the seed wouldn't guarantee the same outcome...


damnit ! I didn't think of that.... Isn't there a way to isolate the generator in its own little cage ?
WooT
 
Posts: 92
Joined: Tue Mar 17, 2009 5:09 pm

Re: A local weather system

Postby Thorsten » Thu Mar 18, 2010 1:07 pm

On the other hand, I have been tinkering with c++ for ridge and thermals, and from this experience I know we can easily write to wind-from-down. I even created a separate property , it is in environment/thermal-lift-fps. environment manager sums it to ridge-lift-fps to give wind-from-down fps.


But they can't be written from the property browser, hence they can't be written from Nasal, hence my problem :(

Let me discuss a few issues I see with the schemes you outlined.

First, you're thinking primarily in terms of thermals. On the other hand, what I'd like to have is a more general thing, the effect volume, a user-specified region in which any weather parameter can be set in a non-standard way - the thermal is a special instance of that in which the weather parameter is thermal lift. I don't particularly mind effect volumes being coded on the C++ side - it's just that I can't do it myself, not because I can't write C++, but simply because I can't get the Flightgear source code compiled for lack of a few libs which would require me to upgrade my whole system, which I can't do easily because a lot of work-related code relies on tweaks I have done to it, and to redo that afer an upgrade usually costs me 2-3 weeks - which I simply don't have right now. But I'd like to see it coded consistently - either effect volumes exist on the C++ side, or on the Nasal side - but a few in C++, a few others in Nasal is not a solution I like.

Having them as a general structure would also allow for a consistent set of rules of what happens in overlapping effect volumes.

So as for now, especially for development, I'd prefer a Nasal solution, especially since speed is not an issue, updating the lift with ~1 Hz is quite sufficient for realism I guess, and transforming the cap cloud model with 20+ Hz is in all likelihood the bottleneck in speed.

Your convection algorithm decides the position of the cloud cap ( given the cloud cap position, my thermal model takes care of thermal foot and everything in between ), its altitude, the thermal strength, and other variables to describe it if we feel the need ( I am thinking diameter for instance, life cycle length... )


The thermal is a bubble of hot air starting from the foot. If the wind is not particularly strong, the initial bubble would not lean much into the wind when the first cloud is formed. Wind would displace it later, and the thermal would stretch more and more between the static foot and the wind-driven cap, till finally the upward stream 'breaks' and a new bubble (and new cap cloud) is formed. So to me it makes not too much sense to fix the cap cloud position from the convection model - the foot is what is fix, and everything else is subject to both wind and thermal dynamics. I don't suppose you can take a thermal and simply displace it with the wind and get a realistic result, there is cross-talk between wind and thermal development. So whatever models the thermal development must also either know or even take care of the wind displacement. Again, I'd like to see wind displacement handled consistently either all in C++ or all in Nasal.

*My thermal writes to new properties that we can add to the tree : cloud existence, cloud height, cloud diameter, even we could think of cloud texture depending of life phase ( fluffy hairy at start, then coliflower when well formed, then destructured at end...), so that your system can draw the cloud.


That would work just fine, I guess. Either the select animation or simply reloading a different model in the same position should do the trick of changing cloud shape, and a material animation can even slowly fade old clouds into transparent.

This version makes the assumption that it is possible to get the thermal height ( cloud cap altitude minus ground level altitude ) from nasal. I don't know if this is possible.


Nasal can get terrain elevation via calls of geoinfo(), so that's not a problem.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: A local weather system

Postby Hooray » Thu Mar 18, 2010 1:39 pm

But I'd like to see it coded consistently - either effect volumes exist on the C++ side, or on the Nasal side - but a few in C++, a few others in Nasal is not a solution I like.


I understand and actually agree with you, it would be better to have some form of unified component model with well-defined responsibilities.
Otherwise, it would not be unlikely to have interfering systems that are not fully 'aware' of each other and may mutually invalidate their state.
So having a clear separation would be a good thing.

But they can't be written from the property browser, hence they can't be written from Nasal, hence my problem


I will need to look into this tonight, it is possible that some properties cannot be modified from "outside C++" because these are so called "tied" properties, i.e. they are not really owned by the property tree, but maintained outside the property tree.
I think, such "tied properties" were introduced to provide better performance for repeatedly updated variables.

This applies to most FDM properties. Tied properties are shown with a 'T' suffix in the property browser, I think (with development extensions enabled).

transforming the cap cloud model with 20+ Hz is in all likelihood the bottleneck in speed.


As I previously mentioned already, there is the very real possibility to use threads for computations
that require higher or more reliable update rates:

Also, I don't know if this is really anywhere documented (or even being so far used in FlightGear?) already, but there is the possibility to invoke Nasal code that is to be run separately from the main loop, inside a Nasal thread.


I only checked very quickly but didn't find any examples on using threads in Nasal, so here's a simple contrived example using one worker thread (I didn't test this yet, it's off the top of my head):

Code: Select all
# for messaging/control purposes (arbitrary variable names):
var thread_message = "run";
var thread_status = "not running";

# a conventional function to be run in a nasal thread
# keep in mind that you may not use any of the FlightGear APIs in
# Nasal threads, without adding explicit locking

var compute = func {
 thread_status = "running"; # change the status of the thread (not necessary, just for inspection by other threads)
 
 # check if there is a thread_message saying "stop" - in order to be able to stop the thread from other threads
 while (thread_message != "stop") {
   # do some very long computations here in the loop body
   var result = 1 + 3 * 4;
 }

 thread_status = "finished"; # change the thread status, because the thread is about to be terminated (for other threads)
}

thread.newthread( compute ); # run the compute function in a new thread

# here you could check for the "thread_status" or send a message to the thread by setting "thread_message"



In order to return data from the worker thread, one could use a queue.
Or even two queues in a conventional producer/consumer fashion: one work queue, and another one for the results.

Such worker threads could then be used to pre-calculate and cache data.

Doing it like this is not that flexible though, the thread will terminate once the computation changes - so one might either want to idle instead or consider using some form of thread pools, especially when you find that you need to spawn many short-lived worker threads repeatedly, then thread pools would be preferable.

If needed, it should be possible to overload Nasal's thread.newthread() to accept a numerical parameter instead of a function name in order to return a vector of threads for later use.

Check out http://plausible.org/nasal/lib.html for more info on the thread module.

I can't get the Flightgear source code compiled for lack of a few libs which would require me to upgrade my whole system, which I can't do easily because a lot of work-related code relies on tweaks I have done to it, and to redo that afer an upgrade usually costs me 2-3 weeks - which I simply don't have right now.

I understand that and can absolutely relate to it.
What libs exactly are affected?

Assuming that you are on Linux: are you aware of the possibility to build and install libraries (and other binaries) into a $HOME directory?

If you are worried, that even that might create additional problems for you, you could just as well create another user ('fgfs') and do all your development in /home/fgfs, this would allow you to install all dependencies in the $HOME directory, without affecting the rest of your system or your private/work folders.
If you need help doing this, please feel free to ask!


My thermal writes to new properties that we can add to the tree : cloud existence, cloud height, cloud diameter, even we could think of cloud texture depending of life phase ( fluffy hairy at start, then coliflower when well formed, then destructured at end...), so that your system can draw the cloud.


Would you consider using an updated binary compiled from a CVS snapshot?
Are you on Linux or Windows?

On Linux, building from source is relatively straight forward, given that there are now a number of helper scripts ("fgfs-construct","fgfs-builder" etc) that largely automates the whole task and downloads from all repositories.

On Windows, things are indeed pretty tricky - but there are excellent instructions provided at http://geoffair.net/fg/
And another very viable short cut would be to simply download the whole set of project files, which are already pre-configured.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: A local weather system

Postby WooT » Thu Mar 18, 2010 3:59 pm

The thermal is a bubble of hot air starting from the foot. If the wind is not particularly strong, the initial bubble would not lean much into the wind when the first cloud is formed. Wind would displace it later, and the thermal would stretch more and more between the static foot and the wind-driven cap, till finally the upward stream 'breaks' and a new bubble (and new cap cloud) is formed. So to me it makes not too much sense to fix the cap cloud position from the convection model - the foot is what is fix, and everything else is subject to both wind and thermal dynamics. I don't suppose you can take a thermal and simply displace it with the wind and get a realistic result,


My understanding of a thermal structure would make me think that what you describe is a particular case.

From what I understand, in a flat country scenario, the whole system is affected by the wind, but the wind velocity decreases at lower levels, due to surface friction. The top of the thermal drifts in the mass of air, at the speed of the flow at the condensation level. the "leg" and "foot" of the thermal lag behind, specially if the surface is quite rough. Eventually some sudden change in the ground structure or surface will break the flow, this is how we often see thermal dying on a hill edge, a river, cliff, building, lake, etc...anything that drastically changes the conditions of the lower layers.

The only moment when a thermal "foot" would remain stationary, would be in an ideal zero wind condition at lower levels, which is rarely the case. In this case,and if there is some wind higher, as you mentioned, only discontinuous bubbles, or very short life thermals are created. Then you will rarely see a good cap cloud above it. You can see it sometimes in the end of summer mornings, this is a nice and funny eructing puffy sign that the "engine" is starting to work.
If there is no wind at all on all layers, the thermal would simply be vertical, and the cloud just above the foot. Rapidly the amount of hot air in this place will all be consumed, replaced by more fresh air from the surroundings. And the source will tarish, until sun rays heat it again.

On the other hand, with wind on all layers, you will most often see thermals repeatedly appear on the same exact spot, like a heated parking or particularly heated region of the ground. But now, the mass of air does move, even in the lower layers. If the air at low level moves too fast for a given ground / low layers heating, the thermal will not activate, and only a bubble will be created and start lift.
If the air at low level moves slowly enough for that same ground / low layers heating , a continuous flow of hot air will be created and will self feed even after the thermal left the "hot parking", the thermal being now fed by new hot air as it travels down wind... until some change in the ground features or surface breaks the phenomenon.

At some point when the thermal is well formed, one needs to see the phenomenon in a wider scale, I think about gently heating sauce in a closed can. At the top part of the thermal, once the cloud is formed, the energy of steam condensation feeds even more the vertical airflow inside the cloud , and this "sucks" the air below it. The air particles then meet the top inversion, where they mix with cold air, cool down, return to gas phase,cool even more, and begin to descend along the thermal body, until they meet the low level layers. As long as the air is heated enough by the surface surrounding the thermal foot so that it is hotter, thus lighter than the flow of cool descending air around the body, the phenomena self feeds.Moreover, as the thermal travels downwind, if constantly finds new hot air from the surface to be alimented with, while it leaves its cold "long skirt" behind it and on the sides.
This is how you can see some thermals travel a good amount of time before dying, despite they don't follow a "hot ribbon" on the ground. Sometimes when waiting at the airfield, you would feel the thermal pass . If there is long enough grass or trees around, you could even see it. Add a sand grain in the machine, and everything breaks.

Extrapolate the phenomenon to a high moisture, high unstable atmosphere with severe wind shear between levels, and the cloud begins to grow more and more, sucks all its neighbors, until it reaches the state of Cb, self alimenting monster that can live even at night or over seas.

In between those extremes , you can see all sort of semi continuous thermals, either clearly "dashed" or simply varying in intensity along the vertical axis, presenting more or less turbulence inside the core itself, but still "in one piece".

Here is a nice essay about thermal structure , which I think confirms my thoughts :
http://narroginglidingclub.org.au/downloads/thermals.pdf

Now, considering a simpler model, which would be a simple leaning truncated cone that drifts in the air could maybe give satisfactory results in the sense that you mentioned much before, that is, giving a sufficient feeling for flight simulation, without the need of a supercomputer at home :)

One characteristic that could be interesting to model is that the thermals are usually not straight cones, but instead follow a more complex curve. The slowdown of the air mass close to the ground is not linear, but rather resembles a square or cubic curve. Also, the vertical airspeed of a particle in the thermal body depends on the strengh of the thermal at this height, giving an even more complicated curve, that depends not only from the ground conditions and heating, but also on the larger scale net of thermal/sinks, amount of moisture in the air, and so on....

I presume as a starter, trying to make an ugly leaned "banana-cone" appear at a realistic place and time, and die in the same way , would be a nice challenge !

Here is an old mockup of how I would see the ideal thing. Green is no lift, no sink. Blue is sink, red is lift. The whole thing drifts in the wind ( not animated in the gif ). The middle period of time when lift is all along the height, could last more or less long. The bending of the thing could vary over time too, I had not modeled it at this time.

Image

In any case, we need to know both the airspeed at cloud level and at ground level. How the in between is interpolated is not really important. I mean, we can build the thing from top to bottom, or from bottom to top, it will build the exact same thing :) If ground wind is low enough / null , and high wind is not too strong, we will see the thermal shape and behavior that you described.
Still my brain wants to see the thing as moving with the global mass of air, with lower layers slowed down by ground friction. Thus my initial idea to plot the cloud first.


About doing the work in nasal, and keeping it consistent, I really don't see any issue. I would be very happy if I can help providing a satisfying algorithm for the precise Thermal part of a wider Effect Volume object, and I think I see why it wouldn't be good to mix both languages/objects in such a system.

Then the problem of writing to wind-from-down or whatever FDM property seems to me to be a show-stopper.
Fortunately , according to Jester in irc, this can be done : http://imagebin.ca/view/wiit9eW.html
The problem doesn't seem to be to write to the property, but rather to prevent it to be reinitialized at every frame, like it seems to occur now.
I know for sure nothing else calculates thermal-lift-fps, as I added the property myself for AIThermal, and no other component uses it ( to my knowledge ).
It could be that the environment reads 'strength' from AIThermals to write to the property. As the AI object is inexistent ( assuming we don't use the thermal scenario ! ), it does not provide any usable value. In turn, environment resets the property to zero ?


I am on linux, so modifying the sources and building it again is really no problem my side. I do it often to stay up to date with CVS. On the other hand, I really don't know how to properly code with a cross platform goal in mind.
WooT
 
Posts: 92
Joined: Tue Mar 17, 2009 5:09 pm

Next

Return to Weather

Who is online

Users browsing this forum: No registered users and 2 guests