Board index FlightGear Development New features

Property rule based road traffic

Discussion and requests for new features. Please note that FlightGear developers are volunteers and may or may not be able to consider these requests.

Property rule based road traffic

Postby Marius_A » Thu Jan 25, 2018 11:05 pm

Preliminary results of my attempt to make Property rule driven road traffic. Video shows 1190 vehicles:


Latitude, longitude, elevation and heading for each car is calculated using interpolation tables. Property rule filter definition file is (pre)generated by Nasal script. Way-points are currently extracted from ufo-model-export file.

I plan to make this as an add-on with tools to generate custom traffic.
Marius_A
 
Posts: 92
Joined: Wed Dec 04, 2013 3:20 pm

Re: Property rule based road traffic

Postby wlbragg » Thu Jan 25, 2018 11:18 pm

Very impressive, especially the models!

I take it these are models in the scene VS shader produced?
Kansas and Ohio/Midwest scenery development.
KEQA, 3AU, KRCP Airport Layout
Intel i7/GeForce RTX 2070/Max-Q
User avatar
wlbragg
 
Posts: 7586
Joined: Sun Aug 26, 2012 12:31 am
Location: Kansas (Tornado Alley), USA
Callsign: WC2020
Version: next
OS: Win10/Linux/RTX 2070

Re: Property rule based road traffic

Postby Thorsten » Fri Jan 26, 2018 7:55 am

Preliminary results of my attempt to make Property rule driven road traffic.


This looks very cool!

Is there a way to get this to handle LOD? In that way, we could think of using fake shader road traffic from far away and actual models from close-by?

Also, not sure whether you're 'bunching' vehicles and moving a whole group with one property update - but that may reduce the needed property updates.

(I am wondering about scalability to a whole city, eventually I guess that's lots of properties to update).
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Property rule based road traffic

Postby xDraconian » Fri Jan 26, 2018 8:23 am

Nice work. Looks awesome!
xDraconian
 
Posts: 406
Joined: Sun Jan 21, 2018 6:53 am
Version: Git
OS: Linux Mint

Re: Property rule based road traffic

Postby Hooray » Fri Jan 26, 2018 10:45 am

Yes, that looks great - if you haven't already, I'd suggest to reach out to ThomasS who's been working on a dedicated "ground services" module and get in touch with the "Traffic shader" folks (Thorsten & OSM team)

Like Thorsten said, doing batching should translate into fewer property accesses (I/O) - basically, traffic could be split into "nets" where each vehicle represents a knot (node) in the net (graph) with different speeds (distances in between vehicles) being modeled by flexing the net/rope - even though, looking at the valleys and bridges, this is probably even a 3D problem, i.e. there might be a lesson or two to be learnt from the way the wingflex algorithm handles the underlying problem of morphing the mesh in 3D - after all, that is what multi-lane traffic between two junctions is also doing basically.

I do think it's a good idea to turn this into an addon.

slightly OFFOPTIC: what exactly are the property rules doing here ?

I am wondering because ground traffic is something that could conceptually run out of the main loop, and even in a background thread to pre-compute position tables asynchronously (which can be done via Nasal) - whereas I don't think that the property rule system allows dynamically instantiating tables on demand or doing such computations asynchronously ?

The conventional way to run such "traffic simations" is indeed in a separate process, e.g. using something like the MP system to inject traffic nodes - but that will obviously not work once we begin using certain optimizations, such as batched updates.

All that is needed to run something like this out of the main loop is access to the vector data (basically OSM data) and a way to inject the corresponding 3D models into the scene and update those.

Note, I am not suggesting that this should be done - but the same people that are usually quick in sugggesting to use "property rules" instead of Nasal, are also the same ones that want a fast "rendering-only main loop":


https://sourceforge.net/p/flightgear/ma ... /34793778/
Torsten Dreyer wrote:Instead of adding just-another-feature
we need to strip it down to getting a fast and constand fps rendering
engine. Everything else needs to run outside the main loop and has to
interact with the core by, say HLA/RTI or whatever IPC we have.

[...]
it introduces an
unpredictable (w.r.t. timing) step in the main loop getting us on step
further away from constant frame rates.


Given that there is no tight main-loop coupling required to pre-compute such traffic, this may be one of those instances where choosing property rules over Nasal code may cause more of an performance impact - especially because property rules always run in the main loop, whereas well-designed Nasal code could conceptually run out of the main loop, and possibly even out-of-process using an IPC mechanism like Nasal

Conceptually, this isn't much differrent from any of the previous "AI Traffic via HLA/RTI or MP" discussions we've previously seen - so, all the pro/contra arguments still hold true.
Anyway, making this an addon seems indeed like the sensible option for now.

PS: Loving already the roundabout handling !! ;-)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Property rule based road traffic

Postby Marius_A » Fri Jan 26, 2018 2:33 pm

Is there a way to get this to handle LOD?

Each ground traffic "scenario" will have its bounding box. Models will be hidden/erased when the viewer is outside this box.

... what exactly are the property rules doing here ?

All the traffic lanes are defined as tracks. For each track:

Calculate global progress [0..1] (0..100% of track length):
Code: Select all
  <filter>
    <type type="string">gain</type>
    <period>
      <min type="double">0</min>
      <max type="double">1</max>
    </period>
    <input>
      <expression>
        <sum>
          <product>
            <property type="string">/sim/time/delta-sec</property>
            <property n="1" type="string">/sim/ground-traffic/traffic/Kaunas/track[0]/speed</property>
          </product>
          <property type="string">/sim/ground-traffic/traffic/Kaunas/track[0]/progress</property>
        </sum>
      </expression>
      <condition>
        <greater-than>
          <value type="double">1</value>
          <property type="string">/sim/ground-traffic/traffic/Kaunas/track[0]/progress</property>
        </greater-than>
      </condition>
    </input>
    <input n="1" type="double">0</input>
    <output type="string">/sim/ground-traffic/traffic/Kaunas/track[0]/progress</output>
  </filter>


For each vehicle:
Calculate local progress [0..1] by adding random offset to the global progress:
Code: Select all
  <filter n="1">
    <type type="string">gain</type>
    <period>
      <min type="double">0</min>
      <max type="double">1</max>
    </period>
    <input>
      <expression>
        <sum>
          <value type="double">0.03050530865</value>
          <property type="string">/sim/ground-traffic/traffic/Kaunas/track[0]/progress</property>
        </sum>
      </expression>
    </input>
    <output type="string">/sim/ground-traffic/traffic/Kaunas/track[0]/vehicle[0]/progress</output>
  </filter>


Use local progress to calculate lat/lon/elevation/heading:
(only showing filter for latitude calculation):
Code: Select all
  <filter n="2">
    <type type="string">gain</type>
    <input>
      <expression>
        <table>
          <entry>
            <ind type="double">0</ind>
            <dep type="double">54.87993809</dep>
          </entry>
          <entry n="1">
            <ind type="double">0.03628652386</ind>
            <dep type="double">54.877513</dep>
          </entry>
          <entry n="2">
            <ind type="double">0.07203049727</ind>
            <dep type="double">54.87512486</dep>
          </entry>
          <entry n="3">
            <ind type="double">0.1351641777</ind>
            <dep type="double">54.87090884</dep>
          </entry>
          <entry n="4">
            <ind type="double">0.2220007542</ind>
            <dep type="double">54.86510657</dep>
          </entry>
          <entry n="5">
            <ind type="double">0.324716894</ind>
            <dep type="double">54.8582448</dep>
          </entry>
          <entry n="6">
            <ind type="double">0.3937575175</ind>
            <dep type="double">54.85363593</dep>
          </entry>
          <entry n="7">
            <ind type="double">0.4520399339</ind>
            <dep type="double">54.84974421</dep>
          </entry>
          <entry n="8">
            <ind type="double">0.513446628</ind>
            <dep type="double">54.84563923</dep>
          </entry>
          <entry n="9">
            <ind type="double">0.5410955635</ind>
            <dep type="double">54.84379321</dep>
          </entry>
          <entry n="10">
            <ind type="double">0.5713047944</ind>
            <dep type="double">54.84173016</dep>
          </entry>
          <entry n="11">
            <ind type="double">0.5960979759</ind>
            <dep type="double">54.84000864</dep>
          </entry>
          <entry n="12">
            <ind type="double">0.6236488057</ind>
            <dep type="double">54.83807064</dep>
          </entry>
          <entry n="13">
            <ind type="double">0.6526109829</ind>
            <dep type="double">54.83601405</dep>
          </entry>
          <entry n="14">
            <ind type="double">0.687024141</ind>
            <dep type="double">54.83355112</dep>
          </entry>
          <entry n="15">
            <ind type="double">0.7159047536</ind>
            <dep type="double">54.83146733</dep>
          </entry>
          <entry n="16">
            <ind type="double">0.757118815</ind>
            <dep type="double">54.82848069</dep>
          </entry>
          <entry n="17">
            <ind type="double">0.7965889984</ind>
            <dep type="double">54.82562313</dep>
          </entry>
          <entry n="18">
            <ind type="double">0.8193138617</ind>
            <dep type="double">54.82399388</dep>
          </entry>
          <entry n="19">
            <ind type="double">0.843357688</ind>
            <dep type="double">54.82230306</dep>
          </entry>
          <entry n="20">
            <ind type="double">0.8663591945</ind>
            <dep type="double">54.82070958</dep>
          </entry>
          <entry n="21">
            <ind type="double">0.8879117056</ind>
            <dep type="double">54.81923556</dep>
          </entry>
          <entry n="22">
            <ind type="double">0.9129669078</ind>
            <dep type="double">54.8175695</dep>
          </entry>
          <entry n="23">
            <ind type="double">0.9322532533</ind>
            <dep type="double">54.81631062</dep>
          </entry>
          <entry n="24">
            <ind type="double">0.9537178073</ind>
            <dep type="double">54.81492095</dep>
          </entry>
          <entry n="25">
            <ind type="double">0.9633238101</ind>
            <dep type="double">54.81428319</dep>
          </entry>
          <entry n="26">
            <ind type="double">0.9709985207</ind>
            <dep type="double">54.81373368</dep>
          </entry>
          <entry n="27">
            <ind type="double">0.9806116083</ind>
            <dep type="double">54.81303852</dep>
          </entry>
          <entry n="28">
            <ind type="double">1</ind>
            <dep type="double">54.81170138</dep>
          </entry>
          <property type="string">/sim/ground-traffic/traffic/Kaunas/track[0]/vehicle[0]/progress</property>
        </table>
      </expression>
    </input>
    <output type="string">/sim/ground-traffic/traffic/Kaunas/track[0]/vehicle[0]/latitude-deg</output>
  </filter>


Nasal is used to load models and to "connect" their lat/lon/elevation/heading properties to filter output properties:
Code: Select all
var vehicle = {
   new: func(mdl_path, traffic, track_id, vehicle_id) {
      ...
   },
   init: func {
      foreach(var a; ["longitude-deg", "latitude-deg", "elevation-ft", "heading-deg"])
         me.model.getNode(a).alias(me.node.getNode(a, 1));
   },
   del: func {
      ...
   },
};
Marius_A
 
Posts: 92
Joined: Wed Dec 04, 2013 3:20 pm

Re: Property rule based road traffic

Postby Thorsten » Fri Jan 26, 2018 2:35 pm

For each vehicle:
Calculate local progress [0..1] by adding random offset to the global progress:


You have 1190 of those running then?
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Property rule based road traffic

Postby Marius_A » Fri Jan 26, 2018 2:49 pm

Yes.
Marius_A
 
Posts: 92
Joined: Wed Dec 04, 2013 3:20 pm

Re: Property rule based road traffic

Postby Hooray » Fri Jan 26, 2018 3:19 pm

That's really cool and very interesting - did you end up using property rules due to actual profiling, i.e. was Nasal's performance insufficient ?
Admittedly, property rules are faster and cause less overhead - but they also run in the main loop as far as I am aware.
Thus, I believe that you may find it interesting to do an actual benchmark running a similar amount of computations in a background thread, note that those would indeed have to use built-in Nasal data structures/library functions and no extension functions (property tree, fgcommands etc)

Code: Select all
var compute = func() {
}


thread.newthread( compute );

Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Property rule based road traffic

Postby Marius_A » Sun Jan 28, 2018 6:56 pm

Is this approach correct?
Code: Select all
var loop = func {
   thread.newthread( compute ); # calculate positions/orientations
   update();                    # move models
}
var timer = maketimer(0.0, loop);
timer.start();
Marius_A
 
Posts: 92
Joined: Wed Dec 04, 2013 3:20 pm

Re: Property rule based road traffic

Postby Hooray » Sun Jan 28, 2018 7:21 pm

NO: you are setting up a timer that is executed at framerate (0.0 / no delay whatsoever) - the callback that is invoked by your timer will create a new thread whenever it is executed, i.e. one thread per created frame! Thus, after 30 frames (~ 1 sec), you will have 30 threads ("tasks") running - i.e. your compute function will be running 30 times in parallel :D

background:
thread.newthread(CALLBACK) will start a new background thread (outside the main loop) running the function specified via callback.
In other words, you could implement an infinite loop in the compute function or use a timer to check if there is any work to be done in the queue (e.g. a vector with tasks).

The thing to keep in mind here is that you need a way to synchronize both threads - i.e. the code running in the main loop, and the one in the background/worker thread.
Imagine it like having two people collaborate - i.e. once creating (producing) a list of positions/orientations and the other one "consuming" that list.
If these two parts (threads) don't communicate properly, they may see invalid/incomplete state.

I am not really suggesting to use any of this, without having done some benchmarking first - otherwise, this could be a dead-end.
Apart from that, there are dedicated sychronization helpers available in the thread module - so called semaphores and mutexes.
For starters, it may be sufficient to use global variables - with one thread reading the variable only, and the other one only writing to it.
That way, you can implement a simple signalling scheme - e.g. the main thread would be polling (checking) the variable to determine the state of the worker thread, which would only update the variable once it is actually fiinished.

But again, don't spend any time playing with this unless you have confirmed that it's worth it - e.g. a simple benchmark would be creating a conventional Nasal function to do your current computations in the main loop (writing only to Nasal variables) - and using debug.benchmark() to see how long this takes, ideally writing everything to a Nasal vector.

Once that is working, the same function could be invoked via thread.newthread()

PS: An inifite loop would be something like this (this would never terminate:
Code: Select all
var compute = func() {
var condition = 1;
while(condition) {
# any code here would never terminate
}

} # compute()

Obviously, condition could also use an external/global variable, e.g. to implement a simple scheduling scheme that would check the task queue for any work, and if there isn't any, wait for 5-10 seconds to check again, and if there is some work to be done, do batches of that work in a background thread, spread across several frames (seconds), and the communicate to the main thread that the work queue is "full"
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Property rule based road traffic

Postby Thorsten » Sun Jan 28, 2018 7:43 pm

I suspect threading this out is ill suited to the problem here, because at the end of the day you need to update properties to move the models, and asynchronously running Nasal can't write properties and remain stable. I suspect compared with the computational load (which seems slim) property I/O is the main issue here.

Anyway - what IS the impact of running 1000+ property rules as compared to not doing it?
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Property rule based road traffic

Postby paju1986 » Sun Jan 28, 2018 7:48 pm

This combined with osm2city should look really awesome :)
paju1986
 
Posts: 246
Joined: Sun Oct 30, 2011 8:42 pm
Location: Badajoz (Spain) - LEBZ
OS: Arch Linux

Re: Property rule based road traffic

Postby Hooray » Sun Jan 28, 2018 8:07 pm

I do agree with Thorsten's point on property I/O likely being the bottleneck - but right now property rules are handling the computations, which is why I suggested to consider using Nasal threading if doing this in the main loop is too expensive. Again, it should suffice to post a corresponding function here for us to take a look.
I suppose that a hybrid approach should work fairly well: doing batched updates using split-frame loops and background processing of the 1000+ position updates.

Even with the current code, the computation overhead can be determined by using debug.benchmark(func compute(arguments) ) - which should log the runtime to the console
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Property rule based road traffic

Postby Thorsten » Mon Jan 29, 2018 7:21 am

but right now property rules are handling the computations


The computations are very simple though - it's all linear interpolation between points - it's a couple of thousand multiplications and additions, nothing that'd impress a modern CPU much.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Next

Return to New features

Who is online

Users browsing this forum: No registered users and 5 guests