Board index FlightGear Development AI Traffic

Populate AI Traffic with real traffic

Intelligent, computer controlled vehicles that drive/fly over the planet!

Re: Populate AI Traffic with real traffic

Postby Hooray » Sun Oct 28, 2012 2:24 pm

Yes, sorry my fault: Forgot to remove the old stuff that used the copied SGThread sources ... pushed a fix, try again.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

Re: Populate AI Traffic with real traffic

Postby F-JJTH » Sun Oct 28, 2012 2:40 pm

ok compilation works fine now :) Thanks you
I will try to use some math function from simgear to see if the link works correctly
User avatar
F-JJTH
 
Posts: 697
Joined: Fri Sep 09, 2011 11:02 am

Re: Populate AI Traffic with real traffic

Postby Hooray » Sun Oct 28, 2012 2:48 pm

No need, I can tell you that you cannot currently directly use the SG stuff yet, because there's a little conflict: The fgms source tree uses outdated sources from SimGear which have been patched by the fgms developers. Also, some of the math header required by fgms have been meanwhile updated in SimGear, so this still needs some work, because the fgms code uses names like sgdVec3 which have been meanwhile renamed in SimGear :)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

Re: Populate AI Traffic with real traffic

Postby F-JJTH » Sun Oct 28, 2012 2:55 pm

Oh, bad new. It's on your todo list ? or should I fix it ? (I don't know how for the moment...)
User avatar
F-JJTH
 
Posts: 697
Joined: Fri Sep 09, 2011 11:02 am

Re: Populate AI Traffic with real traffic

Postby Hooray » Sun Oct 28, 2012 2:58 pm

It's not a big thing, there are several solutions possible. The simplest would be using the "old" simgear headers in fgms for the existing source code and build a separate library, and then simply use the simgear headers in the fgais module. The better (more time consuming) option would be porting the old sources to also use the latest SG headers ...

The first solution would not take very long, but the fg_geometry sources would then use a different (older) version of the SG sources ... so we need to keep that in mind
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

Re: Populate AI Traffic with real traffic

Postby Hooray » Sun Oct 28, 2012 7:27 pm

Okay, for now, I have pushed an updated version (a hack really...) that uses the old SimGear sources and also supports SG 2.9 headers. Please check if it works for you.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

Re: Populate AI Traffic with real traffic

Postby F-JJTH » Sun Oct 28, 2012 7:47 pm

It seems to work : compilation work, FGMS run... SGVec3f is declared without problem. Thanks you !
User avatar
F-JJTH
 
Posts: 697
Joined: Fri Sep 09, 2011 11:02 am

Re: Populate AI Traffic with real traffic

Postby Hooray » Sun Oct 28, 2012 7:59 pm

Okay, thanks for checking.
But please keep in mind that we are basically now using two different versions of the headers, which really should NOT be mixed obviously ;-)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

Re: Populate AI Traffic with real traffic

Postby Thorsten » Mon Oct 29, 2012 8:10 am

Hooray pinged me and asked if I wanted to comment on two issues:

At the moment, there are two main discussions taking place, where some people favor interpolation over extrapolation - and the other one is the question if these computations should be done in the fgms main thread for each connected client (i.e. possibly redundantly) or simply done for the whole time span of 60 seconds in the background thread, so that the main thread could simply use the time offset as a lookup key.


I find my answer in the interpolation/extrapolation question straightforward:

Extrapolation of the current state will mean that planes 'jump' in their position regularly. Extrapolation requires you to make a guess about the future state of a plane which then needs to be updated if the actual state does not match the projected state. The simple need that airplanes should follow continuous trajectories and not teleport is imo a more important consideration than anything else.

Polly wrote:

I hope you will give extrapolation due consideration: the calculations are of identical complexity with extrapolation adding the computed delta to the latest data point ( interpolation would subtract ). Interpolation results will be guaranteed to always mismatch actuality whereas, especially since most flight paths are linear, extrapolation will be close to exact for much of the time.


Flightgear, even feeded with real world data, isn't a real time image of the world. Weather is a similar consideration - the weather is not extrapolated based on an assumption what the next METAR report will possibly say, it is interpolated such that once a new report comes in, the weather is smoothly changed to that state. Even with METAR on and unchanging, the sky Flightgear generates does usually not match the real sky at the location (and if it does, it does so by accident). Lagging behind the real world a minute is not a big issue as far as I am concerned.

Consider, too, the result of interpolating the path of a craft taking just less than one data period between flare and having turned off the runway onto a taxiway heading; extrapolation would at least get the on-runway path bearing correct.


Since the ground interaction is almost guaranteed to be problematic, I would remove this type of AI traffic once it falls below a certain altitude and never actually let it on the runway for the following reasons:

* If either the tracking accuracy or the runway location in Flightgear is off by just a few meters, planes will be seen to land right next to the runway.
* if either runway altitude or tracked altitude are off by a few meters, planes will be seen to sink into the ground or to hover above the runway
* any motion which changes substantially during the 60 seconds interval can't be updated accurately anyway
* real traffic will *not* respect the simulated pilot's presence, so on JFK with real traffic on, you would never get a clear runway

The big plus for extrapolation is for anyone intending to listen on e.g. liveatc.net, who will have a more accurate match between map and audio.


Well, that hardly offsets the teleportation of airplanes when the extrapolated position does not match the update of the real position, does it?

I'm a bit less sure about the second part of the question, since I don't know too much about how the MP protocol works.

Ideally, when asked to render planes of which the position is known every 60 seconds, I would cull the vector down to what is in my vicinity, sort that into a plane-centered quadtree and do updates for whatever is in my field of view on a per-frame basis client-side and update the rest at leisure. However, that's probably not how the MP server works, so...

Next step : update the "AI in area selector" and start to make some complex math (calculate velocity)[/quote[

Once you think about it, you can keep it simple and simply ALWAYS update positions for ALL aircraft.
Imagine a number of aircraft in the same place (i.e. 50+ multiplayer clients at KSFO) - if you always update the whole list, you only need to do the computations once if you only do it for each aircraft separately, then you will always need to re-do the calcuations.


I'm not sure that's true. You have 15.000 planes in the whole world. Even if 50 of them are currently in the vicinity of KSFO and need to be done 50 times, you still only have to compute 2500 position updates, i.e. you're a factor 6 better off.

Wouldn't be the best way to take the raw 15k entries vector and regularly sort this into a relevant vector which contains all the planes which can currently be seen by any connected client? That sorting step would be fairly trivial to do (you just set a boolean flag to the plane info hash once it's been identified to be used for one client and probe the flag before you copy the entry to the vector of interest). That way, you do all position updates you need to do and not more, i.e. in the above situation with 50 clients and 50 AI planes at KSFO, you'd do 50 operations instead of 2500 or 15.000.
Thorsten
 
Posts: 11470
Joined: Mon Nov 02, 2009 8:33 am

Re: Populate AI Traffic with real traffic

Postby Hooray » Mon Oct 29, 2012 12:33 pm

Thorsten wrote in Mon Oct 29, 2012 8:10 am:I'm not sure that's true. You have 15.000 planes in the whole world. Even if 50 of them are currently in the vicinity of KSFO and need to be done 50 times, you still only have to compute 2500 position updates, i.e. you're a factor 6 better off.

That's true, but most people won't be at a single place ... we only need clients to be at 6 different places 100 nm apart, and we'll have to do the same amount of work...

Wouldn't be the best way to take the raw 15k entries vector and regularly sort this into a relevant vector which contains all the planes which can currently be seen by any connected client?


This is sort of what's currently done for all aircraft within 100nm of the fgfs client in the fgms main thread. However, the background thread will currently compute positions for ALL aircraft for the whole 60 seconds for which we don't have any data.
Also, we don't currently have a client-specific vector with relevant aircraft, but traverse the whole vector/list (actually, a std::map) of aircraft.
I was hoping to use a spatial data structure to make this part more efficient. Stuart previously suggested to use a boolean vector instead.

That sorting step would be fairly trivial to do (you just set a boolean flag to the plane info hash once it's been identified to be used for one client and probe the flag before you copy the entry to the vector of interest). That way, you do all position updates you need to do and not more, i.e. in the above situation with 50 clients and 50 AI planes at KSFO, you'd do 50 operations instead of 2500 or 15.000.


Currently, the FGAIS background/worker thread is on average idle for about 50 seconds in between two requests spread over 60 seconds. That will probably be reduced to only ~30 seconds once everything works as expected, but that would still seem like plenty of leeway to run such calculations - the idle time is currently spent sleeping. Which is why the current idea is to do all computations in the background thread, and even create a lookup table with correct multiplayer packets for each time span (0..59)

On average, there are between 12k-15k aircraft in the world. Usually, ~50-100 expire within 60 seconds and another 40-100 per minute are created from scratch. At KSFO, there are usually ~200 active aircraft within a range of 100 nm at any time.

BTW: Ground interaction isn't only going to be "problematic": The data we have does NOT provide any useful ground positions... keep in mind that TCAS is based on MODE-S transponders that encode altitude. It will definitely require a fair amount of thinking to come up with plausible ground movements (gear animations, terrain altitude offset calculation etc)

Like I said earlier, I'm fine focusing on the airborne aspect of the whole issue. And even if were to make it as "real time" as possible, the data we get is definitely lagged behind for security reasons usually, which is an FAA requirement. The same applies to ship AIS information services available online (the free ones).
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

Re: Populate AI Traffic with real traffic

Postby Thorsten » Mon Oct 29, 2012 1:34 pm

That's true, but most people won't be at a single place ... we only need clients to be at 6 different places 100 nm apart, and we'll have to do the same amount of work...


If the clients are not at the same place, you're not doing redundant work :D 50 clients in 50 different places seeing 50 aircraft each means still a load of 2500 aircraft - for separated clients you never get more than the whole list.

The only way to get worse than to process the whole list is to have lots of redundant work by having e.g. 400 clients at KSFO seeing 50 planes in the vicinity - which gives you 20.000 operations.

Which is why the current idea is to do all computations in the background thread, and even create a lookup table with correct multiplayer packets for each time span (0..59)


So multiplayer is transmitted per second - how is it turned into per-frame client-side then?
Thorsten
 
Posts: 11470
Joined: Mon Nov 02, 2009 8:33 am

Re: Populate AI Traffic with real traffic

Postby Hooray » Mon Oct 29, 2012 1:49 pm

Sorry for my poor wording and for not being clear about it: Previously, the idea was to iterate through the list of all aircraft for each connected client and then dynamically create a new extrapolated position/orientation for the aircraft.

In other words, these computations would indeed have been done redundantly/unnecessarily. Which is why we're currently using a "simplifcation" where we always want to compute all positions for all aircraft, for the whole 0..59 timespan, i.e. a space/time trade-off.

The MP system uses the AI system to smoothly adjust aircraft position/orientation, which basically works like the replay system, i.e. using approximation with the data we have.

Basically, the problem is really that the MP protocol and the implementation in FG is much "simpler" than we want it to be. Otherwise, I'd agree that your quadtree approach would be good here. However, there's still a more fundamental problem: Having on average ~200 of these aircraft in the vicinity of each client, means having to do 15x200 = 3000 position updates per minute per client (IIRC the planned update interval was every 4 seconds?)

This is one of those cases where it would really make make sense to do all the position computations at the client side and merely provide "correction packets" to adjust the local/client-side extrapolation in fgfs, otherwise we'll be creating crazy amount of traffic unfortunately.

I'm afraid, that will require changes in the FG MP system - this is going to be pretty obvious once everything works. On average, most servers these days have upstream/downstream capabilities of 10 or 100 MBPS
But 3000 updates per minute per client is a lot given the size of each MP packet (500 bytes)

At the moment, I am thinking in terms of having a new subsystem in FG that handles computation of interpolated/extrapolated positions and which supports "correction" packets to be sent via the server whenever necessary, i.e. once the deviation exceeds a certain threshold. That would mean that we'd get to see much less bandwidth being used/required here.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

Re: Populate AI Traffic with real traffic

Postby Philosopher » Mon Oct 29, 2012 1:56 pm

I have another idea, how about keeping a server running for a while that continuously polls and monitors a couple flight routes (or a lot), and with enough runs we can build a graph of distance vs time vs altitude (since we would be polling the same flight route at enough different times during the route we could eventually have a rough idea of what it looks like for an "average" craft). From there we say "Oh, I have a plane on this route at this altitude and distance" and then move it along the route (which might have to be warped for each aircraft), as we would have gradually generated the route beforehand.

I dunno how far off topic this is, but I just thought I would throw it out there.
Thanks,
Philosopher
(inactive but lurking occasionally...)
Philosopher
 
Posts: 1590
Joined: Sun Aug 12, 2012 6:29 pm
Location: Stuck in my head...
Callsign: AFTI
Version: Git
OS: Mac OS X 10.7.5

Re: Populate AI Traffic with real traffic

Postby Hooray » Mon Oct 29, 2012 2:25 pm

Technically, that would be possible. But the question is if we can find enough contributors who have the corresponding set of skills to implement something like that.

Basically, fgms (the multiplayer server) hasn't been actively maintained/developed in quite a while. F-JJTH's work is the most promising change here, especially because he's working with the fgms code base and because he's becoming more and more familiar with it. And he's more planned changes. So we need to find people who are able to build C++ source code, and who are able to know C++ fairly quickly (like F-JJTH did) if they don't know C++ already ...

So, my thinking is that the fgms/fgais route is going to be our best bet here still. In fact, a "route monitor" like the one you suggest Philosopher, could be trivially implemented in another background task (worker thread) - so the person working on this, would not even need to know very much about fgms or even fgais, it would be a different background task, that's just gather data, possibly writing them to a GIS database.

I still have the idea of adding scripting support to fgms (via Nasal) to make such things easier - but for now, there are more important things to do. However, if anybody is interested in pursuing Philosopher's idea, I am sure that both, F-JJTH and myself can answer all FGAIS related questions

Obviously, the interesting thing is that such a route/traffic database could also be used offline, so it would be kinda cool

Again, building fgms is MUCH easier than building FG and all its dependencies, FGMS is fairly self-contained ... and anybody who already knows advanced Nasal or JavaScript, could obviously also learn C++ basics pretty quickly :-)
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

Re: Populate AI Traffic with real traffic

Postby Hooray » Mon Oct 29, 2012 2:28 pm

Thorsten wrote:Extrapolation of the current state will mean that planes 'jump' in their position regularly. Extrapolation requires you to make a guess about the future state of a plane which then needs to be updated if the actual state does not match the projected state. The simple need that airplanes should follow continuous trajectories and not teleport is imo a more important consideration than anything else.


This is going to remain an issue anyhow: Whenever "real" data comes in, the "projected" (computed) flight path will need to be updated, even though it has already been used for up to 60 seconds by then. So aircraft will definitely "teleport" for the time being ... Just imagine could significantly a flight path may change with 1 minute, some aircraft may change altitude by more than 2000 feet. And groundspeed can also change accordingly, especially while climbing/descending.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11605
Joined: Tue Mar 25, 2008 8:40 am

PreviousNext

Return to AI Traffic

Who is online

Users browsing this forum: No registered users and 1 guest