Board index FlightGear Support Multiplayer

Multiplay Protocol Lag Offset

Trouble getting online, setting up a server?
Forum rules
In order to help you, we need to know a lot of information. Make sure to include answers to at least the following questions in your initial post.

- what OS (Windows Xp/Vista, Mac etc.) are you running?
- what FlightGear version do you use?
- copy&paste your commandline.

Please, also see Requesting Technical Help.

Note: If you did not get a reponse, even after 7 days, you may want to check out the FlightGear mailing lists to ask your question there.

Multiplay Protocol Lag Offset

Postby caged_bird » Fri Feb 05, 2021 4:37 pm

Hey FlightGear community!

I'm an intern for the Air Force Research Labs and am new to FlightGear. I'm working on using FG as a visualization tool and have been writing a python script that can create and send Multiplayer Protocol UDP packets to a FG instance.

I've formatted a custom packet based off of outgoing MP packets from a running FG instance and this wiki page (http://wiki.flightgear.org/Multiplayer_protocol). However, I still cannot get the plane to render where I want in a running FG instance. I'm reaching out with this post to try and find information on the time offset (titled 'lag' in the table for the first part of a positional message described by the wiki) bytes used the the packet.

So far I've tried a lag value of 0x0000000000000000 and 0x43cfdccccccccccd. The former as a dummy var and the latter was copied from the example packet I had. I got these DEBUG messages from my terminal running FG, respectively:

for 0x0000000000000000:
Code: Select all
24.10 [DBUG]:ai         Offset adjust system: time offset = 0.000734691, expected longitudinal position error due to  current adjustment of the offset: 3.54107e-05
   24.10 [DBUG]:ai         Multiplayer vehicle extrapolation: extrapolation time = 0.00330108
   24.10 [DBUG]:ai         Multiplayer position and orientation: [ -2.07581, 0.624261, 969.361 ], [ 0.509965, -0.0750155, -0.847793, -0.124724 ]
   24.10 [DBUG]:atc        Currently at leg : 3
   24.10 [DBUG]:atc        name of previous waypoint : gear-up
   24.10 [DBUG]:atc        Currently under control of BIKF-tower
   24.10 [DBUG]:network    Found Fallback model index in message 516
   24.10 [DBUG]:ai         Offset adjust system: time offset = 0.000733975, expected longitudinal position error due to  current adjustment of the offset: 9.20057e-06
   24.10 [DBUG]:ai         Multiplayer vehicle extrapolation: extrapolation time = 0.000857701
   24.10 [DBUG]:ai         Multiplayer position and orientation: [ -2.06108, 0.624261, 969.333 ], [ 0.50989, -0.0755177, -0.84767, -0.125559 ]
   24.11 [DBUG]:atc        Currently at leg : 3
   24.11 [DBUG]:atc        name of previous waypoint : gear-up
   24.11 [DBUG]:atc        Currently under control of BIKF-tower
   24.11 [DBUG]:ai         Offset adjust system: time offset = 0.000726316, expected longitudinal position error due to  current adjustment of the offset: 9.85101e-05
   24.11 [DBUG]:ai         Multiplayer vehicle extrapolation: extrapolation time = 0.00918338
   24.11 [DBUG]:ai         Multiplayer position and orientation: [ -2.11127, 0.624261, 969.428 ], [ 0.510138, -0.0738327, -0.84808, -0.122757 ]
   24.11 [DBUG]:atc        Currently at leg : 3
   24.11 [DBUG]:atc        name of previous waypoint : gear-up
   24.11 [DBUG]:atc        Currently under control of BIKF-tower
   24.11 [DBUG]:network    Found Fallback model index in message 516
   24.12 [DBUG]:terrain    SGMaterialCache::~SGMaterialCache() size=308
   24.12 [DBUG]:atc        Currently at leg : 3
   24.12 [DBUG]:atc        name of previous waypoint : gear-up
   24.12 [DBUG]:atc        Currently under control of BIKF-tower
   24.12 [DBUG]:network    Found Fallback model index in message 516
   24.12 [DBUG]:ai         Offset adjust system: time offset = 0.000726385, expected longitudinal position error due to  current adjustment of the offset: 8.90902e-07
   24.12 [DBUG]:ai         Multiplayer vehicle interpolation: [57972.6, 57972.6], intervalLen = 0.010808, interpolation parameter = 0.622759
   24.12 [DBUG]:ai         Multiplayer position and orientation: [ -2.05591, 0.624261, 969.323 ], [ 0.509864, -0.0756955, -0.847626, -0.125854 ]
   24.13 [DBUG]:io           Success reading data.
   24.13 [DBUG]:io           Success reading data.
   24.13 [DBUG]:io           Success reading data.
   24.13 [DBUG]:atc        Currently at leg : 3
   24.13 [DBUG]:atc        name of previous waypoint : gear-up
   24.13 [DBUG]:atc        Currently under control of BIKF-tower
   24.13 [DBUG]:ai         Offset adjust system: time offset = 0.000722838, expected longitudinal position error due to  current adjustment of the offset: 4.56173e-05
   24.13 [DBUG]:ai         Multiplayer vehicle extrapolation: extrapolation time = 0.00425256
   24.13 [DBUG]:ai         Multiplayer position and orientation: [ -2.08154, 0.624261, 969.372 ], [ 0.509993, -0.0748217, -0.84784, -0.124402 ]


and 0x43cfdccccccccccd:
Code: Select all
   27.75 [DBUG]:ai         Offset adjust system: time offset = -4.59187e+18, expected longitudinal position error due to  current adjustment of the offset: 0


Major questions I have at the moment are:
a) how significant is the positional adjustment caused by time offset? Some of the offset values seem very small... could it be the bug preventing the plane form being rendered or should I focus elsewhere?
b) why does time offset seem to vary in the Debug output when I set lag = 0x0000000000000000?
c) what is extrapolation time? what is 'current adjustment of the offset'? And how are they calculated?

Tech Stack:
OS - Linux Mint 5.4.0-64
FG - 2020.3.1

Any help or advice would be greatly appreciated!

Regards,
John
caged_bird
 
Posts: 7
Joined: Fri Jan 22, 2021 10:12 pm

Re: Multiplay Protocol Lag Offset

Postby amue » Sun Feb 07, 2021 2:37 pm

caged_bird wrote in Fri Feb 05, 2021 4:37 pm:Major questions I have at the moment are:
a) how significant is the positional adjustment caused by time offset? Some of the offset values seem very small... could it be the bug preventing the plane form being rendered or should I focus elsewhere?
b) why does time offset seem to vary in the Debug output when I set lag = 0x0000000000000000?
c) what is extrapolation time? what is 'current adjustment of the offset'? And how are they calculated?

The lag field in the mp packet is IMHO a little bit misnamed. It actually contains the time spacing (delta time) between mp packets. E.g if the packets are sent at a rate of 10 Hz (FG default) the sender writes 0.1 (seconds) into the lag field .
The lag field is used by the receiver to compute a time offset to determine the (packet) time at which the mp aircraft is displayed. The time offset calculation tries to yield an offset so that the (packet) time at which the mp aircraft is displayed is always before the last received mp packet and the mp aircraft position/velocity can be nicely interpolated to avoid jittery movement (but with the disadvantage, that the aircraft is displayed a little bit in the past).
If for whatever reasons (e.g. mp lag field = 0.0 or using a user defined offset) the computed time for displaying the mp aircraft is after (i.e. in the future of) the last received mp packet, then the mp aircraft position must be extrapolated (and you see extrapolation messages in the debug output).

Andi
amue
 
Posts: 66
Joined: Tue Apr 03, 2018 9:13 am

Re: Multiplay Protocol Lag Offset

Postby jano » Mon Feb 08, 2021 7:26 am

the lag field is only used if you don't compensate for the lag as explained above, and can't be responsible for plane not appearing, and the log suggest the plane's position is updating.

You should first be sure that mp aircraft are visible normally, eg by using 127.O.O.1 as mp server, you'll see yourself as mp plane, there are settings for the mp planes display in one of the view menu where you can choose pixel size or distance at which plane appear, and bad settings here provide no visual mp planes.

Check with multiplayer=>pilot list that the m plane with the correct callsign is present, or find it in the property tree (ai/models/multiplayer[i]) where you can check the model path for exemple .
jano
 
Posts: 199
Joined: Thu Nov 29, 2007 11:32 pm
Location: france
Callsign: jano
Version: git
OS: debian SID

Re: Multiplay Protocol Lag Offset

Postby caged_bird » Fri Feb 12, 2021 6:41 pm

Thanks for the interest in my project! The clarification on lag offset was very helpful and actually solved another issue I was having with my visualization!

So, it turns out that my position data was horribly off. MP protocol takes position and orientation data wrt an Earth Centered / Earth Fixed coordinate system. The data I'm trying to visualize encodes position values in a geodetic, relative coordinate system. I used Pymap3d's enu2ecef() function to convert the values properly and viola! The plane rendered right where I needed it to. And, after fixing my lag offset value to the packet sending rate, the plane would move along the desired path too!

The big issue right now for me is orienting the plane properly. I'm very new to aerospace reference frames / coordinate systems and multivar calc was years ago... I'm having trouble discerning how to translate rotations from ENU into the proper rotations for an Earth-Fixed frame. TBH I'm having difficulties even getting the frames to line up (before applying any of the rotations demanded by the sim data).

Big questions at the moment:
1) What are the units for the x,y,z orientation values in the MP Protocol? I seem to get different results when entering data in degrees vs radians, but it's not entirely obvious just from visually looking at it which is correct. That is to say adding a +90 to the x axis looks visually identical to adding +(math.pi / 2). Yet, if I apply +90 to ALL axis, it looks way different from +(math.pi / 2).

2) Why are there 4 position values in the DBUG output of FG? There are only 3 axis... example output below:

789.70 [DBUG]:ai Multiplayer position and orientation: [ -2.42576e+06, -4.58151e+06, 3.70492e+06 ], [ -1.78814e-07, -5.96046e-08, 0.684848, -0.728686 ]

3) Could anyone point me in the direction of some good resources on coordinate system conversions? I have a feeling I'm under-educated here and am looking to read up on angle-axis representation for rotation, earth-fixed and ENU coord sys and how to convert rotations between the two, and/or where in the src code these rotations are handled.

Thank you all for your interest and time! Any help would be greatly appreciated :D
Last edited by caged_bird on Mon Feb 15, 2021 3:23 pm, edited 1 time in total.
caged_bird
 
Posts: 7
Joined: Fri Jan 22, 2021 10:12 pm

Re: Multiplay Protocol Lag Offset

Postby cgdae » Fri Feb 12, 2021 9:00 pm

[Caveat: i know just about enough maths to be able to copy and paste existing coordinate-transforming code into new code, but not enough to actually understand what's going on in any detail, so might be wrong about things below.]

I'm assuming below that you have a checkout of the main flightgear repository. I think the basic vector and quaternion types and methods are defined in simgear so you might need a checkout of the simgear repository too.

caged_bird wrote in Fri Feb 12, 2021 6:41 pm:Thanks for the interest in my project! The clarification on lag offset was very helpful and actually solved another issue I was having with my visualization!

So, it turns out that my position data was horribly off. MP protocol takes position and orientation data wrt an Earth Centered / Earth Fixed coordinate system. The data I'm trying to visualize encodes position values in a geodetic, relative coordinate system. I used Pymap3d's enu2ecef() function to convert the values properly and viola! The plane rendered right where I needed it to. And, after fixing my lag offset value to the packet sending rate, the plane would move along the desired path too!

The big issue right now for me is orienting the plane properly. I'm very new to aerospace reference frames / coordinate systems and multivar calc was years ago... I'm having trouble discerning how to translate rotations from ENU into the proper rotations for an Earth-Fixed frame. TBH I'm having difficulties even getting the frames to line up (before applying any of the rotations demanded by the sim data).

Big questions at the moment:
1) What are the units for the x,y,z orientation values in the MP Protocol? I seem to get different results when entering data in degrees vs radians, but it's not entirely obvious just from visually looking at it which is correct. That is to say adding a +90 to the x axis looks visually identical to adding +(math.pi / 2). Yet, if I apply +90 to ALL axis, it looks way different from +(math.pi).

2) Why are there 4 position values in the DBUG output of FG? There are only 3 axis... example output below:

789.70 [DBUG]:ai Multiplayer position and orientation: [ -2.42576e+06, -4.58151e+06, 3.70492e+06 ], [ -1.78814e-07, -5.96046e-08, 0.684848, -0.728686 ]


I guess this is because representing an orientation with three values such as (heading, pitch, roll) runs into singularity problems when pitch is 90deg; so it's better to use four values in a quaternion, even though there are only three degrees of freedom.

The multiplayer packet position and orientation is in flightgear:src/MultiPlayer/mpmessages.hxx:FGExternalMotionData:

Code: Select all
  // position wrt the earth centered frame
  SGVec3d position;
  // orientation wrt the earth centered frame
  SGQuatf orientation;


It looks like flightgear:src/AIModel/AIMultiplayer.cxx:FGAIMultiplayer::update() converts the incoming raw multiplayer position and orientation data into useful values. There's a lot of code in that function that seems to be concerned with smoothing/interpolating the incoming data, but near the end there is this code which seems like it might be worth looking at:

Code: Select all
  // The quaternion rotating from the earth centered frame to the
  // horizontal local frame
  SGQuatf qEc2Hl = SGQuatf::fromLonLatRad((float)pos.getLongitudeRad(),
                                          (float)pos.getLatitudeRad());
  // The orientation wrt the horizontal local frame
  SGQuatf hlOr = conj(qEc2Hl)*ecOrient;
  float hDeg, pDeg, rDeg;
  hlOr.getEulerDeg(hDeg, pDeg, rDeg);
  hdg = hDeg;
  roll = rDeg;
  pitch = pDeg;

Hopefully it might be possible to trace through where the raw values in the FGExternalMotionData make it into the inputs to the above code.

caged_bird wrote in Fri Feb 12, 2021 6:41 pm:
3) Could anyone point me in the direction of some good resources on coordinate system conversions? I have a feeling I'm under-educated here and am looking to read up on angle-axis representation for rotation, earth-fixed and ENU coord sys and how to convert rotations between the two, and/or where in the src code these rotations are handled.

Thank you all for your interest and time! Any help would be greatly appreciated :D


There are some methods on the basic vector and quaternion types in simgear, which the code above make use of.

There is also some code in the view system that manipulates viewing angles etc, which might be helpful. See: flightgear:src/Viewer/view.cxx:View::recalcLookFrom() and View::recalcLookAt.

The same kind of calculations goes on in the new Step view system - see flightgear:src/Viewer/sview.cxx, so maybe take a look there too. [The basic transformations were copied verbatim from the view.cxx code.]

Hope that all makes sense and is of some use.

- Jules
cgdae
 
Posts: 62
Joined: Tue May 31, 2016 7:35 pm

Re: Multiplay Protocol Lag Offset

Postby amue » Sun Feb 14, 2021 10:48 am

caged_bird wrote in Fri Feb 12, 2021 6:41 pm:1) What are the units for the x,y,z orientation values in the MP Protocol? I seem to get different results when entering data in degrees vs radians, but it's not entirely obvious just from visually looking at it which is correct. That is to say adding a +90 to the x axis looks visually identical to adding +(math.pi / 2). Yet, if I apply +90 to ALL axis, it looks way different from +(math.pi).

The orientation is defined in angle axis representation and not in euler angles.
The comment in the source header file (https://sourceforge.net/p/flightgear/flightgear/ci/next/tree/src/MultiPlayer/mpmessages.hxx) says:
"orientation wrt the earth centered frame, stored in the angle axis representation where the angle is coded into the axis length".
BTW, just in case you also use velocity and acceleration, the comments in the header file about velocity and acceleration are wrong. Velocity and acceleration are in aircraft local frame and not in ECEF frame.

2) Why are there 4 position values in the DBUG output of FG? There are only 3 axis... example output below:

789.70 [DBUG]:ai Multiplayer position and orientation: [ -2.42576e+06, -4.58151e+06, 3.70492e+06 ], [ -1.78814e-07, -5.96046e-08, 0.684848, -0.728686 ]

The 4 values are the quarternion representation of the rotation.
amue
 
Posts: 66
Joined: Tue Apr 03, 2018 9:13 am

Re: Multiplay Protocol Lag Offset

Postby caged_bird » Tue Feb 23, 2021 3:14 pm

BTW, just in case you also use velocity and acceleration, the comments in the header file about velocity and acceleration are wrong. Velocity and acceleration are in aircraft local frame and not in ECEF frame.


By "local aircraft frame" do you mean something like ENU or DEN? Or is it just wrt to the current aircraft position?
I get the velocity (in feet) from my data as vcas. Could I simply put that in the VelX field of the MP packet? Or do I need to do some trig to get the East and North components of said vcas velocity?


Thank again for all the help everyone! I gave myself a crash course on quaternions and was able to get the plane oriented properly by multiplying two quaternion rotations together. :)
Now I'm working on reducing the classic "jitter" problem. I've already reduced the rate FG expects FDM and MP packets via launch configs:
Code: Select all
fgfs --fdm=null --native-fdm=socket,in,10,,5505,udp --disable-sound --aircraft=f16-block-20 --generic=socket,in,30,,5506,udp,f16 --multiplay=out,10,127.0.0.1,5000 --multiplay=in,10,,5001 --callsign=plz --log-level=debug

The important bits here being the --native-fdm and --multiplay params. And I used wireshark to analyze the packet sending rate of my python script (which was pretty accurate). These changes helped a little, but I'm no where near a useable animation.

Currently I'm investigating the velocities sent in the MP packet to make sure FG has good data to interpolate off of. Next, I'm going to try fiddling with the overall frame rate of FG (if anyone has suggestions on how to do that, im all ears :) ). If that doesn't work, I'm going to look into FG src code.
caged_bird
 
Posts: 7
Joined: Fri Jan 22, 2021 10:12 pm

Re: Multiplay Protocol Lag Offset

Postby caged_bird » Tue Feb 23, 2021 5:04 pm

I wanted to give a quick update with some good news :)

I was re-reading through this forum (viewtopic.php?f=27&t=38421) and took some suggestions from jano and Gijs' posts.

I throttled the frame rate of FG to match the size of timesteps of my data using the launch param --prop:/sim/frame-rate-throttle-hz. This completely eliminated the "jitter" or rubber-banding of the MP plane!

There was still a good degree of lag, however. So I went and manually tampered with the lag adjustment slider (Multiplayer -> Lag settings: master switch on, apply to close mp on) and it worked like a charm. No lag, no jitter. :mrgreen:

I have a few follow up questions that would help me in building out my script into an easy to launch program:
- Is there a way to set the Lag settings procedurally? Perhaps through a FG launch parameter or something? Or can I only access that manually once FG is running?
- where can I find info on how the lag adjustment value is used? I just used the deafult (0.23 sec) and it worked perfectly, but I imagine this will not be the case for all timestep sizes and frame rates, which is likely to change in my data.

Thank you all for your insights and engagement. I think I owe ya'll some beers :)
caged_bird
 
Posts: 7
Joined: Fri Jan 22, 2021 10:12 pm

Re: Multiplay Protocol Lag Offset

Postby jano » Tue Feb 23, 2021 8:07 pm

Hi, the lag parameters are recorded and reused the next session, so tuning them once is enough if that's only the same FG you're running.
to be sure, you can set them in command line, or in the launcher, the props to use are:

Code: Select all
--prop:/sim/multiplay/lag/master=1
--prop:/sim/multiplay/lag/offset=0.06
--prop:/sim/multiplay/lag/apply-close=1


For the lag offset value, anything positif is fine iirc, because you're in a very particular case :)
this lag offset is used to set how late or in the future we display the mp plane, when late, we simply do interpolation between previous received packets, this make regular mp planes smooth, but delayed in time.
when in the future, we use the last packet, and the plane's velocity to guess the plane's position. In your case (no velocity given in the mp packet) it's the same as having a plane immobile for predicted positions. so any lag offset value will give the same plane's position as long as you force it to be displayed in the future!

that would be something bad for a normal fg session :) but running with an external fdm make things differents! if you can maintain a good synch between fg frames, mp and net external input, you'll have simply a serie of "frozen" states, if each fg frame have a new plane's position, and mp plane's position both corresponding to the same "time", you're good, you'll have jitter if FG catch only one update having one plane immobile and the other going to the next position.

Notice that the lag offset value was the old way to address lag, we introduced last year a "real time" mode, using the pc wall clock as reference and displaying other planes in the same "time" , for mp planes obviously out of synch, we revert back to the lag offset system.
managing to have this approach with an external fdm would need a bit of work :) like:
- having a net fdm transmit a time stamp
- having FG able to use this time stamp in external fdm as reference

this (and plane's velocities) would be mandatory if you wanted to see the external driven plane and the mp plane in an other FG session, keeping a perfect synch.

Finally, you must have a good machine to drive FG if the result is smooth with FG other than next, throttle-hz slowly (or not slowly if fps are not achievable :P) drift relative to a time grid for those versions, the patch introduced in next allow a better respect of the running fps, but the net fdm protocol changed too so may be incompatible with your setup.

hope this help, and ok for a beer !
jano
 
Posts: 199
Joined: Thu Nov 29, 2007 11:32 pm
Location: france
Callsign: jano
Version: git
OS: debian SID

Re: Multiplay Protocol Lag Offset

Postby jano » Thu Feb 25, 2021 2:02 pm

You can have a look at the pilot list, lag page, if the absolute lag-ms value for the mp plane is under 200ms (or maybe 250 not sure ) lag adjustement is done using the clock offset field in the lag menu and disregard the slider value.

if you are above, we use the slider with the older lag compensation system, but this is less precise, mainly because we don't know when in the frame arrived the last packet.

if you want to force the real time mode, simply adjust the clock offset to have a mp plane lag value in the 200ms range.

the way to have something working all the time as expected, is to use the wall clock as timestamp in your script.

About the velocity, it's the ECEF velocity, expressed in body frame (u,v,w) ie the 3 plane's axis.

jano
jano
 
Posts: 199
Joined: Thu Nov 29, 2007 11:32 pm
Location: france
Callsign: jano
Version: git
OS: debian SID


Return to Multiplayer

Who is online

Users browsing this forum: No registered users and 1 guest