Board index FlightGear Support Tools FGCom

FGCom-mumble

FGCom is a realtime voice communication system specially designed for FlightGear.

FGCom-mumble

Postby benih » Fri Jun 05, 2020 7:20 am

Split off from from the topic FGCom vs. alternatives (and given the name FGCom-mumble from the readme file in the GitHub repository).

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The project currently lives on github: https://github.com/hbeni/fgcom-mumble/
State: It's working!

FGFS-Wiki page: http://wiki.flightgear.org/FGCom-mumble
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Mubmle lacked the features to support a plugin to enable radio comms for flightgear at the time the thread was started.
Also i was told that fgcom is unmaintained for the time beeing.

The mumble project currently has some plugin framework in the works, that would make this possible [1]; it will be part of the mumble 1.14 release.
In fact, two projects that are very widespread used and serve radio communication teamspeak plugins, ACRE and TFAR (for the MilSim Arma3), are supporting that effort.
The new mumble 1.14 plugin API looks promising.

If that would come to live, we would need something along the line of the following generic architecture:
  • Everyone that would participate principially on the radio will join a dedicated mumble host and join a specific channel.
  • each mumble client loads a specific fgfs-fgcom plugin.
  • that plugin need a very public documented API, so third party software can interact easily with it.
    Ideally this API is platform and language agnostic (some mini REST-Service maybe? Or some very simple TCP socket?) so it can be invoked from third party software (eg: ATC-Pie, OpenRadar, fgcom-standalone) easily.
  • fgfs communicates with this plugin (some arbitary) and provides the following information: position, number of radios in use, frequencies selected, volume of each radio, PTT on each radio pressed. This information is gathered from standard fgfs properties.
  • The plugin now does the following things:
    • calculates client side, based on that information, which clients it currently can hear (based on simple distance at first, but we may introduce arbitary complexity, like realistic radio wave propagation models there like ACRE does already - maybe even code can be used from their effort)
    • Calculate, which clients can be heared, based on the tuned frequencies and distance (and advanced modelling stuff like said above).
    • Clients that can be heared this way are unmuted and may be heared this way (his approach will save bandwith as i understand it, as the server will just send data to unmuted clients).
    • As soon as some client sends some voice data, the plugin intercepts the audio stream and does this stuff:
      - If the user is actively PTTing, the reception is muted (discard received data).
      - If he is not, the incoming voice stream is intercepted. Based on signal strength (->see radio wave model) volume of the voice is adjusted and some white noise introduced, to the extent that on the edge of the possible radio distance it is barely hearable and just nearly all white noise (note that this may already be doable with mumbles positional audio feature).
    When sending, there is just basic sending of the data of the microphone, and this is entirely dealt by from mumble itself. The other clients will receive the data and their plugins will deal with it like outlined above.

For easy transition we might adjust the current fgcom implementation to detect if such a plugin is loaded and then offering that as an alternate "server" in the dropdown menu. Then fgcom may just send the properties there.
The standalone fgcom may also send the data this way, so openradar/atc-pie does not need to be adjusted (r).
Alternatively the plugin itself may hook up the property tree of the fgfs instance since that is already accessible easily [2] and circumvent fgcom entirely (however openradar etc need to implement that interface too then.).

The Pro's of this approach would be the following:
  • All the voice stuff and networking problems are dealt by mumble.
  • Server setup is really easy: just install a murmur server and let people connect. No special server side setup required.
  • The way the plugin is designed above allows for arbitary 3rd party software to use it for radio simulation; it's not fgfs specific.
  • I think it is easy to implement, at least at the level of current fgcom features
  • It is easy to debug from a users perspective: If the user knows he can chat trough mumble, he has confidence that radios will work too.
  • Voice setup is done with mumble, so the user has full control over his input/output device configuration with advanced features from mumble
  • Unlimited frequency support. There are no constraints like now that the selected frequency has to be enabled in asterisk phone book: ANY frequency tuned by two pilots will work without having it to be defined anywhere hardcoded.

What do you think of this approach?
(Please note that sadly my c++ programming skills lack the means to implement most of this that stuff myself).

[1] https://github.com/mumble-voip/mumble/pull/3743
[2] http://wiki.flightgear.org/Property_tre ... perty_tree
Last edited by benih on Mon Oct 05, 2020 11:39 am, edited 6 times in total.
User avatar
benih
 
Posts: 253
Joined: Tue Aug 15, 2017 9:34 am
Callsign: D-EBHX

Re: FGCom vs. alternatives

Postby mickybadia » Fri Jun 05, 2020 1:41 pm

Hi,
I would be really excited if something like that set off to improve or even replace the radio system.
Question: I could not tell for sure if the stuff you propose still had to wait anyway or if you were planning this to get going soon... Are we still waiting for a Mumble plug-in feature or is it already out (albeit beta or whatever)?

A few ideas to keep on the radar scope, from the ATC perspective:
* ATCs may monitor and transmit on multiple (merged) frequencies at once, sometimes with different volumes
* source (position or bearing) and frequency of every transmission must be visible to ATCs (for the moment OK through MP properties but some sync needed there)
* recorded ATIS messages : allow to record on Mumble for automatic looping (like FGCom did)
* of course, allow for 8.33 channels :-)

Besides, I was looking into the Mumble interface too to implement voice landlines between ATCs, but this without a plug-in. I am not saying that to integrate everything in your FG plugin, because: (1) planes cannot call other planes offline with radio so useless in FG terms; and (2) ATC-pie would be needing it even for non-FG sessions. So I still see them as separate, but it is probably better to be aware of related plans before any design choices. For example it would be good if the same connection could be shared transparently.

I will be happy to participate in discussions if they happen and if it makes sense that I do. In any case don't worry about breaking ATC-pie's FGCom. Any new radio system will be interfaced and integrated.
mickybadia
 
Posts: 444
Joined: Tue Sep 24, 2013 9:12 am

Re: FGCom vs. alternatives

Postby benih » Fri Jun 05, 2020 2:34 pm

mickybadia wrote in Fri Jun 05, 2020 1:41 pm:Question: I could not tell for sure if the stuff you propose still had to wait anyway or if you were planning this to get going soon... Are we still waiting for a Mumble plug-in feature or is it already out (albeit beta or whatever)?

For us to start we probably need to wait until the plugin framework wents into stable mumble. However, as i understood, it is really far already and shuld be usable in mumbles dev branch. So theoretically we may start today. I just lack the knowledge to code this unfortunately.

mickybadia wrote in Fri Jun 05, 2020 1:41 pm:A few ideas to keep on the radar scope, from the ATC perspective:
* ATCs may monitor and transmit on multiple (merged) frequencies at once, sometimes with different volumes
* source (position or bearing) and frequency of every transmission must be visible to ATCs (for the moment OK through MP properties but some sync needed there)
* recorded ATIS messages : allow to record on Mumble for automatic looping (like FGCom did)
* of course, allow for 8.33 channels :-)

Good points! Especially the ATIS thing may prove not so easy but that should be solvable within the plugin. All that is needed is some central storage place where the messages get recorded to and replayed from the plugin if tuned on the frequency. Maybe that even is possible in the server instance RAM or something. We probably want to check into mumbles ticket tracker to ask for this feature....
The merged frequencies is just a matter of the plugin code mapping the PTT to several frequencies. That is just a matter of interpreting the data handed over to the plugin (if we send the active PTT in terms of an array with selected frequencies, for example)

mickybadia wrote in Fri Jun 05, 2020 1:41 pm:Besides, I was looking into the Mumble interface too to implement voice landlines between ATCs, but this without a plug-in. I am not saying that to integrate everything in your FG plugin, because: (1) planes cannot call other planes offline with radio so useless in FG terms; and (2) ATC-pie would be needing it even for non-FG sessions. So I still see them as separate, but it is probably better to be aware of related plans before any design choices. For example it would be good if the same connection could be shared transparently.

Good point, but isn't this already solvable via different channels in mumble itself out of the box? I think, mumble already has some feature so you can still monitor the radio channels (dual listening).

mickybadia wrote in Fri Jun 05, 2020 1:41 pm:I will be happy to participate in discussions if they happen and if it makes sense that I do. In any case don't worry about breaking ATC-pie's FGCom. Any new radio system will be interfaced and integrated.

I really do hope this gets some traction and we can get some developer into the project.
For the basic comms stuff as a proof-of-concept (being able to receive and PTT based on frequency selected, starting worldwide range, adding rnge calculation as second step) i think this is not all too much work. We even may fake the interface from fgfs and set the plugins stuff with a simple client manually for a simple POC we can build ontop later.
User avatar
benih
 
Posts: 253
Joined: Tue Aug 15, 2017 9:34 am
Callsign: D-EBHX

Re: FGCom vs. alternatives

Postby benih » Sat Jun 06, 2020 12:46 pm

For interfacing to the plugin we may use fgfs Standards mechanisms: the generic protocol!

http://wiki.flightgear.org/Generic_protocol

Seems to be relatively simple.
User avatar
benih
 
Posts: 253
Joined: Tue Aug 15, 2017 9:34 am
Callsign: D-EBHX

Re: FGCom vs. alternatives

Postby benih » Sat Jun 06, 2020 12:50 pm

... a completely different approach could also be to integrate a complete tailored custom mumble client.

That would be much more work i think but offer a one click solution like it is now in fgcom, because the user doesn’t have the hassle to install mumble and the plugin and open it before flight.
User avatar
benih
 
Posts: 253
Joined: Tue Aug 15, 2017 9:34 am
Callsign: D-EBHX

Re: FGCom vs. alternatives

Postby benih » Sat Jun 06, 2020 6:15 pm

With the plugin approach i would do atis the following way:
Atis recording should be done separately. We can control better who records etc. This may be done with a separate html5 app maybe.

We have two good options then for playback:
A) The plugin checks if there is an atis record for the tuned frequency (web service?) if so it downloads it to memory and playbacks it.
B) We implement a separate bot that chrcks for atis recordings and sends it to the channel. The plugin needs to get some bot position then to determine range and mute the bot. The bot should be governed by some management interface.

I think A) be easier but B) gives more flexibility in implementation method, so in case we can get the additional Position data in there, this may be easier. There seem to be a node.js library for implementing mumble bots, but there are other music playing bots out there. We really just need to implement a atis manager that spawns new bots playing the recording and shut them down again. With the client-name of that bot the bot manager may provide some simple web-api that gives the needet additional info for the plugin.
[edit: there seem to be a pretty easy lua library out there that might do this trick with not so much effort: https://github.com/bkacjios/lua-mumble

The cool stuff about both approaches is that we can record stuff for arbitrary frequencies, like fake radio shows :)
User avatar
benih
 
Posts: 253
Joined: Tue Aug 15, 2017 9:34 am
Callsign: D-EBHX

Re: FGCom vs. alternatives

Postby benih » Sat Jun 06, 2020 9:12 pm

The more i thik about this, the better it looks to make the system as modular as possible.
This enables us to swap implementations of features like ATIS stuff more easily, as long as we stay on the mumble server.

As far as i dug trough some bot code for the ATIS stuff now, it should be no problem (and be the best) when the client-local plugin queries some central (web?)service to get further information about the bots in place.
The central service may be the atis-bot manager, who dynamically spawns bots to play ATIS records when something was recorded (and discards them after some time). In the future we may even make a special bot that can record such ATIS messages directly trough mumble (if the clientplugin synchronizes local tuned frequencies back to the server - which will be done for interplugin comms, but this information is only available trough the mumble plugin, not for the pluginless bots.... Well see, its surely solvable).
The central ati-bot manage also should expose some simple query interface if there is some ATIS for a given station, so fgfs may get that info to be able to mute the in-sim generated ATIS message.

One problem i still have on the radar may be the bandwith. The concept relies on the local muting feature, to mute clients not in range and on the same tuned freqency. But that also means that the server will broadcast any message to all clients, the audio stream is discarded locally it seems.
Serverside muting will not work here, as that is global and affecting all.
With some few dozen clients this surely will be not much of a problem, except your local bandwith is thin. But im not so convinced this will scale very well with BIG (say 100 talking much, like ATIS bots? ->solvable by just creating bots when clients tuned the frequency and are in range) user counts.
However that may really be hypothetical since on a radio net, there should not be excessive talking. But something to keep in mind testing early.


I think this really looks promising. How do we proceed now?
User avatar
benih
 
Posts: 253
Joined: Tue Aug 15, 2017 9:34 am
Callsign: D-EBHX

Re: FGCom vs. alternatives

Postby WoodSTokk » Sun Jun 07, 2020 5:40 am

To save bandwith, all clients should provide a minimum data set.
This is location, how many radios are active and on what frequency they are set.
If one pilot press the PTT, this client sent a audiostream to the server with power output of his radio.
The server looks up all clients that are on the same frequency and calculate signal strength out of distance and power output.
The audiostream will be only sent to clients where the signal strength is above a certain value (maybe the client sent also the squelch setting?) with the signal strength and frequency as data. This way, the client know witch radio receive this audio and can mix noise into the audio based on the signal strength (the noise can be a user option to disable it).
The advantage is, that there can be hundreds client on the same frequency around the world, but the audiostream goes only to the stations that are in range.
If there are two or more pilots transmit audio, the server send only that audiostream with the highest signal strength to the receiving stations.
Maybe the server set also a interferece value based on the other autiostreams. With this value, the client can mix interferece noise into the audio (also based on client setting if the user want simulate interferece or not).

The core data for realistic radio is very small.
The client provide allways position and for every radio the tuned frequencies and squelch value.
If the user press PTT, the client send also the frequency, output power and the audio it self.

The server send out audiostreams only to the users in receiving range and only if they are not trasmitting with signal strength value and a interferece value.
If a user transmit a message, he/she will not receive on that frequency (receive or transmit, not both).

PS: maybe FG support two audio devices in the future. One for environment (cockpit sounds and speakers) and one for headphone.
This will push the audio realism to the next level and the audio panel in most aircrafts become functional.
WoodSTokk
 
Posts: 641
Joined: Tue Oct 17, 2017 2:30 pm
Location: Milky Way/Sol/Earth/Europe
Callsign: OE-WST
IRC name: WoodSTokk
Version: 2020.3.0
OS: Debian Buster

Re: FGCom vs. alternatives

Postby mickybadia » Sun Jun 07, 2020 6:42 am

About recording ATIS, I would rather we did NOT have one more external web-based service to run. There are already a few, and every one added is one more thing that must be maintained and can go down.
If the new Mumble service does not implement the recording and broadcasting loop-over, I suppose the ATC client could do it. But first I find hard to believe that a Mumble plug-in is unable to save an audio piece for playback, then we would have to accept that for ATIS we would be worsening the FGCom experience instead of improving it (FGCom did allow to record ATIS).

WoodSTokk wrote in Sun Jun 07, 2020 5:40 am:The audiostream will be only sent to clients [...] with the signal strength and frequency as data.

...and direction of the signal from the receiving aircraft, please, otherwise RDF is impossible. Alternatively, the callsign or position of the transmitting aircraft but less realistic. (Callsign would allow others to filter out audio trolls in sessions though, which I suspect can be useful).
mickybadia
 
Posts: 444
Joined: Tue Sep 24, 2013 9:12 am

Re: FGCom vs. alternatives

Postby WoodSTokk » Sun Jun 07, 2020 7:04 am

mickybadia wrote in Sun Jun 07, 2020 6:42 am:
WoodSTokk wrote in Sun Jun 07, 2020 5:40 am:The audiostream will be only sent to clients [...] with the signal strength and frequency as data.

...and direction of the signal from the receiving aircraft, please, otherwise RDF is impossible. Alternatively, the callsign or position of the transmitting aircraft but less realistic. (Callsign would allow others to filter out audio trolls in sessions though, which I suspect can be useful).


Oh, yes, good idea, callsign (or session-ID for filterning) and direction for RDF. Sorry, i have missed the possibility of RDF.

EDIT: to save bandwith, it is better to send the filterlist to the server, so that the server doesn't send out a stream that the user will never hear.
Also the trasmission of the ignored user doesn't influence the interferece value.
WoodSTokk
 
Posts: 641
Joined: Tue Oct 17, 2017 2:30 pm
Location: Milky Way/Sol/Earth/Europe
Callsign: OE-WST
IRC name: WoodSTokk
Version: 2020.3.0
OS: Debian Buster

Re: FGCom vs. alternatives

Postby benih » Sun Jun 07, 2020 10:36 am

Well, the idea was with mumble to handle it all client side, which is not how you described the pieces above.

That could surely be done but would involve writing a custom mumble server component. This seem to be alot of work though, why i firstly tough of implementing the features mostly with stock stuff mostly already available. We can always go further from there, but of course if we find a willing developer only, and i suspect that’s our main problem.

Btw got an timeline for the plugin framework: it’s basicly finished. „The plugin framework will definitely be part of the next major release (1.4.0).“
User avatar
benih
 
Posts: 253
Joined: Tue Aug 15, 2017 9:34 am
Callsign: D-EBHX

Re: FGCom vs. alternatives

Postby benih » Sun Jun 07, 2020 11:30 am

Got an idea u want to share: the adf Radio auf could be handled easily from the proposed plugin: It just needs to provide some data back to the fgfs instance, which is easy using the props io interface (telnet)
User avatar
benih
 
Posts: 253
Joined: Tue Aug 15, 2017 9:34 am
Callsign: D-EBHX

Re: FGCom vs. alternatives

Postby benih » Wed Jun 10, 2020 2:57 pm

mickybadia wrote in Sun Jun 07, 2020 6:42 am:About recording ATIS, I would rather we did NOT have one more external web-based service to run.

I wanted to add to that: I think there may still be value to this. Suppose someone is spamming the ATIS frequencies with garbage (or worse: bad stuff!).
Either there is some admin online monitoring ALL the mumble transmissions, or some decentralized mechanism.

The latter may be easily implemented: Just have some web page online (like the current one, btw) where people can downvote an ATIS service. If it has enough downvotes, the system just kills that specific ATIS brodcast. This way, the community currently online can remedy.
It may however also be easy to make the mumble server ACLs such, that all users have permission to kick ATIS-users from the server. This is probably more easy to implement (but easier to abuse, because the proposed voting system may be more self-regulatory).
User avatar
benih
 
Posts: 253
Joined: Tue Aug 15, 2017 9:34 am
Callsign: D-EBHX

Re: FGCom vs. alternatives

Postby benih » Mon Jun 15, 2020 12:14 pm

I did some more thinking and now came up with a (in my opinion) doable concept.
To faciliate development of an PoC, i made a github repository: https://github.com/hbeni/fgcom-mumble

Currently there is the draft for the plugin specification and also a first issue ticket that may be used to coordinate development work.
From what i have read, we may already start experimenting with this - the new mumble plugin stuff shopuld compile on cleient and server, so we may be ready to have a go.
All we need is a capable developer now.
User avatar
benih
 
Posts: 253
Joined: Tue Aug 15, 2017 9:34 am
Callsign: D-EBHX

Re: FGCom vs. alternatives

Postby tdammers » Wed Jun 17, 2020 8:12 am

Some more context about the situation: Now that flying FG on VATSIM is easily possible, I fear that any such effort will have trouble gaining and maintaining momentum, because Swift (the VATSIM client that supports FG) already gives you excellent radio comm simulations, and they're getting better still. And it seems that those who want realistic ATC simulation are a rather large part of the target audience for voice radio, and many of them (myself included) are flying on VATSIM instead of trying to get better voice comm tech into FG.
tdammers
 
Posts: 347
Joined: Wed Dec 13, 2017 10:35 am
Callsign: NL256
IRC name: nl256

Next

Return to FGCom

Who is online

Users browsing this forum: No registered users and 1 guest