Board index FlightGear Development Scenery

Updated navdata without navigraph

Questions and discussion about enhancing and populating the FlightGear world.

Updated navdata without navigraph

Postby SkyClan480 » Wed Jun 09, 2021 4:51 pm

Another user who chooses to go by the name of "TOASTER" and I have written a parser in python with the purpose of parsing through the text files contained in NASR subscriptions freely available from the FAA and writing the necessary data into dat files that flightgear can use. We've uploaded the code to a github repository, and I'm looking to tell other users about it.
This actually came about because I was looking for updated navdata and didn't want to pay for navigraph. I inquired on the FlightGear discord server, and TOASTER was kind enough to let me use a python script he/she had written for parsing fixes. After that I went on to try and write one for navaids, and TOASTER ended up joining me and providing a great deal of assistance since I didn't know python very well. We ended up with 3 python scripts, the original one for fixes, one for navaids, and one for airways. This aims to be an easy method to get up to date navdata for free, without paying for a navigraph subscription.
The parser project is available on github:
https://github.com/SkyClan480/Flightgear-Navdata-Parser
SkyClan480
 
Posts: 12
Joined: Wed Jun 09, 2021 4:42 pm
Version: 2020.3.12
OS: Windows 11 Home

Re: Updated navdata without navigraph

Postby merspieler » Wed Jun 09, 2021 5:16 pm

Hey, welcome to the forum.

Does this cover gobal data or just for the US (since it's from the FAA)?
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: Updated navdata without navigraph

Postby Hooray » Wed Jun 09, 2021 8:40 pm

it would be interesting to turn this into a web based project, so that people can annotate such imported navdata - we could then also provide a web API, for FlightGear to download public domain navdata more easily, without people requiring navigraph

In meantime, feel free to add your news to the newsletter/wiki respectively
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Updated navdata without navigraph

Postby merspieler » Wed Jun 09, 2021 9:07 pm

I've taken a look at to download... it's indees US only... so not a viable solution, if you fly anywhere else.

Looking at the code, it looks more like it was written by a python guru, rather than someone with limited experience in it.

Some recommendations:
* Add commandline args to point to the files/unpacked directory. This way you keep the project directory clean and you won't accidentally add any files to the git repo.
* Add an output commandline arg and create the required directory structure for the user. (For those users, who don't know exactly how to do it)

Hooray: There is already such a project... openAIP (map) ... all we need would be an export to the FG format.
Thisway any changes would go into openAIP and we just need to re-export the data.
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: Updated navdata without navigraph

Postby SkyClan480 » Sat Jun 12, 2021 4:44 am

presently only the US is covered, since the US one is one of the only countries with freely available data. the project may look to expand to other countries in the future.

the code looks like it was written by a python guru because TOASTER helped me out a lot as well as writing the fix and awy scripts. i wrote most of the nav script with much assistance from TOASTER

i think the commandline args are a nice idea, but as you said, there are those people who don't quite know how to do it, and i consider that too. the goal is to make it as simple as possible for the average user, and putting all the files in one folder is generally pretty simple, more so than having to type out a path that many people may not even know how to find. however, the automatic folder structuring is a good idea and i will probably implement that as it does work to make things simpler for the user

this project is just that, a project. it is by no means complete, but its being worked on to be made as close to complete as possible. the main goal here is to provide a free alternative to navigraph for flightgear.
SkyClan480
 
Posts: 12
Joined: Wed Jun 09, 2021 4:42 pm
Version: 2020.3.12
OS: Windows 11 Home

Re: Updated navdata without navigraph

Postby merspieler » Sat Jun 12, 2021 10:40 am

SkyClan480 wrote in Sat Jun 12, 2021 4:44 am:presently only the US is covered, since the US one is one of the only countries with freely available data.


Many other countries have this information available. Just not in such an easily parsable format (most offer .pdf files)

putting all the files in one folder is generally pretty simple, more so than having to type out a path that many people may not even know how to find.


That's the point... they can use any path... nothing they need to find...
And as long as you're documentation is good, it shouldn't be an issue for them anyway.
For an excelent example of good documentation look at the Documentation for the Julia photoscenery generator

this project is just that, a project. it is by no means complete,


Flightgear always is...

but its being worked on to be made as close to complete as possible. the main goal here is to provide a free alternative to navigraph for flightgear.


For that you certainly need more coverage that the US...

If you're interested in the idea of using openAIP for this, and might want some help, let me know.
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: Updated navdata without navigraph

Postby Johan G » Sat Jun 12, 2021 11:23 am

SkyClan480 wrote in Sat Jun 12, 2021 4:44 am:presently only the US is covered, since the US one is one of the only countries with freely available data.

merspieler wrote in Sat Jun 12, 2021 10:40 am:Many other countries have this information available. Just not in such an easily parsable format (most offer .pdf files)

I would actually argue the data is freely available for most countries. It is just that US is one of the few countries where the copyright legislation have exceptions that per default put works by the government in the public domain, so they can be freely reused. :wink: Intellectual property legislation can be a jungle.
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Some YouTube videos
Johan G
Moderator
 
Posts: 6629
Joined: Fri Aug 06, 2010 6:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 2020.3.4
OS: Windows 10, 64 bit

Re: Updated navdata without navigraph

Postby merspieler » Sat Jun 12, 2021 11:27 am

For every country I've been seriously flying from/to, I've found eAIPs online... permission to recompile them is a different thing tho.

One thing that I still see missing, as well for the US, are procedures (SID/STAR). I couldn't see them in the download on a short glance tho.
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: Updated navdata without navigraph

Postby SkyClan480 » Sat Jun 12, 2021 6:03 pm

merspieler wrote in Sat Jun 12, 2021 11:27 am:One thing that I still see missing, as well for the US, are procedures (SID/STAR). I couldn't see them in the download on a short glance tho.

procedures are a different story than the navdata, as they are formatted completely differently in flightgear. the NASR data does include procedures, but i havent even begun to look at how its formatted compared to what flightgear needs. there's also the extremely complicated folder structure for procedures in flightgear. procedures are certainly another thing for the project to look at for the future. in the meantime, getting data for other countries is probably a priority because like you said, having only the us doesn't make this a very viable navigraph alternative.

merspieler wrote in Sat Jun 12, 2021 11:27 am:If you're interested in the idea of using openAIP for this, and might want some help, let me know.

idk what openAIP is, but if you think it's a good idea then ill follow your direction to do whatever it is i need to do with it.
SkyClan480
 
Posts: 12
Joined: Wed Jun 09, 2021 4:42 pm
Version: 2020.3.12
OS: Windows 11 Home

Re: Updated navdata without navigraph

Postby merspieler » Sat Jun 12, 2021 8:03 pm

The directory structure for procedures is actually not relally complicated:
Airports/<first ICAO letter>/<second ICAO letter>/<third ICAO letter>/<full ICAO code>.procedures.xml
Lets take KSFO for example...
Airports/K/S/F/KSFO.procedures.xml

For the openAIP website:
openAIP has the goal to deliver free, current and precise navigational data to anyone. openAIP is a web-based crowd-sourced aeronautical information platform
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: Updated navdata without navigraph

Postby SkyClan480 » Sat Jun 12, 2021 8:14 pm

hmmm, openaip seems like exactly what i spent 3 hours looking for earlier. its what we need, but isn't in file format... potentially we may have to look at revamping the project to enable it to read web-based information, unless the data from openaip can be downloaded as a file. then it would be a matter of parsing it. there also seems to be the problem that the data is navaids only, and doesn't give fix/airway information. it looks promising though.
SkyClan480
 
Posts: 12
Joined: Wed Jun 09, 2021 4:42 pm
Version: 2020.3.12
OS: Windows 11 Home

Re: Updated navdata without navigraph

Postby merspieler » Sat Jun 12, 2021 9:19 pm

yes... you can download the data as files. (needs registration)

I know it's not complete.

Probably the best solution will be to be able to combine multiple sources into the nav data...
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Re: Updated navdata without navigraph

Postby wkitty42 » Mon Jun 14, 2021 12:58 pm

merspieler wrote in Sat Jun 12, 2021 8:03 pm:The directory structure for procedures is actually not relally complicated:
Airports/<first ICAO letter>/<second ICAO letter>/<third ICAO letter>/<full ICAO code>.procedures.xml

FWIW: there are, in FG, also three letter airport designations... if one of them has a procedures.xml file, it would follow the same form except only the first two letters would be the directories...

eg: KQA - Akutan Airport (IATA: KQA, ICAO: PAUT, FAA LID: 7AK) - Airports/K/Q/KQA.procedures.xml

NOTE: i grabbed this one just because it was easy enough and random... i don't know why FG uses the IATA code or when the ICAO code was created/added for this airport... it is listed in the FG apt.dat as KQA and there is no listing for PAUT... this is probably just another "old data" situation... in any case, i just wanted the OP to know that there are some three letter designations and in those cases, the files for them are located in the same fashion as the four letter ones... just one directory shorter...

EDIT: ahhh... this page is the most likely explanation for the above discrepency with KQA... https://en.wikipedia.org/wiki/Akutan_Seaplane_Base
so yeah, another "old data" situation...
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9146
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: Updated navdata without navigraph

Postby www2 » Tue Jun 29, 2021 9:39 pm

@SkyClan480
Thanks for the tip.
An other thing we need to start petition to bring back DAFIF.
Or start our own navigation provider as a corporate sponsor or owner for flight gear project. (i can dream)

Current i working on a demo for a overhol our navigation database (i call project "sim424") and making better models (Project "Virtual Ether") for VOR, TACAN, NDB and include GNSS as GPS/Glonas/Galieo, MLS, and hyperbolic navigation as omega and LORAN in the long run (Project "Next Navigation").
www2
 
Posts: 319
Joined: Thu Apr 16, 2009 2:58 pm
OS: Ubuntu

Re: Updated navdata without navigraph

Postby merspieler » Wed Jun 30, 2021 9:00 am

www2 wrote in Tue Jun 29, 2021 9:39 pm:Or start our own navigation provider as a corporate sponsor or owner for flight gear project. (i can dream)


Why not build on something already existing?
Nia (you&, she/her)

Please use gender neutral terms when referring to a group of people!

Be the change you wish to see in the world, be an ally to all!

Join the official matrix space
merspieler
 
Posts: 2241
Joined: Thu Oct 26, 2017 11:43 am
Location: Wish to be in YBCS
Pronouns: you&, she/her
Callsign: you&, she/her
IRC name: merspieler
Version: next
OS: NixOS

Next

Return to Scenery

Who is online

Users browsing this forum: No registered users and 7 guests