Board index FlightGear Support Flying

METAR not working

Controlling your aircraft, using the autopilot etc.

METAR not working

Postby mathieugouin » Mon Feb 04, 2019 6:29 pm

Hello,

It seems that NOAA changed their server.

Here is the output from the console utility "metar KSFO"

Code: Select all
metar download failed: http://tgftp.nws.noaa.gov/data/observations/metar/stations/KSFO.TXT (302 Object Moved)
ERROR: metar download failed


Also, it does not work within FlightGear.

Thanks!
Mathieu
mathieugouin
 
Posts: 36
Joined: Thu Jan 06, 2011 3:37 am
Location: CYHU
Callsign: MGOUIN
Version: V2018.1.1
OS: Lubuntu 18.04

Re: METAR not working

Postby Lenard Spencer » Mon Feb 04, 2019 7:16 pm

The problem was that noaa.gov switched from http to https. Here is the patch I came up with (also for the devs' benefit).
This patch should work for all 2017/2018 versions.

flightgear-metar.patch
Code: Select all
diff -Naur flightgear-2018.3.2/src/Environment/realwx_ctrl.cxx flightgear-2018.3.2.new/src/Environment/realwx_ctrl.cxx
--- flightgear-2018.3.2/src/Environment/realwx_ctrl.cxx   2019-01-29 08:17:50.000000000 -0500
+++ flightgear-2018.3.2.new/src/Environment/realwx_ctrl.cxx   2019-02-04 13:43:14.938038359 -0500
@@ -426,7 +426,7 @@
 )
 {
   static const std::string NOAA_BASE_URL =
-    "http://tgftp.nws.noaa.gov/data/observations/metar/stations/";
+    "https://tgftp.nws.noaa.gov/data/observations/metar/stations/";
   class NoaaMetarGetRequest:
     public simgear::HTTP::MemoryRequest
   {
diff -Naur flightgear-2018.3.2/src/Main/metar_main.cxx flightgear-2018.3.2.new/src/Main/metar_main.cxx
--- flightgear-2018.3.2/src/Main/metar_main.cxx   2019-01-29 08:17:50.000000000 -0500
+++ flightgear-2018.3.2.new/src/Main/metar_main.cxx   2019-02-04 13:46:20.681671703 -0500
@@ -539,7 +539,7 @@
          try
          {
               static const std::string NOAA_BASE_URL =
-                "http://tgftp.nws.noaa.gov/data/observations/metar/stations/";
+                "https://tgftp.nws.noaa.gov/data/observations/metar/stations/";
                 HTTP::MemoryRequest* mr = new HTTP::MemoryRequest
                 (
                     NOAA_BASE_URL
diff -Naur flightgear-2018.3.2/utils/fgcom/utils/build_fgcom_server.sh flightgear-2018.3.2.new/utils/fgcom/utils/build_fgcom_server.sh
--- flightgear-2018.3.2/utils/fgcom/utils/build_fgcom_server.sh   2019-01-29 08:17:50.000000000 -0500
+++ flightgear-2018.3.2.new/utils/fgcom/utils/build_fgcom_server.sh   2019-02-04 13:47:12.329120380 -0500
@@ -6,7 +6,7 @@
 DAHDI_SRC="http://downloads.asterisk.org/pub/telephony/dahdi-linux-complete/dahdi-linux-complete-current.tar.gz"
 LOGSEP="###########################################"
 GETMETAR_SCRIPT="#!/bin/bash
-#curl http://tgftp.nws.noaa.gov/data/observations/metar/stations/$1.TXT
+#curl https://tgftp.nws.noaa.gov/data/observations/metar/stations/$1.TXT
 echo \"Hello World !\""
 
 ROOT=$PWD
diff -Naur flightgear-2018.3.2/utils/metarproxy/metarproxy flightgear-2018.3.2.new/utils/metarproxy/metarproxy
--- flightgear-2018.3.2/utils/metarproxy/metarproxy   2019-01-29 08:17:50.000000000 -0500
+++ flightgear-2018.3.2.new/utils/metarproxy/metarproxy   2019-02-04 13:43:51.224360207 -0500
@@ -81,7 +81,7 @@
        \$ http_proxy= metarproxy --record LOXL
 
 Sources:
-       http://tgftp.nws.noaa.gov/data/observations/metar/{stations,cycles}/
+       https://tgftp.nws.noaa.gov/data/observations/metar/{stations,cycles}/
        ftp://tgftp.nws.noaa.gov/data/observations/metar/{stations,cycles}/
 EOF
 
@@ -399,7 +399,7 @@
          s/\s+$//;
          &log($DEBUG, $_);
 
-         if (m|^GET\s+http://tgftp.nws.noaa.gov/.*/([A-Z][A-Z0-9]{3}).TXT\s+HTTP/|) {
+         if (m|^GET\s+https://tgftp.nws.noaa.gov/.*/([A-Z][A-Z0-9]{3}).TXT\s+HTTP/|) {
             $icao = $1;
          } elsif (/X-Time: (\d+)/) {
             $epoch = $1;
Lenard Spencer
 
Posts: 5
Joined: Mon Dec 23, 2013 12:36 am

Re: METAR not working

Postby Isaak » Mon Feb 04, 2019 7:41 pm

Thanks for the quick fix. It seems it wasn't cross posted yet to the developers mailing list (quickest way to get something changed). I 'll do that in a minute.
Isaak
 
Posts: 468
Joined: Sat Jun 04, 2011 2:52 pm
Location: Leuven, Belgium
Callsign: OO-ISA
Version: 2019.1.1
OS: Windows 10

Re: METAR not working

Postby V12 » Tue Feb 05, 2019 9:44 am

Will be update for 2018.3.1 available, or need I wait for 2019.x.x ?
Fly high, fly fast - fly Concorde !
User avatar
V12
 
Posts: 1591
Joined: Thu Jan 12, 2017 4:27 pm
Location: LZIB
Callsign: BAWV12

Re: METAR not working

Postby Isaak » Tue Feb 05, 2019 9:56 am

James has just replied on the mailing list that there's an extra issue for FG versions older tan 2017.x, as they cannot connect via https. As for now it's not yet in 2018.3.2, but I guess an intermediate release just for this bug is necessary because it's only a small patch and restores an important functionality. It's easiest to follow on the mailing list if you want to stay updated.
Isaak
 
Posts: 468
Joined: Sat Jun 04, 2011 2:52 pm
Location: Leuven, Belgium
Callsign: OO-ISA
Version: 2019.1.1
OS: Windows 10

Re: METAR not working

Postby Lenard Spencer » Tue Feb 05, 2019 12:17 pm

REVISED patch (forgot the metadata README file, shouldn't affect the build)

flightgear-metar.patch
Code: Select all
diff -Naur flightgear-2018.3.2/src/Environment/realwx_ctrl.cxx flightgear-2018.3.2.new/src/Environment/realwx_ctrl.cxx
--- flightgear-2018.3.2/src/Environment/realwx_ctrl.cxx   2019-01-29 08:17:50.000000000 -0500
+++ flightgear-2018.3.2.new/src/Environment/realwx_ctrl.cxx   2019-02-05 07:11:40.532121269 -0500
@@ -426,7 +426,7 @@
 )
 {
   static const std::string NOAA_BASE_URL =
-    "http://tgftp.nws.noaa.gov/data/observations/metar/stations/";
+    "https://tgftp.nws.noaa.gov/data/observations/metar/stations/";
   class NoaaMetarGetRequest:
     public simgear::HTTP::MemoryRequest
   {
diff -Naur flightgear-2018.3.2/src/Main/metar_main.cxx flightgear-2018.3.2.new/src/Main/metar_main.cxx
--- flightgear-2018.3.2/src/Main/metar_main.cxx   2019-01-29 08:17:50.000000000 -0500
+++ flightgear-2018.3.2.new/src/Main/metar_main.cxx   2019-02-05 07:11:40.533121277 -0500
@@ -539,7 +539,7 @@
          try
          {
               static const std::string NOAA_BASE_URL =
-                "http://tgftp.nws.noaa.gov/data/observations/metar/stations/";
+                "https://tgftp.nws.noaa.gov/data/observations/metar/stations/";
                 HTTP::MemoryRequest* mr = new HTTP::MemoryRequest
                 (
                     NOAA_BASE_URL
diff -Naur flightgear-2018.3.2/utils/fgcom/utils/build_fgcom_server.sh flightgear-2018.3.2.new/utils/fgcom/utils/build_fgcom_server.sh
--- flightgear-2018.3.2/utils/fgcom/utils/build_fgcom_server.sh   2019-01-29 08:17:50.000000000 -0500
+++ flightgear-2018.3.2.new/utils/fgcom/utils/build_fgcom_server.sh   2019-02-05 07:11:40.533121277 -0500
@@ -6,7 +6,7 @@
 DAHDI_SRC="http://downloads.asterisk.org/pub/telephony/dahdi-linux-complete/dahdi-linux-complete-current.tar.gz"
 LOGSEP="###########################################"
 GETMETAR_SCRIPT="#!/bin/bash
-#curl http://tgftp.nws.noaa.gov/data/observations/metar/stations/$1.TXT
+#curl https://tgftp.nws.noaa.gov/data/observations/metar/stations/$1.TXT
 echo \"Hello World !\""
 
 ROOT=$PWD
diff -Naur flightgear-2018.3.2/utils/metarproxy/README flightgear-2018.3.2.new/utils/metarproxy/README
--- flightgear-2018.3.2/utils/metarproxy/README   2019-01-29 08:17:50.000000000 -0500
+++ flightgear-2018.3.2.new/utils/metarproxy/README   2019-02-05 07:10:25.154570061 -0500
@@ -266,13 +266,13 @@
 =====================
 Download addresses for the last 24 hours:
 
-  http://tgftp.nws.noaa.gov/data/observations/metar/cycles/
+  https://tgftp.nws.noaa.gov/data/observations/metar/cycles/
   ftp://tgftp.nws.noaa.gov/data/observations/metar/cycles/
 
 Addresses for the most recent METAR data strings of particular
 stations:
 
-      http://tgftp.nws.noaa.gov/pub/data/observations/metar/stations/
+      https://tgftp.nws.noaa.gov/pub/data/observations/metar/stations/
       ftp://tgftp.nws.noaa.gov/data/observations/metar/stations/
 
 
diff -Naur flightgear-2018.3.2/utils/metarproxy/metarproxy flightgear-2018.3.2.new/utils/metarproxy/metarproxy
--- flightgear-2018.3.2/utils/metarproxy/metarproxy   2019-01-29 08:17:50.000000000 -0500
+++ flightgear-2018.3.2.new/utils/metarproxy/metarproxy   2019-02-05 07:11:40.534121284 -0500
@@ -81,7 +81,7 @@
        \$ http_proxy= metarproxy --record LOXL
 
 Sources:
-       http://tgftp.nws.noaa.gov/data/observations/metar/{stations,cycles}/
+       https://tgftp.nws.noaa.gov/data/observations/metar/{stations,cycles}/
        ftp://tgftp.nws.noaa.gov/data/observations/metar/{stations,cycles}/
 EOF
 
@@ -399,7 +399,7 @@
          s/\s+$//;
          &log($DEBUG, $_);
 
-         if (m|^GET\s+http://tgftp.nws.noaa.gov/.*/([A-Z][A-Z0-9]{3}).TXT\s+HTTP/|) {
+         if (m|^GET\s+https://tgftp.nws.noaa.gov/.*/([A-Z][A-Z0-9]{3}).TXT\s+HTTP/|) {
             $icao = $1;
          } elsif (/X-Time: (\d+)/) {
             $epoch = $1;
Lenard Spencer
 
Posts: 5
Joined: Mon Dec 23, 2013 12:36 am

Re: METAR not working

Postby Lenard Spencer » Tue Feb 05, 2019 12:25 pm

Just to let everyone know, the patch works with 2017.x as well.
Lenard Spencer
 
Posts: 5
Joined: Mon Dec 23, 2013 12:36 am

Re: METAR not working

Postby StuartC » Tue Feb 05, 2019 1:12 pm

where does the patch code go ?
StuartC
 
Posts: 2738
Joined: Fri Jun 18, 2010 8:18 pm
Location: Arse end of the Universe
Callsign: WF01
Version: 2018.3.2
OS: W10 64 bit

Re: METAR not working

Postby SurferTim » Tue Feb 05, 2019 1:57 pm

Patch it to use an external URL in a setup file or startup parameter. If hard coded, this will certainly happen again.
Fly low, fly slow, land on a dime. Twin Otter.
SurferTim
 
Posts: 386
Joined: Sun Dec 09, 2018 5:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2019.2.0
OS: Ubuntu 18.04

Re: METAR not working

Postby lbartik » Tue Feb 05, 2019 5:42 pm

I can tell you how not to fix it quick and dirty.

I tried editing FGFS.exe for windows with a hex editor by inserting an "s" character at offset E9F57C and deleting a NULL character at E9F5B4.

It might have worked around 2016 when the URL changed, but it does not work this time.
lbartik
 
Posts: 5
Joined: Wed Nov 14, 2018 7:46 pm

Re: METAR not working

Postby airzona » Tue Feb 05, 2019 5:53 pm

I'm totally at a loss on how to fix this. :(
Seems like this info should not be in binary, and should be an XML in ../data/Environment/metar.xml or some such. That's been brought up before, and this is a recurring issue. If I knew how to submit a fix, I would.
I'm on a Mac v. 2018.3.2. Thanks!
"power corrupts. powerpoint corrupts absolutely." -edward tufte
User avatar
airzona
 
Posts: 42
Joined: Sat Sep 03, 2011 7:16 pm
Location: Arizona
Callsign: airzona
OS: Max OS X

Re: METAR not working

Postby mathieugouin » Tue Feb 05, 2019 5:57 pm

I'm just throwing an idea here...

How hard would it be to have a small servlet running on flightgear.org that would redirect the metar request to the proper URL and response back to flightgear? This way, users don't care about changing URLs. A new URLs is just a matter of a quick servlet update and automatically without user action, metars are back on track.

A quick example I did a while back using google app engine:

http://mgouin.appspot.com/metar?txtweb-message=KLAX

It's not the exact right format for FG, especially that back then, I also included TAFs...
Mathieu
mathieugouin
 
Posts: 36
Joined: Thu Jan 06, 2011 3:37 am
Location: CYHU
Callsign: MGOUIN
Version: V2018.1.1
OS: Lubuntu 18.04

Re: METAR not working

Postby V12 » Tue Feb 05, 2019 7:31 pm

lbartik wrote in Tue Feb 05, 2019 5:42 pm:I can tell you how not to fix it quick and dirty.

I tried editing FGFS.exe for windows with a hex editor by inserting an "s" character at offset E9F57C and deleting a NULL character at E9F5B4.

It might have worked around 2016 when the URL changed, but it does not work this time.


You can't insert or delete any character. But replace text in some strings is possible.
Fly high, fly fast - fly Concorde !
User avatar
V12
 
Posts: 1591
Joined: Thu Jan 12, 2017 4:27 pm
Location: LZIB
Callsign: BAWV12

Re: METAR not working

Postby SurferTim » Wed Feb 06, 2019 1:58 am

You can replace text in the executable, but that alone won't do it.

The challenge (which I have overcome) is the new weather URL requires a SSL connection (HTTPS). The current code won't handle that. I use a python3 script and modify the fgfs executable. It isn't pretty, but it works, at least on my Ubuntu box. It is good to have real time weather back.

If the script continues to function well, as it did on my last flight, I'll post the changes on the forum tomorrow after another couple flights.

My only question is now: Where should I post the patch?
Fly low, fly slow, land on a dime. Twin Otter.
SurferTim
 
Posts: 386
Joined: Sun Dec 09, 2018 5:49 pm
Location: Miramar Beach, FL
Callsign: Pirate
Version: 2019.2.0
OS: Ubuntu 18.04

Re: METAR not working

Postby V12 » Wed Feb 06, 2019 5:57 am

There is another nasty solution - feed the metar string into the property tree from external application via telnet.
Fly high, fly fast - fly Concorde !
User avatar
V12
 
Posts: 1591
Joined: Thu Jan 12, 2017 4:27 pm
Location: LZIB
Callsign: BAWV12

Next

Return to Flying

Who is online

Users browsing this forum: No registered users and 2 guests