Live WXRadar MapStructure Layer Development

[omega95]

I was thinking of developing a more realistic (works only for real weather/live data, ofcourse) WXRadar layer which uses a radar layer from wunderground to display the weather scenario. To use such an image, you require a subscription to Wunderground but they let you make 500 requests/day (10/min) for free - so if the users register for their API, we might be able to do this! We could possibly get the instrument to make a request every 10/15 minutes and we can rotate/scale the image (and move with aircraft position) to display it.

I'm testing out the API right now, but you submit http GET requests (already possible with FG - used it for an ACARS system I made earlier) and it returns an image. I need to be able to display this image as a MapStructure layer.

Here are some GET requests I made (examples) with my API Key and the layers returned. We can customize the data for position, range, bounds etc.

Code: Select all

http://api.wunderground.com/api/MY_KEY_HERE/radar/image.gif?maxlat=47.709&maxlon=-69.263&minlat=31.596&minlon=-97.388&width=1024&height=1024

Code: Select all

http://api.wunderground.com/api/MY_KEY_HERE/radar/image.gif?maxlat=42.35&maxlon=-109.311&minlat=39.27&minlon=-114.644&width=1024&height=1024

There's also plenty of documentation for the API available here - http://www.wunderground.com/weather/api/d/docs?d=layers/radar&MR=1

I really like how they return images with transparent backgrounds so we can easily overlay it on the ND.

So, I'd say we look up for a position and a range of about 500 miles and move that image until 15 min later, where it makes a new request.

[Hooray]

I'm testing out the API right now, but you submit http GET requests (already possible with FG - used it for an ACARS system I made earlier) and it returns an image. I need to be able to display this image as a MapStructure layer.

Actually, it's even easier than that - the ACARS method you used was probably based on the xml-httprequest fgcommand.
Meanwhile, we have native Nasal/HTTP bindings, but even the Canvas system itself supports http:// directly

Thus, it's just a conventional Canvas::Image, just fetched via http - so it should work as expected. In fact, I used the same method for creating this screen shot:

Subject: NavDisplay & MapStructure discussion (previously via PM)

So, you can just treat it as a normal raster image and use some helper function/class for assembling proper URLs (including login/passwort) for the current position - and if necessary, also deal with tiling that way. Tom is hoping to implement SQLite-level caching for these purposes, too.

For testing purposes, just adapt a Canvas Image example and use a http:// URL for the file location.
A corresponding MapStructure layer would just be a wrapper around 1) CanvasImage (for raster images) and 2) some customizable URL-building scheme, i.e. a class or function that builds proper URLs
Regarding this particular use-case, it would make sense to provide some form of customizable "post-transform" callback that applies texture mapping (extracting a sub-texture) and transforms the image accordingly.

Code: Select all

you require a subscription to Wunderground but they let you make 500 requests/day (10/min) for free - so if the users register for their API, we might be able to do this!

For the sake of efficiency, it would make sense to set up a php script that handles the wunderground access and which simply "mirrors" the same API for FG users, so that there's no API keys or redundant requests involved - i.e. VA folks etc would simply use the hosted script instead of directly using the wunderground API - as long as parity with the original API is maintained (same GET request names/arguments), the whole thing can be easily tested and developed in parallel.

[omega95]

I didn't really understand how mirroring the API helps in this case - If there are 15 pilots, in 15 different parts of the world, and need to get the data at the same time, how can this be done?

[Gijs]

Do note that the weather data FlightGear uses is not as detailed that it will match those plots. Reading FlightGear's weather systems directly (as our, so far crude, WXR layer in the ND does) will give a more accurate view of what you will encounter. It'll certainly proof to be a lot more flexible once we'll implement various features and failures (I've planned to revisit it later this month).

Not saying you should not try to use this API (it does look very nice), but it's not more realistic per se

[omega95]

but it's not more realistic per se

Aha, you're saying that A is not realistic because A does not match B - but B is not realistic, and A is?

Either way, why not try to implement a more realistic (live data based) weather system for FG instead of using a simulated system which goes by the name of Live data but really isn't.

Maybe these wunderMap radar/satellite (the satellite layers are available all around the globe - not just in the US) can help define the weather in FG along with the METAR data.

EDIT -

Actually, discard all of that - the local weather system in FG models all visible weather effects based on METAR data, doesn't it? The wunderMap radar/satellite layers show actual thunderstorms instead of clouds by themselves - which is more realistic. So, if you use the Live Data setting on the local weather system, theoretically when you are approaching a region with say a thunderstorm, it should come up on the wxradar and when are actually in that region, the METAR (of the closest station) should report a thunderstorm there.

Can you explain why you think this will not work? I might be missing something important here.

[omega95]

Code: Select all

t.layer.setFile("http://api.wunderground.com/api/"~t.api_key~"/radar/image.png?centerlat="~t.lat~"&centerlon="~t.lon~"&radius=320&width="~t.res~"&height="~t.res~"&smooth=1");
         
         print("WXR Layer: "~"http://api.wunderground.com/api/"~t.api_key~"/radar/image.png?centerlat="~t.lat~"&centerlon="~t.lon~"&radius=320&width="~t.res~"&height="~t.res~"&smooth=1");

So, I tried just using the GET request URL as the file path but, it doesn't seem to work. I printed the same thing, and when I copy that into my browser window, the image seems to be alright.

[Hooray]

use the property browser to check if the node is a proper Canvas Image node - there should also be some kind of "status" flag somewhere - for testing purposes you can also use some static png/raster image and then use a timer to turn it into a remotely-fetched image

[omega95]

Hmm, interesting -
status = 0
status-msg = 'Ok'
group/image/visible = 'false'
And I can't set any of them to 1/true.

I'm just testing this on a separate canvas display before integrating it as a mapstructure layer, btw.

[Hooray]

Like I said, I'd try to troubleshoot this step-by-step, e.g.:

• use a static LOCAL raster image (on disk/in fgdata - e.g. a splash screen -if this doesn't work, your Canvas/Image isn't set up properly)
• use a http URL (e.g. the google/flightgear logo - if this doesn't work, check for errors/networking issues)
• use a dynamically-built URL - if this doesn't work, check your URL building scheme

and always check the console/property browser for any errors/warnings etc

(I'd also print the status flags to the console for simplicity - including the URL that you've built - and then copy that URL into your browser to check again)

Once this works, you should have all the building blocks in place to turn this into a simple MapStructure layer, so that you can add tiling support later on.
I'd just make sure to assemble the URL only ONCE in some form of helper/wrapper, and not assemble it manually in your print statements - or errors may never show up that way obviously ... (manually concatenating your URL in several places, like your setFile() and printlog() calls is unnecessarily error-prone and violates the DRY-principle, it is better to encapsulate/localize such functionality so that there's only a single place to check/validate)

PS: Regarding the LW/AW system, it's true that it can try to model things based on METAR, but that isn't an "exact" science, like Thorsten keeps pointing out, there's a lot of interpretation/guesswork involved - what /might/ work though is enriching the system, and its heuristics, using additional data, such as those images - i.e. by fetching them via HTTP and post-processing them to get WXR info for certain coordinates - but that is something that isn't currently done, and while we can fetch images via Canvas or Nasal fairly easily, we cannot easily post-process them currently - that would require some image processing support (think OpenCV), or integration with the effects/shader framework - to affect Nasal-space placement heuristics etc. Unlike HTML5 Canvas, there's no such a thing as getPixel() etc available in FG's Canvas AFAIK for pixel manipulation in a setPixel/getImageData fashion(?)
Probably, this is something that we don't want to do in Nasal space - it would be more efficient to really use either shaders or at least OpenCL bindings to accomplish the same in a more generic fashion.

[omega95]

That you very much, I'm making some good progress here - the issue was actually with an incorrect property that triggered visibility of the image.

After I complete this, I'll start making this a MapStructure layer.

[Hooray]

Indeed that's looking very promising, good job !
Besides, I really do appreciate the fact that you're not just trying to integrate it in some "roundabout" fashion, but actually working towards turning this into a generic and reusable MapStructure layer - once your new layer works for independent ND instances .and maybe even in the map-canvas dialog- you'll see that it's well worth it

[omega95]

Alright, so I did most of the work I planned on doing on the WXR and here's what I have so far. I just spawned at Dodge City, Kansas real a storm and I set my weather setting to live data. I'm going to fly into the eye of the storm and see how the weather system responds to that.








I'd say it's pretty neat so far - but FG doesn't show any storms in the distance.

[Hooray]

right, like I said, the existing WXR layer uses the AW system - which puts all "storm" info into the property tree, for details see the WXR layer, which should tell you where to look for the kind of info that AW provides (use the property browser) - overall, this is obviously fairly unlikely to be as detailed/accurate as live imagery fetched online - the other issue may be related to the tiling system used by AW, i.e. I don't know if it even tries to create/render storms far away - I am afraid only Thorsten can answer that reliably.

[omega95]

Ahh ok.

Other than the image itself, I was wondering how I could switch the order of transformations used on a canvas object. To move the weather radar image properly to correspond to the aircraft movement, I need the setTranslation transform to be the first transform and then the setRotation transform - but I read that it's the other way around. So I do I switch them so I don't have to mess with the math and create a new translation?

[TheTom]

What do you want to achieve? You can rotate - optionally set an offset for the rotation center. And then move. So this should allow every possible transformation. Also its not that hard to create two Transformations and use them to rotate/translate/etc.