Board index FlightGear Development Scenery

Breaking down NLCD raster image

Questions and discussion about enhancing and populating the FlightGear world.

Breaking down NLCD raster image

Postby wlbragg » Sat May 07, 2016 7:19 pm

After getting back into the workflow of scenery creation I decided to document this "simple" process so I don't forget again.

Download the raster image of from the USGS.

Using QGIS

Layer/Add Raster Layer
Raster/Conversion/Polygonize (Raster to Vector)
Image

After the above step you have one big vector file (shape) that can be subdivided into many by the DN value.

Vector/Data Management Tools/Split vector layers
Image

Finish by renaming and selecting colors to represent the land class. Add in stream, road and rail layers and you end up with this.
Image

All the data in this project came from the same USGS site. I think it will prove to be superior to the custom scenery available in my sig which was made up of "rounded" NLCD and other hydro and deciduous sources plus OSM road and rail.

Time will tell as I plan to run this in the next week to see how it looks and more importantly how it behaves in FG.

The above work was completed in a few hours (half of that was waiting for processes to complete). If I remembered correctly i should be able to generate the entire state of KS in less than 24 hrs.
Kansas and Ohio/Midwest scenery development.
KEQA, 3AU, KRCP Airport Layout
Intel i7/GeForce RTX 2070/Max-Q
User avatar
wlbragg
 
Posts: 7586
Joined: Sun Aug 26, 2012 12:31 am
Location: Kansas (Tornado Alley), USA
Callsign: WC2020
Version: next
OS: Win10/Linux/RTX 2070

Re: Breaking down NLCD raster image

Postby f-ojac » Sun May 08, 2016 8:48 am

Great!
f-ojac
 
Posts: 1304
Joined: Fri Mar 07, 2008 10:50 am
Version: GIT
OS: GNU/Linux

Re: Breaking down NLCD raster image

Postby Hooray » Sun May 08, 2016 4:47 pm

you should really consider turning your postings into tutorials that we can move over to the wiki, so that other contributors can get involved to help update/maintain such articles
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Breaking down NLCD raster image

Postby wlbragg » Mon May 09, 2016 1:16 am

Some results to share.

This is the new USGS scenery using mostly default FG material mapping and textures.
Image

This image illustrates the concept of using the rounded lake layer as a sand layer with a more accurate lake layer on top of it. You end up with what looks like beaches from low water in the lake. It also give you a good view at altitude of the complexity of the scenery.
Image

From a different angle.
Image

Here is a comparison of ws2.0 and what the ws3.0 will hopefully look like.
Image

Here is the difference in vertices between ws2.0 and the custom scenery. Most of the layers need to be ran through the rounding script to lower the resolution and also the road network needs to be reduced as it is in ws2.0. Hopefully we can do a little more than what was in ws2.0, but I think what is in this scenery is going to prove to be to much unless we can include a robust LOD system for it.
Image

The inflow to the lake, notice the sand layer and how good much it enhances the look of the lake,
Image

A close up of the lake shoreline. Notice the "un-rounded" layer on the inside of the sand layer. Using a rounded version of this land class would look much better. Any roads that run close to the lake shoreline (no good examples here) that would normally end up in the rounded water would in this case be on sand instead.
Image

This is with a more custom material file and textures.
Image

Same here. The USGS NLCD has at least two separate layers of town or impervious. If you look at the urban or town area in this shot, I mapped the residential area to Town/SubUrban and the downtown commercial areas of the town to BuiltUpCover. It gave it a really distinguishing difference between the two different areas of the town. This is an area that we could use some new professional textures. Although what we have actually work pretty well as they are.
Image

You can see what I am talking about better in this close up of the town.
Image

Overall, I am impressed with this data, keeping in mind this is straight off the site with no editing other than mapping and a few texture in some cases.
Kansas and Ohio/Midwest scenery development.
KEQA, 3AU, KRCP Airport Layout
Intel i7/GeForce RTX 2070/Max-Q
User avatar
wlbragg
 
Posts: 7586
Joined: Sun Aug 26, 2012 12:31 am
Location: Kansas (Tornado Alley), USA
Callsign: WC2020
Version: next
OS: Win10/Linux/RTX 2070

Re: Breaking down NLCD raster image

Postby wlbragg » Mon May 09, 2016 1:28 am

turning your postings into tutorials that we can move over to the wiki

I'm not ignoring you Hooray. I new that was coming before you did.
After being away from the wiki system for some time it becomes more of an effort to re-learn how to start a new file and figure out how to format it , with what headings, etc. Sometimes it is almost too much to do the original write up of the original post and all the work of that layout and effort to have to also duplicate it in a wiki article. If I was a little more current and affluent with the wiki editor, I might not be so hesitant to create the work there instead of the forum.
Kansas and Ohio/Midwest scenery development.
KEQA, 3AU, KRCP Airport Layout
Intel i7/GeForce RTX 2070/Max-Q
User avatar
wlbragg
 
Posts: 7586
Joined: Sun Aug 26, 2012 12:31 am
Location: Kansas (Tornado Alley), USA
Callsign: WC2020
Version: next
OS: Win10/Linux/RTX 2070

Re: Breaking down NLCD raster image

Postby erik » Mon May 09, 2016 8:58 am

This starts to look great. It's about on par with the CORINA data. Even the raster effect doesn't detract too much (at least from this altitude). The rounded shore line compensates quite nicely for it.

Erik
Current: Parachutist, Paraglider, Pterosaur, Pilatus PC-9M and variants, ERCO Ercoupe, Fokker Dr.1, Fokker 50, Fokker 100
Less active: Cessna T-37, T-38, Santa Claus. Previous: General Dynamics F-16. Worked on: Wright Flyer
erik
 
Posts: 2244
Joined: Thu Nov 01, 2007 2:41 pm

Re: Breaking down NLCD raster image

Postby statto » Mon May 09, 2016 4:02 pm

wlbragg wrote in Mon May 09, 2016 1:16 am:Some results to share.


Something you might do is to simulate the "base" layer by creating just a big square of data, pull out individual layers from the raster, smooth them, remove small areas (areas that cover one or two pixels in the NLCD data), and then try recompiling using the "base" layer as the lowest layer (put everything else on top) - you'll lose something in the scenery, but you'll lose the blocks and you'll decrease the triangle count.

Your scenery looks great, but you're starting to notice why I was willing to sacrifice a bit of precision for the cs_ nationwide vector NLCD layer ;)
Custom Scenery available from http://www.stattosoftware.com/flightgear
statto
 
Posts: 2106
Joined: Fri Jan 25, 2008 10:57 pm

Re: Breaking down NLCD raster image

Postby jaxsin » Tue May 10, 2016 11:50 am

erik wrote in Mon May 09, 2016 8:58 am:This starts to look great. It's about on par with the CORINA data. Even the raster effect doesn't detract too much (at least from this altitude). The rounded shore line compensates quite nicely for it.

Erik


I also like what I see. Much improved even with the older material and textures.

On that Note, I once heard someone say the problem with scenery generation was holes and other anomalies and because of this the whole lot had to be done at once to avoid these issues. Start with the same data more or less.
But my question is, why could you not break down the world scenery into a couple sections using the ocean as the binding point. Zero elevation, one texture and one land class(OCEAN). I am not certain that this matters any more with Tosten's new announcement about custom scenery making it into TS
jaxsin
 
Posts: 395
Joined: Mon Dec 28, 2015 4:54 pm

Re: Breaking down NLCD raster image

Postby wlbragg » Wed May 11, 2016 6:13 pm

statto wrote in Mon May 09, 2016 4:02 pm:Something you might do is to simulate the "base" layer by creating just a big square of data

This part I already do, the base (grassland) is just one big square.

pull out individual layers from the raster, smooth them

Is it documented anywhere how to run the smoothing script? Is it done in Grass, QGIS or do you execute it in a command line environment?

Just for kicks I am running tgconstruct on a combination of the smoothed cs land classes - smoothed cs deciduous + new NLCD water and Woodland using the smoothed cs water as a sand layer, thus no blocks. It should produce the best results to date and have a reasonable chance at being light enough to be useful. I also used an edited version of the ws2.0 OSM layer (edited to include a little more detail) and a custom stream layer.

I've been having some "unknown" issues when trying to tgconstruct the entire state. It bails after some time with no debug flag and no output. If I gen a smaller area (I'm trying around 1/5 of the state) it seems to be OK. I don't know if it is a piece of data it doesn't like or a memory or even a space issue. I am down to 13 gig on the drive I am generating this on.
Kansas and Ohio/Midwest scenery development.
KEQA, 3AU, KRCP Airport Layout
Intel i7/GeForce RTX 2070/Max-Q
User avatar
wlbragg
 
Posts: 7586
Joined: Sun Aug 26, 2012 12:31 am
Location: Kansas (Tornado Alley), USA
Callsign: WC2020
Version: next
OS: Win10/Linux/RTX 2070

Re: Breaking down NLCD raster image

Postby wkitty42 » Wed May 11, 2016 7:42 pm

wasn't there some discovery a few weeks back that generating new scenery times needed to encompass all of the tiles being built? making the selection of the area a little larger or smaller ""fixed"" the problem (included the rest of or dropped the partial tile section) and allowed the build to complete...
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9146
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: Breaking down NLCD raster image

Postby Hooray » Wed May 11, 2016 7:45 pm

I think what is in this scenery is going to prove to be to much unless we can include a robust LOD system for it.


I have't been following the LOD discussion, but at some point, a few people were working on an osg::simplifier based approach, I think Stuart (or James) tossed it out because they weren't too impressed with it - but others have also raised the idea to disable certain features, detail, at runtime, i.e. by skipping nodes - which is basically what the simplifier is doing by coming up with down-sampled mesh, even without a revamped LOD system in place.

https://sourceforge.net/p/flightgear/ma ... /32016452/
http://wiki.flightgear.org/FlightGear_N ... xperiments
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: Breaking down NLCD raster image

Postby statto » Wed May 11, 2016 7:49 pm

wlbragg wrote in Wed May 11, 2016 6:13 pm:
pull out individual layers from the raster, smooth them

Is it documented anywhere how to run the smoothing script? Is it done in Grass, QGIS or do you execute it in a command line environment?


I do everything in GRASS and there's a program within GRASS called v.generalize. I have found the "snakes" works best. There may be a QGIS generalizer, not sure how it works though.
Last edited by bugman on Wed May 11, 2016 9:10 pm, edited 2 times in total.
Reason: Fixed the quoting.
Custom Scenery available from http://www.stattosoftware.com/flightgear
statto
 
Posts: 2106
Joined: Fri Jan 25, 2008 10:57 pm

Re: Breaking down NLCD raster image

Postby statto » Wed May 11, 2016 7:51 pm

wkitty42 wrote in Wed May 11, 2016 7:42 pm:wasn't there some discovery a few weeks back that generating new scenery times needed to encompass all of the tiles being built? making the selection of the area a little larger or smaller ""fixed"" the problem (included the rest of or dropped the partial tile section) and allowed the build to complete...


It's not a discovery - it's been known for years, you have to build tiles in their entirety, and it's for scenery suddenly ending when you're flying over it, not, I believe, a tgconstruct issue.
Custom Scenery available from http://www.stattosoftware.com/flightgear
statto
 
Posts: 2106
Joined: Fri Jan 25, 2008 10:57 pm

Re: Breaking down NLCD raster image

Postby wkitty42 » Wed May 11, 2016 8:32 pm

i used the term "discovery" because some time in the last few weeks, two someones were building an area and the builds kept failing on both systems... someone else mentioned to maybe enlarge the selected area being worked on (like you do in terramastergui to choose the area you want to work with) and try building again... that worked for one of those someones and helped the other one to get their's to build, too... they "discovered" that partial tiles could cause the build tool(s) to fail at some point... in this case of building "the entire state", it may be a matter of needing to enlarge the selected area around "the entire state" so that it contains complete tiles... then it may work unless the system in question is running out of resources (RAM and/or drive space)...
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 9146
Joined: Fri Feb 20, 2015 4:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 20.04

Re: Breaking down NLCD raster image

Postby wlbragg » Wed May 11, 2016 8:35 pm

I do everything in GRASS and there's a program within GRASS called v.generalize.

OK, thanks. I think QGIS has something like that or maybe it is the grass plugin that has it that I am remembering. I'll have to give that a try.

In the mean time my run failed again, so I am moving it over to Linux to narrow down whether it is a data issue or a tool chain issue. It takes too much time to run each class independent to see if one is causing a problem (and that doesn't always catch it if it is a combination of classes) but if this works in Linux then I can chalk it up to the Windows build. If not then it is going to be a very long and time consuming process of elimination.

it may be a matter of needing to enlarge the selected area around "the entire state" so that it contains complete tiles...

It's just that I am using the same boundaries that I have always used successfully. But it is worth looking at if I don't get anywhere in Linux. Thanks for the suggestion.
Kansas and Ohio/Midwest scenery development.
KEQA, 3AU, KRCP Airport Layout
Intel i7/GeForce RTX 2070/Max-Q
User avatar
wlbragg
 
Posts: 7586
Joined: Sun Aug 26, 2012 12:31 am
Location: Kansas (Tornado Alley), USA
Callsign: WC2020
Version: next
OS: Win10/Linux/RTX 2070

Next

Return to Scenery

Who is online

Users browsing this forum: No registered users and 5 guests