Board index FlightGear Development Spaceflight

Earthview - an orbital terrain rendering engine [v2.0]

Discussion about development and usage of spacecraft

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby Thorsten » Thu Nov 03, 2016 6:54 pm

Hi Chris,

for experimenting and implementing, I guess a lower resolution than 8k would be preferable (I typically have to start FG 50 to 100 times to implement rendering stuff - so it matters a lot whether loading time is 20 seconds or 2 minutes...)

The processing should be like for the textures to get the same mapping, i.e. create normal map first, then do the border trick. As for re-sizing, I don't really know. You're probably altering the normals somewhat in the process, but resizing always leads to information loss, so I don't know whether it matters either way.

(Not really an expert in normal maps).

Perhaps let's try a simple re-size first.

(Still might be a few days - I've finished flight-testing the Shuttle today, but documentation is lacking...)
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby chris_blues » Thu Nov 03, 2016 10:53 pm

Yea, I'll give it another try tomorrow, and resize first, generate normal maps, the add the borders... Then we'll see, how it goes from there.
Don't hesitate to let me know if I'm incorrect or just annoying! As long as you do it gently! :)
Debian stable 64bit - i7 8x2.8GHz - 20GB RAM - GeForce GTS 450
Citation II
User avatar
chris_blues
Retired
 
Posts: 1577
Joined: Mon May 03, 2010 2:30 pm
Location: claws of real life
Callsign: chris_blues
Version: GIT
OS: Debian stable 64

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby Thorsten » Sun Nov 06, 2016 5:05 pm

Chris, is your site down? I can't download anything...
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby chris_blues » Sun Nov 06, 2016 5:46 pm

Works here...

Wasn't able to do anything. RL again... I'll report, when I got something.
Don't hesitate to let me know if I'm incorrect or just annoying! As long as you do it gently! :)
Debian stable 64bit - i7 8x2.8GHz - 20GB RAM - GeForce GTS 450
Citation II
User avatar
chris_blues
Retired
 
Posts: 1577
Joined: Mon May 03, 2010 2:30 pm
Location: claws of real life
Callsign: chris_blues
Version: GIT
OS: Debian stable 64

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby chris_blues » Mon Nov 07, 2016 4:03 pm

Ok, got it!

Here are the normalmaps in 2k, 4k and 8k resolution: https://musicchris.de/public_pics/FGearthview/normalmaps_earth.7z (ca 130MB)
Procedure in general was:
  1. Resize to 2k, 4k and 8k minus borderwidth ( Size / 128 )
  2. Generate normalmaps
  3. propagate the borders with last pixels

This is the bash-script used to generate the above normalmaps: https://musicchris.de/public_pics/FGearthview/convertHeightToNormalMaps.7z (ca 1.5kB)
This script needs imageMagick and GIMP with the gimp-data-extras package.

Here are the source files I used (by NASA): https://musicchris.de/public_pics/FGearthview/HeightMapsEarth.7z (ca 90MB)

At this moment it's still uploading. It'll be done in 1/2 hour. (16:30 german time / 15:30 GMT) [ Done ]

Enjoy! :)
Don't hesitate to let me know if I'm incorrect or just annoying! As long as you do it gently! :)
Debian stable 64bit - i7 8x2.8GHz - 20GB RAM - GeForce GTS 450
Citation II
User avatar
chris_blues
Retired
 
Posts: 1577
Joined: Mon May 03, 2010 2:30 pm
Location: claws of real life
Callsign: chris_blues
Version: GIT
OS: Debian stable 64

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby Thorsten » Mon Nov 07, 2016 6:37 pm

You might want to know this - the reason I can't access your page seems to be browser-version related. I've installed a brand-new Firefox here and it does the trick, but anything much older and your page is just gone.

Not sure whether that's the behavior you want... It used to work a few months ago even on the oldtimers.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby chris_blues » Mon Nov 07, 2016 8:10 pm

Thanks for the info. Do you happen to have ther version number of "stone-age"-firefox? Might be, that the SSL security policy is too much for old browsers... I'll take a look, and rather try to get my hands on old browsers. :mrgreen:

And do the normalmaps work?
Don't hesitate to let me know if I'm incorrect or just annoying! As long as you do it gently! :)
Debian stable 64bit - i7 8x2.8GHz - 20GB RAM - GeForce GTS 450
Citation II
User avatar
chris_blues
Retired
 
Posts: 1577
Joined: Mon May 03, 2010 2:30 pm
Location: claws of real life
Callsign: chris_blues
Version: GIT
OS: Debian stable 64

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby Thorsten » Tue Nov 08, 2016 7:25 am

Firefox version 18.0 definitely doesn't work.

As for the normal maps, I got as far as unzipping them, but I haven't started on the shader yet. Since they're going to mean 50% more loading time, I wanted to explore some ideas to reduce loading times as well...
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby chris_blues » Tue Nov 08, 2016 1:31 pm

:shock: Firefox 18 ??? In that case I'm glad you're on a newer one now. This is kind of ancient! Its support ended Sebtember 2013 with FF24.

Can't wait to see the normalmapped earth and clouds in action. 8)

Edit:
BTW, does it make sense to use dds-files for normalmaps? I have no clue, but since it's not directly a texture but a computed shader thing, my guess would be no.
Don't hesitate to let me know if I'm incorrect or just annoying! As long as you do it gently! :)
Debian stable 64bit - i7 8x2.8GHz - 20GB RAM - GeForce GTS 450
Citation II
User avatar
chris_blues
Retired
 
Posts: 1577
Joined: Mon May 03, 2010 2:30 pm
Location: claws of real life
Callsign: chris_blues
Version: GIT
OS: Debian stable 64

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby Thorsten » Tue Nov 08, 2016 2:57 pm

This is kind of ancient! Its support ended Sebtember 2013 with FF24.


I need working software, not supported software.... xterm probably exists unchanged since X11 first appeared on Linux, but it works perfectly fine for me.

Can't wait to see the normalmapped earth and clouds in action.


Well. here's the teaser for all your hard work:

This only shows up in low light, but the relief comes out nicely:

Image

Image

Here's my best shot of how clouds look normal-mapped:

Image

And here's an impression of it all combined:

Image


Now for the hard part, because loading the hires maps of course takes too long, so I need some cleverness with the effect predicates to force it only for the parts we actually need.

And the whole thing is quirky. Because of the atmosphere, it may not act as a normal normalmap - we need the normal of the underlying sphere to determine whether something is in light and to shade the fog layer, whereas we need the normal as distorted by the normalmap only for the terrain if it is in light - otherwise the normal map leaks into the darkness zone. And the effect of the map needs to be toned down, because while terrain behind the planet's curvature is really dark during the night, the atmosphere during the day takes care that shadow isn't deep black (we're not rendering the moon...)

So this needs to be sorted out carefully...
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby chris_blues » Tue Nov 08, 2016 3:10 pm

Wow! Impressive shots! Now I'm even more impatient! :D

As for FF, I'm using software like this, which needs to be actual, a bit like in win. There are stand-alone versions even for Linux. So I get one, unzip it in my home-folder and have a ready to go FF. It updates itself quite gracefully and just works. So I have a brand-new FF on my rather conservative Debian8... An unsupported out-dated FF is quite a security risk.
Don't hesitate to let me know if I'm incorrect or just annoying! As long as you do it gently! :)
Debian stable 64bit - i7 8x2.8GHz - 20GB RAM - GeForce GTS 450
Citation II
User avatar
chris_blues
Retired
 
Posts: 1577
Joined: Mon May 03, 2010 2:30 pm
Location: claws of real life
Callsign: chris_blues
Version: GIT
OS: Debian stable 64

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby Thorsten » Tue Nov 08, 2016 6:51 pm

Yeah, sorry... This is more complicated than I had anticipated...

Well - once we do use a normal map, we have one texture channel available for additional fun - so we might pack a specular map in... Or the sea ice in some form... But the normal map alpha is unused, so we can use it for whatever takes your fancy.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby chris_blues » Tue Nov 08, 2016 11:50 pm

Well, I've been riding along that ice / north-pole thing quite a while. And I'm sick of seeing the results. For science, it seems to be enough to have sth like 2km squares, to measure the ice sheets data, but for the eye it's repulsive! You see me given up on that. Though I very much appreciate you motivating for it. Yes, we can go for it. And if I can persuade myself a little bit more to just doing it (I just do), no matter, what it looks like, then I'll come up with an ice map. Hopefully, I'll have some time on my hands to dig into it the next days. Promised: I'll come up with this - as soon as I find the time to actually do it. (meaning: i have put this on my todo list - on the very top (since I brought it up actually))

Come to think of it, don't we have more channels already free for stuff like ice? Planet's texture is taken with all 4 channels (RGB + nightlights), but clouds could actually live with one of four channels (though, that might take the depth out of 'em (just a thought)). And all the normal maps also have the alpha channel free...

About the clouds in general:
Orbiting earth is one of the greatest things for me! And seeing the cloud textures as they are now, even without your normalmaps, I always found them quite spectacular. Which made me ask: why does he want to make it even better? - Seeing your screenies answer that!
Though I really found the textures most plastique, it wasn't possible to do some depth-math like where does the light get red and stuff like that. Most impressive!
Also: most interesting to learn by your work: how orbital visuals are governed by rather simple maths, compared to the math, sitting on the surface and watching the same scene from below...

Anyway; I think I'll try one more time to get the best ice sheets possible...

Howdy!
Don't hesitate to let me know if I'm incorrect or just annoying! As long as you do it gently! :)
Debian stable 64bit - i7 8x2.8GHz - 20GB RAM - GeForce GTS 450
Citation II
User avatar
chris_blues
Retired
 
Posts: 1577
Joined: Mon May 03, 2010 2:30 pm
Location: claws of real life
Callsign: chris_blues
Version: GIT
OS: Debian stable 64

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby Thorsten » Wed Nov 09, 2016 7:51 am

but clouds could actually live with one of four channels (though, that might take the depth out of 'em (just a thought)).


I guess that's true.

We are accessing the cloud textures anyway in the planet shader to build the cloud shadows, so we could potentially encode terrain features there as well (the cloud shader just gets to discard those).

Night lights we have. Specular map is kind of the classic. The ice is workable. Haze density distributions would be cool (right now we can just change overall haze, but some spatial modulation would look more compelling). I suspect for good seasonal distributions we're actually going to need the different seasonal textures.

(The cloud depth is derived from their alpha under the assumption that thick, high clouds have high opacity - which isn't always true, but it's good enough in practice- no extra channel needed since we don't have cloud normalmaps anyway).

Which made me ask: why does he want to make it even better?


Meh - I have a few books on the Space Shuttle and Apollo with these nice pictures of everything from orbit - I'm always aiming to get it like these. Right now I'm still missing clouds casting shadows into haze (ought to be doable). But I need to make it all efficient first, currently loading times and memory needs are just not acceptable.

I'll also try rgb format for the textures - if I remember some old tests of mine correctly, it ought to load faster then png (though not as fast as dds). rgb is frowned upon on FGData, but as long as we have it elsewhere...
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: Earthview - an orbital terrain rendering engine [v0.1]

Postby Thorsten » Wed Nov 09, 2016 6:42 pm

Okay, this is going to take a bit... With the normal maps, it becomes really texture-loading heavy. I've discovered two ways how one can not prevent first the whole planet from loading and then letting OSG sort out what we actually see. Now I have a third which might work (I get compelling loading times at least, but other quirks - the question is whether I can overcome them...)

Also, right now we have the planet sliced into 8 pieces. A further subdivision into 32 would clearly help with loading times and memory occupancy - but I need a 3d modeler to do the required model and texture cutting, I have no clue how this would have to be organized.

Finally: Chris, that's mainly a question to you since you host the textures. I believe using rgb format instead of png would speed up loading quite a bit. But it's twice as large on harddisk. Maybe you can run a few tests how this works out for you, and then make a decision what format we want to distribute.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

PreviousNext

Return to Spaceflight

Who is online

Users browsing this forum: No registered users and 5 guests