Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby portreekid » Tue Apr 28, 2020 8:04 pm

vanosten wrote in Tue Apr 28, 2020 3:51 pm:Since then, nobody has ever submitted anything.
I did, which makes flying such a joke as one building is [51.321625, 12.293311] taken by me and [51.343471, 12.364624] taken by radi. Leipzig is over represented. Generated ones are really far better. The effort to produce just one of those photo textures is immense.
portreekid
 
Posts: 448
Joined: Tue Jan 14, 2014 3:36 pm
Location: Leipzig
Callsign: PORTREE
Version: 2020.2.1
OS: Windows 10

Re: osm2city.py development

Postby vnts » Tue Apr 28, 2020 9:41 pm

vnts wrote in Sun Apr 26, 2020 1:07 pm:(In the /very long/ term (e.g. future years) there should be a 2nd iteration of buildings using different atlases that are easy for people to use (with a gimp file with layers & instructions), and easy to extract from available GPL2 compatible photos, parts of photos, photos of building parts, and texture packs. A shader based effect that builds partially randomised building facades from small parts of buildings like: wall textures, roof tiles, dirt maps, windows, doors, skylights, air-conditioning units, ornamental detail and, seasonal changes/decorations. It shouldn't need a completely unobstructed building facade photo taken under overcast conditions at the correct head-on angle. Existing texture photos have hard to avoid objects in front of buildings like: trees, cars, streetlamps, fences, bins. A /far/ smaller atlas for the same number of buildings means people only need work on a small part at a time to edit lighting for photo sources not taken under overcast conditions. It can be done without any changes in OSM2City data or scripts, just c++ & GPU side changes.)


vanosten wrote in Tue Apr 28, 2020 3:51 pm:Until then my plan remains to create a new texture atlas with better synthetic materials sometime in the future.


Firstly I was not saying that any plans for this iteration be changed. :mrgreen:

I was talking about a 2nd iteration up-to "few years in the future."

Any proof of concept is just a start. The rest requires input from graphics experts like Thorsten (who is mostly away, and will be for the foreseaable future if you read the effects&shaders forum). Looking ahead there are some big graphics projects that graphics programmers and core programmers have waiting. The compositor. After that, I imagine the next gen scenery will be a big project, and any big OSM2City work might have to fit while that's being worked on.

There has been some confusion in the past, so I'll try to be clear and detailed:

vanosten wrote in Tue Apr 28, 2020 3:51 pm:[A] the code in osm2city and the texture atlas are directly interlinked - osm2city need to "know" what is in a slot in the atlas to be able to use heuristics to be smart when applying textures (e.g. a sand-stone facade should not be on a skyscraper, a glass facade not on a warehouse, ...).


The way GPU programming, and OSM2CIty data from models and building shader lsits works is like this:

(There has been some recurring confusion with the way GPU/CPU.data works in the past, and a clarification should help for future issues too)

What the vertex shader sees as input: Model space X , Y, Z. Vertex attribuites: Variable1, variable2,variable3.. variable N

What the vertex shader outputs: Vertex position. Interpolated variables: iv1,iv2,iv3..ivN. This could just be Variable1 to N, or an arbitraty derived quantity from inputs.

Between input and output the shader is programmable with a great degree of freedom. Like writing a python function. Except it's written in C (which GLSL is based on).

OSM2City Data:
- 3d models in AC files: vertex coords, colours rgb, normals xyz, texcoords uv, ..
- Building shader lists: vertex coords, data 1, data 2 .... data N..

Vertex shader input data:
- Vertex coords are x,y,z.
- The rest of the data are just variables (called vertex attributes). It just so happens that 3 colours , normals, etc. are commonly needed vertex attributes. Varaible 1-3 could be a colour. Variable 4-6 could be normals. Variable

The flightgear core sees the OSM2City data, textures, any other data source, and regional material definitions. Flightgear has control over what it sends as per vertex data (vertex attribuites), what 2d/3d texture data files it sends based on regional definitions, and constants it sends from regional definitions and environment things like sun position.

Fragment shader:

What the fragement shader sees as input: Interpolated position x , y, z. Any textures that are relevant for the object as 2d or 3d data: tex1, tex2..texn. Interpolated quantities that the vertex shader has chosen to share: iv1, iv1 .. ivN.

Fragment shader outputs the colour at a position in the screen mainly.

Fragment shader input data from vertex shader:
These could include colours, normals, or texture coordinates directly from vertex attribuites, or derived as needed.

Between input and output the fragment shader is programmable. For instance there's complete control over what data to read from 2d/3d texture tables and how many lookups happen. Texture lokups can be just the original vertex attribuites, or a lookups based on model position, or lookups altered pseudo-randomly by an object ID.

(For clarity I left out constants called uniforms that C++ can change every frame that has useful info like the sun position at that frame, and a bunch of stuff like setting depth)

-------

To return to the case at hand the texture coordinate that OSM2City sets is just a variable to the shader. The shaders can consider all the data supplied and decide what texture positions to look up. The shader can look at the texture coordinates and see a lookup is for the front of a medium sized office building in the current texture atlas. Then the shader can just use that 'medium office building fronts' classification and look up a future texture atlas, and also psuedo-randomly choose between 3 medium front texture variants for that region.

There's complete detachment between texture coords and other data from OSM2City, and what the shader looks up and the colours at a screen position it outputs.

That's what I meant by "It can be done without any changes in OSM2City data or scripts".
vanosten wrote in Tue Apr 28, 2020 3:51 pm:[B] The possibility to use real photos and submit them has been implemented ca. 5 years ago by radi. Since then, nobody has ever submitted anything. [quote


The difficulty in getting photos of complete building fronts, and people not being photographers/artists, is why I came up with a design around parts of buildings. That way any/all sources can be combined, including things like dirt maps to show weathering. This way there's huge variation, and it's quicker as there's a smaller texture space to fill with textures people create and/or find.

"It /shouldn't/ need a completely unobstructed building facade photo taken under overcast conditions at the correct head-on angle. Existing texture photos have hard to avoid objects in front of buildings like: trees, cars, streetlamps, fences, bins." - i.e the NOT current way :) .

vanosten wrote in Tue Apr 28, 2020 3:51 pm:And IMO there is no reason to do so. Synthetic textures will almost always beat photos and will not have IP problems.

It doesn't matter to the shader if the sources are synthetic (e.g. based on noise textures, or texture packs), or parts of photos from where ever they can be found.

The idea is to get parts of buildings created synthetically or otherwise.
The current texture and any improvements can be chopped up into a new atlas in a second iteration.

-------

vnts wrote in Sun Apr 26, 2020 1:07 pm:A shader based effect that builds partially randomised building facades from small parts of buildings like: wall textures, roof tiles, dirt maps, windows, doors, skylights, air-conditioning units, ornamental detail and, seasonal changes/decorations.

Again this is for some distant future iteration, when people with relevant core/GPU expertise are around. It shouldn't change plans in the immediate future and not for 2020.1.

vnts wrote in Wed Mar 18, 2020 1:01 pm:In my old experiments I used a building version of the model ALS ultra fragment-shader.
It's enough to specify face number of each quad & corner number of each vertex as vertex attributes. The vertex coords can be stored shader side in an array of vec3s representing vertices. An array of ivec4s can contain the index number for each vertex at the 4 corners of each quad. It's then a simple matter of using the face & corner number to look up the index number of the vertex for each corner. Normals can be derived in shader saving 3 attributes. The shader can do a cross product of two quad vectors, or look up a table. If the face and corner number isn't available in explicit form, then it has to be reverse engineered from the tex coords and vertex positions in a future iteration that uses a custom texture sheet (when I did some experiments I just assumed a front face for all walls as the face was convoluted to derive from tex coords and vertex positions)
[/quote]

At around the time the example building lists for EHLE/EHAM was released at the end of last year, I spent 3-4 weeks doing some experiments prototyping and looking at conceptual difficulties of a new iteration.

This is more a prototype of a proof of concept :)

Don't take the presence of screenshots as a implemented version, it's just the concept. These are old screenshots.

I'll follow the way it was developed. These textures are all quick copies from the existing texture atlas.

Print of the new texture sheet format on a wall. Each row is a building part with variants. Ignore the big grey windows that were from the old light map or normal map.

Wall, window:
Image

Change building parts by changing the PRNG seed manually:

Take the wall texture and apply it (this is a super low res texture and the non-resyntesiser tiling I used at the time blurred it):
Image

(The wall needs to be mixed with a map of weathering which changes saturation and/or hue to make it less uniform.)

Tile huge windows + red wall:
Image

Tile huge windows+dark wall:
Image

Tile different window+red wall:
Image

Tile different window+yellow wall:
Image

Dividing building faces into different stories based on height: ground floor+repeated stories (shades of green)+left over space+top ornamental decoration area:
Image

Change repeated floor heights and the shader adjusts:
Image

Putting it together, doing multiple stories with the 3rd window variant and adding doors (I had a door for the second story at that time):
Image

The blue is just the light volume (circle) of the window for illumination at night. The same circle can be used to place stains and weathering around windows. It shows the type of control that the shader has when the shader the building facades from parts, as the shader knows what's being drawn and where it is in relation to other objects.

This is more for general interest, and what's as an example of possible without any change to OSM2City. As I said with people being busy or away because of covid-19, and upcoming big graphical projects to FG, it may take quite some time before another iteration is done.

Nothing about the current way needs to change as far as OSM2CIty is concerned (the texture atlas can be chopped up and reused).
vnts
 
Posts: 213
Joined: Thu Apr 02, 2015 12:29 am

Re: osm2city.py development

Postby merspieler » Tue Apr 28, 2020 10:46 pm

@vnts: Just out of curriosity: What's stopping you from getting this poc into working form and then into fg?
Love at first flight A<380
Attempting an osm2city worldbuild... Coming to YOUR terrasync....... when ever they start pulling it in, so go nag them on the mailing list (I totally didn't say that)!
merspieler
 
Posts: 696
Joined: Thu Oct 26, 2017 10:43 am
Location: Wish to be in YBCS
Callsign: JST935. ORI1711
IRC name: merspieler
Version: next
OS: Debian Bullseye

Re: osm2city.py development

Postby vnts » Wed Apr 29, 2020 12:47 pm

I'm not at all a programmer professionally let alone a graphics programmer (or a digital artist for that matter) : ).

A prototype of a proof of concept showing shader capabilities still needs a phenomenal amount of work :

- Finding which building features to make, and which features to keep. Needs artistic knowledge of buildings around the world, as well as graphics performance knowledge.
- Having some idea of where shader buildings fits in the overall scheme of things to figure out how much performance should be allocated. Then trimming features down or finding fast lower quality approximations of algorithms.
- Finding fast algorithms - for instance the light circle is overkill. It'd be better to distort a sine wave or something to do lighting if possible. I don't know all the algorithmic tricks used by ALS
- Dealing with various quirks needs expertise. For example: the GPU selects texture LoD (which mipmap to pick) by working on blocks of 4 pixels. It looks at how far apart in the texture coordinates each pixel samples to determine the resolution needed. When you have a texture atlas objects next to each other, at lower lod levels the edges get blurred in (image). To work around it required research and choosing lod level manually , and sampling the full resolution texture close to the edges.
- There are various quirks as each GPU is very different under the hood. There were some quirks that were probably due to my drivers at the time, and disproportionate changes in performance etc. that are unexplained.
- Knowing what works fast on various GPUs and what GLSL language things are prone to compiler issues on AMD etc.
- Optimisation and usability related to testing various ways of defining material properties - the amount of properties, the split between what's hardcoded and what's defined through regional definitions properties. For my prototype I used large constant register tables.
- Core support - Something like this also needs core support. Examples: a) It would be simpler if the texture lookups were moved to shaderside entirely, and there were simple face ids for each quad (it also makes it possible to reduce the number of varyings). As I wrote earlier, the shader building texture definitions were convoluted, and I used the front face for every wall. b) A way of specifying large arrays of values in regional definitions concisely. A lot of these would be set out in the form of a table with the ability to add new rows. To change the number of variants of textures and material/seasonal properties of each variant, and the layout specifications of buildings. c) What to do about object IDs for seeds. I used a hack in the vertex shader by combining (mixing bits of) rounded off position and building properties like house rotation/size, and then rounding of the itnerpolated seed varying in the fragment shader. I'm not sure how reliable that is.
- Versioning plans - this might need some attention sooner rather than later. Is there a way to maybe change the data format a long time in the future e.g. 3-5 years without crashing older versions of FG? Maybe it's easy, just by changing the folder the new data is stored in, or making terrasync servers not send files to old clients. It also may help if there was a system where building shader lists had versions. The idea being that subsequent versions add a new set of properties to each row after the current set of properties. Then old clients would be set to ignore X properties based on information in the file header.
- Art - the existing textures are fairly small(?), and most have light/shadows. When-ever a full implementation gets done art people would need to be involved to figure out the usability aspect, and to establish & document a workflow with useful plugins etc.

It's like a lot of things, the implementation takes a huge amount of time to do and polish compared to a prototype concept. This is a prototype of a prototype. (Possibly the compositor is an example, and also things like getting the infrastructure and everything ready to do world builds regularly with a system for people to submit shapefile improvements. It always takes longer than expected.).

What I wrote before wasn't at all a full writeup of how a possible scheme would work starting with the design goals to make it quicker/easier to get texture contributions, and from the perspectives of the core side, OSM2City side, GPU programming side, and art/regional definition side. :D It was just a demo of the type of things which were possible with shaders, with the technical bits and some solutions I had found to various problems taken out to avoid confusion.

Kind regards

Edit: as far as immediate term things go, I think everything is constrained by looking at space considerations for: hosting hard drive sizes, bandwidth for terrasync clients&servers, space on peoples hard drives on old laptops etc. I wrote a bunch of possible optimisation approaches earlier, there seem to be opportunities for a lot of space saving.

If space is a concern, then even the number of properties can be reduced/sacrificed (unlikely to ever get that bad).

I think your (Merspeliers) work shows the final terrasync hosting size to be greater than 2 TB. Maybe 3-4 TB? Not sure if that means organisations & people who donate hosting need new hard drives. It depends on what is a very comfortable size range for servers&clients, what is straining capability of people who donate servers or download files, and what is impossible (or might only be possible if a hosting service like google decides to help).
Last edited by vnts on Wed Apr 29, 2020 1:26 pm, edited 1 time in total.
vnts
 
Posts: 213
Joined: Thu Apr 02, 2015 12:29 am

Re: osm2city.py development

Postby merspieler » Wed Apr 29, 2020 1:16 pm

Well... I might be wrong, but to my point of view (and knowledge) you sound like a pro...

vnts wrote in Wed Apr 29, 2020 12:47 pm:It's like a lot of things, the implementation takes a huge amount of time to do and polish compared to a prototype concept. This is a prototype of a prototype. (Possibly the compositor is an example, and also things like getting the infrastructure and everything ready to do world builds regularly with a system for people to submit shapefile improvements. It always takes longer than expected.).


This kinda exists already... You can submit your shapes to osm and then ask me to rebuild the area... But a rebuild doesn't make too much sense yet, as osm2city doesn't get pulled into terrasync yet.
Love at first flight A<380
Attempting an osm2city worldbuild... Coming to YOUR terrasync....... when ever they start pulling it in, so go nag them on the mailing list (I totally didn't say that)!
merspieler
 
Posts: 696
Joined: Thu Oct 26, 2017 10:43 am
Location: Wish to be in YBCS
Callsign: JST935. ORI1711
IRC name: merspieler
Version: next
OS: Debian Bullseye

Re: osm2city.py development

Postby vnts » Wed Apr 29, 2020 2:29 pm

It's more like the only things formally taught is things at a very fundamental level (bits of math, and some tiny bits of physics underlying the optical problem being solved). Everything else is sort of out of interest and applying, applying/interest, applying/interest...until I get to things relevant for FG out of interest. :? :) What it means is that there can be huge holes in what I know, and just because I can understand some specific technical things (or just the math behind them) doesn't mean I have years of all-round knowledge. It's more like I'm learning and solving a string of problems from first principles in a far way area, and they just happen to be add up to something in FG. Usually people think of an expert if they know the technical/mathy stuff, and then assume they know the more superficial stuff and the rest, which doesn't work in this case.

Recall this post (link) where Thorsten explained to me what the alpha channel terminology was. :mrgreen:

This kinda exists already... You can submit your shapes to osm and then ask me to rebuild the area... But a rebuild doesn't make too much sense yet, as osm2city doesn't get pulled into terrasync yet.


I meant regular terrain. From previous discussions IIRC the source data was not public, and there wasn't a established system to contribute back improvements to landcover, water bodies, elevation etc. It's taken a long time to get towards the final stages for the next terrain world scenery build.

Kind regards
vnts
 
Posts: 213
Joined: Thu Apr 02, 2015 12:29 am

Re: osm2city.py development

Postby merspieler » Wed Apr 29, 2020 2:52 pm

Terrain is another thing... But if you want to scedule regular rebuilds, you've got to keep in mind, that it requires a machine to run on... and while my server is very capable of running the terrasync chain, it still costs me quite a bit for operating costs....
If I'm able to crowd fund running it, I've got no problem at all having quaterly rebuilds or so running on it.
Love at first flight A<380
Attempting an osm2city worldbuild... Coming to YOUR terrasync....... when ever they start pulling it in, so go nag them on the mailing list (I totally didn't say that)!
merspieler
 
Posts: 696
Joined: Thu Oct 26, 2017 10:43 am
Location: Wish to be in YBCS
Callsign: JST935. ORI1711
IRC name: merspieler
Version: next
OS: Debian Bullseye

Re: osm2city.py development

Postby vnts » Wed Apr 29, 2020 3:08 pm

Yeah exactly. Similar thing with people/organisations hosting OSM2City data. If they need to buy hardisks to store 2-4 TB, then the current scheme needs (?) to be tweaked and made less space hungry.

(IIRC for terrain Torsten was also looking at using Amazon cloud servers (maybe crowdfunded) with an AI that runs builds when prices are cheap. Might be simpler to just ask google if they want to donate some cloud server time regularly - I guess they can technically get PR when FG acknowledges their contribution in the changelog or something.)

Kind regards
vnts
 
Posts: 213
Joined: Thu Apr 02, 2015 12:29 am

Re: osm2city.py development

Postby montagdude » Wed Apr 29, 2020 3:22 pm

vnts wrote in Wed Apr 29, 2020 2:29 pm:I meant regular terrain. From previous discussions IIRC the source data was not public, and there wasn't a established system to contribute back improvements to landcover, water bodies, elevation etc. It's taken a long time to get towards the final stages for the next terrain world scenery build.

Kind regards
This would definitely be helpful. As it currently stands, you can't contribute back terrain unless you are able to avoid creating seams in the existing terrain, which basically makes it impossible unless you are doing an island. I would be more than happy to contribute landcover and waterbody shapefiles for the next worldbuild if there were a way to do that.
montagdude
 
Posts: 225
Joined: Tue Dec 31, 2019 6:04 am

Re: osm2city.py development

Postby merspieler » Wed Apr 29, 2020 4:09 pm

Similar thing with people/organisations hosting OSM2City data. If they need to buy hardisks to store 2-4 TB,


Osm2city is less than 100GB right now... osm2city + current terrasync will be less than 300GB... for the hosters.

using Amazon cloud servers (maybe crowdfunded) with an AI that runs builds when prices are cheap.


Using an AI for sceduling isn't a good idea... too complex... better use the funds to support those in the community who already have a server... I'll be cheaper.
Just play around with the AWS price calculator. On my server I can provide 80GB Ram, 28 vCPU and easily 2-3TB storage... At a running cost of about 50-60€/month... Tell me if you can get the same price with AWS.
In addition, an capacity expansion is a one time cost for me opposed by increased monthly cost when running in the cloud. The break even point is usually in less than a year, when compared.

... I've taken a look at cloud solutions before buying the server for osm2city... On the long run the cloud is always more expensive.

Might be simpler to just ask google if they want to donate some cloud server time regularly


Ofc... if you can convince anyone to provide something for free, I wouldn't say no.

montagdude wrote in Wed Apr 29, 2020 3:22 pm:I would be more than happy to contribute landcover and waterbody shapefiles for the next worldbuild if there were a way to do that.


IMO it would be better, if that would be contributed to osm and then use osm to build our landcover... The result of that is amazing (just look at the australian scenery build)
Love at first flight A<380
Attempting an osm2city worldbuild... Coming to YOUR terrasync....... when ever they start pulling it in, so go nag them on the mailing list (I totally didn't say that)!
merspieler
 
Posts: 696
Joined: Thu Oct 26, 2017 10:43 am
Location: Wish to be in YBCS
Callsign: JST935. ORI1711
IRC name: merspieler
Version: next
OS: Debian Bullseye

Re: osm2city.py development

Postby montagdude » Wed Apr 29, 2020 4:27 pm

merspieler wrote in Wed Apr 29, 2020 4:09 pm:
montagdude wrote in Wed Apr 29, 2020 3:22 pm:I would be more than happy to contribute landcover and waterbody shapefiles for the next worldbuild if there were a way to do that.


IMO it would be better, if that would be contributed to osm and then use osm to build our landcover... The result of that is amazing (just look at the australian scenery build)
I was thinking in terms of being able to contribute in a way that the result could be included in TerraSync in the future. I'm already able to build good looking custom scenery with it; see, for example, my Virginia, Maryland, Delaware, and DC scenery thread. Unless you are implying that the next worldbuild will pull landcover data from OSM? If so, I wouldn't mind contributing it to OSM, but how would I make sure it doesn't stomp on the existing OSM landcover data?
montagdude
 
Posts: 225
Joined: Tue Dec 31, 2019 6:04 am

Re: osm2city.py development

Postby merspieler » Wed Apr 29, 2020 5:07 pm

I haven't got any evidence that the next build will use osm but it's quite likely if you just look what the community has done already...

I'm not sure, about how to edit osm landclasses...
Love at first flight A<380
Attempting an osm2city worldbuild... Coming to YOUR terrasync....... when ever they start pulling it in, so go nag them on the mailing list (I totally didn't say that)!
merspieler
 
Posts: 696
Joined: Thu Oct 26, 2017 10:43 am
Location: Wish to be in YBCS
Callsign: JST935. ORI1711
IRC name: merspieler
Version: next
OS: Debian Bullseye

Re: osm2city.py development

Postby frtps » Thu Apr 30, 2020 12:08 am

merspieler wrote in Wed Apr 29, 2020 4:09 pm:
montagdude wrote in Wed Apr 29, 2020 3:22 pm:I would be more than happy to contribute landcover and waterbody shapefiles for the next worldbuild if there were a way to do that.


IMO it would be better, if that would be contributed to osm and then use osm to build our landcover... The result of that is amazing (just look at the australian scenery build)


Note that both the Maryland and Australian scenery builds make extensive use of govt landcover data as well as OSM. In the Australian case we also have 100mx100m pixelated vegetation cover data for the entire country (some places 10mx10m), so every patch of trees corresponds to an actual patch of trees (in theory). We have access to 27 different vegetation classes. I don't know if that sort of coverage is on the OSM radar, if it were this data would probably have been imported into OSM already. Likewise, the actual boundaries of built-up areas are not indicated routinely in OSM, whereas landcover data do contain this.

So I think non-OSM data will be with us for a long time.
frtps
 
Posts: 112
Joined: Sun Aug 05, 2018 9:58 am

Re: osm2city.py development

Postby vnts » Thu Apr 30, 2020 1:26 pm

merspieler wrote in Wed Apr 29, 2020 4:09 pm:Osm2city is less than 100GB right now... osm2city + current terrasync will be less than 300GB... for the hosters.


Ah, that's pretty good. I was put off by the 2.32 TB size quoted in the other thread, but that seems to be for a database.

merspieler wrote in Tue Apr 21, 2020 10:42 am:To bridge the waiting time, some current stats:

Global database:
Total import time: 34 days
-> of that for indexing: 18 days
Size: 2.32 TiB

Scenery:
Download size: 86.1 GiB
Size: 506.6 GiB

Build:
Duration: 50 days
Total logfile size: 15.9 GiB

Scenery breakdown (disk size/apparent size/number of files):
Code: Select all
               disk size        apparent size    number of files
Total:         506.6 GiB        419.6 GiB        4858967
Roads:         286.9 GiB        238.1 GiB        2510535
Pylons:        156.7 GiB        127.8 GiB        1283881
Buildings:     50.0 GiB         43.4 GiB         865330
Details:       13.1 GiB         10.4 GiB         199217


Time remaining until build will be continued: uncertain.


What do you estimate the compressed download size, and uncompressed size (~5.8x download size?), to be when it's done? You should probably let Torsten know, and if there are no no other blockers, then it's upto C++ side (Stuart..) and OSM2City side (Vanosten..) on how/when to proceed(?).

Only things to tick off might be
- People using previous versions? 2018.3 and earlier doesn't accidentally download building lists & crash. Will 2019.1 with it's slightly buggy building shader be served OSM2City?
- Versioning plans for building lists? Mentioned in this post (link). There are one or two optional/bonus bits of data which ([url=https://forum.flightgear.org/viewtopic.php?f=5&t=22809&start=1215]link[/link], for example, might help with making L, U, or more complex shaped buildings with compatible roofs out of rectangular buildings in a way that works well with future iterations. These could be added later too if the list file format is made extensible/backwards compatible.

montagdude wrote in Wed Apr 29, 2020 4:27 pm:As it currently stands, you can't contribute back terrain unless you are able to avoid creating seams in the existing terrain, which basically makes it impossible unless you are doing an island. I would be more than happy to contribute landcover and waterbody shapefiles for the next worldbuild if there were a way to do that.


The sourceforge website allows searching mailing lists (left hand side) and sorting by date. Searching for the latest report on world scenery terrain update (see link):

Torsten on the mailinglist wrote:over the last couple of weeks, I finally succeeded to at least partly
process/regenerate the cs_* landcover data from Martins database. Recap:
This is the "best of breed" data set combining vmap0, corine and manual
edits of worldwide landcover. NLCD is not part of the dataset.
...
I now have approx. 100GB of clipped polygon data that we eventually can use to recreate ws 2.x from.
...
I'd really love to make this processed data available online, which I
believe is in line with the restrictions Martin put on his database.
Unfortunately, I don't know of any free and public service to host that
amount of data and allow the bandwidth to handle it.
...
As stated, OSM line data has not been processed and can easily be added as
other landclass directories.

NLCD is for the US what CORINE is for Europe. Once we can process that
source, we can replace the cs_* generated polys in 1x1 deg chunks per
landclass.
...


- Your NLCD based data should be a big improvement for USA.
- There's was/is(?) a lack of a host for the combined landcover data set. (Once this is solved, there might be a manual way to contribute to a best-of-all-sources landcover database??)

merspieler wrote in Wed Apr 29, 2020 4:09 pm:... I've taken a look at cloud solutions before buying the server for osm2city... On the long run the cloud is always more expensive.

I don't know what the state of thinking with regards to cloud hosting is for the terrain build is, I only recall it was considered (prototyped?) sometime.

merspieler wrote in Wed Apr 29, 2020 4:09 pm:On my server I can provide 80GB Ram, 28 vCPU and easily 2-3TB storage... At a running cost of about 50-60€/month.

You should possibly mention these stats in the mailing list as an available option for doing regular terrain builds.

merspieler wrote in Wed Apr 29, 2020 4:09 pm:Ofc... if you can convince anyone to provide something for free, I wouldn't say no.

Google are the type of company that may possibly help out, as Flightgear is a volunteer project that benefits the public good, and it's an insignificant amount of cloud server time for terrain builds or hosting space for them. The big benefit of striking an official relationship with google might be that FG gets access to bits of their streetview photo database (and maybe any close-up aerial images they own rights to for terrain textures). The idea being that FG could upload some extracts with trees (or maybe buildings) into a repo, and google signs off periodically (e.g. every 3 months) releasing the content into GPL2+. Google generally likes opensource. The reason for their restrictive license is seemingly mainly so it doesn't enable a competitor to run a rival service using their collected data. Flightgear's use is only a few bits of images for textures, so they can't be hurt financially. It may be something worth trying out.

Kind regards
vnts
 
Posts: 213
Joined: Thu Apr 02, 2015 12:29 am

Re: osm2city.py development

Postby Johan G » Thu Apr 30, 2020 7:04 pm

vnts wrote in Thu Apr 30, 2020 1:26 pm:Google are the type of company that may possibly help out, [...] The big benefit of striking an official relationship with google might be that FG gets access to bits of their streetview photo database (and maybe any close-up aerial images they own rights to for terrain textures).

What would they want in return? And have something like that even happened before? I would not hold my breath waiting for that to happen. :wink:
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator
 
Posts: 5899
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: No registered users and 3 guests