Board index FlightGear Development Scenery

Procedural buildings in OSM before part of Scenery

Questions and discussion about enhancing and populating the FlightGear world.

Procedural buildings in OSM before part of Scenery

Postby vanosten » Sat Sep 15, 2012 12:47 pm

Hi

In http://wiki.flightgear.org/OpenStreetMap_buildings it is rightly stated that the coverage of buildings in OSM is not very good - and I know that in Switzerland the statistics are not encouraging at all. However having buildings in realistic places and realistic geometry in relation to streets is an important visual aspect for me.
So the idea would be to procedurally generate buildings based on OSM data (not uploading to OSM!) based on other OSM data like streets, area classifications, ... I thought of enriching offline OSM data with buildings - which could then in a later step be used to actually generate buildings in Flightgear (or X-Plane with http://osm2xp.com/). Due to differences in landuse patters between e.g. cnetral Europe and USA the algorithm would naturally have to be configurable - however somewhere we need to start.

Would that be a good idea? Has this already been attempted (I mean based on OSM data, not generic procedural cities)?
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 416
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: Procedural buildings in OSM before part of Scenery

Postby Hooray » Sat Sep 15, 2012 2:01 pm

So the idea would be to procedurally generate buildings based on OSM data (not uploading to OSM!) based on other OSM data like streets, area classifications, ... I thought of enriching offline OSM data with buildings - which could then in a later step be used to actually generate buildings in Flightgear


Have you looked at: http://wiki.flightgear.org/OpenStreetMap_buildings

FlightGear does have support for random buildings (i.e. procedurally created buildings): http://wiki.flightgear.org/Random_Buildings
Image

This is a feature added fairly recently:
viewtopic.php?f=5&t=16083

The development discussion can be read at: http://www.mail-archive.com/flightgear- ... 37003.html

The memory consumption of the system was recently improved:
viewtopic.php?f=18&t=17589
http://wiki.flightgear.org/FlightGear_N ... pment_news

There's a fairly long-standing idea to improve FG's "autogen" support using OSM data: http://wiki.flightgear.org/AutoGen_Scen ... FlightGear

However, at the moment the "random buildings" system can merely be customized via editing materials.xml and by providing custom texture-sets: https://gitorious.org/fg/fgdata/blobs/m ... .materials

In other words, to use OSM data for the creation of procedural buildings, you would currently need to know C++ - at the moment, there's no dedicated property tree or Nasal (scripting) interface to affect the creation and placement heuristics dynamically.

The C++ code you need to look at is mostly in SimGear in: http://gitorious.org/fg/simgear/trees/n ... scene/tgdb
Also see: viewtopic.php?f=5&t=16083&start=15#p157022

While I am not sufficiently familiar with the random buildings code, I am fairly familiar with Nasal scripting, and I can help people who are interested in exposing the system to Nasal space, like I mentioned here: viewtopic.php?f=5&t=16083#p155667

This would then make it possible to manipulate and customize the random buildings code via Nasal, so that OSM data (and other sources) could be used.
Exposing new bindings/hooks to/from Nasal is fairly simple actually: http://wiki.flightgear.org/Howto:Extend_Nasal

Some people have had success using OSM data and creating pre-compiled scenery:
viewtopic.php?f=5&t=16083#p155668
viewtopic.php?f=5&t=10016

There's also the "OSM2XP" effort: viewtopic.php?f=5&t=16083&start=15#p157020
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11336
Joined: Tue Mar 25, 2008 8:40 am

Re: Procedural buildings in OSM before part of Scenery

Postby vanosten » Sun Sep 16, 2012 7:18 am

Hi

Thank you very much for your reply - it is more comprehensive than I even had hoped for :-)

I know Java and Phyton, so lokking into C++ and Nasal should be doable. Thanks to your pointers I will have plenty to read and understand.

However I have to be convinced by reading the code and the articles, that generating buildings on the fly is better suitable than having the buildings' location, geometry and potentially properties calculated on the fly. I might not be able to express myself precicely enough, but I intend to
  • build a pre-processor to enrich OSM data offline, such that OSM data seems to appear to be more complete in terms of buildings coverage than it is now. Basically I am aiming at what is written in the chapter "Challenges" in http://wiki.flightgear.org/OpenStreetMap_buildings
  • This "fake data could then be used by a Flightgear enhanced osm2xp or http://wiki.flightgear.org/OpenStreetMap_buildings to actually create pre-complied scenery in Flightgear format
  • The more "real" buildings OSM contributers over time would add, then less "enriched" buildings would have to be created over time
  • I have to check the feasibility, but theoretically it should be possible to take care of manually added buildings in WorldScenery and include them aspart of the "enriched" buildings

Well, give me some time to read all articles, try to understand it and write a proof-of-concept in Java or Python.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 416
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: Procedural buildings in OSM before part of Scenery

Postby stuart » Mon Sep 17, 2012 10:42 am

Hi Vanosten,

This is a really interesting idea. A couple of thoughts based on my experience writing the current random buildings code and general FG experience:

1) I think the right place for this information is in the scenery models database, or probably some similar off-shoot of that DB. We already have the OSM data in a PostGIS DB there, so you could retrieve data from that, do some complex off-line processing and then write the resulting data into our global database. Martin Spott is probably the best person to talk to about this. However, he's not on these forums and has little time available right now. If you get some code to read the DB and write the data, I can talk to him about creating some new data structures for this effort. However, one fly in the ointment is the OSM license. I understand it's changing to be more FG compatible, but we'd need to make sure the work is GPL compatible. For some people this isn't very important, but for myself and Martin it is.

2) At present we use line data (VMPA0, hopefully OSM once the licensing is sorted) for major roads and have specific textures for different types of built up areas (Urban, Town etc.), which include smaller roads. The random buildings code uses this information to place and rotate the buildings so that they match up with the textures (using another .png as an mask). Obviously using OSM data would create buildings in different locations, which wouldn't match up with the existing textures. If we used the full set of OSM data to create complete road networks, the textures could simply be a mixture of concrete and grass, without any roads. However, using the entire OSM data generates huge scenery files, which IIRC terragear struggles to process and FG struggles to display.

3) The main problem I had to solve when creating the random buildings was finding a way to display them without taking up too much FG resource, as a typical urban area (say around KSFO) has 300k+ buildings. There were two aspects to this: minimizing the number of objects (which use up CPU and memory), and minimizing the memory footprint. The first was solved by grouping large numbers of buildings together into a single object, which is something you'd want to do with your buildings as well and should be straightforward (I'd suggest storing them individually in the scenery DB, then having an exporter that writes them out in blocks). The second problem was more difficult to solve. In the end I created a small number of buildings and then use specialist GPU shader code to instantiate them multiple times. That will be much trickier for you to do, and will probably require some specific C++ code in FG to handle your buildings as a special case.

I'd be very happy for my random buildings to be supersede by proper procedural buildings, and would be happy to discuss the technicalities further.

Good luck!

-Stuart
G-MWLX
User avatar
stuart
Moderator
 
Posts: 1469
Joined: Wed Nov 29, 2006 9:56 am
Location: Edinburgh
Callsign: G-MWLX

Re: Procedural buildings in OSM before part of Scenery

Postby Gijs » Mon Sep 17, 2012 9:04 pm

Hi all,

from my experiments with OSM buildings so far (yes, I wrote that wiki), the biggest problem is the impact that thousands of buildings have on any system. I still need to test what the effect of grouping the buildings into larger models is, but first I need to slightly rewrite bob. If this proofs to be sufficient, we can simply add some script to the database that groups those models when it creates the scenery.

But I'm afraid that grouping the models isn't sufficient. We probably need to loose the details and create a relatively small range of generic buildings (of different sizes), like Stuart did for the random buildings. Based on the OSM floorplan, a best fitting model would then added to the scenery. Don't know what would be the best way to feed the floorplans into FlightGear though... The "facades" from X-Plane look like a good method, see http://scenery.x-plane.com/library.php? ... acades.php

Btw, importing non-procedural objects (eg. windturbines, power pylons etc.) is documented in http://wiki.flightgear.org/OpenStreetMap_import.

Cheers,
Gijs

@Stuart, the license change was completed last week. Martin is currently importing the new OSM planet into the database.
Airports: EHAM, EHLE, KSFO
Aircraft: 747-400
User avatar
Gijs
Moderator
 
Posts: 9377
Joined: Tue Jul 03, 2007 2:55 pm
Location: Amsterdam/Delft, the Netherlands
Callsign: PH-GYS
Version: Git
OS: Windows 10

Re: Procedural buildings in OSM before part of Scenery

Postby vanosten » Tue Sep 18, 2012 6:30 am

Hi

Thanks all for your replies. One related idea, which I have in my head, is to use a OSM floorplan (great term :-) ) in order to generate customized tiles for each town/village to be able to include them in (custom) sceneries. I am mostly flying in Switzerland and the generic tiles do not fit well (the geometry is too regular). So basically the OSM floorplan could be used both as a basis for rendering 3D buildings and for rendering adapted tiles for towns/villages.

Well: right now I have to find time, sit down and write some code to make a proof of concept. Based on that experience I can then commit to more.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 416
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: Procedural buildings in OSM before part of Scenery

Postby Hooray » Tue Sep 18, 2012 5:09 pm

Sorry, I only just noticed that you replied ...
As you can see, we have had a fair share of discussions related to this. Stuart's work has certainly paved the way for even more improvements.
Like I mentioned, I would be interested in exposing the creation of procedural scenery to scripting space (or even just the property tree) so that customizing this aspect of FG is no longer possible purely from C++ space.

With your background in Java and Python, understanding the C++ and Nasal side of things should definitely be pretty easy - even though Nasal is in fact closer to Java/JavaScript syntax-wise ... but many Nasal concepts will seem familiar to you due to Python.

Stuart is the developer of the random buildings code (and the random trees), so he's got plenty of experience with procedural creation of scenery elements.
To understand the C++ side of things, you'll definitely also need to be familiar with OpenGL and preferably OSG, i.e. 3D graphics and 3D maths.
But even if that's not your thing, reusing Stuart's code to the largest possible degree would definitely be a good idea, because he's already tackled many of the difficult issues - and creating an interface (scripting or property tree) is definitely simpler than writing such a system from scratch.

Once you do take a look, you'll also find a number of threads related to the open source PixelCity project, which uses a fixed rendering pipeline (i.e. no shaders) - but which is nonetheless extremely impressive. Like Torsten mentioned a while ago, this would need to be ported to OSG (and shaders) to be usable in FG, but this could probably be done by merging it with Stuart's code, because that's a very good (and
working!) example of how the FlightGear/SimGear side of things works.



If you are interested in pre-processing and enriching vector data, you'll probably want to get in touch with some of the other folks who have done similar things, such as the osm2xp project.
Also, if that's definitely your main interest, you may also want to look at TerraGear, which is the "FG scenery compiler suite" to build scenery.

That said, a "pre-processor" could also be implemented as a native FG component, i.e. within the main fgfs executable - quite probably even as a fully threaded component, so that pre-processing could take place in a simple worker thread using exclusive read/write access.

I'm just saying this because "compiling" scenery from scratch is generally understood not to be a simple task for most people here. In other words, even if you should come up with such an OSM-pre-processor to enrich FG scenery, the resulting data may not be usable by most people - not without an official scenery rebuild at least.

Thus, having a component that would be running inside (or next to) FG would be easier from a deployment point of view.
FG already has fairly extensive networking support, and it would be comparatively simple to define an I/O interface to enrich scenery with metadata for scenery vector information, whereas said metadata could be created by a separate process, running in parallel with FG - that would mean that it would be up to us to provide such an interface, and that it would be up to you to provide an application that talks to FG in order to provide a "metadata feed".

The corresponding data would not necessarily need to be propagated via sockets, it could just well use pipes on most OS.

So there are really various options here and it really only depends on your background and your main area of interest.

Obviously, being able to build FG from source would still be useful regardless, because most approaches would inevitably require modifications to the C++ side of things - even though you may not necessarily need to implement them yourself.

Well, give me some time to read all articles, try to understand it and write a proof-of-concept in Java or Python.

Before you spend any serious amount of time coding, I'd suggest to report back here and let us know about your ideas and approach - it's pretty likely that some of us can provide further pointers that could turn out to be real time savers when getting started hacking FG.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11336
Joined: Tue Mar 25, 2008 8:40 am

Re: Procedural buildings in OSM before part of Scenery

Postby vanosten » Sat Sep 22, 2012 11:24 am

Hi
Thank you very much for your suggestions. I agree that we might have to go with less details and a few / couple of buildings - maybe even replacing existing OSM floorplans in place. It is not about matching reality - it is about making something that looks realistic.
I have built some paper based algorithms and the next thing is to try the stuff in programming. I will most probably go for a Java based prototype using the JTS Topology Suite library.
If the prototype prooves sucessful and estimates for the next step do not look too bad, then I need to take a decision with your input, whether to port it to C++ / GEOS or C++ / Simgear. However this would mean that i need to learn C++.
Which leads to a question, which I could not find answered on the development related wiki pages: what is the most recent C++ standard supported?

(The rest of the weekend will go trying to build a scenery from scratch - despite TerraGear GUI I am still going nuts)
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 416
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: Procedural buildings in OSM before part of Scenery

Postby ckruetze » Sat Sep 22, 2012 6:08 pm

vanosten wrote in Sat Sep 22, 2012 11:24 am:Hi
Thank you very much for your suggestions. I agree that we might have to go with less details and a few / couple of buildings - maybe even replacing existing OSM floorplans in place. It is not about matching reality - it is about making something that looks realistic.


Are you sure about that? Right now my town simply dosn't have buildings, no big deal. But I think I would find it more irretating if there were buildings that didn't match what I see when I look out of the window.
ckruetze
 
Posts: 32
Joined: Sat Jun 11, 2011 12:06 pm

Re: Procedural buildings in OSM before part of Scenery

Postby Soitanen » Thu Oct 25, 2012 1:37 pm

Question about script, described here: http://wiki.flightgear.org/OpenStreetMap_buildings

After successful work of script I have *.stg file with text (tried to do only 1 building):
Code: Select all
OBJECT_STATIC osm27871893.ac 30.3842007 59.8470129 -9999 180

And it appears in game at 9999 meters depth. Is it right?
Boeing 737-300. Reworked cockpit, FDM, autopilot and much more. WIP.
Boeing 737-800. WIP. Canvas PFD and ND.
Antonov An-24B. Made from scratch. Very good FDM. 3D model by Adrian. WIP.
Project Russia (some cities, based on OSM with custom objects).
Soitanen
 
Posts: 489
Joined: Sat Jun 16, 2012 6:50 am
Location: Saint-Petersburg, Russia
Version: git
OS: Linux Mint 17

Re: Procedural buildings in OSM before part of Scenery

Postby Gijs » Fri Oct 26, 2012 7:28 pm

Yes. The script does not know about scenery elevation, it just defaults to -9999 for all objects. The correct altitude strongly depends on the terrain that you're using, so you need to calculate the altitudes for a specific terrain data (eg. from TerraSync).
You can run a script to get the altitudes, eg. this one: http://wiki.flightgear.org/Howto:Get_a_ ... of_objects
Airports: EHAM, EHLE, KSFO
Aircraft: 747-400
User avatar
Gijs
Moderator
 
Posts: 9377
Joined: Tue Jul 03, 2007 2:55 pm
Location: Amsterdam/Delft, the Netherlands
Callsign: PH-GYS
Version: Git
OS: Windows 10

Re: Procedural buildings in OSM before part of Scenery

Postby Soitanen » Mon Nov 26, 2012 3:59 pm

I have made straight convert from OSM to FG using bob.pl script, which is built-in in my own om2fg converter. See, what is result. Place - Sochi, Russia.
Image
Image
Image
Image
I use for this non-recommended algorithm, that places objects on ground, because I don't know elevation of each building, when converting is in progress. So I have FPS drop. But it's nice looking!
Boeing 737-300. Reworked cockpit, FDM, autopilot and much more. WIP.
Boeing 737-800. WIP. Canvas PFD and ND.
Antonov An-24B. Made from scratch. Very good FDM. 3D model by Adrian. WIP.
Project Russia (some cities, based on OSM with custom objects).
Soitanen
 
Posts: 489
Joined: Sat Jun 16, 2012 6:50 am
Location: Saint-Petersburg, Russia
Version: git
OS: Linux Mint 17

Re: Procedural buildings in OSM before part of Scenery

Postby Gijs » Mon Nov 26, 2012 4:06 pm

Looks good! I see a few shape-closing issues here and there, but overall it looks quite convincing I think. Did you use the latest version (from Git, newer than May 2012)? I'm asking, because I see a few rectangular buildings that don't have a gable roof. For example the post office building at http://osm.org/go/x~F1V1E8B--

EDIT: Just built that building (51696113) myself and I did get a gable roof...
Airports: EHAM, EHLE, KSFO
Aircraft: 747-400
User avatar
Gijs
Moderator
 
Posts: 9377
Joined: Tue Jul 03, 2007 2:55 pm
Location: Amsterdam/Delft, the Netherlands
Callsign: PH-GYS
Version: Git
OS: Windows 10

Re: Procedural buildings in OSM before part of Scenery

Postby Soitanen » Mon Nov 26, 2012 4:10 pm

I use script, downloaded from git somewhere in October 2012. I made little modification of it (OBJECT_STATIC_AGL, buildings without defined height or levels cannot be higher then 2 levels, sloped roof only on 1 level buildings).

EDIT: translation problem - gable is roof with slope? Not horizontal. Right?
Last edited by Soitanen on Mon Nov 26, 2012 4:17 pm, edited 1 time in total.
Boeing 737-300. Reworked cockpit, FDM, autopilot and much more. WIP.
Boeing 737-800. WIP. Canvas PFD and ND.
Antonov An-24B. Made from scratch. Very good FDM. 3D model by Adrian. WIP.
Project Russia (some cities, based on OSM with custom objects).
Soitanen
 
Posts: 489
Joined: Sat Jun 16, 2012 6:50 am
Location: Saint-Petersburg, Russia
Version: git
OS: Linux Mint 17

Re: Procedural buildings in OSM before part of Scenery

Postby Gijs » Mon Nov 26, 2012 4:15 pm

Soitanen wrote in Mon Nov 26, 2012 4:10 pm:sloped roof only on 1 level buildings

Ah, there it is. Good to know that it's not a failure of the script :-)
I like the idea, will add it as an option to the original script.

EDIT: Yes, "gable roof" is a triangle shaped (as seen from the side) roof.
Airports: EHAM, EHLE, KSFO
Aircraft: 747-400
User avatar
Gijs
Moderator
 
Posts: 9377
Joined: Tue Jul 03, 2007 2:55 pm
Location: Amsterdam/Delft, the Netherlands
Callsign: PH-GYS
Version: Git
OS: Windows 10

Next

Return to Scenery

Who is online

Users browsing this forum: No registered users and 2 guests