Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby wkitty42 » Sat Sep 24, 2016 5:59 pm

vanosten wrote in Sat Sep 24, 2016 8:58 am:I am still a bit surprised, how much delayed scenery objects load - despite the fact that I have set LOG_DETAILED in FGFS to 15000.

is this supposed to be LOD_DETAILED? what is your --visibility set to? i use the following...

Code: Select all
# the following distance numbers are based on the horizon distance of 369km @ 35000ft altitude
# visibility 370000 takes fgfs up to ~9.5G of RAM usage with these current settings
# visibility 100000 takes fgfs up to ~4G of RAM usage with these current settings
--visibility=150000
--prop:/sim/rendering/max-paged-lod=400
--prop:double:/sim/rendering/static-lod/ai-detailed=0
--prop:double:/sim/rendering/static-lod/ai-range-mode-pixel=1
--prop:double:/sim/rendering/static-lod/detailed=3000
--prop:double:/sim/rendering/static-lod/rough=65000
--prop:double:/sim/rendering/static-lod/bare=370000
"You get more air close to the ground," said Angalo. "I read that in a book. You get lots of air low down, and not much when you go up."
"Why not?" said Gurder.
"Dunno. It's frightened of heights, I guess."
User avatar
wkitty42
 
Posts: 5757
Joined: Fri Feb 20, 2015 3:46 pm
Location: central NC, USA
Callsign: wk42
Version: git next
OS: Kubuntu 14.04.5

Re: osm2city.py development

Postby stuart » Mon Sep 26, 2016 9:05 am

Re: Delayed loading.

The new OBJECT_OSM_BUILDING_DETAILED does have slightly different behaviour from what you had previously using LOD animations.

(For the following discussion we'll ignore the element of randomness that FG uses when determining the LOD range of STG objects)

We need to differentiate between when FG loads an model from disk, and when it is made visible within the sim. Previously the entire mesh of buildings were loaded at /sim/rendering/static-lod/rough range, but some were only visible within /sim/rendering/static-lod/detailed due to the use of LOD animations.

Now, those meshes are split into OBJECT_OSM_BUILDING_DETAILED and OBJECT_OSM_BUILDING_ROUGH. Some objects are being _loaded_ at /sim/rendering/static-lod/rough and some at /sim/rendering/static-lod/detailed range. This should mean that there are smaller meshes so all things being equal the buildings at OBJECT_OSM_BUILDING_ROUGH should appear slightly sooner (less to load), and those at OBJECT_OSM_BUILDING_DETAILED slightly later (a new object has to be loaded before it can be displayed).

Another factor is that the LOD is calculated from the center of the object. So with a 2kmx2km mesh object rather than 1kmx1km, you'll be ~500m closer before it starts to get loaded.

-Stuart
G-MWLX
User avatar
stuart
Moderator
 
Posts: 1469
Joined: Wed Nov 29, 2006 9:56 am
Location: Edinburgh
Callsign: G-MWLX

Re: osm2city.py development

Postby vanosten » Mon Oct 03, 2016 7:52 pm

I have implemented a new overlap check with parameter OVERLAP_CHECK_CONVEX_HULL. Basically it reads the ac-file of static objects and creates a convex hull around the points discovered in the first object's vertices. You can see some pictures in the documentation.

You can see the result for SBRJ by install the following object folder. Some pictures with existing algorithm:
Image
Image

Some pictures with new algorithm (I have omitted textures, roads etc. to make it more transparent what is osm2city buildings.py and what is static):
Image
Image

I believe that the new algorithm is better and does not need a buffer like OVERLAP_RADIUS. Most probably just because the rotation is applied against the correct spot. The existing algorithm contains the possibility to respect inside structures - but is this really needed?

Please test. And please tell me whether there should be a similar test for shared objects.
Last edited by vanosten on Tue Oct 04, 2016 4:33 pm, edited 1 time in total.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Mon Oct 03, 2016 8:05 pm

stuart wrote in Fri Sep 23, 2016 9:11 pm:Also, I will be part of the team demoing FG at FSWeekend in the Netherlands on 5/6 November. We're likely to demonstrate the Rio scenery, and I'd like to include osm2city output as part of that. So if you want a "target date" for any development, that would be a good one :)

-Stuart


Could you please consider showing LSZH (which currently looks like the next default airport) instead of SBRJ? The reason is that in order for SBRJ to look somewhat ok with OMS data, more development needs to be done, than what can fit in the next month (at least my time is limited). Looking at pictures from Rio it is clear that we lack a lot of "white" textures of different sizes. So we would need more textures - and that also might require to revamp texGUI.py as well as creating parameters to better filter / select textures (not difficult, but still needs to be done).

Looking at OSM data for LSZH the scenery looks "complete" already now - and the airport buildings are also quite nice with lot of progress.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Tue Oct 04, 2016 6:29 pm

vanosten wrote in Mon Oct 03, 2016 7:52 pm:I have implemented a new overlap check with parameter OVERLAP_CHECK_CONVEX_HULL. Basically it reads the ac-file of static objects and creates a convex hull around the points discovered in the first object's vertices.


OVERLAP_CHECK_CONVEX_HULL now also takes into account shared objects depending on parameter OVERLAP_CHECK_CONSIDER_SHARED plus possibility to use buffer (parameters OVERLAP_CHECK_CH_BUFFER_STATIC and OVERLAP_CHECK_CH_BUFFER_SHARED) - however as convex hull already is a conservative approach, buffers might not be needed.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Sat Oct 08, 2016 12:31 pm

I have added 5 new parameters to be able to filter textures. If nothing specific is specified, then just all available textures are used. See the documentation.

Given that the available number of textures is rather low (and adding new might be a bit difficult because the texture UI is broken), you might need to compromise between having plausible textures for your scenery and having a larger diversity of textures within a scenery.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby saul » Fri Oct 14, 2016 9:48 pm

Hi guys!

I have tested the latest version of osm2city in Stockholm area. The results are just amazing! This is just like flying in a new simulator. I can recognize all the places that I drive daily. This is an overwhelming improvement over the standard scenery. Many thanks for the effort spent in this fantastic tool!
Here down you can see my first experiments. I have added an original picture and the result of osm2city for comparison. Also, my params.ini file is copied below. This my first try, so if you know of a better configuration of the parameters please let me know. I receive a lot of errors due to missing textures due to colors, etc. Accordingly, many objects, some of them representative of the city, are not placed.

My long term ambition is to create a public repository with major Swedish cities.

Pictures:

Gamlastan (old city downtown)
http://www.ladda-upp.se/bilder/eggasbtvwlafqh/
http://www.ladda-upp.se/bilder/qwrfipabzxerst/

Kista Science City (North of Stockholm)
http://www.ladda-upp.se/bilder/vjqciswdqmbbq/
http://www.ladda-upp.se/bilder/ghwypxfnmsglfx/

Arlanda Airport (~20 km north of Stockholm)
http://www.ladda-upp.se/bilder/oeptlyemrkjpis/
http://www.ladda-upp.se/bilder/tolbsxhinfbpda/

Bromma Airport (inside Stockholm)
http://www.ladda-upp.se/bilder/hlzkutlzqltpjy/
http://www.ladda-upp.se/bilder/cbgspenaumdzy/

Stockholm osm2city pictures in a sunny day
http://www.ladda-upp.se/bilder/tubuivrrywumfp/
http://www.ladda-upp.se/bilder/vjcagqwtnhxmm/
http://www.ladda-upp.se/bilder/ooaotrwmwlaoy/

"params.ini"


PREFIX = "STOCKHOLM"

BOUNDARY_WEST = 17.535
BOUNDARY_SOUTH = 59.066
BOUNDARY_EAST = 18.51
BOUNDARY_NORTH = 59.686

ELEV_RASTER_X = 10
ELEV_RASTER_Y = 10

PATH_TO_SCENERY = "/home/saul/.fgfs/TerraSync"
PATH_TO_OUTPUT = "/home/saul/Dropbox/personal/flightgear/custom_objects/OUT"

PATH_TO_OSM2CITY_DATA = "/home/saul/flightgear/osm2city/osm2city-data"

NO_ELEV = False
ELEV_MODE = "FgelevCaching"
FG_ELEV = "/home/saul/flightgear/stable/install/flightgear/bin/fgelev"

OVERLAP_CHECK = True
OVERLAP_RADIUS = 5

TILE_SIZE = 1000

OSM_FILE = "buildings.osm"
MAX_OBJECTS = 1000000
CONCURRENCY = 1
USE_PKL = False
SKIP_LIST = []

BUILDING_REDUCE_RATE = 0.01

C2P_PROCESS_POWERLINES = True
C2P_PROCESS_AERIALWAYS = True
C2P_PROCESS_OVERHEAD_LINES = True
C2P_PROCESS_STREETLAMPS = False

MIN_ABOVE_GROUND_LEVEL = 0.3
MAX_SLOPE_RAILWAY = 0.15
MAX_SLOPE_ROAD = 0.15
OBSTRUCTION_LIGHT_MIN_LEVELS = 30

TEXTURES_REGIONS_EXPLICIT = ["de"]

# wget -O buildings.osm http://overpass.osm.rambler.ru/cgi/xapi_meta?*[bbox=17.535,59.066,18.51,59.686]
saul
 
Posts: 38
Joined: Tue Nov 26, 2013 9:57 pm

Re: osm2city.py development

Postby wlbragg » Sat Oct 15, 2016 4:34 am

Wow, that is absolutely outstanding! It appears that Europe has a much better OSM contributor base than here in North America.

Also, really nice job on the documentation!
Last edited by wlbragg on Sat Oct 15, 2016 4:48 am, edited 1 time in total.
Kansas(2-27-15)/Ohio/Midwest scenery development.
KEQA (2-27-15), 3AU, KRCP Airport Layout
Intel i5 3570K AMDRX480
User avatar
wlbragg
 
Posts: 4988
Joined: Sat Aug 25, 2012 11:31 pm
Location: Kansas (Tornado Alley), USA
Callsign: WC2020
Version: next
OS: Win10/Linux/AMDRX480

Re: osm2city.py development

Postby D-ECHO » Sat Oct 15, 2016 4:42 am

Well, Europe is somehow smaller
User avatar
D-ECHO
 
Posts: 1553
Joined: Sat May 09, 2015 12:31 pm

Re: osm2city.py development

Postby stuart » Wed Oct 26, 2016 7:59 pm

Hi All,

In preparation for FSWeekend, I've generated OSM buildings in a 4x1 degree area from Zurich (LSZH) to Innsbruck (LOWI) using the new STG Verbs

It's available here: https://www.dropbox.com/s/qp59a3d3w40qf ... ar.gz?dl=0

Don't forget to set --prop:/sim/rendering/building-mesh=true

One thing I've noticed is that some textures are just a block of texture with some noise. Is that just a limitation of the textures available in the atlas right now?

-Stuart
G-MWLX
User avatar
stuart
Moderator
 
Posts: 1469
Joined: Wed Nov 29, 2006 9:56 am
Location: Edinburgh
Callsign: G-MWLX

Re: osm2city.py development

Postby vanosten » Thu Oct 27, 2016 7:31 am

Hi Stuart

I guess you are referring to what can be seen ca. 4 km south of LSZH in the commercial area (around 47*25'10.1"N, 8*32'12.5"E). I took the following screenshot: Image.

The yellow area shows the texture problems on large roofs reported in previous posts in this thread. The same for the red area - with the difference that here a facade texture is used as the roof texture. There seems to be a work-around for this discussed between radi and pommesschranke - I have not yet bothered to look at it.

The blue area is a texture flickering - actually it looks like two different textures competing with each other. I am not sure why that can be.

The green areas are "correct" in the sense that they just pick up textures in osm2city-data\tex.src\generic\industrial. So to get of this you would need to use some of the texture related parameters - or just on your local copy remove/mask the *.py files in that directory.

This proofs that fixes to textures and having more/better textures is an area, which should have more focus - I would love to discuss this area at FSWeekend as has not been an area, where I have put energy into learning the needed stuff.

PS: I will generate a maxed-out version of LSZH with all osm2city features for the FSWeekend.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby stuart » Sun Oct 30, 2016 8:05 pm

Hi,

Thanks for the analysis. I had spotted this issue you highlighted in green.

I'd be very interested in you "max-out" version of LSZH - I've just used the default options so I'm sure there are better results possible!

Looking forward to seeing you at the weekend. James and Torsten are also very interested in having a discussion about where we go forward with this.

-Stuart
G-MWLX
User avatar
stuart
Moderator
 
Posts: 1469
Joined: Wed Nov 29, 2006 9:56 am
Location: Edinburgh
Callsign: G-MWLX

Re: osm2city.py development

Postby vanosten » Mon Nov 07, 2016 7:55 pm

Hi

I would like to reduce the code/documentation to mimimize maintenance. Is anyone still using a different elevation probing mode than FGelevCache? I guess that anyone actually knowing how to generate scenery with osm2city is also using a recent version of Flightgear and therefore can use FGelev.

=> Any reason why we should keep the other options as described at http://osm2city.readthedocs.io/en/latest/preparation.html#available-elevation-probing-modes?
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Thu Nov 10, 2016 6:21 pm

Well - I just took a bold decision and removed all elevation probing mechanisms but FGElevCaching. If someone for some reason needs the old version, then branch BEFORE_LAYERS can be used. A copy of the documentation at that state can be found in a zip-file.

Using a FlightGear version from the 9th of November or younger respectively the upcoming 2016.4.1 version gives the possibility to set a new parameter PROBE_FOR_WATER to True. Currently in buildings.py an din upcoming versions in e.g. roads we can make sure that osm2city stuff is not placed into the water (currently in buildings.py a test is only made against one node of a building - theoretically parts of a building can still be in the water).
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Thu Nov 10, 2016 6:40 pm

In the post above I write about BEFORE_LAYER - so what does then LAYER mean?

I was at the FSWeekend, where a version of osm2city around Zurich was shown. Basically it drew quite a bit of attention due to the increased realism. There has been some discussions on the fg-devel list recently about the future of osm2city, which I had the pleasure to talk about with some of the core developers..

The high level idea is as follows:
  • osm2city scenery will get distributed by means of Terrasync for the whole world.
  • There will be options in FG for scenery detail selection and in Terrasync as to what will get downloaded/shown.
  • A combination of STG-verbs and Terrasync datalayers will make this finer grained osm2city consumption possible.
  • We will take one step at a time - a bit of infrastructure updates, some updates in osm2city and FG etc. - and we will learn on the way.
  • The layers are not decided yet. It could look like the following from my point of view (subject to further discussions and learning):
    • Buildings (from buildings.py):
      • rough
      • details
      • (most probably no "bare")
    • Roads:
      • roads and rails (from roads.py) plus platforms (from platforms.py)
      • railway overhead lines and pylons (from pylons.py)
      • streetlamps (from pylons.py)
    • Powerlines: all power lines no matter the size (from pylons.py)
    • Windmills: (not yet programmed)
    • Piers and ships: from piers.py
    • "Furniture": everything else, e.g. aerialways from pylons.py
  • Once we have new layers in Terrasync, which replace existing shared objects like electrical power lines, then remove existing shared objects from database to avoid collisions. Assumption is that recent OSM data processed by osm2city is better than hand-placed or old imports.
  • The processing pipeline could be as follows:
    • New OSM-data is loaded into a central FG PostGIS database at scheduled intervals
    • Parametrization is either file-based in a directory structure (maybe like regional textures) or in database tables
    • Pre-processing data with https://gitlab.com/vanosten/owbb in order to get better coverage for buildings (once it is more stable/feature-rich) with direct merge into the database (instead of osc-files and then via Osmosis into osm-files)
    • Run osm2city programs and generate texture atlas, *.ac, *.xml and stg-entries
    • Maybe convert output in a separate process into osg-compatible binary formats
    • Update Terrasync data in layers
    • (Using a database approach will also make it easier to collect statistics from osm2city processing and compare it e.g. with OSM-deltas etc.)
  • Given a processing pipeline new builds for a given tile might be triggered by:
    • scheduled OSM updates
    • more than X new static objects registered (to avoid collisions)
    • significant update of osm2city programs ("significant" to be defined, as it is also depending on layer)
    • change in parameters (more "regionalized")
    • significant enhancement of available textures or shared models for a given region
  • For a longer while we should be able to have access to OSM data from both files and DB. I will learn to handle OSM data in DB within OWBB first.
  • It is an assumption that if we make consumption of osm2city data much more easy and therefore more people start using it, then people "annoyed" with default textures will get motivated to contribute building-textures. And/or they will contribute with needed updates/parameters for heuristics. Still texture processing of large roofs we should fix soon (based on Forum thread).

These of course are ideas and plans - and despite high motivation these are not committments. It will mean that in the shorter run a bit more effort will go into "infrastructure" than new features. But the hope is that osm2city generated scenery will become much more accessible to the average FlightGear users - and might help to attract more users (and thereby maybe developers) to FlightGear.

And of course we would like you to continue to provide input and check stuff out. The journey is long for a little team.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: No registered users and 1 guest