Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby vanosten » Tue Mar 06, 2018 8:46 pm

the short answer is: everything in your last paragraph is already implemented. So yes: it only populates areas, where the density of OSM-buildings seems to be too low based on parametrized heuristics.

Currently I am changing the whole land-use implementation. Lit zones is already done and will give fewer on/off street lighting annoyances plus thereby hopefully fewer residuals. The next step is to drop land-use zones as mapped in OSM and instead create city blocks (using simple cycles in a street network graph), which then get a land-use type assigned. This is especially important in city/town like zones, such that all buildings in a block get the same height/levels (unless OSM values tell otherwise by building). Right now city zones actually look rather odd compared to real life. And when this is available we can work with more aligned textures instead of the once available right now.

When this is done (and I have no time plan due to paid work and family), then I will return to building generation and make it more intelligent in dense areas (residential family house type of zones are easy and therefore "done").
Maintaining osm2city
vanosten
 
Posts: 305
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Sat Apr 07, 2018 5:38 pm

A new push of features, most of them hidden, as they are infrastructure for later changes.

You might notice that roof types dome and onion are now implemented (quite visible in Prague) and that lit streets are more consistent in larger areas.

Next step is to profit from new city blocks and make buildings more aligned in down-town areas.
Maintaining osm2city
vanosten
 
Posts: 305
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Fri May 11, 2018 2:34 pm

For the osm2city scenery generation, I made some performance experiments comparing standard CPython 3.6 on Linux and PyPy3.6 version 6.0. It unfortunately shows that PyPy does not improve speed, actually it decreases run-time in osm2city (i.e. no easy fix without touching the code). However if you actually for some reason need to use PyPy, then it actually works. The manual has been updated accordingly.

As the runtime of osm2city constantly increases due to the addition of features (meaning more geometric data crunching), one might have to look into using Cython or reprogramming (parts) in C/C++. However currently that is not on my priority list.
Maintaining osm2city
vanosten
 
Posts: 305
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

KeyError: 'building:part'

Postby powoflight » Tue May 15, 2018 12:35 pm

Now i have done my first tests after a long time now in ubuntu 16.04 with the postgisDB

1 I download from geofabrik.de/europe/germany bremen-latest.osm.pbf.
2 Write in DB: ./osmosis --read-pbf /media/sf_VMOrdner/OSM/BREMEN/bremen-latest.osm.pbf --log-progress --write-pgsql database=kbos user=gisuser password=xxxx

3 Then Start: python ./build_tiles.py -f BREMEN/params.ini -b 8.7726_53.0569_8.8443_53.099 -p 1 -l DEBUG -o
and get the KeyError: 'building:part'

Code: Select all
******* Exception in tile 3089347 - to reprocess use boundaries: 8.7726_53.0569_8.8443_53.099 ******* at 2018-05-15_123333 - 
Traceback (most recent call last):
  File "/home/bernhard/development/osm2city/build_tiles.py", line 145, in process_scenery_tile
    generated_buildings, file_lock)
  File "/home/bernhard/development/osm2city/buildings.py", line 641, in process_buildings
    _clean_building_parents_with_one_child(the_buildings)
  File "/home/bernhard/development/osm2city/buildings.py", line 457, in _clean_building_parents_with_one_child
    building.make_building_from_part()
  File "/home/bernhard/development/osm2city/building_lib.py", line 136, in make_building_from_part
    part_value = self.tags['building:part']
KeyError: 'building:part'
******* Exception in tile 3089347 - to reprocess use boundaries: 8.7726_53.0569_8.8443_53.099 ******* at 2018-05-15_134446 - 
Traceback (most recent call last):
  File "/home/bernhard/development/osm2city/build_tiles.py", line 145, in process_scenery_tile
    generated_buildings, file_lock)
  File "/home/bernhard/development/osm2city/buildings.py", line 641, in process_buildings
    _clean_building_parents_with_one_child(the_buildings)
  File "/home/bernhard/development/osm2city/buildings.py", line 457, in _clean_building_parents_with_one_child
    building.make_building_from_part()
  File "/home/bernhard/development/osm2city/building_lib.py", line 136, in make_building_from_part
    part_value = self.tags['building:part']
KeyError: 'building:part'


Here is the osm2city_main_2018-05-15_133603.log and osm2city_process_SpawnPoolWorker-1_2018-05-15_133607.log
https://c.gmx.net/@327459158418791162/R ... -YY6t7l1Og

Do i something wrong?

Bernhard
powoflight
 
Posts: 61
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2018.2
OS: win7 Ubuntu 16.04VM

Re: osm2city.py development

Postby vanosten » Tue May 15, 2018 7:48 pm

My best guess is that you are not running the latest code, at this has been corrected some time back.
Maintaining osm2city
vanosten
 
Posts: 305
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby powoflight » Wed May 16, 2018 3:28 pm

I have download the latest version and now it works. The scenery is very nice now.

Thank you .
powoflight
 
Posts: 61
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2018.2
OS: win7 Ubuntu 16.04VM

Whole area (tile?) of OSM buildings disappears

Postby powoflight » Wed May 23, 2018 8:19 am

I've now generated wien (Vienna), but I have the same problem as slawekmikula (whole area (tile?) Of OSM buildings disappears:). See viewtopic.php?f=5&t=22809&p=318286#p318274
I read the thread but there was no solution about this item-

I fly with the UFO from the airport LOWW to the west 320 degrees. A tile is visible. Then northbound 010 degrees to the next tile. As I get closer (houses visible) the streets appear and then immediately the whole tile disappears. Even the first tile west of the airport is no longer visible.
The remaining Tiles of Vienna are never visible. Only restart helps.

I use FG Version 2017.3
LOD setting In-sim: detailed: 1500, Rough: 9000, Bare: 30000
Code: Select all
params.ini
PREFIX = "WIEN"
PATH_TO_SCENERY = "/media/sf_SC/SC_HD"
PATH_TO_OUTPUT = "/media/sf_VMOrdner/OSM/WIEN_OUT1"
PATH_TO_OSM2CITY_DATA = "/home/berni/development/osm2city-data"

NO_ELEV = False
FG_ELEV = '/usr/games/fgelev'

DB_HOST = "localhost"
DB_PORT = 5432
DB_NAME = "wien"
DB_USER = "gisuser"
DB_USER_PASSWORD = "bern"

python script
python ./build_tiles.py -f WIEN/params.ini -b 16.2388_48.0838_16.5272_48.3213 -p 2 -l DEBUG -o

You can download
WIEN_OUT1.zip and osm2city-exceptions.log (very short) from
https://c.gmx.net/@327459158418791162/R ... -YY6t7l1Og
powoflight
 
Posts: 61
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2018.2
OS: win7 Ubuntu 16.04VM

Re: Whole area (tile?) of OSM buildings disappears

Postby vanosten » Wed May 23, 2018 5:52 pm

powoflight wrote in Wed May 23, 2018 8:19 am:I fly with the UFO from the airport LOWW to the west 320 degrees. A tile is visible. Then northbound 010 degrees to the next tile. As I get closer (houses visible) the streets appear and then immediately the whole tile disappears. Even the first tile west of the airport is no longer visible.


Well, I do not know. One thing to try is to disable the osm2city stuff and see whether the scenery loads normally. Also I guess when you write "the whole tile disappears" then you mean even the default scenery.

Maybe you could have a look at http://osm2city.readthedocs.io/en/lates ... ry-objects and play a bit with --prop:/sim/rendering/max-paged-lod=400. I have a theory that the higher the value the more stuff is retained, but I actually do not know.

Unfortunately I do not have a capable computer to test these days.. I have tested your scenery on an old Samsung laptop with integrated graphic card and also after flying ca. for 10 minutes in Linux/FG2018.1. I have not seen an issue. So my best guess are some settings on your side.
Maintaining osm2city
vanosten
 
Posts: 305
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: Whole area (tile?) of OSM buildings disappears

Postby powoflight » Thu May 24, 2018 1:57 pm

I obviously did not explain it properly.The default Scenery (tiles)loads normally ...
But many buildings are missing

Now i am using these parameters too for the Sim.
Code: Select all
--prop:double:/sim/rendering/static-lod/detailed=5000^
--prop:double:/sim/rendering/static-lod/rough=10000^
--prop:/sim/rendering/max-paged-lod=400^

But no difference.

I miss all inner densely populated districts. After living here, it's easy to spot missing buildings / areas. Only a few areas (buildings) in the periphery are visible.
Now I divided Vienna into 6 subareas and regenerated the osmcity. Now I have all buildings in some subareas (Center of Vienna) !!! I will investigate this case.

Question: Should I import a larger area into the database than the osm area query calculation?

I will contact you again if I have results.
powoflight
 
Posts: 61
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2018.2
OS: win7 Ubuntu 16.04VM

Re: osm2city.py development

Postby vanosten » Thu May 24, 2018 5:35 pm

Hi

Ok, I got confused by "then immediately the whole tile disappears", which is different from certain buildings missing at all instead of just disappearing. Looking at the log it appears that there is a flaw in the code, which is why an uncaught exception is thrown. And it is in a part of the code, where I have not much experience and which is going to change. And I am in the middle of something ...

You could try around line 1229 to change as a workaround, which maybe works. Sorry

Code: Select all
try:
    b.write_to_ac(ac_object, cluster_elev, cluster_offset, roof_mgr, face_mat_idx, roof_mat_idx, stats)
except:
    pass
Maintaining osm2city
vanosten
 
Posts: 305
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby powoflight » Thu May 24, 2018 6:10 pm

It's ok . it was my mistake. I am not a native speaker. however i will try the patch.
powoflight
 
Posts: 61
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2018.2
OS: win7 Ubuntu 16.04VM

Re: osm2city.py development

Postby powoflight » Sat May 26, 2018 8:58 am

Patch is ok . No errors. But as a sideeffect. I miss a lot of buildings . I think, it stops the whole calculationprozess and do not calculating the rest of an area. I will send examples later.
powoflight
 
Posts: 61
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2018.2
OS: win7 Ubuntu 16.04VM

Re: osm2city.py development

Postby vanosten » Sun Jun 17, 2018 1:47 pm

I have pushed a new version. The big difference is that the building levels and height are more consistent within city blocks - which in real life at least in European cities is also the case.

A bit of explanation is in the docs at https://osm2city.readthedocs.io/en/late ... e-handling.
Maintaining osm2city
vanosten
 
Posts: 305
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

OSM Buildings disappear

Postby powoflight » Mon Jun 18, 2018 9:14 am

Dear Vanosten

I have already written a forum entry: see viewtopic.php?f=5&t=22809&start=900#p332391
Upgrade drivers from GTX1060 . But it is the same.

Now I have recorded this phenomenon.
See youtube: https://youtu.be/7sg0G-U30tI
Image

Description of the Video:
Code: Select all
0:26 Load Scenery
0:33 OSM Buildings disappear heading=175
0:43 Relaod Scenery
0:55 OSM Buildings disappear

turning ufo to 355 degree
1:11 Relaod Scenery                        Loading of OSM Buildings OK !
moving camera back to original Direction heading=175
3:24  Buildings disappear
Relaod Scenery -> Buildings disappear


Startfile *.bat
Code: Select all
D:\FG\FlightGear2018_2_1\bin\fgfs.exe --launcher  --fg-aircraft=D:\FG\Aircraft^
     --fg-scenery=D:\VMOrdner\OSM\GRAZ_OUT_2;^
     --fg-scenery=D:\FG\SC\SC_HD^
     --airport=LOWG^
     --aircraft=ufo^
     --timeofday=noon^
 --altitude=1100^
 --heading=175^
 --lat=47.0764^
 --lon=15.4404^
 --prop:double:/sim/rendering/static-lod/detailed=5000^
 --prop:double:/sim/rendering/static-lod/rough=10000^
 --prop:/sim/rendering/max-paged-lod=400^
 --httpd=8080


Rendering options:
Image
Image

This is realy very strange. Because this happens seldom, but allways in the same area; means I can reproduce it.
I have also tried to change LOD Detail and Rough to really strange values but this makes no difference.
Like
Detail : from 200 - to 10000
Rough : from 1000 - 30000



If you want to test it , i can upload the OSM Scenery for this area..


P:S: By the way. I will downlad the new version and test it. Because especially in vienna, tall buildings are often displayed with only one floor.
Last edited by powoflight on Thu Jun 21, 2018 8:39 am, edited 2 times in total.
powoflight
 
Posts: 61
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2018.2
OS: win7 Ubuntu 16.04VM

Re: osm2city.py development

Postby miguel » Tue Jun 19, 2018 2:34 pm

I follow with great interest the development osm2city.py development when you have something new to publish in http://wiki.flightgear.org/Areas_popula ... ty_scenery Thanks
miguel
 
Posts: 16
Joined: Wed Aug 19, 2015 4:05 pm

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: No registered users and 4 guests