Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby Johan G » Sun Jul 19, 2020 7:20 pm

Some posts were split off to the new topic Typical specifications of computers running FlightGear?
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator
 
Posts: 5782
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

Re: osm2city.py development

Postby vanosten » Sat Jul 25, 2020 1:54 pm

I have pushed some updates to the logic for building exclusions/reduction. More importantly, there is now code to better detect and handle L-shaped roofs with neighbour buildings (in central Reykjavik these are around 4% of all buildings).
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 463
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby saul » Thu Jul 30, 2020 2:24 pm

Hi,

I just came back from a few months pause in FG and pulled the latest changes on the osm2city builder, but now I got some problems that I can not solve by myself. Can you please give me a hand to see what I am doing wrong?

I created a virtual environment for python 3.7 by using:

Code: Select all
python3.7 -m venv work3.7


and installed all the needed dependencies:
Code: Select all
(work3.7) saul@viking:~/flightgear/development/osm2city$ pip freeze
attrs==19.3.0
certifi==2020.6.20
chardet==3.0.4
click==7.1.2
click-plugins==1.1.1
cligj==0.5.0
cycler==0.10.0
decorator==4.4.2
descartes==1.1.0
Fiona==1.8.13.post1
idna==2.10
kiwisolver==1.2.0
matplotlib==3.3.0
munch==2.5.0
networkx==2.4
numpy==1.19.1
Pillow==7.2.0
psycopg2-binary==2.8.5
pyparsing==2.4.7
pyproj==2.6.1.post1
python-dateutil==2.8.1
requests==2.24.0
scipy==1.5.2
Shapely==1.7.0
six==1.15.0
urllib3==1.25.10


When I run the builder from within the environment I have the following error:

Code: Select all
SpawnPoolWorker-2 root ERROR    Exception occurred while processing tile 1726160.
Traceback (most recent call last):
  File "/home/saul/flightgear/development/osm2city/build_tiles.py", line 131, in process_scenery_tile
    lit_areas, water_areas, osm_buildings = ol.process(the_coords_transform, my_airports)
  File "/home/saul/flightgear/development/osm2city/osm2city/owbb/landuse.py", line 783, in process
    btg_reader = btg.read_btg_file(transformer, None)
  File "/home/saul/flightgear/development/osm2city/osm2city/utils/btg_io.py", line 445, in read_btg_file
    out_proj = pyproj.Proj('epsg:4326', ellps='WGS84', datum='WGS84')
  File "/home/saul/work3.7/lib/python3.7/site-packages/pyproj/proj.py", line 107, in __init__
    self.crs = CRS.from_user_input(projparams, **kwargs)
  File "/home/saul/work3.7/lib/python3.7/site-packages/pyproj/crs/crs.py", line 440, in from_user_input
    return CRS(value, **kwargs)
  File "/home/saul/work3.7/lib/python3.7/site-packages/pyproj/crs/crs.py", line 296, in __init__
    super().__init__(projstring)
  File "pyproj/_crs.pyx", line 2338, in pyproj._crs._CRS.__init__
pyproj.exceptions.CRSError: Invalid projection: epsg:4326 +ellps=WGS84 +datum=WGS84 +type=crs: (Internal Proj Error: proj_create: crs not found)
SpawnPoolWorker-2 root ERROR    ******* Exception in tile 1726160 - to reprocess use boundaries: -75.0_1.25_-74.93_1.375 *******
Traceback (most recent call last):
  File "/home/saul/flightgear/development/osm2city/build_tiles.py", line 131, in process_scenery_tile
    lit_areas, water_areas, osm_buildings = ol.process(the_coords_transform, my_airports)
  File "/home/saul/flightgear/development/osm2city/osm2city/owbb/landuse.py", line 783, in process
    btg_reader = btg.read_btg_file(transformer, None)
  File "/home/saul/flightgear/development/osm2city/osm2city/utils/btg_io.py", line 445, in read_btg_file
    out_proj = pyproj.Proj('epsg:4326', ellps='WGS84', datum='WGS84')
  File "/home/saul/work3.7/lib/python3.7/site-packages/pyproj/proj.py", line 107, in __init__
    self.crs = CRS.from_user_input(projparams, **kwargs)
  File "/home/saul/work3.7/lib/python3.7/site-packages/pyproj/crs/crs.py", line 440, in from_user_input
    return CRS(value, **kwargs)
  File "/home/saul/work3.7/lib/python3.7/site-packages/pyproj/crs/crs.py", line 296, in __init__
    super().__init__(projstring)
  File "pyproj/_crs.pyx", line 2338, in pyproj._crs._CRS.__init__
pyproj.exceptions.CRSError: Invalid projection: epsg:4326 +ellps=WGS84 +datum=WGS84 +type=crs: (Internal Proj Error: proj_create: crs not found)
SpawnPoolWorker-2 root INFO     ******* Finished tile 1726160 - 2914/2915 ******



There are no output files created. My params.ini is very short:

Code: Select all
PREFIX = "ECUADOR"

MAX_SLOPE_ROAD = 0.20
MIN_ABOVE_GROUND_LEVEL = 0.02

PATH_TO_SCENERY = "/home/saul/flightgear/scenery"
PATH_TO_OUTPUT = "/home/saul/flightgear/fg_customscenery/ECUADOR"

PATH_TO_OSM2CITY_DATA = "/home/saul/flightgear/development/osm2city-data"

DB_HOST = "localhost"
DB_PORT = 5432
DB_NAME = "ecuador"
DB_USER = "giuser"
DB_USER_PASSWORD = "hola"

NO_ELEV = False
FG_ELEV = "/home/saul/flightgear/dnc-managed/install/flightgear/bin/fgelev"


Is there something that I am doing wrong?

Best regards,

Saul
Last edited by Johan G on Thu Jul 30, 2020 3:55 pm, edited 1 time in total.
Reason: Added [code] tags. They can be added manually or by clicking the [^_ ] button above the text editing box.
saul
 
Posts: 40
Joined: Tue Nov 26, 2013 9:57 pm

Re: osm2city.py development

Postby merspieler » Thu Jul 30, 2020 2:32 pm

Hey Saul,

known issue caused by pyproj 2.6.0.

There's a requirements.txt which you can use to create your venv. This will install all dependencies in working versions.

Code: Select all
$ python3.7 -m venv work3.7/
$ source work3.7/bin/activate
$ pip install -r osm2city/requirements.txt
Love at first flight A<380
Attempting an osm2city worldbuild... Coming to YOUR terrasync....... when ever they start pulling it in, so go nag them on the mailing list (I totally didn't say that)!
Take the August 2020 Flightgear Community Survey
merspieler
 
Posts: 633
Joined: Thu Oct 26, 2017 10:43 am
Location: Wish to be in YBCS
Callsign: JST935. ORI1711
IRC name: merspieler
Version: next
OS: Debian Bullseye

Re: osm2city.py development

Postby saul » Fri Jul 31, 2020 10:46 pm

Many thanks Merspieler,

I got it working all again. I launched a build on a complete country, Ecuador, by using the latest osm.pbf file which is 67 MB. It took around a day to complete in a i7-4790k@4 GHz, 4 processors enabled. The osm2city scenery is a little more than 1 GB. The script actually did not exit. It just released all processors and stayed in the very last tile, but all the files are there.

Code: Select all
SpawnPoolWorker-3 root INFO     No platforms found -> aborting
SpawnPoolWorker-3 root INFO     Generating clusters -3895.1668000 -6910.9286484 3895.1668000 6910.9286484 for object_static
SpawnPoolWorker-3 root INFO     Reading OSM way data for ['man_made=>pier'] from db took 0.0076 seconds.
SpawnPoolWorker-3 root INFO     ways: 0
SpawnPoolWorker-3 root INFO     No piers found -> aborting
SpawnPoolWorker-3 root INFO     ******* Finished tile 1726160 - 2914/2915 *******


I have tested some cities and everything is there. Now I am thinking to build Sweden, but I am a bit skeptical since the osm.pbf is 484 MB and at the same speed than Ecuador it may take 1 week (and some good 7-8 GB, probably more). Are these numbers the same you are having or I am doing something wrong?

Saul
saul
 
Posts: 40
Joined: Tue Nov 26, 2013 9:57 pm

Re: osm2city.py development

Postby merspieler » Sat Aug 01, 2020 7:42 am

I didn't look at my exact numbers... but it sounds plausible...
Love at first flight A<380
Attempting an osm2city worldbuild... Coming to YOUR terrasync....... when ever they start pulling it in, so go nag them on the mailing list (I totally didn't say that)!
Take the August 2020 Flightgear Community Survey
merspieler
 
Posts: 633
Joined: Thu Oct 26, 2017 10:43 am
Location: Wish to be in YBCS
Callsign: JST935. ORI1711
IRC name: merspieler
Version: next
OS: Debian Bullseye

Previous

Return to Scenery

Who is online

Users browsing this forum: No registered users and 1 guest