Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby statto » Mon Dec 19, 2016 9:48 pm

So we'd lose the 3-D airports that currently get generated? I'm all for progress, but that seems like a shame.
Custom Scenery available from http://www.stattosoftware.com/flightgear
statto
 
Posts: 2112
Joined: Fri Jan 25, 2008 9:57 pm

Re: osm2city.py development

Postby psadro_gm » Tue Dec 20, 2016 1:13 am

no, the altitude within the smoothing boundary of the airport would be calculated via the pre-calculated smoothing polynomial instead of reading from the mesh. Just the layout genapt creates would be 2-d.
So there wouldn't need to be holes cut into the terrain, and you wouldn't need to re-run terragear when changing airport boundaries / layout. I also want to experiment with smoothing the raw DEM and airport polynomial calculated elevation. Perhaps this could be customized per airport ass well.
8.50 airport parser, textured roads and streams...
psadro_gm
 
Posts: 751
Joined: Thu Aug 25, 2011 2:23 am
Location: Atlanta, GA USA
IRC name: psadro_*
Version: git
OS: Fedora 21

Re: osm2city.py development

Postby pommesschranke » Mon Jan 02, 2017 4:29 pm

I'm using an older (Oct 2015) version and it works very well.

Today I'd like to try the latest version...
( git clone https://gitlab.com/fg-radi/osm2city.git )

my first problem: I cannot find the osm2city.py file.
SOLVED: It was renamed to buildings.py.

second issue: when I try any of the scripts I get this error:
Traceback (most recent call last):
File "./roads.py", line 82, in <module>
from typing import List
ImportError: No module named 'typing'

SOLVED: pip3 install typing

more installation issues:

pip3 install matplotlib
...
* The following required packages can not be built:

* freetype, png

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/matplotlib

SOLVED: apt install python3-matplotlib

pip3 install pil
Downloading/unpacking pil
Could not find any downloads that satisfy the requirement pil
Cleaning up...
No distributions at all found for pil
Storing debug log for failure in /root/.pip/pip.log

??? not needed for python3?

pip3 install scipy
...
File "scipy/linalg/setup.py", line 20, in configuration

raise NotFoundError('no lapack/blas resources found')

SOLVED: apt install python3-scipy


pip3 install shapely
...
WARNING:/tmp/pip_build_root/shapely/setup.py:The C extension could not be compiled, speedups are not enabled.


pip3 install psycopg2
...
Error: pg_config executable not found.

SOLVED: apt install python3-psycopg2

SUCCESS!

I get many of those:
Unknown token 'OBJECT_SHARED'
Unknown token 'OBJECT_STATIC'

but
python3 ./buildings.py -f params.ini
did run successfully.

next I'll try the batch_processing...
Last edited by pommesschranke on Mon Jan 02, 2017 5:48 pm, edited 6 times in total.
pommesschranke
 
Posts: 1104
Joined: Sat Apr 27, 2013 7:58 pm
Location: EDLM & LJCE
Callsign: d-laser
IRC name: laserman
Version: git
OS: Linux Lubuntu 18.04

Re: osm2city.py development

Postby vanosten » Mon Jan 02, 2017 4:48 pm

pommesschranke wrote in Mon Jan 02, 2017 4:29 pm:...

Today I'd like to try the latest version
( git clone https://gitlab.com/fg-radi/osm2city.git )

...

second issue: when I try any of the scripts I get this error:
Code: Select all
Traceback (most recent call last):
  File "./roads.py", line 82, in <module>
    from typing import List
ImportError: No module named 'typing'



You need to use Python 3.5+. For the other errors: I am not sure, I just use Ubuntu 16.04 and the dependencies work out of the box. The same with the Windows version if following http://osm2city.readthedocs.io/en/lates ... n-packages.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby pommesschranke » Mon Jan 02, 2017 6:01 pm

I have python3.4 (3.4.3-1ubuntu1~14.04.5)
and buildings.py seems to work ok in my first short test run.
pommesschranke
 
Posts: 1104
Joined: Sat Apr 27, 2013 7:58 pm
Location: EDLM & LJCE
Callsign: d-laser
IRC name: laserman
Version: git
OS: Linux Lubuntu 18.04

Re: osm2city.py development

Postby powoflight » Wed Jan 04, 2017 1:53 pm

Sorry for my bad english but i am austrian.........

i have installed the latest version like pommesschranke

I use WIn7 64bit so i installed the 64bit versions of python with the extentions.
My first try was to install WinPython-64bit-3.5.2.3.exe with extensions but Gunzip does not work. Dont ask why.

The I installed python-3.5.0-amd64.exe with extension packages

matplotlib-1.5.3-cp35-cp35m-win_amd64.whl
matplotlib (1.5.3)

networkx-1.11-py2.py3-none-any.whl
networkx (1.11)

numpy-1.11.3+mkl-cp35-cp35m-win_amd64.whl
numpy (1.11.3+mkl)

psycopg2-2.6.2-cp35-cp35m-win_amd64.whl
psycopg2 (2.6.2)

scipy-0.18.1-cp35-cp35m-win_amd64.whl
scipy (0.18.1)

Shapely-1.5.17-cp35-cp35m-win_amd64.whl
Shapely (1.5.17)

pil does not work so i installed "pip3.5 install pillow"
Pillow (4.0.0)
------------------------------------------------------------------------------------------------------------------
i start python D:\FG\SC\OSM_Development35\osm2city\buildings.py -f D:\FG\SC\OSM_Development35\osm2city\TEST\params.ini -l DEBUG

INFO:root:3 MAIN reading elevation data
INFO:root:Loading TEST\elev.pkl
INFO:root:Loading elev cache failed ([Errno 2] No such file or directory: 'TEST\\elev.pkl')
##############################################################
This is ok. I use version with python 2.7 on an old PC with xp .
##############################################################

But the i get an error

INFO:root: *** SUB 1 aptdat_io powo 102 utils aptdat_io file_name : D:\FG\FlightGear36\data\Airports\apt.dat.gz
Traceback (most recent call last):
File "D:\FG\SC\OSM_Development35\osm2city\buildings.py", line 934, in <module>
parameters.BOUNDARY_EAST, parameters.BOUNDARY_NORTH)
File "D:\FG\SC\OSM_Development35\osm2city\utils\aptdat_io.py", line 149, in get_apt_dat_blocked_areas
airports = _read_apt_dat_gz_file(apt_dat_gz_file, min_lon, min_lat, max_lon, max_lat)
File "D:\FG\SC\OSM_Development35\osm2city\utils\aptdat_io.py", line 111, in _read_apt_dat_gz_file
with gzip.open(file_name, 'rt', encoding="latin-1") as f: # orig
File "C:\Python35\lib\gzip.py", line 53, in open
binary_file = GzipFile(filename, gz_mode, compresslevel)
File "C:\Python35\lib\gzip.py", line 163, in __init__
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
OSError: [Errno 22] Invalid argument: ' D:\\FG\\FlightGear36\\data\\Airports\\apt.dat.gz'

In aptdat_io.py gzip can not fine the apt.dat.gz file because there is a space between ' D:
For debuging i hardcoded the directorystructure to apt.dat.gz in utils\aptdat_io.py -> def _read_apt_dat_gz_file

def _read_apt_dat_gz_file(file_name: str, min_lon: float, min_lat: float,
max_lon: float, max_lat: float) -> List[Airport]:
file_name = 'D:/FG/FlightGear36/data/Airports/apt.dat.gz'

ok now the python find now the apt.dat.gz
#########################################################


IN if parameters.OVERLAP_CHECK:
there are some warnings
WARNING:root:stg_io:read: Ignoring unreadable file [Errno 2] No such file or directory: 'D:/FG/SC/SC_HD\\Objects\\e010n40\\e016n47\\3220082.stg'
because the file do not exist

then

IN if parameters.OVERLAP_CHECK_CONVEX_HULL
there will be called building_lib.py

HERE I GET AN ERROR
Traceback (most recent call last):
File "D:\FG\SC\OSM_Development35\osm2city\buildings.py", line 974, in <module>
prepare_textures.facades, prepare_textures.roofs)
File "D:\FG\SC\OSM_Development35\osm2city\building_lib.py", line 538, in analyse
if b.tags['building'] in ['glasshouse', 'greenhouse'] or (
KeyError: 'building'
powoflight
 
Posts: 84
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2018.3.2
OS: win7 Ubuntu 18.04

Re: osm2city.py development

Postby vanosten » Wed Jan 04, 2017 6:26 pm

I must admit that I have not tried running osm2city from Windows for a while. However given your explanation above I do not see an immediate need to do changes - unless you tell me otherwise (also in German).

For the last thing. Yes, this is a bug fixed on Monday (i.e. please pull the latest version from GIT). The error was triggered, when the building was tagged "building:part", which tends to happen only in well modeled cities (and my default test range did not include it :-( ).
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby powoflight » Thu Jan 05, 2017 9:38 am

I download the latest version from GIT.

Got an error python.exe -> can't start MSVCP140.dll missing.
Download and install Visual C++ Redistributable for Visual Studio 2015

Thanks, now it works.
powoflight
 
Posts: 84
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2018.3.2
OS: win7 Ubuntu 18.04

Re: osm2city.py development

Postby Eskimo » Sat Jan 07, 2017 1:36 pm

I can generate the buildings, roads, pylons.., but they are -1000 below the surface.
I've red the suggestions in this thread and also tried the hints in the wiki, but with no success.
I am quite sure, that it is a stupid little thing, but I just cant see it.
Maybe someone can help.

It was mentioned, that
Code: Select all
 ELEV_MODE = "FgelevCaching"
should be used in the params.ini file.
But in my case, this parameter is not known...
Code: Select all
 WARNING:root:Unknown parameter: ELEV_MODE=FgelevCaching


My params.ini is:
Code: Select all
PREFIX = "/home/roman/osm2city-input"
PATH_TO_SCENERY = "/home/roman/.fgfs/TerraSync"
PATH_TO_OUTPUT = "/home/roman/osm2city-output"
PATH_TO_OSM2CITY_DATA = "/home/roman/osm2city-data"
OSM_FILE = "lachen.osm"

BOUNDARY_WEST = 8.8360
BOUNDARY_SOUTH = 47.1837
BOUNDARY_EAST = 8.9034
BOUNDARY_NORTH = 47.210754

NO_ELEV = False
ELEV_MODE = "FgelevCaching"
FG_ELEV = "/usr/bin/fgelev"


If I try to run fgelev directly as suggested in the wiki, I get the following:
Code: Select all
[roman@frosch ~]$ export FG_ROOT=/home/roman/FGCHECKOUT/fgdata
[roman@frosch ~]$ export FG_SCENERY=/home/roman/.fgfs/TerraSync
[roman@frosch ~]$ /usr/bin/fgelev --expire 1000 --fg-root $FG_ROOT --fg-scenery $FG_SCENERY
0 8.8360 47.1873
Loading tile 3088960.stg, no scenery path suffixes were configured so giving up
Loading tile 3088968.stg, no scenery path suffixes were configured so giving up
...
Loading tile 3105395.stg, no scenery path suffixes were configured so giving up
Loading tile 3105403.stg, no scenery path suffixes were configured so giving up
Found hole of minimum diameter 1.31072m at lon = 8.836deg lat = 47.1873deg
0: -1000


Any help what could be wrong?
Eskimo
 
Posts: 42
Joined: Thu Jan 21, 2016 6:37 am

Re: osm2city.py development

Postby vanosten » Sun Jan 08, 2017 9:41 am

Hi Eskimo

You are most probably using old code and are not looking at the linked documentation. I disabled the different possibilities of elevation probing, so only one is left. Please fetch the newest code and have a look at http://osm2city.readthedocs.io/en/lates ... ation-data and http://osm2city.readthedocs.io/en/lates ... ters-label.

PS: I know that the wiki article needs an update - but it is just not a priority for me right now.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby Eskimo » Sun Jan 08, 2017 11:14 am

Vanosten, thanks for your help.
Sorry if I did not write it in my post, but I used the documentation you pointed at.
After the problem with fgelev, I did read the wiki.
I added the "ELEV_MODE" after going through the thread and trying to find the fault...
Eskimo
 
Posts: 42
Joined: Thu Jan 21, 2016 6:37 am

Re: osm2city.py development

Postby Eskimo » Wed Jan 11, 2017 12:02 pm

I was able to get it to work, but only when I installed flightgear from the arch main repository.
When I install flightgear from git, it runs flawless, but with the same working params.ini and xxx.osm file, all the buildings are below the surface.
It must have something to do with FG_ROOT, since thats the only thing I changed in params.ini.

The working FG_ROOT is /usr/share/flightgear/data
The not working FG_ROOT is /home/grn/FGCHECKOUT/fgdata

Any ideas?
Eskimo
 
Posts: 42
Joined: Thu Jan 21, 2016 6:37 am

Re: osm2city.py development

Postby pommesschranke » Fri Jan 13, 2017 11:40 am

I just tried to place some osm buildings at KOAK and maybe found a bug.
osm2city created this stg file: 0942059.stg - but Flightgear ingnores it. Flightgear only reads this one: 942059.stg

line 37 in stg_io2.py
Code: Select all
 self.file_name = self.path_to_stg + "%07i.stg" % tile_index


Code: Select all
XX@i3:~/scenery/2.0/Objects$ find . -name "?????.stg" | wc -l
21
XX@i3:~/scenery/2.0/Objects$ find . -name "??????.stg" | wc -l
2137
XX@i3:~/scenery/2.0/Objects$ find . -name "???????.stg" | wc -l
34832
pommesschranke
 
Posts: 1104
Joined: Sat Apr 27, 2013 7:58 pm
Location: EDLM & LJCE
Callsign: d-laser
IRC name: laserman
Version: git
OS: Linux Lubuntu 18.04

Re: osm2city.py development

Postby vanosten » Sun Jan 15, 2017 4:12 pm

pommesschranke wrote in Fri Jan 13, 2017 11:40 am:I just tried to place some osm buildings at KOAK and maybe found a bug.
osm2city created this stg file: 0942059.stg - but Flightgear ingnores it. Flightgear only reads this one: 942059.stg

You are right. I found actually more areas, where the same assumption is made. I have fixed it. I need to check the actual calculation of tile indexes another day (looks different from c++ code in Simgear).
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 417
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby powoflight » Tue Jan 17, 2017 3:55 pm

I have generated the osmcity from Wien (Vienna) ( a little big about 220 000 )
If i start from LOWW the osmcity will be loaded very slowly and the worst think is. it takes a long time to load the neighbor tiles like it is described in the report
viewtopic.php?f=37&t=29012

but i have NO lags (stuttering) while i am flying. The framerate is constant by 50.

I tried this with the ufo and the dhc6/twinotter.

RAM usage for flightgear.exe is about 2GB. (NOT to much)
CPU load 58%
Video RAM usage is about 11%. (ridiculous !). (I have 15% if i am flying over the alps without OSM Buildings)

I tried
LOD
Detailed 3000, Rough 65000, Bare 370000

and the Default
Detailed 1500, Rough 9000, Bare 30000

and
--prop:/sim/rendering/texture-compression=off
and
/sim/tile-cache/enable=true' and '/sim/rendering/max-paged-lod=200'

but there is no big difference in performance.


The console output shows about a couple of minutes

Code: Select all
prepare_ground_cache(): scenery_available returns false at lon = 16.334deg, lat = 48.2499deg, elev = 268.452m [ 4.08346e+006, 1.19672e+006, 4.73563e+006 ] 10
prepare_ground_cache(): scenery_available returns false at lon = 16.334deg, lat = 48.2499deg, elev = 268.452m [ 4.08346e+006, 1.19672e+006, 4.73563e+006 ] 10
..
..
prepare_ground_cache(): scenery_available returns false at lon = 16.334deg, lat = 48.2499deg, elev = 268.452m [ 4.08346e+006, 1.19672e+006, 4.73563e+006 ] 10
Loading tile 3220104.stg
Loading stg file D:/FG/SC/SC_HD/Objects/e010n40/e016n48/3220104.stg
Loading stg file D:/FG/SC/SC_HD/Terrain/e010n40/e016n48/3220104.stg
prepare_ground_cache(): scenery_available returns false at lon = 16.334deg, lat = 48.2499deg, elev = 268.452m [ 4.08346e+006, 1.19672e+006, 4.73563e+006 ] 10
prepare_ground_cache(): scenery_available returns false at lon = 16.334deg, lat = 48.2499deg, elev = 268.452m [ 4.08346e+006, 1.19672e+006, 4.73563e+006 ] 10
prepare_ground_cache(): scenery_available returns false at lon = 16.334deg, lat = 48.2499deg, elev = 268.452m [ 4.08346e+006, 1.19672e+006, 4.73563e+006 ] 10
SGMaterialCache::~SGMaterialCache() size=286
SGMaterialCache::~SGMaterialCache() size=286
SGMaterialCache::~SGMaterialCache() size=286
Loading tile 3220096.stg
Loading stg file D:/FG/SC/SC_HD/Terrain/e010n40/e016n48/3220096.stg
SGMaterialCache::~SGMaterialCache() size=286
New model with attached script(s) (branch = 0000000044FCAE00, path = /'D:/FG/SC/SCENERY_OSM/WIEN/Objects/e010n40/e016n48/lightmap-switch.xml')
Loading tile 3220112.stg
Loading stg file D:/FG/SC/SC_HD/Objects/e010n40/e016n48/3220112.stg
Loading stg file D:/FG/SC/SC_HD/Terrain/e010n40/e016n48/3220112.stg
SGMaterialCache::~SGMaterialCache() size=286
SGMaterialCache::~SGMaterialCache() size=286
Loading tile 3220114.stg
Loading stg file D:/FG/SC/SCENERY_OSM/WIEN/Objects/e010n40/e016n48/3220114.stg
Loading stg file D:/FG/SC/SC_HD/Objects/e010n40/e016n48/3220114.stg
Loading stg file D:/FG/SC/SC_HD/Terrain/e010n40/e016n48/3220114.stg
SGMaterialCache::~SGMaterialCache() size=286
SGMaterialCache::~SGMaterialCache() size=286
SGMaterialCache::~SGMaterialCache() size=286
Loading tile 3220121.stg
Loading stg file D:/FG/SC/SC_HD/Objects/e010n40/e016n48/3220121.stg
Loading stg file D:/FG/SC/SC_HD/Terrain/e010n40/e016n48/3220121.stg
SGMaterialCache::~SGMaterialCache() size=286
Loading tile 3203723.stg
Loading stg file D:/FG/SC/SC_HD/Terrain/e010n40/e015n48/3203723.stg
SGMaterialCache::~SGMaterialCache() size=286
Loading tile 3220088.stg
Loading stg file D:/FG/SC/SC_HD/Objects/e010n40/e016n47/3220088.stg
Loading stg file D:/FG/SC/SC_HD/Terrain/e010n40/e016n47/3220088.stg
SGMaterialCache::~SGMaterialCache() size=286

---------------------------
Hardware
---------------------------
win7 64Bit
CPU i5-6600 3.3GHZ
RAM 16GB
GPU: Geforce 1060 6GB RAM

Flightgear version 3.6
Scenery 2.0.1 HD (Download with Graphical interface and Terramaster)


---------------------------
params.ini
---------------------------
PREFIX = "WIEN"   
# PREFIX = "D:\FG\SC\OSM_Development35\osm2city\WIEN"

BOUNDARY_WEST =  16.2433 
BOUNDARY_SOUTH = 48.0974
BOUNDARY_EAST =  16.6427
BOUNDARY_NORTH = 48.3251

ELEV_RASTER_X = 10      # ------- NOT in parameters ********************
ELEV_RASTER_Y = 10      # ------- NOT in parameters  ********************

PATH_TO_SCENERY = "D:/FG/SC/SC_HD"                                #ok
PATH_TO_OUTPUT = "D:/FG/SC/SCENERY_OSM/WIEN"            #ok
PATH_TO_OSM2CITY_DATA = "D:/FG/SC/OSM_Development35/osm2city-data"  #ok
OSM_FILE = "buildings.osm"           #ok

NO_ELEV = False                           # -- skip elevation probing
ELEV_MODE = "FgelevCaching"                             # ------- NOT in parameters ********************
FG_ELEV = '"D:/FG/FlightGear36/bin/fgelev.exe"'    #ok

TILE_SIZE = 2000                # -- 2000 tile size in meters for clustering of buildings
SKIP_LIST = []  #    E.g. SKIP_LIST = ["Theologische Fakultät", "Rhombergpassage", 55875208]

---------------------------
Here is the log of generating the buildings
---------------------------

LOD
    LOD rough       95147 (42 %)
    LOD detail      132543 (58 %)

area >=
     1 m^2      0 |
    10 m^2      0 |
    20 m^2      0 |
    50 m^2  42226 |##########################
   100 m^2  53898 |##################################
   200 m^2  88616 |########################################################
   500 m^2  26732 |################
  1000 m^2   9632 |######
  2000 m^2   4876 |###
  5000 m^2   1216 |
 10000 m^2    382 |
 20000 m^2     96 |
 50000 m^2     16 |

number of corners >=
      3     24 |
      4  91928 |########################################################
      5  10434 |######
      6  34217 |####################
      7   7637 |####
      8  27166 |################
      9  56284 |##################################
 complex     0 |
Last edited by Johan G on Thu Jan 19, 2017 6:25 pm, edited 1 time in total.
Reason: Please use [code] tags for long listings. They can be added manually or by using the [>_ ] button.
powoflight
 
Posts: 84
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2018.3.2
OS: win7 Ubuntu 18.04

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: No registered users and 2 guests