Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby legoboyvdlp » Tue Jan 02, 2018 8:45 pm

Just a note to you all, I was trying to run osm2city having finally gotten everything done and the database set up. Ensure you download numpy+MKL, or your scipy installation will not work, failing with

Code: Select all
Traceback (most recent call last):
  File "../../../GitHub/osm2city/build_tiles.py", line 18, in <module>
    import pylons
  File "C:\Users\Redpath\Documents\GitHub\osm2city\pylons.py", line 30, in <module>
    import roads
  File "C:\Users\Redpath\Documents\GitHub\osm2city\roads.py", line 93, in <module>
    import linear_bridge
  File "C:\Users\Redpath\Documents\GitHub\osm2city\linear_bridge.py", line 10, in <module>
    import scipy.interpolate
  File "C:\Users\Redpath\Documents\Jonathans Folder\FlightGear Reference Files\WinPython-64bit-3.5.4.1Zero\python-3.5.4.amd64\lib\site-packages\scipy\interpolate\__init__.py", line 175, in <module>
    from .interpolate import *
  File "C:\Users\Redpath\Documents\Jonathans Folder\FlightGear Reference Files\WinPython-64bit-3.5.4.1Zero\python-3.5.4.amd64\lib\site-packages\scipy\interpolate\interpolate.py", line 20, in <module>
    import scipy.linalg
  File "C:\Users\Redpath\Documents\Jonathans Folder\FlightGear Reference Files\WinPython-64bit-3.5.4.1Zero\python-3.5.4.amd64\lib\sitepackages\scipy\linalg\__init__.py", line 186, in <module>
    from .misc import *
  File "C:\Users\Redpath\Documents\Jonathans Folder\FlightGear Reference Files\WinPython-64bit-3.5.4.1Zero\python-3.5.4.amd64\lib\site-packages\scipy\linalg\misc.py", line 5, in <module>
    from .blas import get_blas_funcs
  File "C:\Users\Redpath\Documents\Jonathans Folder\FlightGear Reference Files\WinPython-64bit-3.5.4.1Zero\python-3.5.4.amd64\lib\site-packages\scipy\linalg\blas.py", line 196, in <module>
    from scipy.linalg import _fblas
ImportError: DLL load failed: The specified module could not be found.


Installing NumPy+MKL instead of Numpy, from the unofficial extensions page solves this issue.
User avatar
legoboyvdlp
 
Posts: 7981
Joined: Sat Jul 26, 2014 2:28 am
Location: Northern Ireland
Callsign: G-LEGO
Version: next
OS: Windows 10 HP

Re: osm2city.py development

Postby vanosten » Tue Jan 02, 2018 10:12 pm

legoboyvdlp wrote in Tue Jan 02, 2018 8:45 pm:JEnsure you download numpy+MKL, or your scipy installation will not work ... on Windows ...

Installing NumPy+MKL instead of Numpy, from the unofficial extensions page solves this issue.


Thanks. I have updated the docs accordingly.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Tue Jan 02, 2018 10:14 pm

I have removed the parameter FLAG_2017.2, which means that you now can only generate sceneries for version 2017.2.1 and onward of FlightGear. If you still need the old version, then please checkout the code for git tag = LAST_COPY_TEX.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby legoboyvdlp » Wed Jan 03, 2018 5:27 pm

Code: Select all
C:\Users\Redpath\Documents\FlightGear\Custom Scenery\projects>python ../../../GitHub/osm2city/build_tiles.py -f NI/params.ini -b *8.888_53.963_*5.350_55.466 -p 1
MainProcess root INFO     Reading parameters from file: NI/params.ini
MainProcess root INFO     Overall boundary 8.888, 53.963, 5.35, 55.466
MainProcess root INFO     Read 34074 airports, 0 having runways/helipads within
the boundary
MainProcess root INFO     Execution time: 7.471034
MainProcess root INFO     Total time used 3.463611364364624


Have you any ideas? I think my boundary might be wrongly formatted...
User avatar
legoboyvdlp
 
Posts: 7981
Joined: Sat Jul 26, 2014 2:28 am
Location: Northern Ireland
Callsign: G-LEGO
Version: next
OS: Windows 10 HP

Re: osm2city.py development

Postby vanosten » Wed Jan 03, 2018 6:38 pm

The boundary is wrong, as West must be less than East. I guess you wanted either *-8.888_53.963_-5.350_55.466 or *5.350_53.963_8.888_55.466.

Unfortunately the current check of boundary is too late in the code. That is corrected now.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby legoboyvdlp » Wed Jan 03, 2018 8:03 pm

OK. Thanks, yes, it was a negatives confusion.
User avatar
legoboyvdlp
 
Posts: 7981
Joined: Sat Jul 26, 2014 2:28 am
Location: Northern Ireland
Callsign: G-LEGO
Version: next
OS: Windows 10 HP

Re: osm2city.py development

Postby vnts » Thu Jan 04, 2018 2:41 am

Something odd with OSM(?) objects & lighting.

Screenshots: the dark area (cloud shadow?) appears to move as the camera moves. e.g. Starts near the 2 buildings and moves forward as the camera moves forward 1, 2, 3, zoomed out showing clouds.

Does not happen with cloud shadows off. Does not happen on terrain - when objects are off. Just objects including roads (building interaction with cloud shadows?).

Screenshots testing recent nightly with spain OSM (LEMD). Happened in testing with 2017.2 OSM from wiki near KBOS. Never sure if it's just my FG setup (still very unfamiliar with FG).

Very quick look for bugs in new spain OSM outside LEMD & nightly (translucent road showing texture beneath - it might just be something I've done wrong).


lat 40.45798208
lon -3.624850158

Noticed that as the camera gets closer the problem resolves itself (z-buffer related?). Affected sections seem close to terrain. Generated roads often seem to go under terrain and re-emerge in the OSM. There's also some visible z-fighting flickering.
vnts
 
Posts: 409
Joined: Thu Apr 02, 2015 1:29 am

Re: osm2city.py development

Postby radi » Thu Jan 04, 2018 11:41 am

Same here, has been like that ever since Thorsten implemented ALS cloud shadows. I suspect it's due to the fact that osm2city objects (roads and buildings) are clustered into comparably huge meshes, on the order of 1x1 km. Perhaps the shadow calculation only takes into account the origin of an osm2city mesh? Thorsten?
OSM buildings for LOWI, EDDC
Custom scenery for VHXX YMML
Edit .stg via the FG Object Placement Tool
radi
 
Posts: 659
Joined: Mon Aug 25, 2008 5:24 pm
Location: YMML, EDDC

Re: osm2city.py development

Postby Thorsten » Thu Jan 04, 2018 1:28 pm

Perhaps the shadow calculation only takes into account the origin of an osm2city mesh? Thorsten?


Actually the clouds shadows should be computed by relative distance of fragment to the eye in the fragment shader - which should make them resilient against transformations, geometry extrusions or vertex resolution or in fact about any assumption about coordinate origin.

So the finding comes as a bit of a surprise.

I promised Rick to look at the dark areas in the road network anyway, so I'll just add that one to the to-do list.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: osm2city.py development

Postby legoboyvdlp » Thu Jan 04, 2018 7:07 pm

Script fails with
Code: Select all
C:\Users\Redpath\Documents\FlightGear\Custom Scenery\projects>python ../../../Gi
tHub/osm2city/build_tiles.py -f NI/params.ini -b *-8.888_53.963_-5.350_55.466 -p
 1
MainProcess root INFO     Reading parameters from file: NI/params.ini
Traceback (most recent call last):
  File "../../../GitHub/osm2city/build_tiles.py", line 197, in <module>
    parameters.read_from_file(args.filename)
  File "C:\Users\Redpath\Documents\GitHub\osm2city\parameters.py", line 343, in
read_from_file
    exec(compile(open(filename).read(), filename, 'exec'), file_globals)
  File "NI/params.ini", line 2
    PATH_TO_SCENERY = "C:\Users\Redpath\Documents\FlightGear\TerraSync-b"
                     ^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in positio
n 2-3: truncated \UXXXXXXXX escape

C:\Users\Redpath\Documents\FlightGear\Custom Scenery\projects>


Any suggestions?

I'll try forward slashes instead.
User avatar
legoboyvdlp
 
Posts: 7981
Joined: Sat Jul 26, 2014 2:28 am
Location: Northern Ireland
Callsign: G-LEGO
Version: next
OS: Windows 10 HP

Re: osm2city.py development

Postby legoboyvdlp » Thu Jan 04, 2018 7:11 pm

Seems to work. Last time it said "0 airports within area" and had no output, I tried \\ as someone else suggested. But now / seems to work.

Code: Select all
SpawnPoolWorker-1 root WARNING  stg_io:read: Ignoring unreadable file [Errno 2]
No such file or directory: 'C:/Users/Redpath/Documents/FlightGear/TerraSync-b\\O
bjects\\w010n50\\w009n54\\2810890.stg'


Try using a raw string or forward slashes:
https://stackoverflow.com/questions/18084554/why-do-i-get-a-syntaxerror-for-a-unicode-escape-in-my-file-path


For some reason the script works... but I don't see any main or details folders anywhere. What is going on!

I do see "permission error, access is denied" at the end of each tile. So I will try it in administrator mode.
User avatar
legoboyvdlp
 
Posts: 7981
Joined: Sat Jul 26, 2014 2:28 am
Location: Northern Ireland
Callsign: G-LEGO
Version: next
OS: Windows 10 HP

Re: osm2city.py development

Postby legoboyvdlp » Thu Jan 04, 2018 7:19 pm

An example tile process:

Code: Select all
 use boundaries: -6.5_54.625_-6.25_54.75 *******
Traceback (most recent call last):
  File "C:\Users\Redpath\Documents\GitHub\osm2city\utils\utilities.py", line 442
, in probe
    elev_is_solid_tuple = self._cache[key]
KeyError: (-6.3271271, 54.7430085)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Redpath\Documents\GitHub\osm2city\build_tiles.py", line 136, in
 process_scenery_tile
    buildings.process_buildings(the_coords_transform, my_fg_elev, my_blocked_are
as, my_stg_entries, file_lock)
  File "C:\Users\Redpath\Documents\GitHub\osm2city\buildings.py", line 603, in p
rocess_buildings
    prepare_textures.facades, prepare_textures.roofs, stats)
  File "C:\Users\Redpath\Documents\GitHub\osm2city\building_lib.py", line 1155,
in analyse
    if not b.analyse_elev_and_water(fg_elev):
  File "C:\Users\Redpath\Documents\GitHub\osm2city\building_lib.py", line 506, i
n analyse_elev_and_water
    temp_ground_elev = fg_elev.probe_list_of_points(self.pts_outer)
  File "C:\Users\Redpath\Documents\GitHub\osm2city\utils\utilities.py", line 459
, in probe_list_of_points
    elev_is_solid_tuple = self.probe(ve.Vec2d(point))
  File "C:\Users\Redpath\Documents\GitHub\osm2city\utils\utilities.py", line 445
, in probe
    elev_is_solid_tuple = really_probe(position)
  File "C:\Users\Redpath\Documents\GitHub\osm2city\utils\utilities.py", line 393
, in really_probe
    self._open_fgelev()
  File "C:\Users\Redpath\Documents\GitHub\osm2city\utils\utilities.py", line 365
, in _open_fgelev
    bufsize=1, universal_newlines=True)
  File "C:\Users\Redpath\Documents\Jonathans Folder\FlightGear Reference Files\W
inPython-64bit-3.5.4.1Zero\python-3.5.4.amd64\lib\subprocess.py", line 676, in _
_init__
    restore_signals, start_new_session)
  File "C:\Users\Redpath\Documents\Jonathans Folder\FlightGear Reference Files\W
inPython-64bit-3.5.4.1Zero\python-3.5.4.amd64\lib\subprocess.py", line 957, in _
execute_child
    startupinfo)
PermissionError: [WinError 5] Access is denied
SpawnPoolWorker-1 root INFO     ******* Finished tile 2843690 *******


Edit: I have worked out what is going on. My import to the database using osmosis was failing because I was not using the "superuser". So I had to manually grant my user permission to access schema_info table in pgadmin in order to actually be able to import the data

I will test now, and it should work better.


Once I finally see the scenery in FlightGear I will make a Windows tutorial on how to use it. Your documentation is great, but is best on Linux. Mine will show how to do it on Windows using pgadmin.
User avatar
legoboyvdlp
 
Posts: 7981
Joined: Sat Jul 26, 2014 2:28 am
Location: Northern Ireland
Callsign: G-LEGO
Version: next
OS: Windows 10 HP

Re: osm2city.py development

Postby vanosten » Fri Jan 05, 2018 2:49 pm

legoboyvdlp wrote in Thu Jan 04, 2018 7:19 pm:Once I finally see the scenery in FlightGear I will make a Windows tutorial on how to use it. Your documentation is great, but is best on Linux. Mine will show how to do it on Windows using pgadmin.


Could you please consider submitting it in almost any format to me, so I can include it directly into the guide? I would prefer one place where people look into. Thanks in advance.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby legoboyvdlp » Fri Jan 05, 2018 4:12 pm

I certainly would :)
Now that I have the data in the database it should be a matter of running build_tiles.py

most of my problems have been due to the user, not the software :P

Code: Select all
SpawnPoolWorker-1 root WARNING  stg_io:read: Ignoring unreadable file [Errno 2]
No such file or directory: 'C:/Users/Redpath/Documents/FlightGear/TerraSync-b\\O
bjects\\w010n50\\w009n53\\2810872.stg'


What's with this?

I could only use forward slashes, because otherwise it gave me a Unicode error, \U is to do with Unicode in python.

ie C:\Users


Edit:

I tried to run osm2city, and I got only a whole pile of elevxxxxxx.pkl files, each 1KB and a 330KB log...
https://pastebin.com/PZ64qcr6

No output produced either main or details folder. Not sure what is going on. It looks as if it can't access fgelev, permission is denied?
User avatar
legoboyvdlp
 
Posts: 7981
Joined: Sat Jul 26, 2014 2:28 am
Location: Northern Ireland
Callsign: G-LEGO
Version: next
OS: Windows 10 HP

Re: osm2city.py development

Postby vanosten » Sat Jan 06, 2018 6:55 pm

legoboyvdlp wrote in Fri Jan 05, 2018 4:12 pm:
Code: Select all
SpawnPoolWorker-1 root WARNING  stg_io:read: Ignoring unreadable file [Errno 2]
No such file or directory: 'C:/Users/Redpath/Documents/FlightGear/TerraSync-b\\O
bjects\\w010n50\\w009n53\\2810872.stg'


What's with this?

Well, it is just a warning - and if the program continues and you do not get anything into the osm2city exception log, then every thing is alright. In general I need much more info (e.g. the whole log) to be able to deduct some knowledge.

I tried to run osm2city, and I got only a whole pile of elevxxxxxx.pkl files, each 1KB and a 330KB log...
https://pastebin.com/PZ64qcr6

No output produced either main or details folder. Not sure what is going on. It looks as if it can't access fgelev, permission is denied?

Well, it looks like an access problem. You can try running with NO_ELEV = True.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 540
Joined: Sat Sep 25, 2010 6:38 pm
Location: Denmark - but I am Swiss
Pronouns: he/his
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: No registered users and 11 guests