Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby vanosten » Tue Jan 17, 2017 6:41 pm

Hi powoflight

I do not understand whether you just make a statement or report a problem.

What I can say though based on your params.ini file (also commented upon in earlier posts in this thread) is that you should not generate sceneries across tile boundaries (see http://wiki.flightgear.org/Tile_Index_Scheme). The easiest way to see tiles boundaries is by using https://scenery.flightgear.org/map/, input the airport (LOWW in this case) and then observe the blue grid as well as the tile-number at the bottom. The obvious alternative right now is using batch processing.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 483
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby powoflight » Wed Jan 18, 2017 3:16 pm

Hi vanosten

I think slowly wien is a special case because I
downloded LOWI with Insbruck , zirl in the west and Schwaz in the east , a big area over 4 tiles and it works perfect !!!

BOUNDARY_WEST = 11.16
BOUNDARY_SOUTH = 47.20
BOUNDARY_EAST = 11.79
BOUNDARY_NORTH = 47.38

Before we get into details about errors , the best is that you answer a few questions, to clarify something to me.

----------------------------------------------------------------------------------------------------------------
Question 1:
----------------------------------------------------------------------------------------------------------------
If i download OSM data and generate the buildings in different areas , do i have (is it a must) to start prepare_textures.py allways? I guess it only influence the textures but nothing more.

----------------------------------------------------------------------------------------------------------------
Question 2:
----------------------------------------------------------------------------------------------------------------
I tried the batch_processing but i get an error

D:\FG\SC\OSM_PROJECTS> python3 D:\FG\SC\OSM_Development35\osm2city\batch_processing\build_tiles.py -t e009n47 -f CH_OSM\params_kp.ini -o params.ini
Traceback (most recent call last):
File "D:\FG\SC\OSM_Development35\osm2city\batch_processing\build_tiles.py", line 14, in <module>
import parameters
ImportError: No module named 'parameters'

do i have forget something for python ??


Directory Structure what i use:
D:\FG\SC\OSM_Development35\osm2city
D:\FG\SC\SCENERY_OSM\CH_OSM This is the output Directory
D:\FG\SC\OSM_PROJECTS Here i start: python3 D:\FG\SC\OSM_Development35\osm2city\batch_processing\build_tiles.py -t e009n47 -f CH_OSM\params_kp.ini -o params.ini
D:\FG\SC\OSM_PROJECTS\CH_OSM\params_kp.ini

This is params_kp.ini
PREFIX = "CH_OSM"
BOUNDARY_WEST = 16.2433
BOUNDARY_SOUTH = 48.0974
BOUNDARY_EAST = 16.6427
BOUNDARY_NORTH = 48.3251
PATH_TO_SCENERY = "D:/FG/SC/SC_HD"
PATH_TO_OUTPUT = "D:\FG\SC\SCENERY_OSM\CH_OSM"
PATH_TO_OSM2CITY_DATA = "D:/FG/SC/OSM_Development35/osm2city-data"
OSM_FILE = "buildings.osm"
NO_ELEV = False # -- skip elevation probing
ELEV_MODE = "FgelevCaching"
FG_ELEV = '"D:/FG/FlightGear36/bin/fgelev.exe"'

----------------------------------------------------------------------------------------------------------------
Question 3:
----------------------------------------------------------------------------------------------------------------
Is it possible to downlod a big area from osm with wget. lets say from vienna and suburbs
and generate the buildings only for a small area like the first district ( I think i have done this with the old python 2.7 Version)?
----------------------------------------------------------------------------------------------------------------

Now I have 2 problems.
A)
I download as you said only 1 tile from Vienna and i get no output (means no buildings).
ERROR:root:Failed to parse building (A LinearRing must have at least 3 coordinate tuples) WayID 357085032 {'building:part': 'yes'} Refs [3624570540, 1842593375]

B)
Then i tried to generate only the first district from vienna and get the KeyError: 1936175569
I have downloaded your bugfix before. As you mentioned before.
" For the last thing. Yes, this is a bug fixed on Monday (i.e. please pull the latest version from GIT). The error was triggered, when the building was tagged "building:part", which tends to happen only in well modeled cities (and my default test range did not include it :-( ). "

# first district
BOUNDARY_WEST = 16.356801
BOUNDARY_SOUTH = 48.199601
BOUNDARY_EAST = 16.385801
BOUNDARY_NORTH = 48.220301

INFO:root:Reading OSM data from xml took 140.8212 seconds.
Traceback (most recent call last):
File "D:\FG\SC\OSM_Development35\osm2city\buildings.py", line 600, in <module>
process()
File "D:\FG\SC\OSM_Development35\osm2city\buildings.py", line 444, in process
the_buildings = _process_osm_building(osm_nodes_dict, osm_ways_dict, clipping_border)
File "D:\FG\SC\OSM_Development35\osm2city\buildings.py", line 212, in _process_osm_building
first_node = nodes_dict[way.refs[0]]
KeyError: 1936175569

Hope this was not to much for you. I slowly think may if i should go back to python version 2.7 and try it , what happens :)-
powoflight
 
Posts: 154
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2020.1
OS: win7 Ubuntu 18.04

Re: osm2city.py development

Postby vanosten » Wed Jan 18, 2017 6:37 pm

Question 1: no, you do not have to. Just remember to do it if you change the data or if you pull new osm2city-data from git.

Question 2: I guess you run it outside of an IDE and have forgotten to set the PYTHONPATH: http://osm2city.readthedocs.io/en/lates ... pythonpath

Question 3: yes, but there is a memory/performance tax to pay. Doing stuff like this is the reason we are looking at using data from a DB. See also http://osm2city.readthedocs.io/en/lates ... ing-region

Issues: I would be interested to get a link to the osm-files and parameter-files you used, so I can try to reproduce the problem.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 483
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby powoflight » Thu Jan 19, 2017 10:08 am

You can download the Scenario WIEN_TILE.zip file from
https://c.gmx.net/@327459158418791162/R ... -YY6t7l1Og
powoflight
 
Posts: 154
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2020.1
OS: win7 Ubuntu 18.04

Re: osm2city.py development

Postby powoflight » Thu Jan 19, 2017 10:57 am

Sorry . I forgot to mention , that if i generated the 1 district from vienna and got the KeyError: 1936175569
This only happens if the osm file is bigger than 1 tile !!!

If i use a osmfile this is not bigger or equel than 1 tile i get no buildings (see Scenario WIEN_TILE.zip)
Maybe this info helps.
powoflight
 
Posts: 154
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2020.1
OS: win7 Ubuntu 18.04

Re: osm2city.py development

Postby powoflight » Thu Jan 19, 2017 12:39 pm

I have uploaded wien1.zip file too. (just small)
Here i have only downloaded the osm data from the 1 district from vienna and tried to build the data for the 1 district. see WIEN1_02_BUILDING.log.
Result are no buildings.
area >=
1 m^2 0 |
10 m^2 0 |
20 m^2 0 |
50 m^2 0 |
100 m^2 0 |
200 m^2 0 |
500 m^2 0 |
1000 m^2 0 |
2000 m^2 0 |
5000 m^2 0 |
10000 m^2 0 |
20000 m^2 0 |
50000 m^2 0 |
powoflight
 
Posts: 154
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2020.1
OS: win7 Ubuntu 18.04

Re: osm2city.py development

Postby vanosten » Thu Jan 19, 2017 5:16 pm

I have looked at WIEN1. There was a bug when there is no blocked area (i.e. no airport) taken into account. I have pushed a fix. WIEN_TILE I will have a look later.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 483
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby powoflight » Thu Jan 19, 2017 6:40 pm

I will pull the last git and test it.
powoflight
 
Posts: 154
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2020.1
OS: win7 Ubuntu 18.04

Re: osm2city.py development

Postby vanosten » Thu Jan 19, 2017 7:02 pm

WIEN_TILES: there was no problem running it on my machine with your params.ini and osm-file. Please have a look at http://osm2city.readthedocs.io/en/lates ... etmap-data.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 483
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby powoflight » Fri Jan 20, 2017 8:27 am

WIEN1 works now.

For WIN7 users . Once again i get an error:
OSError: [Errno 22] Invalid argument: ' D:\\FG\\FlightGear36\\data\\Airports\\apt.dat.gz'
There is a SPACE between ' D

as a result gzip.open can not find the file .
As a short patch i hardcoded the file_name = 'D:/FG/FlightGear36/data/Airports/apt.dat.gz'
in aptdat_io.py in the function

def _read_apt_dat_gz_file(file_name: str, min_lon: float, min_lat: float,
max_lon: float, max_lat: float) -> List[Airport]:
start_time = time.time()
airports = list()
total_airports = 0

logging.info(" *** SUB WRONG")
logging.info(" *** SUB 1 aptdat_io powo 102 utils aptdat_io file_name : " + file_name )

file_name = 'D:/FG/FlightGear36/data/Airports/apt.dat.gz' # THIS IS MY PATCH
logging.info(" *** SUB WORKS NOW 1 aptdat_io powo 102 utils aptdat_io file_name : " + file_name )

with gzip.open(file_name, 'rt', encoding="latin-1") as f:

for WIEN_TILES i will do more investigations .
Did you use this params?
BOUNDARY_WEST = 16.250201
BOUNDARY_SOUTH = 48.125100
BOUNDARY_EAST = 16.499800
BOUNDARY_NORTH = 48.249800
powoflight
 
Posts: 154
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2020.1
OS: win7 Ubuntu 18.04

Re: osm2city.py development

Postby powoflight » Fri Jan 20, 2017 9:27 am

With the last git WIEN_TILES too working.
powoflight
 
Posts: 154
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2020.1
OS: win7 Ubuntu 18.04

Re: osm2city.py development

Postby vanosten » Sat Jan 21, 2017 2:28 pm

powoflight wrote in Fri Jan 20, 2017 8:27 am:For WIN7 users . Once again i get an error:
OSError: [Errno 22] Invalid argument: ' D:\\FG\\FlightGear36\\data\\Airports\\apt.dat.gz'
There is a SPACE between ' D

as a result gzip.open can not find the file .


Just in case I do now trim the value returned from the OS. Could you please have a try? And after you tried: could you please go into your environment variables and check, whether you accidentally set a leading blank in the FG_ROOT environment variable?
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 483
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby powoflight » Sun Jan 22, 2017 1:15 pm

OSError: [Errno 22] Invalid argument: ' D:\\FG\\FlightGear36\\data\\Airports\\apt.dat.gz'
Yes . now it works. Perfect

By the way . I have found 1 bug more , but my wife is at home, so no time :-) . Tommorow i will check once again and send a report.. You know how you eat an elephant? Answer: Step by step ...

PS Slowly i fell like a Betatester, but it makes fun.
powoflight
 
Posts: 154
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2020.1
OS: win7 Ubuntu 18.04

Re: osm2city.py development

Postby vanosten » Sun Jan 22, 2017 2:35 pm

Well - you should feel like a beta tester, because I cannot test on both Linux, Windows, OS X and on huge data sets -- and still have time to do some coding. Btw I still think that the blank is due to a setting on your machine - as it works correctly on my seldom used Windows.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 483
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby powoflight » Mon Jan 23, 2017 8:26 am

Subject : SKIP_LIST = ["Donauturm"] ERROR

In Wien there is a static object called Donauturm and in the osm data there is the same object too. Because the 2 objects are a little far away buildings.py
generates the tower. See picture fgfs-Donauturm.png (is in DONAUTURM.zip included)

So i put in parans.ini SKIP_LIST = ["Donauturm"]

But this gives an error.

INFO:root:SKIPPING Donauturm
INFO:root:min/max 16.4070900 48.2380900 16.4140800 48.2413000
INFO:root:Created 45 buildings.
Traceback (most recent call last):
File "D:\FG\SC\OSM_Development35\osm2city\buildings.py", line 618, in <module>
process(args.uninstall, args.create_atlas)
File "D:\FG\SC\OSM_Development35\osm2city\buildings.py", line 440, in process
tex.screen_osm_tags_for_colour_spelling(b.osm_id, b.tags)
AttributeError: 'bool' object has no attribute 'osm_id'


Here will be skipped the data
the_buildings = _process_osm_building(osm_nodes_dict, osm_ways_dict, clipping_border)

Here ocours the error . There is an object no ID (is skipped ? before)
# clean up "color" in tags
for b in the_buildings:
tex.screen_osm_tags_for_colour_spelling(b.osm_id, b.tags)


You can download the DONAUTURM.zip (params.ini,buildings.osm eg) from
https://c.gmx.net/@327459158418791162/R ... -YY6t7l1Og
for testing.

By the way how can i debug a python program ?
I was used to do it for c programms very comfortable with an ide.
powoflight
 
Posts: 154
Joined: Fri Mar 25, 2016 10:04 am
Location: LOWW
Version: 2020.1
OS: win7 Ubuntu 18.04

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: YandexBot [Bot] and 1 guest