Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby merspieler » Wed Mar 20, 2019 10:13 am

vanosten... have you taken a look into the issues with the NoneType and out of bounds?

I'm almost in place to start the first try at doing a world wide build...
I've got to wait tho until i can build most tiles relyable without exceptions cause i trigger the success/fail dependent on the existance of that file
If i can help you in any way, please tell me
Love at first flight A<380
Checkout Autopush. An improvment to the pushback to make your life easier.
Attempting an osm2city worldbuild... Testers needed!
merspieler
 
Posts: 276
Joined: Thu Oct 26, 2017 10:43 am
Location: YBCS
Callsign: LH454, HG3840, LX64
IRC name: merspieler
Version: 2019.1.1
OS: Debian Bullseye

Re: osm2city.py development

Postby legoboyvdlp » Thu Mar 28, 2019 11:05 am

Has anyone ever got this?

Code: Select all
 Unable to process building relation osm_id 1566802
Traceback (most recent call last):
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\buildings.py", line 170, in _process_osm_relations
    my_buildings, coords_transform)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\buildings.py", line 248, in _process_multipolygon_buildings
    a_building = _make_building_from_way(nodes_dict, all_tags, outer_way, coords_transform, inner_rings)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\buildings.py", line 510, in _make_building_from_way
    refs=way.refs)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\building_lib.py", line 266, in __init__
    self.update_geometry(outer_ring, inner_rings_list, refs)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\building_lib.py", line 315, in update_geometry
    self.roll_inner_nodes()
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\building_lib.py", line 381, in roll_inner_nodes
    outer_nodes_avail.remove(min_o)
ValueError: list.remove(x): x not in list


It seems to continue fine.
User avatar
legoboyvdlp
 
Posts: 7001
Joined: Sat Jul 26, 2014 1:28 am
Callsign: YV-LEGO
Version: 2018.3.1
OS: Windows 10 HP

Re: osm2city.py development

Postby legoboyvdlp » Thu Mar 28, 2019 2:11 pm

Also

Code: Select all
SpawnPoolWorker-3 root ERROR    ******* Exception in tile 2941786 - to reprocess use boundaries: -0.5_51.375_-0.25_51.5 *******
Traceback (most recent call last):
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\build_tiles.py", line 152, in process_scenery_tile
    pylons.process_details(the_coords_transform, lit_areas, my_fg_elev, file_lock)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 1908, in process_details
    highway.calc_and_map(fg_elev, coords_transform)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 749, in calc_and_map
    _calc_heading_nodes(self.shared_pylons)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 1572, in _calc_heading_nodes
    next_pylon = nodes_array[1]
IndexError: list index out of range
...
SpawnPoolWorker-1 root ERROR    Exception occurred while processing tile 2941794.
Traceback (most recent call last):
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\build_tiles.py", line 152, in process_scenery_tile
    pylons.process_details(the_coords_transform, lit_areas, my_fg_elev, file_lock)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 1908, in process_details
    highway.calc_and_map(fg_elev, coords_transform)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 709, in calc_and_map
    p = shg.Polygon(self.highway.linear)
  File "C:\Users\redpa\AppData\Local\Programs\Python\Python37\lib\site-packages\shapely\geometry\polygon.py", line 240, in __init__
    ret = geos_polygon_from_py(shell, holes)
  File "C:\Users\redpa\AppData\Local\Programs\Python\Python37\lib\site-packages\shapely\geometry\polygon.py", line 494, in geos_polygon_from_py
    ret = geos_linearring_from_py(shell)
  File "shapely\speedups\_speedups.pyx", line 239, in shapely.speedups._speedups.geos_linearring_from_py
ValueError: A LinearRing must have at least 3 coordinate tuples
SpawnPoolWorker-1 root ERROR    ******* Exception in tile 2941794 - to reprocess use boundaries: -0.5_51.5_-0.25_51.625 *******
Traceback (most recent call last):
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\build_tiles.py", line 152, in process_scenery_tile
    pylons.process_details(the_coords_transform, lit_areas, my_fg_elev, file_lock)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 1908, in process_details
    highway.calc_and_map(fg_elev, coords_transform)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 709, in calc_and_map
    p = shg.Polygon(self.highway.linear)
  File "C:\Users\redpa\AppData\Local\Programs\Python\Python37\lib\site-packages\shapely\geometry\polygon.py", line 240, in __init__
    ret = geos_polygon_from_py(shell, holes)
  File "C:\Users\redpa\AppData\Local\Programs\Python\Python37\lib\site-packages\shapely\geometry\polygon.py", line 494, in geos_polygon_from_py
    ret = geos_linearring_from_py(shell)
  File "shapely\speedups\_speedups.pyx", line 239, in shapely.speedups._speedups.geos_linearring_from_py
ValueError: A LinearRing must have at least 3 coordinate tuples
User avatar
legoboyvdlp
 
Posts: 7001
Joined: Sat Jul 26, 2014 1:28 am
Callsign: YV-LEGO
Version: 2018.3.1
OS: Windows 10 HP

Re: osm2city.py development

Postby vanosten » Sat Mar 30, 2019 7:18 pm

I ran -0.5_51.375_-0.25_51.5 without problems. Are you sure to use proper data like described in https://osm2city.readthedocs.io/en/late ... etmap-data? I.e. no other way of getting and cutting data?
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 411
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby legoboyvdlp » Sat Mar 30, 2019 9:51 pm

Yes, certainly - using geofabrik data for London - it is untouched, not even cut by osmosis.

Seems to be specifically in detailed pylons?
User avatar
legoboyvdlp
 
Posts: 7001
Joined: Sat Jul 26, 2014 1:28 am
Callsign: YV-LEGO
Version: 2018.3.1
OS: Windows 10 HP

Re: osm2city.py development

Postby vanosten » Sun Mar 31, 2019 7:13 am

Most probably you used STREETLAMPS. It has always been experimental and unsupported - and I have now updated the docs and totally disabled it.

Your first issue "Unable to process building relation osm_id 1566802" I will have to investigate another day.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 411
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby legoboyvdlp » Sun Mar 31, 2019 8:27 am

Ok - thank you ;)
User avatar
legoboyvdlp
 
Posts: 7001
Joined: Sat Jul 26, 2014 1:28 am
Callsign: YV-LEGO
Version: 2018.3.1
OS: Windows 10 HP

Re: osm2city.py development

Postby merspieler » Sun Mar 31, 2019 9:48 am

I need your help... I'm not sure, if it's my fault or not. I'm trying to run the same build as before with exception of the FLAG_STG_BUILD_LIST=True option in the params.ini.
You can see the printout here.

the build script that I'm using is just changing into the project dir and then running the following:
time ( python3 ../../osm2city/build_tiles.py -f params.ini -b "$bounds" -p 3 2>&1 ) 2> exec-time

($bounds is read from another file)

This has worked so far without any problems... but now it hangs... no CPU nor IO usage (I've killed it in the end cause of that)

Can you help me to determinate what i'm doing wrong
Love at first flight A<380
Checkout Autopush. An improvment to the pushback to make your life easier.
Attempting an osm2city worldbuild... Testers needed!
merspieler
 
Posts: 276
Joined: Thu Oct 26, 2017 10:43 am
Location: YBCS
Callsign: LH454, HG3840, LX64
IRC name: merspieler
Version: 2019.1.1
OS: Debian Bullseye

Re: osm2city.py development

Postby vanosten » Mon Apr 01, 2019 6:40 pm

Sorry, I do not understand your printout. Maybe running with only one process instead of 3 parallel processes would help understanding the output. I am not even sure "where it hangs".
(It might be that the building list file is locked - but I need a proper output first.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 411
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby legoboyvdlp » Tue Apr 02, 2019 8:32 am

Its somewhat confusing - is there a need for osm2city to parse everything, create settlement clusters when I only want details (no buildings)? That takes over three hours for complex tiles... I am now re-running with -e details to try and generate the piers etc after disabling streetlamps as you suggested but it seems to be restarting the landuse.py script from the beginning - and if it starts doing those settlement clusters I think I will forget about it...


Also, may be a bug causing the exception in processing relations:

Code: Select all
for inner in self.polygon.interiors:
            min_r = 1e99
            for i, node_i in enumerate(list(inner.coords)[:-1]):
                node_i = Vec2d(node_i)
                for o in outer_nodes_avail:
                    r = node_i.distance_to(Vec2d(self.pts_outer[o]))
                    if r <= min_r:
                        min_r = r
                        min_i = i
                        min_o = o
            new_inner = shg.polygon.LinearRing(np.roll(np.array(inner.coords)[:-1], -min_i, axis=0))
            new_inner_rings_list.append(new_inner)
            self.outer_nodes_closest.append(min_o)
            outer_nodes_avail.remove(min_o)


What happens if min_o is never set to o, therefore can't be removed from the outer_nodes?
User avatar
legoboyvdlp
 
Posts: 7001
Joined: Sat Jul 26, 2014 1:28 am
Callsign: YV-LEGO
Version: 2018.3.1
OS: Windows 10 HP

Re: osm2city.py development

Postby merspieler » Tue Apr 02, 2019 10:01 am

Now it works again... .oO(Have you turned it off and on again?)
Or was it a change that came with the last pull?
fb1f963..2681f9f master -> origin/master

For now it's running so... i'll keep it that way
Love at first flight A<380
Checkout Autopush. An improvment to the pushback to make your life easier.
Attempting an osm2city worldbuild... Testers needed!
merspieler
 
Posts: 276
Joined: Thu Oct 26, 2017 10:43 am
Location: YBCS
Callsign: LH454, HG3840, LX64
IRC name: merspieler
Version: 2019.1.1
OS: Debian Bullseye

Re: osm2city.py development

Postby legoboyvdlp » Tue Apr 02, 2019 10:11 am

Your output suggested no data was found - was your database correct?

Code: Select all
Found 0 buildings out of 0 buildings


and so on?
User avatar
legoboyvdlp
 
Posts: 7001
Joined: Sat Jul 26, 2014 1:28 am
Callsign: YV-LEGO
Version: 2018.3.1
OS: Windows 10 HP

Re: osm2city.py development

Postby vanosten » Tue Apr 02, 2019 5:42 pm

legoboyvdlp wrote in Tue Apr 02, 2019 8:32 am:Its somewhat confusing - is there a need for osm2city to parse everything, create settlement clusters when I only want details (no buildings)?


I have pushed a fix, so landuse processing is only called if absolutely necessary.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 411
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby legoboyvdlp » Tue Apr 02, 2019 10:07 pm

Thanks :)
This should save me six hours tomorrow...
User avatar
legoboyvdlp
 
Posts: 7001
Joined: Sat Jul 26, 2014 1:28 am
Callsign: YV-LEGO
Version: 2018.3.1
OS: Windows 10 HP

Re: osm2city.py development

Postby merspieler » Wed Apr 03, 2019 12:01 am

@lego ofc is the database correct... i've never touched it... it worked before and after... no clue why it hang.
Love at first flight A<380
Checkout Autopush. An improvment to the pushback to make your life easier.
Attempting an osm2city worldbuild... Testers needed!
merspieler
 
Posts: 276
Joined: Thu Oct 26, 2017 10:43 am
Location: YBCS
Callsign: LH454, HG3840, LX64
IRC name: merspieler
Version: 2019.1.1
OS: Debian Bullseye

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: No registered users and 8 guests