osm2city.py development

[merspieler]

vanosten... have you taken a look into the issues with the NoneType and out of bounds?

I'm almost in place to start the first try at doing a world wide build...
I've got to wait tho until i can build most tiles relyable without exceptions cause i trigger the success/fail dependent on the existance of that file
If i can help you in any way, please tell me

[legoboyvdlp]

Has anyone ever got this?

Code: Select all

 Unable to process building relation osm_id 1566802
Traceback (most recent call last):
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\buildings.py", line 170, in _process_osm_relations
    my_buildings, coords_transform)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\buildings.py", line 248, in _process_multipolygon_buildings
    a_building = _make_building_from_way(nodes_dict, all_tags, outer_way, coords_transform, inner_rings)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\buildings.py", line 510, in _make_building_from_way
    refs=way.refs)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\building_lib.py", line 266, in __init__
    self.update_geometry(outer_ring, inner_rings_list, refs)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\building_lib.py", line 315, in update_geometry
    self.roll_inner_nodes()
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\building_lib.py", line 381, in roll_inner_nodes
    outer_nodes_avail.remove(min_o)
ValueError: list.remove(x): x not in list

It seems to continue fine.

[legoboyvdlp]

Also

Code: Select all

SpawnPoolWorker-3 root ERROR    ******* Exception in tile 2941786 - to reprocess use boundaries: -0.5_51.375_-0.25_51.5 *******
Traceback (most recent call last):
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\build_tiles.py", line 152, in process_scenery_tile
    pylons.process_details(the_coords_transform, lit_areas, my_fg_elev, file_lock)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 1908, in process_details
    highway.calc_and_map(fg_elev, coords_transform)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 749, in calc_and_map
    _calc_heading_nodes(self.shared_pylons)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 1572, in _calc_heading_nodes
    next_pylon = nodes_array[1]
IndexError: list index out of range
...
SpawnPoolWorker-1 root ERROR    Exception occurred while processing tile 2941794.
Traceback (most recent call last):
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\build_tiles.py", line 152, in process_scenery_tile
    pylons.process_details(the_coords_transform, lit_areas, my_fg_elev, file_lock)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 1908, in process_details
    highway.calc_and_map(fg_elev, coords_transform)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 709, in calc_and_map
    p = shg.Polygon(self.highway.linear)
  File "C:\Users\redpa\AppData\Local\Programs\Python\Python37\lib\site-packages\shapely\geometry\polygon.py", line 240, in __init__
    ret = geos_polygon_from_py(shell, holes)
  File "C:\Users\redpa\AppData\Local\Programs\Python\Python37\lib\site-packages\shapely\geometry\polygon.py", line 494, in geos_polygon_from_py
    ret = geos_linearring_from_py(shell)
  File "shapely\speedups\_speedups.pyx", line 239, in shapely.speedups._speedups.geos_linearring_from_py
ValueError: A LinearRing must have at least 3 coordinate tuples
SpawnPoolWorker-1 root ERROR    ******* Exception in tile 2941794 - to reprocess use boundaries: -0.5_51.5_-0.25_51.625 *******
Traceback (most recent call last):
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\build_tiles.py", line 152, in process_scenery_tile
    pylons.process_details(the_coords_transform, lit_areas, my_fg_elev, file_lock)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 1908, in process_details
    highway.calc_and_map(fg_elev, coords_transform)
  File "C:\Users\redpa\Documents\FlightGear\osm2city-work\osm2city\pylons.py", line 709, in calc_and_map
    p = shg.Polygon(self.highway.linear)
  File "C:\Users\redpa\AppData\Local\Programs\Python\Python37\lib\site-packages\shapely\geometry\polygon.py", line 240, in __init__
    ret = geos_polygon_from_py(shell, holes)
  File "C:\Users\redpa\AppData\Local\Programs\Python\Python37\lib\site-packages\shapely\geometry\polygon.py", line 494, in geos_polygon_from_py
    ret = geos_linearring_from_py(shell)
  File "shapely\speedups\_speedups.pyx", line 239, in shapely.speedups._speedups.geos_linearring_from_py
ValueError: A LinearRing must have at least 3 coordinate tuples

[vanosten]

I ran -0.5_51.375_-0.25_51.5 without problems. Are you sure to use proper data like described in https://osm2city.readthedocs.io/en/late ... etmap-data? I.e. no other way of getting and cutting data?

[legoboyvdlp]

Yes, certainly - using geofabrik data for London - it is untouched, not even cut by osmosis.

Seems to be specifically in detailed pylons?

[vanosten]

Most probably you used STREETLAMPS. It has always been experimental and unsupported - and I have now updated the docs and totally disabled it.

Your first issue "Unable to process building relation osm_id 1566802" I will have to investigate another day.

[legoboyvdlp]

Ok - thank you

[merspieler]

I need your help... I'm not sure, if it's my fault or not. I'm trying to run the same build as before with exception of the FLAG_STG_BUILD_LIST=True option in the params.ini.
You can see the printout here.

the build script that I'm using is just changing into the project dir and then running the following:
time ( python3 ../../osm2city/build_tiles.py -f params.ini -b "$bounds" -p 3 2>&1 ) 2> exec-time

($bounds is read from another file)

This has worked so far without any problems... but now it hangs... no CPU nor IO usage (I've killed it in the end cause of that)

Can you help me to determinate what i'm doing wrong

[vanosten]

Sorry, I do not understand your printout. Maybe running with only one process instead of 3 parallel processes would help understanding the output. I am not even sure "where it hangs".
(It might be that the building list file is locked - but I need a proper output first.

[legoboyvdlp]

Its somewhat confusing - is there a need for osm2city to parse everything, create settlement clusters when I only want details (no buildings)? That takes over three hours for complex tiles... I am now re-running with -e details to try and generate the piers etc after disabling streetlamps as you suggested but it seems to be restarting the landuse.py script from the beginning - and if it starts doing those settlement clusters I think I will forget about it...


Also, may be a bug causing the exception in processing relations:

Code: Select all

for inner in self.polygon.interiors:
            min_r = 1e99
            for i, node_i in enumerate(list(inner.coords)[:-1]):
                node_i = Vec2d(node_i)
                for o in outer_nodes_avail:
                    r = node_i.distance_to(Vec2d(self.pts_outer[o]))
                    if r <= min_r:
                        min_r = r
                        min_i = i
                        min_o = o
            new_inner = shg.polygon.LinearRing(np.roll(np.array(inner.coords)[:-1], -min_i, axis=0))
            new_inner_rings_list.append(new_inner)
            self.outer_nodes_closest.append(min_o)
            outer_nodes_avail.remove(min_o)

What happens if min_o is never set to o, therefore can't be removed from the outer_nodes?

[merspieler]

Now it works again... .oO(Have you turned it off and on again?)
Or was it a change that came with the last pull?
fb1f963..2681f9f master -> origin/master

For now it's running so... i'll keep it that way

[legoboyvdlp]

Your output suggested no data was found - was your database correct?

Code: Select all

Found 0 buildings out of 0 buildings

and so on?

[vanosten]

Its somewhat confusing - is there a need for osm2city to parse everything, create settlement clusters when I only want details (no buildings)?

I have pushed a fix, so landuse processing is only called if absolutely necessary.

[legoboyvdlp]

Thanks
This should save me six hours tomorrow...

[merspieler]

@lego ofc is the database correct... i've never touched it... it worked before and after... no clue why it hang.