Board index FlightGear Development Scenery

osm2city.py development

Questions and discussion about enhancing and populating the FlightGear world.

Re: osm2city.py development

Postby radi » Sun Sep 11, 2016 3:31 pm

Excellent job, thank you very much! I hope I can test in about two weeks' time.
OSM buildings for LOWI, EDDC
Custom scenery for VHXX YMML
Edit .stg via the FG Object Placement Tool
radi
 
Posts: 643
Joined: Mon Aug 25, 2008 4:24 pm
Location: YMML, EDDC

Re: osm2city.py development

Postby pommesschranke » Sun Sep 11, 2016 7:04 pm

radi wrote in Wed Sep 07, 2016 12:23 pm: change the dimension of ALL roof textures to something huge, e.g. 400x400m. This should get rid of most/all roof bugs.


I did. And it works! :D
Thank you very much!
pommesschranke
 
Posts: 1104
Joined: Sat Apr 27, 2013 7:58 pm
Location: EDLM & LJCE
Callsign: d-laser
IRC name: laserman
Version: git
OS: Linux Lubuntu 18.04

Re: osm2city.py development

Postby stuart » Tue Sep 13, 2016 6:53 pm

Hi Guys,

As you may have seen from the -devel list, there's some interested in integrating osm2city output into the scenery toolchain. Ideally, it would be good to get a couple of the main developers on the -devel list to discuss this further.

As a starter point, I've been looking at how to integrate this nicely into the .stg format while minimizing the scenegraph depth, I'd like to split the LOD_detail and LOD_rough objects into separate .ac files. We can then apply some PagedLOD in the .STG file handling so that they get loaded only when within range.

I don't want to step on your toes here, but it would probably also be good to have osm2city under flightgear/utils as well.

Please let me know if you are interested.

Best regards,

-Stuart
G-MWLX
User avatar
stuart
Moderator
 
Posts: 1469
Joined: Wed Nov 29, 2006 9:56 am
Location: Edinburgh
Callsign: G-MWLX

Re: osm2city.py development

Postby portreekid » Wed Sep 14, 2016 6:19 am

I'm happy to help if I can find time. The split of the LOD ac files should be straightforward. Regarding the move to utils I have no objections.
portreekid
 
Posts: 336
Joined: Tue Jan 14, 2014 3:36 pm
Location: Dresden
Version: 3.0
OS: Windows 7

Re: osm2city.py development

Postby vanosten » Wed Sep 14, 2016 3:09 pm

I will try to read devel maillist archives today. I agree regarding LOD, however I need to understand the mechanics better to do it right (not the spli, but e.g if curren clustering for osm2pylons is "correct".

Before we move to a new home I would at least like to improve the current directory structure and remove more dead code.

Feature wise osm2city might be too European-style to be in a global toolchain.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 416
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Thu Sep 15, 2016 4:53 am

As I can only subscribe to, but not post to the devel-list (something about my reply-to-address etc. which I do not have the time to fix now), I will repost my answer here:

Please allow me to give my personal view on the discussion regarding the integration of osm2city (same disclaimer as Keith).
* I agree that it is a good idea to have new STG verbs. I would like to ask to consider whether they could get more generic names. I guess we just need to distinguish between [A] STATIC as in unique objects manually added, [B] SHARED individual objects based on objects in the scenery database, [C] clusters of facades for volume objects like buildings and [D] clusters of facades for linear objects like roads. It is naming w.r.t. what we handle, and not so much an association to where it comes from (e.g. osm2city could also have competition from procedural city ...).
* If it makes sense to do clustering by LOD, then let us do it and have it as part of the verb. 2 or 3 levels? I do not know - osm2city (building part) uses 3 levels.
* osm2city related scripts do a lot more than buildings. Maybe this thread is only about buildings, So just for the records: volume like objects for (train) platforms, volume like objects for piers, line like objects for roads and railways, line and volume objects for power lines, aerialways etc. Actually osm2pylon is "competing" with pylons (without the wires) already in the scenery.
* Currently the number of available textures for buildings is limited. And support to apply textures by region is in its beginnings - but not difficult to extend (actually on my todo list). Meaning: it all looks very Western European.
* IMHO facade clusters generated out of OSM information alone works fine in cities (ergo osm2"city"). For residential areas or villages it does not give very good results - here actual "shared objects" should be included into clusters (maybe / maybe not shared objects from scenery database - I am experimenting).
* It is not ideal that the textures need to be copied around in each scenery object sub-folder. There should be a possibility to store them more centrally - a place in FGDATA or somewhere else, which is reserved for outside generated textures etc. Actually it will be a bit more complex, because once we would have lots of regional textures, then we do not want to fill memory with textures for the whole global area (unless we assume that whoever actually wants to use osm2city also has a graphics card, where lets say 50-100 MB of textures does not matter).
* osm2city related scripts do not make a diff between what has been in an OSM-extract used last time and the currect OSM-extract. For several reasons: ease of extract logic, difficulties to decide when a change in geometry or even tags are enough to require an update - and most importantly because you would have to go back and find out, in which existing cluster to place an object (ok, we could preserve the OSM ID in the AC-objects). In each case providing a facility like terrasync to just push out deltas for som2city stuff would be non-trivial.
* osm2city related scripts are slow - profiling might reveal some easy to fix stuff - and hard-core Python coders might also find easy optimizations. What has not been implemented for good reasons is concurrency: [A] atomicy is not so easy due to relations and [B] it is much easier to parallelize just by tiles. osm2city has a batch mode with possibility to parallelize - however I do not find it safe. So just trigger batch mode for whole 1x1 degree areas in parallel can gain speed.
* There are some glitches in OSM data vs. the rest of the scenery: e.g. buildings in the water because the cost line of OSM and e.g. Corine data differ. The fix is easy by reading FG scenery data into the same process as generating OSM-data. However that makes the process even more complicated than it currently is.
* Before osm2city related scripts could be merged into FG, at least some clean-up of dead code and directory structure should be done. Also I do not want to loose commit rights.
* For the records: I am infrequently working on OpenStree Map Would-Be Buildings (https://gitlab.com/vanosten/owbb). This might once in the future address the most probably biggest problem right now: too few objects/buildings have been mapped in OSM - and coverage varies a lot even within smaller areas. So OSM can actually look worse than without.

Therefore from my point of view:
* I very welcome new STG verbs. I will help the other osm2city developers to implement the changes.
* Maturity of osm2city related scripts to include into the FG toolchain plus low predictability of outcome given lack of mapped buildings around the globe -> might be a bit early.
* If I can be of any help in explaining how the scripts work or if the documentation (http://osm2city.readthedocs.io/en/latest/index.html) is ambiguous: I am happy to help.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 416
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby Catalanoic » Thu Sep 15, 2016 1:30 pm

Something important too is how to solve the problem of mixing OSM buildings with hand-made static ones, i think the 3d structure made by a scenery developer must remain than a osm-building, because is better modeled and textures (more realistic), how we can check that and not overlap existing static buildings and easy to add more ones? i know, a osm-building file compounds of a large group of structures and isn't possible to remove one or another to do place for a single new static object. Thanks
User avatar
Catalanoic
 
Posts: 1055
Joined: Mon Mar 05, 2012 12:33 am
Location: Barcelona (LEBL)
Callsign: Catalanoic
Version: 2017.3
OS: Lubuntu/Windows 7

Re: osm2city.py development

Postby portreekid » Thu Sep 15, 2016 1:38 pm

vanosten wrote in Thu Sep 15, 2016 4:53 am:Therefore from my point of view:
* I very welcome new STG verbs. I will help the other osm2city developers to implement the changes.
* Maturity of osm2city related scripts to include into the FG toolchain plus low predictability of outcome given lack of mapped buildings around the globe -> might be a bit early.
* If I can be of any help in explaining how the scripts work or if the documentation (http://osm2city.readthedocs.io/en/latest/index.html) is ambiguous: I am happy to help.


Yes I totally agree. The STG verbs will be a very welcome incentive to also get the piers and platforms into the LOD scheme.

Catalanoic wrote in Thu Sep 15, 2016 1:30 pm:Something important too is how to solve the problem of mixing OSM buildings with hand-made static ones, i think the 3d structure made by a scenery developer must remain than a osm-building, because is better modeled and textures (more realistic), how we can check that and not overlap existing static buildings and easy to add more ones? i know, a osm-building file compounds of a large group of structures and isn't possible to remove one or another to do place for a single new static object. Thanks


There is collision code in osm2city and I have have fixed some quirks in it. The bigger the static object and the smaller the osm building, the more likely it will fail. The only I see to way to fix that would be reading the ACs as we cannot know how much area it actually covers.
portreekid
 
Posts: 336
Joined: Tue Jan 14, 2014 3:36 pm
Location: Dresden
Version: 3.0
OS: Windows 7

Re: osm2city.py development

Postby pommesschranke » Sat Sep 17, 2016 9:21 am

as we cannot know how much area it actually covers.


And sometimes the reference point of the object is not at it's center, sometimes even outside the building footprint.

If such an overlap between osm and static building does not happen too often then we may add the OSM IDs to the Skip List to solve that.
pommesschranke
 
Posts: 1104
Joined: Sat Apr 27, 2013 7:58 pm
Location: EDLM & LJCE
Callsign: d-laser
IRC name: laserman
Version: git
OS: Linux Lubuntu 18.04

Re: osm2city.py development

Postby vanosten » Tue Sep 20, 2016 7:29 pm

I have done some renaming of the python modules:
  • setup.py -> prepare_elev.py (because setup.py has a special meaning in Python and because the module name should tell what it is
  • osm2city.py -> buidlings.py (because there is constant confusion about osm2city.py the module and osm2city the package with several modules generating scenery objects based on OSM data)
  • osm2pylon -> pylons.py (due to consistency)
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 416
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby vanosten » Fri Sep 23, 2016 8:13 pm

I have done some more clean-up:
  • A lot of non-Python files like shaders, effects etc. have been moved to osm2city-data.
  • Setting the FG_ROOT environment variable is now in most situations required (if you use FGElev* or want support in copying fgdata directory content to $FG_ROOT.
  • I have introduced a new mandatory variable: PATH_TO_OSM2CITY_DATA. On the other side making symlinks between osm2city and osm2city-data/tex(.src) is not needed anymore.
  • Copying shaders, effects etc. has now been centralized into copy_data_stuff.py (which got renamed from copy_texture_stuff.py)

Therefore: please remember to pull the newest changes from osm2city-data, when you pull the latest changes from osm2city.

Documentation at ReadTheDocs has been updated accordingly.

PS: for those with a preference for pdf documentation: if you look at the lower right corner of the documentation, there is a black box with green text. If you click on it, you can choose more formats incl. pdf.
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 416
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby stuart » Fri Sep 23, 2016 9:11 pm

Hi All,

Just in case you're not subscribed to the -devel list, I've committed the STG verbs:

https://sourceforge.net/p/flightgear/mailman/flightgear-devel/thread/CAP3ntysfcB8WJ48KXmKTZ5wcu%3DrFBWmqXbQjjxjyhRkZvzujsg%40mail.gmail.com/#msg35385866

You'll need to update simgear, flightgear and fgdata to make use of them.

Also, I will be part of the team demoing FG at FSWeekend in the Netherlands on 5/6 November. We're likely to demonstrate the Rio scenery, and I'd like to include osm2city output as part of that. So if you want a "target date" for any development, that would be a good one :)

-Stuart
G-MWLX
User avatar
stuart
Moderator
 
Posts: 1469
Joined: Wed Nov 29, 2006 9:56 am
Location: Edinburgh
Callsign: G-MWLX

Re: osm2city.py development

Postby vanosten » Sat Sep 24, 2016 5:39 am

Cool. A deadline is always good :-) Let me start by generating a scenery for Rio and make them available as is. OSM good around SBRJ is pretty - I will see how far I can get with OpenStree Map Would-Be Buildings (https://gitlab.com/vanosten/owbb).
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 416
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

Re: osm2city.py development

Postby portreekid » Sat Sep 24, 2016 6:31 am

I hate deadlines ;-) Are there any issues open?
portreekid
 
Posts: 336
Joined: Tue Jan 14, 2014 3:36 pm
Location: Dresden
Version: 3.0
OS: Windows 7

Re: osm2city.py development

Postby vanosten » Sat Sep 24, 2016 8:58 am

I have uploaded a zip-file containing the scenery and the related params.ini file for the tile containing SBRJ. Still using the current verbs to be compatible with 2016.3.1.

There are lots of things to do. Fly around and it will become quite obvious.
  • I will most probably start looking into better collision detection with static objects. After having looked at some roof errors.
  • The heuristics for piers could get better - and maybe parameters for what types of ships to include (e.g. only small boats).
  • Bigger problem is the difference between OSM and FG scenery coastlines. One possibility is to clip OSM according to FG scenery coastlines. The other possibility is to overlay the sea with a texture (configurable material) in those places, where OSM has e.g. buildings and roads in the water. It is quite easy to read FG scenery shapefiles - I do it in OWBB. Piers would have to be clipped (right now some piers and boats are on land :-( )
  • I am still a bit surprised, how much delayed scenery objects load - despite the fact that I have set LOG_DETAILED in FGFS to 15000. Maybe new verbs can help with that. If someone with more knowledge about scenegraph in FG knows what I should change in clustering etc, I am happy to make code corrections.
  • The biggest problem is the lack of OSM data for buildings. Once I am done with further clean-up of the code base and have looked at collission detection, I will return to OWBB.

We could need a helping hand with more texture. I get some errors in the generation due to lack of matching textures (and yes, I am not done with validations in texture generation).
Maintaining osm2city. Contributing with ground attack stuff to the OPRF FlightGear military-simulation community.
vanosten
 
Posts: 416
Joined: Sat Sep 25, 2010 5:38 pm
Location: Denmark - but I am Swiss
Callsign: HB-VANO
Version: latest
OS: Win 10 and Ubuntu

PreviousNext

Return to Scenery

Who is online

Users browsing this forum: No registered users and 2 guests

cron