Board index FlightGear Development New features

Screen warping / edge blending for projectors

Discussion and requests for new features. Please note that FlightGear developers are volunteers and may or may not be able to consider these requests.

Screen warping / edge blending for projectors

Postby Vik » Wed Nov 23, 2016 4:06 am

Hi all,

I have a proof of concept implementation of screen warping / edge blending for Flightgear.

I wrote this Python utility to produce the necessary calibration data: https://github.com/viktorradnai/screenwarp

The following patch adds support for reading this calibration data in Flightgear. This has been tested with 2016.3.1 and 2016.4.1 on Linux only, although it should work on the other platforms as well.

Code: Select all
--- flightgear-2016.3.1-orig/src/Viewer/CameraGroup.cxx   2016-09-08 07:52:29.000000000 +0100
+++ flightgear-2016.3.1/src/Viewer/CameraGroup.cxx   2016-11-23 02:15:15.632152390 +0000
@@ -602,7 +602,6 @@
 }
 
 // Mostly copied from osg's osgViewer/View.cpp
-
 static osg::Geometry* createPanoramicSphericalDisplayDistortionMesh(
     const Vec3& origin, const Vec3& widthVector, const Vec3& heightVector,
     double sphere_radius, double collar_radius,
@@ -678,11 +677,16 @@
 
 
             osg::Vec3 v = screenCenter + osg::Vec3(sin(alpha)*gamma*2.0/osg::PI, -cos(alpha)*gamma*2.0/osg::PI, 0.0f)*screenRadius;
+            osg::Vec3 v2;
 
             if (flip)
-                vertices->push_back(osg::Vec3(v.x(), top.y()-(v.y()-origin.y()),v.z()));
+                v2 = osg::Vec3(v.x(), top.y()-(v.y()-origin.y()),v.z());
             else
-                vertices->push_back(v);
+                v2 = v;
+
+            OSG_WARN<<"coords x: "<< v2.x() <<" y: "<< v2.y() <<" z: "<< v2.z()<<std::endl;;
+
+            vertices->push_back(v2);
 
             texcoords0->push_back( texcoord );
 
@@ -746,6 +750,93 @@
     return geometry;
 }
 
+
+static osg::Geometry* createCustomDistortionMesh(const osg::Vec3& widthVector, const osg::Vec3& heightVector, string fileName)
+{
+    bool flip = true;
+    bool texcoord_flip = true;
+
+    // create the quad to visualize.
+    osg::Geometry* geometry = new osg::Geometry();
+
+    geometry->setSupportsDisplayList(false);
+
+    osg::Vec3 xAxis(widthVector);
+    float width = widthVector.length();
+    xAxis /= width;
+
+    osg::Vec3 yAxis(heightVector);
+    float height = heightVector.length();
+    yAxis /= height;
+
+    osg::Vec3Array* vertices = new osg::Vec3Array;
+    osg::Vec2Array* texcoords0 = new osg::Vec2Array;
+    osg::Vec2Array* texcoords1 = new osg::Vec2Array;
+    osg::Vec4Array* colors = new osg::Vec4Array;
+
+    geometry->getOrCreateStateSet()->setMode(GL_CULL_FACE, osg::StateAttribute::OFF | osg::StateAttribute::PROTECTED);
+
+    int rows, cols;
+    // xb (Xbase) and yb (Ybase) are the original undistorted grid coordinates
+    // xc (Xcorrected) and yc (Ycorrected) are the corrected (distorted) grid coordinates
+    // i is the intensity value at the given point
+    double xb, yb, xc, yc, i;
+
+    std::ifstream infile(fileName);
+    if(!infile) {
+        OSG_WARN<<"Could not open distortion map file '"<<"'"<<std::endl;
+        return geometry;
+    }
+    infile >> rows >> cols;
+    while (infile >> xb >> yb >> xc >> yc >> i) {
+
+        if(flip) yc = 1 - yc;
+        if(texcoord_flip) yb = 1 - yb;
+        osg::Vec3 v(xc*width, yc*height, 0.0);
+        osg::Vec2 texcoord(xb, yb);
+
+        vertices->push_back(v);
+        texcoords0->push_back( texcoord );
+        texcoords1->push_back( texcoord );
+        colors->push_back(osg::Vec4(i, i, i, i));
+    }
+
+    // pass the created vertex array to the points geometry object.
+    geometry->setVertexArray(vertices);
+
+    geometry->setColorArray(colors);
+    geometry->setColorBinding(osg::Geometry::BIND_PER_VERTEX);
+
+    geometry->setTexCoordArray(0, texcoords0);
+    geometry->setTexCoordArray(1, texcoords1);
+
+    osg::DrawElementsUShort* elements = new osg::DrawElementsUShort(osg::PrimitiveSet::TRIANGLES);
+    geometry->addPrimitiveSet(elements);
+
+
+    for(int i=0;i<rows-1;++i)
+    {
+        for(int j=0;j<cols-1;++j)
+        {
+            int i1 = j+(i+1)*cols;
+            int i2 = j+(i)*cols;
+            int i3 = j+1+(i)*cols;
+            int i4 = j+1+(i+1)*cols;
+
+            elements->push_back(i1);
+            elements->push_back(i2);
+            elements->push_back(i3);
+
+            elements->push_back(i1);
+            elements->push_back(i3);
+            elements->push_back(i4);
+        }
+    }
+
+    return geometry;
+}
+
+
 void CameraGroup::buildDistortionCamera(const SGPropertyNode* psNode,
                                         Camera* camera)
 {
@@ -760,6 +851,7 @@
         // error
         return;
     }
+
     Viewport* viewport = camera->getViewport();
     float width = viewport->width();
     float height = viewport->height();
@@ -767,9 +859,19 @@
     double radius = psNode->getDoubleValue("radius", 1.0);
     double collar = psNode->getDoubleValue("collar", 0.45);
     Geode* geode = new Geode();
+
+
+    const SGPropertyNode* fileNode = psNode->getNode("file");
+    if (fileNode) {
+        string fileName = fileNode->getStringValue();
+        OSG_INFO<<"Loading distortion map from file: "<<fileName<<std::endl;
+        geode->addDrawable(createCustomDistortionMesh(
+                           Vec3(width,0.0f,0.0f), Vec3(0.0f,height,0.0f), fileName));
+    } else {
     geode->addDrawable(createPanoramicSphericalDisplayDistortionMesh(
                            Vec3(0.0f,0.0f,0.0f), Vec3(width,0.0f,0.0f),
                            Vec3(0.0f,height,0.0f), radius, collar));
+    }
 
     // new we need to add the texture to the mesh, we do so by creating a
     // StateSet to contain the Texture StateAttribute.
@@ -795,6 +897,7 @@
     camera->setName("DistortionCorrectionCamera");
 }
 
+
 CameraInfo* CameraGroup::buildCamera(SGPropertyNode* cameraNode)
 {
     WindowBuilder *wBuild = WindowBuilder::getWindowBuilder();


Then the warped display can be configured using the following XML file:
Code: Select all
<PropertyList>
  <sim>
    <rendering>
      <camera-group>
        <camera>
          <window>
            <name type="string">main</name>
            <host-name type="string"></host-name>
            <display>0</display>
            <screen>0</screen>
            <!-- <fullscreen type = "bool">true</fullscreen>-->
            <width>1920</width>
            <height>1080</height>
          </window>
          <view>
            <heading-deg type="double">0.0</heading-deg>
            <roll-deg type="double">0.0</roll-deg>
            <pitch-deg type="double">0</pitch-deg>
          </view>
          <physical-dimensions>
            <width>1920</width>
            <height>1080</height>
          </physical-dimensions>
          <master-perspective>
            <eye-distance>450</eye-distance>
            <x-offset>0</x-offset>
            <y-offset>130</y-offset>
          </master-perspective>
          <texture>
            <name>mainview</name>
            <width>1920</width>
            <height>1080</height>
          </texture>
        </camera>
        <camera>
          <window><name>main</name></window>
          <ortho>
            <top>1080</top>
            <bottom>0</bottom>
            <left>0</left>
            <right>1920</right>
            <near>-1.0</near>
            <far>1.0</far>
          </ortho>
          <panoramic-spherical>
            <texture>mainview</texture>
            <file>calibration_points.txt</file>
          </panoramic-spherical>
        </camera>
        <gui>
          <window>
            <name type="string">main</name>
          </window>
        </gui>
      </camera-group>
    </rendering>
  </sim>
</PropertyList>


Just save the above as warp.xml and start Flightgear as fgfs --config=warp.xml

Here are a couple of images showing it in action:

Image
Image

Image
Image

Image
Image
Vik
 
Posts: 12
Joined: Wed Jul 13, 2011 10:35 pm

Re: Screen warping / edge blending for projectors

Postby Johan G » Wed Nov 23, 2016 5:05 pm

That is a very useful feature. :D

If you have not done that yet, please also post on the developer mailing list about this. :wink:

Edit: Noticed that you already have posted about it on the dev list. :D
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator
 
Posts: 5276
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

Re: Screen warping / edge blending for projectors

Postby Hooray » Tue Nov 29, 2016 7:36 pm

My suggestion would be to document the whole approach using a dedicated wiki article and adding a link to it to the upcoming newsletter, including links to sources/patches to preserve the whole thing
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11100
Joined: Tue Mar 25, 2008 8:40 am

Re: Screen warping / edge blending for projectors

Postby ludomotico » Tue Nov 29, 2016 7:53 pm

Johan G wrote in Wed Nov 23, 2016 5:05 pm:That is a very useful feature.


Indeed! The same approach can be done to render the deformation the VR headsets need.
User avatar
ludomotico
 
Posts: 992
Joined: Tue Apr 24, 2012 1:01 pm
Version: git
OS: Debian GNU/Linux

Re: Screen warping / edge blending for projectors

Postby rustemy » Thu Dec 22, 2016 12:53 pm

Did this ever get submitted and merged? I'd love to be able to test this feature out.
rustemy
 
Posts: 23
Joined: Fri Jun 21, 2013 1:15 pm
Version: PPA-Edge
OS: Xbuntu 17.04

Re: Screen warping / edge blending for projectors

Postby Vik » Mon Jan 02, 2017 2:39 am

rustemy wrote in Thu Dec 22, 2016 12:53 pm:Did this ever get submitted and merged? I'd love to be able to test this feature out.


Not yet.

I've been asked to modify the patch so it uses a model file that OSG can load, instead of using my own format. But I've been too busy working on my projection calibration utility (as well as other things such as the revamped ASK21) to have the time for this.
Vik
 
Posts: 12
Joined: Wed Jul 13, 2011 10:35 pm

Re: Screen warping / edge blending for projectors

Postby Johan G » Sun Jan 08, 2017 6:51 pm

Thanks for the update. :)
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator
 
Posts: 5276
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

Re: Screen warping / edge blending for projectors

Postby Vik » Mon Jul 03, 2017 11:08 am

Hi all,

I'm close to merging this into Flightgear. First, here's a preview of this working with a cylindrical screen, to show some potential :)

https://www.youtube.com/watch?v=h4afj8YJTMU

Now, I would need some help from someone who understands OpenSceneGraph and 3D file formats. I would really like to find a 3D file format that is supported well enough by OpenSceneGraph and is capable of the following:

    - Texture coordinates for each vertex rather than each face
    - Colour and intensity for each vertex
The first requirement rules out the .ac format most commonly used by Flightgear. I have tried .ply, which seems ideal but OSG doesn't seem to support texture coordinates. :(

I have also tried .obj which is a bit clunky (materials need to be stored in a separate file, and you need a different material for each colour / intensity value) but I couldn't get texture mapping to work on it.

If someone could jump in and help me with this, that would be very much appreciated, in the absence of this I will have to stick to my own file format and read it rather than using OSG. This will at least allow people to start using this but would require file format conversions from some popular 3D formats, which is no big deal. I'd just like to avoid letting another 3D format loose on the world, there are enough of them already :)
Vik
 
Posts: 12
Joined: Wed Jul 13, 2011 10:35 pm

Re: Screen warping / edge blending for projectors

Postby Johan G » Sun Jul 16, 2017 5:16 pm

Thanks for the update. :)

Vik wrote in Mon Jul 03, 2017 11:08 am:[...] here's a preview of this working with a cylindrical screen, to show some potential :)

That looks amazing! :D 8)
Low-level flying — It's all fun and games till someone looses an engine. (Paraphrased from a YouTube video)
Improving the Dassault Mirage F1 (Wiki, Forum, GitLab. Work in slow progress)
Johan G
Moderator
 
Posts: 5276
Joined: Fri Aug 06, 2010 5:33 pm
Location: Sweden
Callsign: SE-JG
IRC name: Johan_G
Version: 3.0.0
OS: Windows 7, 32 bit

Re: Screen warping / edge blending for projectors

Postby Hooray » Sun Jul 16, 2017 5:20 pm

looking at the video, most folks on the devel list should be all ears once you share the video with them and pose your questions there - so that's what I'd suggest to do.
And please DO add this to the newsletter !
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 11100
Joined: Tue Mar 25, 2008 8:40 am


Return to New features

Who is online

Users browsing this forum: No registered users and 2 guests