Board index Other Hangar talk

Concurrent game engines - is this even a good idea?

Talk about (almost) anything, as long as it is no serious FlightGear talk and does not fit in the other subforums.
Forum rules
Please refrain from discussing politics.

Concurrent game engines - is this even a good idea?

Postby Parnikkapore » Wed Jun 12, 2019 12:54 pm

Hi all!

Reading the MSFS2019 thread, there seems to be a lot of people knowledgeable about threading. I'm planning to write a game engine that utilizes Go(lang) to run concurrently many for loops that would've been executed sequentially in a classical engine. (For FG for example, the physics, OSG, MP client, and autopilot can run in parallel.)

Some of the things I worked out:
  • Only one thread can write to a variable (that, or it had to be mutexed.)
  • All drawing takes place in one thread. (Concurrent drawing makes no sense. Vulkan might change this though.)
  • It might be possible for a thread to get a partial update (reads don't have to be mutexed), and that might lead to race condition glitches.

Is this a good idea, or would it likely be not worth the effort or have a large overhead?
There are free alternatives to (almost) every program you encounter. You just have to find them.
Parnikkapore
 
Posts: 929
Joined: Thu Oct 29, 2015 11:16 am
Callsign: HS-FGS
Version: next
OS: Kubuntu

Re: Concurrent game engines - is this even a good idea?

Postby bugman » Wed Jun 12, 2019 1:40 pm

The overhead is very large. What you need is practicality. You need to identify the specific part in which parallelisation could theoretically improve performance. You then need to consider the penalty of passing data between threads verses the fast accessing a global data structure. Detailed benchmarking of each step is also 100% essential for the process of parallelisation.

In any case, the benefit is more likely to come from internally parallelising one specific component rather that parallelising between different components. This is called 'granularity' of the problem. Designing successful parallelisation is a fine balance between the level granularity and the high costs of data transfer. You should also be mentally prepared that putting in a lot of effort in one chosen parallelisation scheme could end in failure, with benchmarking showing a slow down rather than speed up of the code. Parallelisation can be very hard to get right! And note that many problems are simply not suitable for parallelisation.

Regards,
Edward
bugman
Moderator
 
Posts: 1808
Joined: Thu Mar 19, 2015 10:01 am
Version: next

Re: Concurrent game engines - is this even a good idea?

Postby icecode » Wed Jun 12, 2019 1:57 pm

It's a very hot topic right now in the game industry as CPUs have more and more cores by the year. It's definitely something worth looking into, even if you fail. Knowledge in general is never a waste of time, even less so when it's such a hot topic. :)

Job scheduling systems synergize very well with game engines. Work can usually divided in small independent jobs - e.g. prepare batch of scene objects to render, update AI components etc. These jobs are sent to the scheduler asynchronously and are assigned a thread. Basically the problem then is to divide the work into as many independent tasks as possible. The more isolated tasks you find, the more performance you gain.
icecode
 
Posts: 709
Joined: Thu Aug 12, 2010 1:17 pm
Location: Spain
Version: next
OS: Fedora

Re: Concurrent game engines - is this even a good idea?

Postby Parnikkapore » Wed Jun 12, 2019 2:14 pm

@bugman
So you say writing something like this is a good idea?

Code: Select all
// Update everyone's states
wg := new WaitGroup
foreach object in objects {
    wg.add(1)
    go func{
        object.update()
        wg.done()
    }
}
wg.wait()
// Something else, probably drawing
// go func{} asynchronously runs it


Take note that, as I have a C++ background, each "object" is essentially a struct instance, shared with every thread. I don't know if this is actually safe (with proper de-racing) or is actually possible at a low level...
There are free alternatives to (almost) every program you encounter. You just have to find them.
Parnikkapore
 
Posts: 929
Joined: Thu Oct 29, 2015 11:16 am
Callsign: HS-FGS
Version: next
OS: Kubuntu

Re: Concurrent game engines - is this even a good idea?

Postby bugman » Wed Jun 12, 2019 2:55 pm

Parnikkapore wrote in Wed Jun 12, 2019 2:14 pm:So you say writing something like this is a good idea?


If you eliminate a real and observable bottleneck, yes. Otherwise, no. You can only tell if you vigorously benchmark.

Regards,
Edward
bugman
Moderator
 
Posts: 1808
Joined: Thu Mar 19, 2015 10:01 am
Version: next

Re: Concurrent game engines - is this even a good idea?

Postby curt » Wed Jun 12, 2019 3:31 pm

For the purposes of thinking about threaded architectures and performance here is a little story:

Going back a ways, I took over some UAV autopilot code that was highly threaded. It had a thread for reading the IMU (gyros, accelerometers, magnetometers), a thread to read the gps, a thread to compute the roll, pitch, yaw estimate and location estimate, a thread to compute the PID's (flight control outputs), a thread to command the actuators, etc. All these threads were woven together in a fairly complicated architecture using a signal and wait structure with mutex's on the shared structures.

After struggling for a while, I took a deep dive into the execution order of the threads. The IMU thread would send a signal that new data was ready. The Nav/EKF thread would wait on that signal, compute the new attitude estimate and send it's own signal. The flight control thread would wait for a new attitude solution, then do it's work, and send it's own signal out. The actuator thread would wait for the PID thread signal and then send the new actuator positions to the servos. I light bell went on in my head and I realized this was nothing more than a single threaded "grand loop" architecture, just implemented in a very complicated way using thread primitives.

it's just a small story and every situation is different. The point is that sometimes there is a natural information flow that is worth thinking about. Sometimes simpler is better.

If you think you must have a thread to watch a sensor for input so you can feed your system the most current information as quickly as possible, consider the information flow and dependencies in your system.
Perhaps a grand-loop architecture and non-blocking IO will work just as well (and be much simpler and much more understandable.)

If you do schedule a thread for a certain update rate, make sure you understand the tick resolution of your system scheduler (often 100hz) and that you understand how signals and wakeups work on your system. Measure and test the result to verify you are actually getting what you think you are getting. Another story I have is from a time where a student scheduled a 40hz task on a system with a 100hz tick rate. He wrote his thread as a simple loop with a 2500 us sleep(). Years later I looked at that code and said "nope". I convinced him to actually measure the loop time and suggested he may see something closer to 33 hz ... and it came back just as I predicted. In this system (like most others) a sleep() call will sleep at least the requested amount of time, and then only wake up at the next tick ... so you could possibly sleep for ~1000us longer than you intended. He also just wrote a while loop as a standalong thread, so the time to execute his loop + the sleep time + whatever else was going on in the system determined the update rate which might not have been perfectly consistent. So for quite some time he had been flying a system that wasn't performing like he expected it was, because of a simple misunderstanding of how the system worked and he never actually checked he was really getting what he wanted. Not picking on you Raghu, you are way smarter than me! Just using this as an example that things don't always work the way we thought or hoped.

In a real time system, you typically would use some hardware timer or external hardware signal to schedule a function to run at a very precise and consistent update rate. But you don't always have enough timers or the necessary timer resolution to do what you really want. So often these systems require a lot of creativity and cleverness to get the necessary performance in the end.

Another thing I've discovered. Often due to the natural information flow through a system, you can pick one sensor or one signal to drive the heart beat of your system. You'll need to think about what is the most important aspect. In an autopilot, IMU, GPS, air data, pilot inputs, etc. may all be sampled at different rates. The sensors themselves may be running on different clocks. It's impossible to sync all these things together perfectly (unless you are someone like honeywell that builds your own hardware and sensors and designs the entire system so you can strobe all the sensors at exactly the same time, but I digress...)

So my point is, that even with highly threaded systems there are information and timing dependencies that can matter if you want to produce the best and most consistent output. (And don't fall into the trap of thinking you can just boost the main loop update rate and timing problems and dependencies all go away.)

If you are designing a threaded architecture, at least be conscious of your goals. Do you want something like a computer science operating system scheduler designed to be "fair" so there is no thread starvation. Or do you want something that has very deterministic timing. I will suggest it's difficult to guarantee both simultaneously. For FlightGear which I would call real time simulation and visualization, timing is not unimportant. Consistent frame rates make a difference and any change is detectable by our human eyes.
Aerospace Engineering and Mechanics
University of Minnesota
curt
Administrator
 
Posts: 1168
Joined: Thu Jan 01, 1970 1:00 am
Location: Minneapolis, MN

Re: Concurrent game engines - is this even a good idea?

Postby Parnikkapore » Thu Jun 13, 2019 12:16 am

light bell went on in my head and I realized this was nothing more than a single threaded "grand loop" architecture, just implemented in a very complicated way using thread primitives.

Yes, this is why I feel a bit leery about threading out a group of short functions like the one earlier in the thread. The gain/(pain+overhead) ratio would be pretty low. Threads need to be as independent as possible.

However, jitter is a problem, and I forgot about the player noticing. I thought this wouldn't be a problem as the renderer thread would be locked onto vsync regardless, but physics engine inconsistencies (and races) can also make things buggy...
There are free alternatives to (almost) every program you encounter. You just have to find them.
Parnikkapore
 
Posts: 929
Joined: Thu Oct 29, 2015 11:16 am
Callsign: HS-FGS
Version: next
OS: Kubuntu


Return to Hangar talk

Who is online

Users browsing this forum: No registered users and 7 guests