by curt » Wed Jun 12, 2019 3:31 pm
For the purposes of thinking about threaded architectures and performance here is a little story:
Going back a ways, I took over some UAV autopilot code that was highly threaded. It had a thread for reading the IMU (gyros, accelerometers, magnetometers), a thread to read the gps, a thread to compute the roll, pitch, yaw estimate and location estimate, a thread to compute the PID's (flight control outputs), a thread to command the actuators, etc. All these threads were woven together in a fairly complicated architecture using a signal and wait structure with mutex's on the shared structures.
After struggling for a while, I took a deep dive into the execution order of the threads. The IMU thread would send a signal that new data was ready. The Nav/EKF thread would wait on that signal, compute the new attitude estimate and send it's own signal. The flight control thread would wait for a new attitude solution, then do it's work, and send it's own signal out. The actuator thread would wait for the PID thread signal and then send the new actuator positions to the servos. I light bell went on in my head and I realized this was nothing more than a single threaded "grand loop" architecture, just implemented in a very complicated way using thread primitives.
it's just a small story and every situation is different. The point is that sometimes there is a natural information flow that is worth thinking about. Sometimes simpler is better.
If you think you must have a thread to watch a sensor for input so you can feed your system the most current information as quickly as possible, consider the information flow and dependencies in your system.
Perhaps a grand-loop architecture and non-blocking IO will work just as well (and be much simpler and much more understandable.)
If you do schedule a thread for a certain update rate, make sure you understand the tick resolution of your system scheduler (often 100hz) and that you understand how signals and wakeups work on your system. Measure and test the result to verify you are actually getting what you think you are getting. Another story I have is from a time where a student scheduled a 40hz task on a system with a 100hz tick rate. He wrote his thread as a simple loop with a 2500 us sleep(). Years later I looked at that code and said "nope". I convinced him to actually measure the loop time and suggested he may see something closer to 33 hz ... and it came back just as I predicted. In this system (like most others) a sleep() call will sleep at least the requested amount of time, and then only wake up at the next tick ... so you could possibly sleep for ~1000us longer than you intended. He also just wrote a while loop as a standalong thread, so the time to execute his loop + the sleep time + whatever else was going on in the system determined the update rate which might not have been perfectly consistent. So for quite some time he had been flying a system that wasn't performing like he expected it was, because of a simple misunderstanding of how the system worked and he never actually checked he was really getting what he wanted. Not picking on you Raghu, you are way smarter than me! Just using this as an example that things don't always work the way we thought or hoped.
In a real time system, you typically would use some hardware timer or external hardware signal to schedule a function to run at a very precise and consistent update rate. But you don't always have enough timers or the necessary timer resolution to do what you really want. So often these systems require a lot of creativity and cleverness to get the necessary performance in the end.
Another thing I've discovered. Often due to the natural information flow through a system, you can pick one sensor or one signal to drive the heart beat of your system. You'll need to think about what is the most important aspect. In an autopilot, IMU, GPS, air data, pilot inputs, etc. may all be sampled at different rates. The sensors themselves may be running on different clocks. It's impossible to sync all these things together perfectly (unless you are someone like honeywell that builds your own hardware and sensors and designs the entire system so you can strobe all the sensors at exactly the same time, but I digress...)
So my point is, that even with highly threaded systems there are information and timing dependencies that can matter if you want to produce the best and most consistent output. (And don't fall into the trap of thinking you can just boost the main loop update rate and timing problems and dependencies all go away.)
If you are designing a threaded architecture, at least be conscious of your goals. Do you want something like a computer science operating system scheduler designed to be "fair" so there is no thread starvation. Or do you want something that has very deterministic timing. I will suggest it's difficult to guarantee both simultaneously. For FlightGear which I would call real time simulation and visualization, timing is not unimportant. Consistent frame rates make a difference and any change is detectable by our human eyes.
Aerospace Engineering and Mechanics
University of Minnesota