I had so much fun lately playing with extreme scenario and failures that I wanted to do a proper sum up of it.
There are around 50 pictures,it might be a bit long to load the thread if you have a low bandwidth like me, but I think it is worth it
I condensed and highlighted some part of the ascent pocket checklist in order to have a good guide in order to follow the failure checklists
It is available here, and it is better to follow the thread
https://drive.google.com/open?id=1uZPOso6LROCYFCFIn79bMmvJxpdQ1-ab
A bit more infos about checks here:
http://wiki.flightgear.org/Flying_the_Shuttle_-_Space_Shuttle_Checklists
And normal Mission Phases there:
http://wiki.flightgear.org/Space_Shuttle#Mission_phases
Let's start our terrific journey.
Another forecasted mission, months if not years of training for the crew.
Veteran Commander, first flight for the Pilot and 2 Mission Specialists.
No payload for the ascent, aim was to take back a so called " observation satellite" for maintenance.
Launch at Sunrise, for a polar Orbit.
Cavok over the pad.
Last 6 mn before Launch, APU are fired up, and last checks accomplished.
Go for Launch, and final coutdown before the 3 G's danger zone.
That was an almost perfect first stage, Atlantis already way above our heads.
Entering the second stage on schedule, heads still down
After 8mn and 30 seconds of roller coaster, MECO time and ET separation
No need for OMS 1, next meeting into the check for OMS 2
End of propellant Dump, annonciated among other things by the body flap light extinction
Approaching Higher latitudes, and what a show of lights
Pre OMS 2 actions are performed ( APU and MPS shutdown, Umbilical doors closure, ..)
Burn is loaded into the computer, Ignition in 17 mn to circularize the orbit around 130 Nm
Splendid Scenic while waiting...
For sure, not waiting for that shrill Alarm, and all those lights shining on the Caution and Warning center panel.
Amongst them, MAIN BUS UNDERVOLT is quite bad
Time to take the first checklist in the doc I linked, page 2, MN BUS UNDERVOLS/FC VOLTS and to have a look like suggested into the BFS , system management , sum up 1 ( same troubleshooting page for all failures whith only the BFS as system management GPC)
We can see three things that are not normal.
It concerns fuel cell 2 and Main bus B.
Low voltage for FC 2 and MnB ( 6.1 V) and crazy amps ( superior at 1500 A)
Probably a short, but where ?
Let's enter into the check.
I highlighted in Green the first two conditionnal choices, either we are in one case or the other, and then we follow what is written below it.
For us, it is the first case ( Mn Bus and Fc volts below 26.6 V, and amps superior at 480 )
We don't have bus tied to each other, so no need for step 2 and 3
Step 4,5 and 6 would be to unload the Payload bus from fuel cell 2 if we had a payload ( Space Hab, Satelitte ..) , but it is not our case
Step 7 is for us.
We will remove the FC 2 power from the essential bus 2 and Main bus B to see where is the short
Result here :
Fuel cell voltage recovered ( 32.5), no volts on Mn B and essential bus powered ( by main A and C buses, triple redundancy)
Also amps is now zero, as fuel cell 2 is not connected to any main buses.
Verdict: Main Bus B short
Impossible to tie it to another fuel cell, bus is lost and as directed by the step 10, we will have to perform Main B bus loss action
That means we will have to reorganize all the systems that were controlled by Main bus B and AC2 coming from that bus. ( Everything in the cockpit where it is written Mn B or AC 2 for power source)
In reality, it was possible to put cable into the cockpit to tie AC lost bus to a good AC bus, quite crazy maintenance action
Let's have a look on that nice sunset before diving into the next checklist
Page 3 of our checklist now.
MNB DA2 (Entire BUs)(Includes AC2)
That means we lost all the Main Bus B ( we could have lost just a part of it, a lot of subdivisions behind the term Main Bus) and the AC bus associated ( AC2)
Let's go for some cockpit reconfiguration.
Step 1 for a back up valve that will allow Remaining Gaseous Hydrogen to be vented outside ( we need to be sure that this valve will be opened in case of another failure that would prevent it to be opened)
Step 2 to change the alimentation source of the boiler Heater of APU 1
Step 4 to turn off AC bus sensor. That sensor allows AC bus to be shed automatically in case of overload ( no need for it now that we don't have AC 2 anymore)
And Step 5 , telling us to shutdown FC2 as we don't have any use of it.
Cue Card means that we can directly look to some Papers in the cockpit near the electric panel to read the action ( no need to dive into the book, that was used for time efficiency in case of important failure before reaching orbit)
Cue card there:
Let's go then.
No step 2 ( bus tie) as bus B is short, it would lead to bad things ( shortage of other main buses tied to the B )
Step 3 and 4 to stop the FC and close the reactant alimentation ( O2 and H2)
Step 5 direct us to LOSS of 1 FC PWRDWN
We have 2 FC remaining, we will have to shutdown some systems and reorganized a bit to avoid unnecessary power consumption.
Let's meet on Page 4 of the checks.
It is for loss of either 1 FES ( Flash evaporator) or 1 fuel cell.
Step 2 and 3 tell us to dim the cockpit and use just one screen for the computer and 2 for instruments ( PFD and engine settings for example)
Step 5 and 6 are for navaids shutting down.
Step 7 through 12 are for ECLSS ( Environment, conditionning and life support system) reorganisation.
We already mainly did it before.
We make sure we have on IMU fan working and that all freon cooling sytems and evaporator heaters are not powered by AC 2
End of that check, rest of it is in case of Abort once around, or if we were already in orbit, plus some changes for entry etc, not for us, at least not now
View of our reorganized cockpit
After some Nasa analysis and crew concertation, decision was made to continue the mission at least post OMS 2
Loss of 1 Main Bus is not critical, as Shuttle is supposed to work with only one FC with heavy load shedding, and as we have no payload to power, we can continue the mission futher
However, only the critical objectives will be accomplished, and mission time duration will be decrease at its minimum.
It happened a couple of times in real ( STS 2, 83) ( Fuel cell problem)
You can have a look there if you are interested
https://spaceflightblunders.wordpress.com/2017/05/24/in-space-no-one-should-see-you-belch/
Let's continue.
Next step, OMS 2 burn and nominal checklist for Post insertion.
Awkward monitoring
Safely on Orbit now, 40 mn after Launch and a Bus failure.
130 Nmish above the Earth
Another unpleasant alarm is ringing.
What now....
New alarm for FUELL CELL PUMP
That is linked to a fuel cell pump that is used to cool it, with liquid circulating around the fuel cell, cooling it and exchanging its heat with freon loop.
Not good then.
We can see the little ( delta P) below FC 1, with Stack temperature increasing.
Checklist page 5, FC COOL P
Pressure to the cooling sytem is lost, no more cooling liquid movement.
Step1: We can Tie the Main A Bus to the good one ( Main C ) to not lost systems powered by main A when we will shutdown the second fuel cell.
Then another conditionnal in green , either cooling Pump Pressure high or low.
For us , it is low ( 0) as we lost the pump.
Step 3 then, another FC Shutdown.
Checklist page 6, 2nd FC SHUTDN
First a step that is not written there, I untied Mn A and Mn C
As we will shut down a cell, load will be heavy on just ONE cell to supply 2 main buses ( voltage below 24 V hard lower boundary)
It works in the sim with a lot of shedding, but it is very tight.
In reality, it was Ok ish, even with 3 mains buses on a single FC because they could open circuit breaker, and shed many more secondary systems not modelled here ( useless stuff for a sim, but that take power, like rcs breaker, water tank heaters etc)
To sum up, it is better to leave on the remaining FC just one main bus and then to try to tie them with other good buses afterwards ( if we don't lose all the screens, it is ok )
Step 5 and 6, like we did before, we disconnect the cell from its essential and main bus, then we shut if off and secure the reactant alimentation.
Step 7, go to LOSS OF 2nd FC (PWRDN)
We will have to shed and check even more systems now that we just have one fuel cell remaining,one Main Bus ( max two with bus tie and chance) and one AC Bus .
That is becoming a quite critical situation.
A view on the very degraded cockpit
Electrical sum up
FC 3, Main bus C and AC 3 are working ok
Essential buses one, two and three also, thanks to the triple redundancy
You can note the total amps ( 365) for 10 ish kw, max sustained of 12
We will try to decrease that with the next checklist
Page 7 for the next checklist, LOSS OF 2nd FC
It will be very similar to what we see before.
To check that some systems are down, to check that other ones are well powered by Main C or AC3, and to see which systems will be inoperative ( very few thanks to redundancy)
And mainly to shed some energy ogre systems ( like fans, hydraulic pumps,...)
Step 2 for low use of screens ( we don't have the choice anyway)
Step 3 and 4 to check that navaids are off.
MS for mission specialist task (page 11 of the check , mainly circuit breakers to open, and APU/OMS/RCS/Hydraulics Heaters to deactivate)
Step 5 for dimming the light ( already done)
Step 7 to stop the cabin fan ( gonna get hot inside the Shuttle)
Step 9: Important one.
We will shut down some GPC computer.
We need to reorganize the computer buses on the only GPC that will remain, the one
More info and links on GPC handling stuffs here:
http://wiki.flightgear.org/Flying_the_Shuttle_-_Launch_And_Post_Insertion_Advanced#Post_Insertion:_Advanced_GPC_reconfiguration
Done, every buses on GPC 1
Step 12: Shutting down GPC 2,3 and 4. Powering off 2 and 3 ( 4 will be use for a minimum of redundancy during entry, 5 for the backup computer)
Step 21 ( steps before are for on Orbit reconfiguration, not for us)
MPS engine power already off, it was part of post meco actions ( and huge , huge power demand on fuel cells)
Step 23 through 32 are for ECLSS reconfiguration and check ( like we did during the loss of one FC)
Basically, one fan will still power the IMU, one water pump, one flash evaporator and one freon pump are working ( very important for cooling)
Every avionic fans work, except AV bay 2 ( either AC 1 or 2, not 3) We will monitor the temperature, near the limit during all the rest of the flight.
Step 35 is for checking that cryo heaters are still working for H2 and O2 storage.
Step 39 for deactivation of the fes feedline heaters ( heaters that avoid the water line going from the cells to the fes to freeze) . Huge demand on power also, better to have it off.
Step 41 to 44: Other checks for APU Boiler heaters and recirculation pumps, already moved to off after APU shutdown earlier on.
And no bus tie for us as one of them had a short.
Page 10. Almost there. Quite long and exhaustive procedure. But the guys were three to do it .
We will switch our computer to entry software ( OPS 3)
That is the purpose of step 59 to 63
Every bus on GPC 1 for entry, and we can then type OPS 301 PRO on keyboard to transition towards OPS 3
Finally step 65
Go to LOSS OF 2 FC LAUNCH DAY DEORBIT PREP
(ORBIT 2 OR ORBIT 3)
Quite frightening
Thats means we will have to deorbit as soon as possible, either to an Emergency Landing Site or normal one during second or third orbit.
While we are crossing the South Pole, MCC is checking the available landing sites, weather, deorbit opportunities , etc
Alright, decision is taken.
It will be a deorbit in 30 mn for Diego Garcia, in Indian Ocean.
Cross range will be perfect, deorbit burn forecasted South of Mexico.
Flight dynamics ofccier and his team came up with a deorbit burn solution.
Peg 4 guidance, Almost no radial velocity, 3200 Nm for rentry range, and Time of Ignition at 01h35
Not too bad
More infos on Peg 4 here:
http://www.science-and-fiction.org/science/leo_05.html
Time to tackle our last checklist, the LOSS OF 2 FC LAUNCH DAY DEORBIT PREP
That is part of a document called Contigency Deorbit, where an off nominal deorbit was needed due to severe failures after last opportunity for an Abort Once Around ( after OMS 2 basically)
It covers fuel cells, freon /water loops, cabin pressu .. failures
The menu
Initial Conditions are met.
We performed the loss of 2 FC check.
And we didn't started the Post Insertion activities ( Payload bay door opening mainly)
The checklist is long, shared between Commander, Pilot and one Mission Specialist. It is supposed to be entered 1h30 before TIG. We have 30 mn, challenge accepted.
It is very similar to what we have done until now.
Reassigned some systems for entry, shed other ones. 3 screens usage , minimum lighting, ...
To sum up, a thin layer between the vital equipment needed for entry and the max load one FC could handled ( a kind of Apollo 13 scenario electrical wise)
I am not going to share that check as it is not publicly available ( L2 membership on nasaspaceflight.com forum), but make a sum up of it.
It is following a nominal deorbit preparation check with some power differences.
First, we activate the boiler heaters and controller for APU start up later on
Mission Specialist instals all the checklists and cue cards for deorbit burn and entry,something close to that:
He will also check that all checklists related to electrical problems have been well done and will eventually connect some lost AC buses ( AC 1 or 2) to the remaining good one ( AC3) with some AC power transfer cable ( helped by the pilot)
Space Mac Gyver
A lot of CB reactivation ( Mainly for RCS valves and drivers, flight control systems during entry, accelerometers and rate gyros assembly that allow a far better refresh rate than IMU for dynamic phases like entry, etc)
Then action are very similar to a normal entry preparation.
Landing site preparation on Spec 50 horizontal display.
Burn Datas entered into our only GPC left
Purge of the fuel cell to have optimal power with no contaminants in it for entry.
That take us normally 45 mn before the burn ( 15 mn for us), and there we go through the entry check
Similar to that tutorial:
http://wiki.flightgear.org/Flying_the_Shuttle_-_Deorbit_Burn_and_Final_Entry_Preparation_Advanced
Main difference is the reactivation of GPC 4, to have two redundant GPC for entry ( instead of 4 normally)
GPC 1 and 4 for Guidance, Control and Navigation
GPC 5 for Back up flight system
After reconfig.
2 strings on GPC 1, 2 on GPC 4 and 3 running GPC
Time to Deorbit.
01h35mn into the mission
Monitoring is really tough on 3 screens
For that task, Mission Specialist seat is really the best, like an instructor in a simulator
Good burn, a steep entry angle ( low perigee and short entry interface)
Peacefully derivating towards its Entry Interface in 20 mn
Last look from Commander to the Aurora Borealis
Apu put back online and full hydraulic pressure powers.
Just two are available for Entry, as they have a dual redundancy Main bus wise.
Entry Interface as forecasted by the LEO tool
Burning Phoenix out there, overhead Saudi Arabia
We bit low on path due to the steeper than usual Rentry angle, but we will catch up
Out of the danger zone, no more plasma
Diego Garcia insight, finally
Long final, good state vector thanks to GPS update ( no MLS there)
Final Flare and max concentration
Made it, not too bad!
What a journey, awesome adventure, really.
I spent a lot of time reading through real life procedures to see how such failures would have been handled , that is fascinating to see the number of things they thought about.
Dozen if not hundred of pages for just a specific scenario.
Crazy STS program
And that is truly amazing to be able to repoduce it with a nice degree of fidelity in the sim.
Of course , there are some discrepancies with real stuff, as it is so complex in real for system intrication and handling in such a severe case.
Next journey, one with absolutely no failure in it