Since the first article in January 2007 a few questions have arrived to our mailbox, some of them to send data (not a lot) some of them requesting where to see this data (most of them) and how to calculate this data.

I think we all agree that formulas and concepts are not easy to get specially when it seems there are quite a lot of details that come to mind, exceptions to rules, and special cases.

The following article is going to try to illustrate how to calculate and provide data following the guidelines already published in the previous article.

First of all I would like to express my appreciation and gratitude to those that so far have provided data and are quite enthusiastic with the idea, yes I know you are all waiting to see your data and results and see others as well. Also I would like to remind you all that we are not trying to compete or replace the official version on how to make this calculations or present them, this option is well presented into the ARINC documentation and is called ARINC-433 report, there is a lot of good info and as far as we do know they are currently reviewing the document to provide even more details on how they do understand this data, how should be presented and do not forget that once those recommendations are ready and well established most likely will become part of the authorities recommendations within the JAA or FAA regulations for simulators.

So lets explain our concept once more, so far we have call it the SRP for Simulator Reliability Program, basically what we are trying to achieve is to share and present data from different operators so they can all look at it. Seems obvious that you cannot measure anything within a good reference, and the SRP intention is to provide the measurement of several operators so an average within the industry will be available to all and for free.

Our first target is to present at least 3 operators and at least 20 devices, for each of them we want to provide the MTBF and Usage for the complete 2006 period and here it is where most of the questions do come … how to calculate this data. The end date for first target is end of March to provide all of us with 3 months to provide data.

The Busy Simulator Example (The Ideal Situation)

The profile of a busy simulator is quite simple, works most of the day and there is just time enough to maintain it, it is a well worked out machine with not a lot of updates and with a limited amount of free time, quick turnaround between crews and not a lot of failures…we hope. It is so busy that any time that there is a problem it does get reflected as downtime since there is no time to hide any possible downtime during turnaround, maintenance, etc. No I am not suggesting that we do hide time, but it does happen that problems get solved during crew changes or crew ends as long as there is maintenance time after or not a following crew. This machine is the ideal one to measure reliability and usage is high.

So let’s go for the numbers, the simulator works on a pattern based on a 365 days a year. Every day the simulator gets ready by maintenance and it is expected it will produce 20 hours of training a day, leaving 4 hours for maintenance a day.

This means in numbers 365 * 20 = 7,300 hours available to be sold, or in other words we can sell and use 7,300 hours this year.

There are no updates to be done this year, no specific maintenance to be done outside the 4 hours a day already scheduled, please note that the 4 hours of maintenance may include 10 or 15 or 5 minutes between crews for change over, there are no configuration changes on this machine and also there is no need of any extra slots for any other means.

In our example our sales department sold 6,000 hours of training. We didn't have any cancellations, we didn't even have any no shows - it is a perfect machine after all.

We can calculate the Usage right now...

Usage = (6,000/7300) * 100 = 82.19%

Now moving over to the maintenance side of things, we did have 150 calls from the crew, of which, 15 caused downtime and a loss of 10 hours of training time.

10 hours lost in 15 incidents and the machine used for a total of 6,000 hours.

It is a simple approach to say that the actual MTBF (mean time between failures causing downtime) is:

MTBF = (6,000-10)/15 = 5,990/15 = 399 hours

Because the downtime is normally low this can be approximated as: MTBF = 6,000/15 = 400 hours.

Reliability can now be calculated as:

Reliability = ((6,000-10)/6,000) * 100 = (5,990/6,000) * 100 = 99.83%

we can provide MTTR (mean time to repair or most precisely mean time to return to operation or mean time of crew delays in training):

MTTR = 10/15 = 600/15 = 40 minutes

I am sure that by now you can come up with a lot of other definitions and useful detail but please stick with us so we can progress to other data. So far we have:

The Busy Simulator Example (The Reality)

Let's now look at a more realistic example. It's here where things can get a bit confusing. This time our simulator is still very busy, but we also have a lot of different costumers; cancellations; no shows; maintenance are going to have a certification session; a simulator testing profile session; and also 10 snag clearance sessions. It is a configurable machine so we also need configuration change time, and of course the regular maintenance time.

We'll keep to the concept of 365 days of 20 hours in every day (24 hours) to keep with the 7300 hours of saleable time.

Our sales department has informed us that they've managed to sell another 6,000 hours this year.

In order to achieve this we needed 20 hours of configuration changes, and scheduling told us we had 40 hours cancelled, and 12 hours recorded as no shows by crews. Maintenance consumed 44 hours during 11 snag clearance sessions (total of 44 hours). Certification and simulator test profiles are considered as training time.

• Possible to sell after discounted maintenance time: 7300 hours
• Sold and booked for training: 6,000 hours.
• Extra required to performed configuration changes: 20 hours.
• Cancelled hours: 40 hours.
• No shows: 12 hours.

Therefore, this year it's possible to sell:

7,300 - 20 (configuration changes) = 7,280 hours.

Sold: 6,000 - 40 (cancelled) = 5,960 hours.

Note: The No Show time is not accounted for since it is un-expected - the simulator is still available and sitting idle and ready for training, and not touched by maintenance personnel. The snag clearance sessions, certification and simulator test profiles are still accounted for as trained time so this has to be added to the sold time as time trained on the machine. This time will be added if the sales department or scheduling are not including these hours in their calculations.

Training time: 5,960 + 44 (snag clearance) + 4 (certification) + 4 (test profile) = 6,012 hours.

Usage: (6,012/7,280) * 100 = 85.58%

During this period the maintenance department has collected 200 write-ups, 50 snags or defects, and 20 incidents with downtime, with a total downtime of 15 hours.

We can now recalculate the MTBF:

MTBF = 6,012/20 = 300hours.

Meaning that for every 300 hours of training there is a downtime. Or in other words we cannot do more than 300 hours of training without an interrupt!

Reliability: ((6,012 - 20)/6,012) * 100 = 99.66%

MTTR = 15/20 = 45 minutes. Average time to return to operation after a downtime.

The Busy Simulator Example (The Ugly Truth)

In the real world we don't always have all the data as simple and straight as we have shown: it's a battle between the sales department; the scheduling; and the maintenance departments to provide accurate details in order to present a proper set of data.

So far we have not covered all defects reported; trends; or even repeat ofenders on the crew side. All the examples above are centered on the downtime , and only the downtime, but we have to start somewhere.

Remember, and this is the important point, we are trying to show that your simulator is as good or bad as the rest of the industry - there is no point in trying to present a simulator with 99.99% reliability if the rest of the industry can only achieve 85.50%. If this happens you are doing something wrong, or the rest of the world is working with a different set of metrics. It is time to find out because you can be misled into believing you are right, and what you are doing cannot be done better.

As usual, any metrics are subject to varying interpretation by different departments at your work place. While a low MTBF might be seen as a poor maintenance team by some, others might see the need for more training or parts to support the simulator. In the same way poor reliability can be seen as an indication of poor maintenance; while maintenance might see it as the indication of a repeated problem pointing to a design fault; or the need for a simulator update to improve an unreliable system. High MTTR might mean you do not have tools in place; not enough engineers; or not enough knowledge to service the machine quickly.

But how high is your MTTR? MTBF is only relative to the rest of the industry and that is why presenting and sharing the data is so necessary.

We hope, by now, that you realize how important is to provide and share the data. We also hope you are starting to prepare that data to send it to us. We will need Usage Index;, Reliability Index; MTBF in hours per device for the period of the complete year 2006; we are requesting a complete year number so we can smooth out any bad months. Once presented SimUser will provide you a an operator code so you can identify your own data amongst all the rest.

As usual for any comments or to supply your data you can contact SimUser directly.
This e-mail address is being protected from spam bots, you need JavaScript enabled to view it

(0) No Comments. Start a discussion...