Systems Configuration and Reliability Analysis (20 pts.)
Systems configuration and integration is pivotal to achieve the desired performance. Both series and
parallel configurations have pros and cons and implications in the system performance. Explain which
configuration provides the best reliability if you do not have physical and resources constraints. Also,
briefly explain under which conditions systems in series will provide the most adequate configuration.
A system has five (5) components in series with a reliability of 0.99999, each. What is the overall system
reliability? If the system engineer request to you to change the configuration to parallel to increase the
reliability; what is the difference (%) in reliability? What is the minimum reliability each component (in
parallel) must have to have an overall system reliability of 0.99999 (i.e., the same reliability each
component in series has)?

 

2. Operation and Maintenance and Operation and Support (10 pts.)
Briefly define the concept of “Sustainment Engineering” and describe its elements. Explain the
importance of “obsolescence management” in the and provide an example.

Software Engineering: Code and Unit Test (25 pts.)
You are the software systems engineer for the US Transportation Security Administration (TSA) project
“Acquisition of New Simultaneous Explosives and Narcotics Trace Detectors”. After the evaluation of
multiple systems and vendors during the RFP process, the ISOSCAN 500 DT manufactured by Point
Security, Inc. was selected. The specifications require a factory acceptance test (FAT) to assess the
performance of the system. The code and unit testing protocol require a sample of 10,000 simulations to test
the classification model. The results are summarized in the form of a confusion matrix, where true positives
(TP) are 2,387, the true negatives (TN) are 4,939, the false positives (FP) or type-1 error are 721, and the
false negatives (FN) or type-2 error are 1,953. The performance and functional requirements, the specific
metric, and the minimum acceptance criteria are listed. Estimate the actual metrics, and based on your results
determine whether you accept or reject the ISOSCAN 500 DT. Explain the implications of the two types of
errors and which one is the most critical.

 

4. Decision-making and Model Validation (25 pts.)
ROC curve (AUC) is a measure of prediction performance of classification models by comparing the
true positives (TP) vs false positive (FP) classifications. The figure below represents the curves for a
model performance before (baseline) and after (updated) new knowledge or evidence was available.
What can be a reason why the improvement of the model is only 4%? What is the importance of the
model validation process? What does the red line represent? What happen if the model performance
coincides with or is below the red line?

 

5. Software Engineering and Modeling (10 pts.)
The reliability engineer (RE) makes a request to develop a prognostic health management (PHM) program
for critical components of a complex system to support the sustainment engineering program to develop
appropriate maintenance management strategies. The RE recommends a piece of built-in test equipment, or
passive fault management and diagnosis equipment, to monitor a list of failure modes and mechanisms
resulting from the Failure Mode and Criticality Analysis (FMECA) during the engineering development
phase. The RE recommends the use of machine learning (ML) for the equipment to analyze the failures.
What type of ML (supervised, unsupervised, reinforcement) would you use? Why?
Then, the integrated logistic support (ILS) team wants to leverage the use of ML to develop an interactive
training program to test the maintenance crew troubleshooting and diagnostic skills. What type of ML
(supervised, unsupervised, reinforcement) would you use? Why?

 

6. Risk Assessment (15 pts.)
You have the following table summarizing the risk assessment data of the project you are currently
working on:
Scenario/Event Likelihood (L) Consequence (C)
(Millions)
Risk Score
(L x C)
1 1 in 10 years (1/10) $100 10
2 1 in 500 years (1/500) $1,000 2
3 1 in 50 years (1/50) $500 10
4 1 in 10 years (1/10) $20 2
5 1 in 100 years (1/100) $300 3
6 1 in 10 years (1/10) $800 80
Perform the following:
a. Construct a risk matrix (i.e., a “Likelihood vs. Consequence” matrix)
b. Provide your analysis about the risk matrix and the risk prioritization to recommend mitigation
strategies (you do not have to recommend specific mitigation strategies). Try also to prioritize
the events based on the risk score and explain differences and challenges.
c. Can one use risk-based decision-making only in this process? Justify your answer.