The first was to compare the average by obtaining the prediction
accuracy of each of the 98 ship engines. To model the 3-layer dataset,
only one option exists for hierarchical model. This is because the
hierarchical model predicts 3-layer data using information of all
layers. For Prophet and ARIMA, which are unable to represent the
hierarchical structure, input data should be preprocessed, by averaging,
to learn the parameters. As can be seen from Table 1, RMSE of HS model
was the lowest. Since the HS model pools the information of the layers,
the overall average RMSE is low. In Table 2, the HS model has a lower
RMSE than the comparative model in the engine type of type 2
~ type 4, which has a relatively large number of data.
Type 1 was less accurate than ARIMA, and type 5 had the worst
performance. Since ARIMA or Prophet has no pooling effect, the smaller
the number of data, the more noise is reflected in the prediction.
Therefore, when the number of data is small, the prediction accuracy of
ARIMA and Prophet can be high. This will be further explained in section
5.2.
5.2. Forecasting a new type of ship or engine
When we fit the hierarchical model with failure rates of 98 ships, the
learned results are stored in the model in the form of each parameter’s
distribution, i.e. posterior. For example, whose prior had exponential
form would evolve into a posterior distribution. Bayes formula explains
this mechanism. As discussed in the introduction, engine failure rate of
a new type of engine or ship is frequently needed. Depending on its
engine type, the way by which the hierarchical model should be applied
differs. If its engine type is present among the data, the posterior of
parameters corresponding to layer 2 could be used for the forecast
(5.2.1). On the other hand, if the engine type is new as well, the only
information we could borrow from the previous
data
are posteriors of layer1 parameters (5.2.2).
The test set data is shown in Fig. 5. Types 1 ~ 5 are
the same as the 5 engine types included in train set data. One engine
data was obtained for each engine type and prepared as a test set. Types
6 ~ 10 are new engine types not included in train set
data. Ship engine data corresponding to 5 new engine types were prepared
as a test set for each type.
5.2.1. New ship type
Previously learned posterior of \(\overset{\overline{}}{\alpha_{e}}\ \)and \(\overset{\overline{}}{w_{e}}\) , could be directly used for
predicting engine failure of a new ship engine, but whose engine type is
not new; in other words not among the 5 trained engine types. As in
section 5.1, Prophet and Arima were used as comparative models. The
results are shown in Table 3.
Table 3. Test set data RMSE (New ship type which engine type was
trained)