Remaining useful life for all!

Before you ask yourself, no, we do not intend in this article to quantify how much usefulness you – the reader – still have. That would be Orwellian and morbid in the least. Instead, with this article we would like to briefly outline some of the research we are doing on our topic, remaining useful life (RUL). The “all” in the title will refer to the ability of everyone being able to estimate the RUL for a piece of machinery they might be interested in, as we will see. What you would need is some minimal knowledge of Python. We will show how to do that, the difficulties and the advantages. But first, a quick recap.

The remaining useful life (RUL) of an asset or system is defined as the length from the current time to the end of the useful life [1]. Because the adjective “useful” is subjective, the previous definition can be extended to the time when the extent of deviation or degradation of the component from its expected normal operating conditions exceeds a pre-defined threshold [2]. The estimation of the RUL is of paramount importance for safety-critical systems and lies in the heart of prognostics and health management (PHM) [3]. The estimation of the RUL can be done in various ways. Model-based and data-driven are the most prominent [3], and in general all methods make some use of the sensor data of the equipment. In this article we will talk about data-driven methods, which are essentially machine learning methods. For more information on the possible methods that can be used we direct the reader to [3].

Estimating the RUL using data-driven methods requires, as the name suggests, historical data. And specifically, failure data and nominal data to train our models from multiple units/equipment. For now, we will assume that we have these data. What we would like is given sensor data of an equipment at the current point in time, to estimate how much more of a useful life our equipment still has. In other words, we are asking the question for how long can we expect our equipment to behave as it should behave? A graspable example would be: for how long can I drive my car before I need to change its distribution belt, relying though on my data and my data only? Why would my car need maintenance at 10K km (like most) and not at 150K km? This is the difference between preventive (or preventative/scheduled/routine/time-based) and predictive maintenance. It saves time and money and is as safe (theoretically at least) as preventative maintenance.

From a data science perspective, the previous translates to discovering a mapping from a time-series (our observed sensor data) to a real number, which in our case is the RUL. Thus, it can be thought of as a regression problem. There is just one problem. What labels do I have on my training data? This brings us to our first difficulty.

If we are lucky, a field expert might be able to label every time-step for us, but most of the times that is not the case. It’s one of the intrinsic difficulties of PHM and the reason it’s so intriguing. There are a few ways you can do that. Since you know the end-of-life (EoL) of multiple training trajectories you can subtract at every time-step of the data “1” from the EoL, creating this way a linear degradation [4]. Of course, this is not the case in real-life as degradation will probably be negligible for the first cycles (constant RUL) and will start manifesting itself after some time. A solution to this is to create exactly this type of degradation [4]. But again, who is to say what my initial value for the RUL is, and after which point do I start observing degradation? These are the other difficulties. Unless, we have an expert or some other information for the degradation, the only solution is trial-and-error or some other kind of hyperparameter optimization (HO). Unfortunately, like everything else, there is no free lunch.

For the sake of discussion, we will now assume that we have the data transformed into a regression problem and we will move on to the more exciting part. Modeling. There are a multitude of ways to model our problem having our data. The list contains, DNNs (deep neural networks), RVR (relevance vector regression), GPR (gaussian process regression) RF (random forests) and other abbreviations that might be overwhelming. However, as we said in the introduction this is remaining useful life for all. Thankfully, there is a field called AutoML, which stands for automated machine learning.

Automated machine learning (AutoML) deals with the automation of the process of applying machine learning to real-world problems. In general, AutoML covers the complete pipeline from the raw data to the deployment of the model, and it was proposed as an artificial intelligence-based solution to the ever-growing challenge of applying machine learning [5]. The high degree of automation in AutoML allows non-experts to make use of machine learning models and techniques without needing to be an expert in the field first. The user can select from a big variety of libraries, such as TPOT [6], AUTO-SKLEARN [7], Hyperopt-Sklearn [8], Auto-WEKA [9] and the list goes on. For more information on the available libraries please see [10].

This brings us to the pros of such an approach. First, AutoML allows practitioners and non-experts to test their ideas quickly and formulate/update further their hypotheses. What is more, it allows non experts to use a variety of machine learning algorithms to estimate the RUL of their equipment without the need of hyperparameter tuning of their models and selecting the models (the so-called CASH problem standing for combined algorithm selection and hyperparameter optimization [11]). The latter advantage is the reason that AutoML is so popular and important.

In the CIMPLO project and in the context of designing methods to estimate the RUL we are investigating AutoML approaches for the RUL prediction in order to make it available for all. We are investigating how to extract the available degradation information from sensor signals in a generic way that can be applied to most domains and can be used with AutoML methods. Currently, we are developing a platform where AutoML (and more problem specific methods) are employed for predictive maintenance coupled with schedule optimization.

References:

[1]: X.-S. Si, W. Wang, C.-H. Hu, and D.-H. Zhou, “Remaining useful life estimation–a review on the statistical data driven approaches,” European Journal of Operational Research, vol. 213, no. 1, pp. 1–14, 2011.

[2]: A. Saxena, K. Goebel, D. Simon, and N. Eklund, “Damage propagation modeling for aircraft engine run-to-failure simulation,” in Proc. of IEEE International Conference on Prognostics and Health Management (PHM 2008), pp. 1-9, 2008.

[3]: Nguyen, D., Kefalas, M., Yang, K., Apostolidis, A., Olhofer, M., Limmer, S., & Back, T. (2019). A review: ¨ Prognostics and health management in automotive and aerospace. International Journal of Prognostics and Health Management, 10(2), 35.

[4]: Sateesh Babu, G., Zhao, P., Li, X.L.: Deep Convolutional Neural Network BasedRegression Approach for Estimation of Remaining Useful Life. In: Navathe, S.B.,Wu, W., Shekhar, S., Du, X., Wang, X.S., Xiong, H. (eds.) Database Systems forAdvanced Applications, vol. 9642, pp. 214–228. Springer International Publishing,Cham (2016).

[5]: Hutter, F., Kotthoff, L., Vanschoren,J. (eds.): Automated MachineLearning:Methods,Systems,Challenges.The Springer Se-rieson Challenges in Machine Learning, Springer International Publishing, Cham(2019).

[6]: Olson, R.S., Moore, J.H.: Tpot: A tree-based pipeline optimization tool for au-tomating machine learning. Proceedings of Machine Learning Research, vol. 64,pp. 66–74. PMLR, New York, New York, USA (24 Jun 2016).

[7]: Feurer, M. and Klein, A. and Eggensperger, K. and Springenberg, J. and Blum, M. and Hutter, F. Efficient and Robust Automated Machine Learning In: Advances in Neural Information Processing Systems 28.

[8]: Bergstra, J., Yamins, D., Cox, D. D. (2013) Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures. To appear in Proc. of the 30th International Conference on Machine Learning (ICML 2013).

[9]: Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms Chris Thornton, Frank Hutter, Holger Hoos, and Kevin Leyton-Brown. In Proc. of KDD 2013, 2013.

[10]: Jonathan Waring, Charlotta Lindvall, Renato Umeton, Automated machine learning: Review of the state-of-the-art and opportunities for healthcare, Artificial Intelligence in Medicine, Volume 104, 2020, 101822, ISSN 0933-3657.

[11]: F. Hutter, H. H. Hoos, and K. Leyton-Brown, “Sequential model-based optimization for general algorithm configuration,” in Learning and Intelligent Optimization, C. A. C. Coello, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 507–523.