Time-Aware and Feature Similarity Self-Attention in Vessel Fuel Consumption Prediction
Source abstract
Abstract
An accurate vessel fuel consumption prediction is essential for constructing a ship route network and vessel management, leading to efficient sailings. Besides, ship data from monitoring and sensing systems accelerate fuel consumption prediction research. However, the ship data consist of three properties: sequential, irregular time interval, and feature importance, making the predicting problem challenging. In this paper, we propose Time-aware Attention (TA) and Feature-similarity Attention (FA) applied to bi-directional Long Short-Term Memory (LSTM). TA acquires time importance by nonlinear function from irregular time intervals in each sequence and emphasizes data depending on the importance. FA emphasizes data based on similarities of features in the sequence by estimating feature importance with learnable parameters. Finally, we propose the ensemble model of TA and FA-based BiLSTM. The ensemble model, which consists of fully connected layers, is capable of simultaneously capturing different properties of ship data. The experimental results on ship data showed that the proposed model improves the performance in predicting fuel consumption. In addition to model performance, visualization results of attention maps and feature importance help to understand data properties and model characteristics.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Maritime Transport Emissions and Efficiency · Maritime Navigation and Safety · Advanced Chemical Sensor Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Industrial and Management Engineering, Korea University, 145, Anam-Ro, Seongbuk-Gu, Seoul 02841, Korea
Seavantage, Nonhyeon-dong 201-6beon-ji, Seoul 06120, Korea
Funding
Korea TechnoComplex Foundation Grant · Korea Institute for Advancement of Technology · Korea Government(MOTIE)
References (45)
Show source-provided references
1. What price of speed? A critical revision through constructal optimization of transport modes10.1007/s40095-015-0160-6 ↗
2. Vessel pattern knowledge discovery from AIS data: A framework for anomaly detection and route prediction10.3390/e15062218 ↗
3. An artificial neural network based decision support system for energy efficient ship operations10.1016/j.cor.2015.04.004 ↗
4. Prediction of ship fuel consumption by using an artificial neural network10.1007/s12206-018-1126-4 ↗
5. Prediction of fuel consumption for enroute ship based on machine learning10.1109/ACCESS.2019.2933630 ↗
6. Neural network-based fuel consumption estimation for container ships in Korea10.1080/03088839.2020.1729437 ↗
7. Prediction of vessel propulsion power using machine learning on AIS data, ship performance measurements and weather data
8. Uyanik, T., Arslanoglu, Y., and Kalenderli, O. (2019, January 25–27). Ship Fuel Consumption Prediction with Machine Learning. Proceedings of the 4th International Mediterranean Science and Engineering Congress, Antalya, Turkey.
9. A multi-task analysis and modelling paradigm using LSTM for multi-source monitoring data of inland vessels10.1016/j.oceaneng.2020.107604 ↗
10. Panapakidis, I., Sourtzi, V.M., and Dagoumas, A. (2020). Forecasting the Fuel Consumption of Passenger Ships with a Combination of Shallow and Deep Learning. Electronics, 9.10.3390/electronics9050776 ↗
11. The input vector space optimization for LSTM deep learning model in real-time prediction of ship motions10.1016/j.oceaneng.2020.107681 ↗
12. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
13. Assessment of ship’s engine effective power, fuel consumption and emission using the vessel speed
14. Bidirectional recurrent neural networks10.1109/78.650093 ↗
15. Learning representations by back-propagating errors10.1038/323533a0 ↗
16. Backpropagation through time: What it does and how to do it10.1109/5.58337 ↗
17. Learning long-term dependencies with gradient descent is difficult10.1109/72.279181 ↗
18. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.10.3115/v1/D14-1179 ↗
19. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
20. Luong, M.T., Pham, H., and Manning, C.D. (2015). Effective approaches to attention-based neural machine translation. arXiv.10.18653/v1/D15-1166 ↗
21. Yao, X. (2017). Attention-based BiLSTM neural networks for sentiment classification of short texts. Proc. Int. Conf. Inf. Sci. Cloud Comput., 110–117.
22. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
23. Merity, S. (2019). Single headed attention rnn: Stop thinking with your head. arXiv.
24. Gao, Y., Fang, C., and Ruan, Y. (2019, January 6–7). A novel model for the prediction of long-term building energy demand: LSTM with Attention layer. Proceedings of the IOP Conference Series: Earth and Environmental Science, Tokyo, Japan.10.1088/1755-1315/294/1/012033 ↗
25. Song, H., Rajan, D., Thiagarajan, J.J., and Spanias, A. (2017). Attend and diagnose: Clinical time series analysis using attention models. arXiv.10.1609/aaai.v32i1.11635 ↗
26. An attentive survey of attention models10.1145/3465055 ↗
27. Wu, N., Green, B., Ben, X., and O’Banion, S. (2020). Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case. arXiv.
28. TAPER: Time-Aware Patient EHR Representation10.1109/JBHI.2020.2984931 ↗
29. Li, J., Wang, Y., and McAuley, J. (2020, January 3–7). Time Interval Aware Self-Attention for Sequential Recommendation. Proceedings of the 13th International Conference on Web Search and Data Mining, Houston, TX, USA.10.1145/3336191.3371786 ↗
30. An introduction to variable and feature selection
31. A survey on feature selection methods10.1016/j.compeleceng.2013.11.024 ↗
32. How to generate a good word embedding10.1109/MIS.2016.45 ↗
33. Iqbal, R.A. (2011). Using feature weights to improve performance of neural networks. arXiv.
34. Deep feature weighting for naive Bayes and its application to text classification10.1016/j.engappai.2016.02.002 ↗
35. GSSA: Pay attention to graph feature importance for GCN via statistical self-attention10.1016/j.neucom.2020.07.098 ↗
36. Škrlj, B., Džeroski, S., Lavrač, N., and Petkovič, M. (2020). Feature importance estimation with self-attention networks. arXiv.
37. Gui, N., Ge, D., and Hu, Z. (February, January 27). AFS: An attention-based mechanism for supervised feature selection. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
38. Lee, K.H., Park, C., Oh, J., and Kwak, N. (2021). LFI-CAM: Learning Feature Importance for Better Visual Explanation. arXiv.10.1109/ICCV48922.2021.00139 ↗
39. Xiao, J., Ye, H., He, X., Zhang, H., Wu, F., and Chua, T.S. (2017). Attentional factorization machines: Learning the weight of feature interactions via attention networks. arXiv.10.24963/ijcai.2017/435 ↗
40. Baytas, I.M., Xiao, C., Zhang, X., Wang, F., Jain, A.K., and Zhou, J. (2017, January 13–17). Patient subtyping via time-aware lstm networks. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.10.1145/3097983.3097997 ↗
41. Dietterich, T.G. (2000). Ensemble methods in machine learning. International Workshop on Multiple Classifier Systems, Springer.10.1007/3-540-45014-9_1 ↗
42. Qiu, X., Zhang, L., Ren, Y., Suganthan, P.N., and Amaratunga, G. (2014, January 9–12). Ensemble deep learning for regression and time series forecasting. Proceedings of the 2014 IEEE Symposium on Computational Intelligence in Ensemble Learning (CIEL), Orlando, FL, USA.10.1109/CIEL.2014.7015739 ↗
43. Tan, Z., Wang, M., Xie, J., Chen, Y., and Shi, X. (2017). Deep semantic role labeling with self-attention. arXiv.10.1609/aaai.v32i1.11928 ↗
44. Kim, W., Goyal, B., Chawla, K., Lee, J., and Kwon, K. (2018, January 8–14). Attention-based ensemble for deep metric learning. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01246-5_45 ↗
45. Wang, W., Wu, T., Zhao, D., Guo, C., Luo, W., and Pang, Y. (2019). Experimental–numerical analysis of added resistance to container ships under presence of wind–wave loads. PLoS ONE, 14.10.1371/journal.pone.0221453 ↗