journal article·2019-11-07Open access

Multi-Task Learning for Multi-Dimensional Regression: Application to Luminescence Sensing

Umberto Michelucci, Francesca Venturini

Applied Sciences· Volume 9· Issue 22

Abstract

The classical approach to non-linear regression in physics is to take a mathematical model describing the functional dependence of the dependent variable from a set of independent variables, and then using non-linear fitting algorithms, extract the parameters used in the modeling. Particularly challenging are real systems, characterized by several additional influencing factors related to specific components, like electronics or optical parts. In such cases, to make the model reproduce the data, empirically determined terms are built in the models to compensate for the difficulty of modeling things that are, by construction, difficult to model. A new approach to solve this issue is to use neural networks, particularly feed-forward architectures with a sufficient number of hidden layers and an appropriate number of output neurons, each responsible for predicting the desired variables. Unfortunately, feed-forward neural networks (FFNNs) usually perform less efficiently when applied to multi-dimensional regression problems, that is when they are required to predict simultaneously multiple variables that depend from the input dataset in fundamentally different ways. To address this problem, we propose multi-task learning (MTL) architectures. These are characterized by multiple branches of task-specific layers, which have as input the output of a common set of layers. To demonstrate the power of this approach for multi-dimensional regression, the method is applied to luminescence sensing. Here, the MTL architecture allows predicting multiple parameters, the oxygen concentration and temperature, from a single set of measurements.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Machine Learning in Materials Science · Neural Networks and Reservoir Computing · Air Quality Monitoring and Forecasting

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

TOELT LLC, Birchlenstr. 25, 8600 Dübendorf, Switzerland

Institute of Applied Mathematics and Physics, Zurich University of Applied Sciences, Technikumstrasse 9, 8401 Winterthur, Switzerland

References (34)

Show source-provided references
1. Glynn, P., and Robinson, S.M. (1999). Numerical Optimization, Springer.
2. Boyd, S.P., and Vandenberghe, L. (2019). Convex Optimization, Cambridge University Press.
3. James, G., Witten, D., Hastie, T., and Tibshirani, R. (2013). An Introduction to Statistical Learning, Springer.10.1007/978-1-4614-7138-7 ↗
4. Papkovsky, D.B., and Dmitriev, R.I. (2018). Fundamentals of Quenched Phosphorescence O2 Sensing and Rational Design of Sensor Materials. Quenched-phosphorescence Detection of Molecular Oxygen, Royal Society of Chemistry. [1st ed.].10.1039/9781788013451 ↗
5. Dual Fluorescence Sensor for Trace Oxygen and Temperature with Unmatched Range and Sensitivity
6. Collier, B.B., and McShane, M.J. (2011). Simultaneous, accurate lifetime determination of two luminophores using time-domain techniques. Sensors, 943–946.10.1109/ICSENS.2011.6127121 ↗
7. Compact optical instrument for simultaneous determination of oxygen and carbon dioxide10.1007/s00604-010-0520-0 ↗
8. Dual Optical Sensor for Oxygen and Temperature Based on the Combination of Time Domain and Frequency Domain Techniques10.1016/j.talanta.2010.12.016 ↗
9. A novel planar optical sensor for simultaneous monitoring of oxygen, carbon dioxide, pH and temperature10.1007/s00216-010-4617-4 ↗
10. Zhang, Y., and Yang, Q. (2018). A Survey on Multi-Task Learning. arXiv.
11. A brief review on multi-task learning10.1007/s11042-018-6463-x ↗
12. Is learning the n-th thing any easier than learning the first?
13. A model of inductive bias learning10.1613/jair.731 ↗
14. Multi-task learning10.1023/A:1007379606734 ↗
15. Michelucci, U. (2018). Applied Deep Learning—A Case-Based Approach to Understanding Deep Neural Networks, Apress Media, LLC.10.1007/978-1-4842-3790-8 ↗
16. Irie, B., and Miyake, S. (1988, January 24–27). Capabilities of three-layered perceptrons. Proceedings of the IEEE International Conference on Neural Networks, San Diego, CA, USA.10.1109/ICNN.1988.23901 ↗
17. Approximation Capabilities of Multilayer Feedforward Networks10.1016/0893-6080(91)90009-T ↗
18. Approximation by Superpositions of a Sigmoidal Function10.1007/BF02551274 ↗
19. Hanin, B. (2019). Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations. Mathematics, 7.10.3390/math7100992 ↗
20. Lu, Z., Pu, H., Wang, F., Hu, Z., and Wang, L. (2017, January 4–9). The expressive power of neural networks: A view from the width. Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA.
21. Rojas, R. (1996). Neural Networks—A Systematic Introduction, Springer.10.1007/978-3-642-61068-4 ↗
22. Bishop, C.M. (2005). Neural Networks for Pattern Recognition, Oxford University Press.
23. On the structure of Continuous Functions of Several Variables10.1090/S0002-9947-1965-0210852-X ↗
24. Argyriou, A., Evgeniou, T., and Pontil, M. (2006, January 4–7). Multi-task feature learning. Proceedings of the 19th International Conference on Neural Information Processing Systems (NIPS’06), Vancouver, BC, Canada.10.7551/mitpress/7503.003.0010 ↗
25. Kingma, D.P., and Ba, J. (2015, January 7–9). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA.
26. Optical methods for sensing and imaging oxygen: materials, spectroscopies and applications10.1039/C4CS00039K ↗
27. Narayanaswamy, R., and Wolfbeis, O.S. (2004). Optical Technology until the Year 2000: An Historical Overview. Optical sensors: Industrial Environmental and Diagnostic Applications, Springer. [1st ed.].
28. Lakowicz, J.R. (2006). Principles of Fluorescence Spectroscopy, Springer. [3rd ed.].10.1007/978-0-387-46312-4 ↗
29. Oxygen Sensors Based on Luminescence Quenching10.1021/ac9908546 ↗
30. Michelucci, U., Baumgartner, M., and Venturini, F. (2019). Optical oxygen sensing with artificial intelligence. Sensors, 19.10.20944/preprints201901.0047.v1 ↗
31. Burnham, K.P., and Anderson, D.R. (2002). Model Selection and Multimodel Inference, Springer. [2nd ed.].
32. Variations of Box Plots10.1080/00031305.1978.10479236 ↗
33. Hastie, T., Tibshirani, R., and Friedman, J. (2013). The Elements of Statistical Learning, Springer Science-Business Media, LLC.
34. Waskom, M., Botvinnik, O., O’Kane, D., Hobson, D., Lukauskas, S., Gemperline, D., and Qalieh, A. (2019, October 21). A Python Visualisation Library. Available online: http://doi.org/10.5281/zenodo.883859.