Numerical Simulation of an InP Photonic Integrated Cross-Connect for Deep Neural Networks on Chip
Source abstract
Abstract
We propose a novel photonic accelerator architecture based on a broadcast-and-weight approach for a deep neural network through a photonic integrated cross-connect. The single neuron and the complete neural network operation are numerically simulated. The weight calibration and weighted addition are reproduced and demonstrated to behave as in the experimental measurements. A dynamic range higher than 25 dB is predicted, in line with the measurements. The weighted addition operation is also simulated and analyzed as a function of the optical crosstalk and the number of input colors involved. In particular, while an increase in optical crosstalk negatively influences the simulated error, a greater number of channels results in better performance. The iris flower classification problem is solved by implementing the weight matrix of a trained three-layer deep neural network. The performance of the corresponding photonic implementation is numerically investigated by tuning the optical crosstalk and waveguide loss, in order to anticipate energy consumption per operation. The analysis of the prediction error as a function of the optical crosstalk per layer suggests that the first layer is essential to the final accuracy. The ultimate accuracy shows a quasi-linear dependence between the prediction accuracy and the errors per layer for a normalized root mean square error lower than 0.09, suggesting that there is a maximum level of error permitted at the first layer for guaranteeing a final accuracy higher than 89%. However, it is still possible to find good local minima even for an error higher than 0.09, due to the stochastic nature of the network we are analyzing. Lower levels of path losses allow for half the power consumption at the matrix multiplication unit, for the same error level, offering opportunities for further improved performance. The good agreement between the simulations and the experiments offers a solid base for studying the scalability of this kind of network.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Neural Networks and Reservoir Computing · Photonic and Optical Devices · Optical Network Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute for Photonic Integration, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands
Funding
Nederlandse Organisatie voor Wetenschappelijk Onderzoek
References (35)
Show source-provided references
1. McAfee, A., and Brynjolfsson, E. (2012). Big Data: the Management Revolution. Harv. Bus. Rev., 90.
2. Data-Intensive Applications, Challenges, Techniques and Technologies: A Survey on Big Data10.1016/j.ins.2014.01.015 ↗
3. Honkela, T., Duch, W., Girolami, M., and Kaski, S. (2011). Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction. Proceedings of the Artificial Neural Networks and Machine Learning-ICANN 2011, Springer.10.1007/978-3-642-21735-7 ↗
4. Face Recognition: A Convolutional Neural-Network Approach10.1109/72.554195 ↗
5. Neural Network Models for Time Series Forecasts10.1287/mnsc.42.7.1082 ↗
6. Global Optimization of Absorption Chiller System by Genetic Algorithm and Neural Network10.1016/S0378-7788(01)00085-8 ↗
7. Convolutional Neural Network Architectures for Predicting DNA–Protein Binding10.1093/bioinformatics/btw255 ↗
8. Data Mining and Machine Learning in Astronomy10.1142/S0218271810017160 ↗
9. Disruption Forecasting at JET Using Neural Networks10.1088/0029-5515/44/1/008 ↗
10. Artificial Neural Networks: A New Approach to Modeling Interregional Telecommunication Flows10.1111/j.1467-9787.1994.tb00880.x ↗
11. TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip10.1109/TCAD.2015.2474396 ↗
12. Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations10.1109/JPROC.2014.2313565 ↗
13. The SpiNNaker Project10.1109/JPROC.2014.2304638 ↗
14. Braindrop: A Mixed-Signal Neuromorphic Architecture with a Dynamical Systems-Based Programming Model10.1109/JPROC.2018.2881432 ↗
15. Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., and Cong, J. (2015, January 22–24). Optimizing FPGA-Based Accelerator Design for Deep Convolutional Neural Networks. Proceedings of the FPGA 2015–2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA.10.1145/2684746.2689060 ↗
16. Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M.A., and Dally, W.J. (2016, January 18–22). EIE: Efficient Inference Engine on Compressed Deep Neural Network. Proceedings of the 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), Seoul, Korea.10.1109/ISCA.2016.30 ↗
17. Jouppi, N.P., Borchers, A., Boyle, R., Cantin, P., Chao, C., Clark, C., Coriell, J., Daley, M., Dau, M., and Dean, J. (2017, January 24–28). In-Datacenter Performance Analysis of a Tensor Processing Unit. Proceedings of the Proceedings-International Symposium on Computer Architecture, Toronto, ON, Canada.10.1145/3079856.3080246 ↗
18. Ultrafast All-Optical Implementation of a Leaky Integrate-and-Fire Neuron10.1364/OE.19.002133 ↗
19. Reinforcement Learning in a Large-Scale Photonic Recurrent Neural Network10.1364/OPTICA.5.000756 ↗
20. Laser Dynamical Reservoir Computing with Consistency: An Approach of a Chaos Mask Signal10.1364/OE.24.008679 ↗
21. Monolithically Integrated 8 × 8 Space and Wavelength Selective Cross-Connect10.1109/JLT.2013.2290322 ↗
22. An Introduction to InP-Based Generic Integration Technology10.1088/0268-1242/29/8/083001 ↗
23. Parallel Reservoir Computing Using Optical Amplifiers10.1109/TNN.2011.2161771 ↗
24. Experimental Demonstration of Reservoir Computing on a Silicon Photonics Chip10.1038/ncomms4541 ↗
25. Deep Learning with Coherent Nanophotonic Circuits10.1038/nphoton.2017.93 ↗
26. Neuromorphic Photonic Networks Using Silicon Photonic Weight Banks10.1038/s41598-017-07754-z ↗
27. Deep Neural Network Through an InP SOA-Based Photonic Integrated Cross-Connect10.1109/JSTQE.2019.2945548 ↗
28. Neuromorphic Photonic Integrated Circuits10.1109/JSTQE.2018.2868185 ↗
29. Nahmias, M.A., Peng, H.-T., de Lima, T.F., Huang, C., Tait, A.N., Shastri, B.J., and Prucnal, P.R. (October, January 30). A TeraMAC Neuromorphic Photonic Processor. Proceedings of the 2018 IEEE Photonics Conference (IPC), Reston, VA, USA.10.1109/IPCon.2018.8527089 ↗
30. An all-Optical Neuron with Sigmoid Activation Function10.1364/OE.27.009620 ↗
31. Gradient-Based Learning Applied to Document Recognition10.1109/5.726791 ↗
32. Amplified Spontaneous Emission in Semiconductor Laser Amplifiers. Validity of the Transmission-Line Laser Model10.1049/ip-j.1990.0041 ↗
33. Dynamic Multi-Path WDM Routing in a Monolithically Integrated 8 × 8 Cross-Connect10.1364/OE.22.000435 ↗
34. The Use of Multipe Measurements in Taxonomic Problems10.1111/j.1469-1809.1936.tb02137.x ↗
35. PCANet: A Simple Deep Learning Baseline for Image Classification?10.1109/TIP.2015.2475625 ↗