Semi-Supervised Convolutional Neural Network for Law Advice Online
Source abstract
Abstract
With the rapid developments of Internet technology, a mass of law cases is constantly occurring and needs to be dealt with in time. Automatic classification of law text is the most basic and critical process in the online law advice platform. Deep neural network-based natural language processing (DNN-NLP) is one of the most promising approaches to implement text classification. Meanwhile, as the convolutional neural network-based (CNN-based) methods developed, CNN-based text classification has already achieved impressive results. However, previous work applied amounts of manually-annotated data, which increased the labor cost and reduced the adaptability of the approach. Hence, we present a new semi-supervised model to solve the problem of data annotation. Our method learns the embedding of small text regions from unlabeled data and then integrates the learned embedding into the supervised training. More specifically, the learned embedding regions with the two-view-embedding model are used as an additional input to the CNN’s convolution layer. In addition, to implement the multi-task learning task, we propose the multi-label classification algorithm to assign multiple labels to an instance. The proposed method is evaluated experimentally subject to a law case description dataset and English standard dataset RCV1 . On Chinese data, the simulation results demonstrate that, compared with the existing methods such as linear SVM, our scheme respectively improves by 7.76%, 7.86%, 9.19%, and 2.96% the precision, recall, F-1, and Hamming loss. Analogously, the results suggest that compared to CNN, our scheme respectively improves by 4.46%, 5.76%, 5.14% and 0.87% in terms of precision, recall, F-1, and Hamming loss. It is worth mentioning that the robustness of this method makes it suitable and effective for automatic classification of law text. Furthermore, the design concept proposed is promising, which can be utilized in other real-world applications such as news classification and public opinion monitoring.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Artificial Intelligence in Law · Artificial Intelligence Applications · Imbalanced Data Classification Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
The School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
The College of Automation, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
References (63)
Show source-provided references
1. Legal advice on the smartphone10.1145/2933414 ↗
2. Platform Economy in Legal Profession: An Empirical Study on Online Legal Service Providers in China
3. A Framework for Combining a Motion Atlas with Non-Motion Information to Learn Clinically Useful Biomarkers: Application to Cardiac Resynchronisation Therapy Response Prediction10.1016/j.media.2016.10.002 ↗
4. An efficient Wikipedia semantic matching approach to text document classification10.1016/j.ins.2017.02.009 ↗
5. Eger, S., Youssef, P., and Gurevych, I. (2019). Is it time to swish? comparing deep learning activation functions across NLP tasks. arXiv.10.18653/v1/D18-1472 ↗
6. Linkify: Enhancing Text Reading Experience by Detecting and Linking Helpful Entities to Users10.1109/MIS.2018.111144233 ↗
7. Kurita, S., Kawahara, D., and Kurohash, S. (2018). Neural Adversarial Training for Semi-supervised Japanese Predicate-argument Structure Analysis. arXiv.10.18653/v1/P18-1044 ↗
8. Recent Trends in Deep Learning Based Natural Language Processing10.1109/MCI.2018.2840738 ↗
9. Online suicide prevention through optimised text classification10.1016/j.ins.2018.02.014 ↗
10. Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data10.1093/bioinformatics/btg210 ↗
11. Brodeur, J., Coetzee, S., Danko, D., Garcia, S., and Hjelmager, J. (2019). Geographic Information Metadata—An Outlook from the International Standardization Perspective. ISPRS Int. J. Geo-Inf., 8.10.3390/ijgi8060280 ↗
12. Mehmood, A., Jia, S., Mahmood, R., Yan, J., and Ahsan, M. (2019). Non-Stationary Bayesian Modeling of Annual Maximum Floods in a Changing Environment and Implications for Flood Management in the Kabul River Basin, Pakistan. Water, 11.10.3390/w11061246 ↗
13. Gene Selection for Cancer Classification using Support Vector Machines10.1023/A:1012487302797 ↗
14. Sun, D., Zhu, Y., Xu, H., He, Y., and Cen, H. (2019). Time-Series Chlorophyll Fluorescence Imaging Reveals Dynamic Photosynthetic Fingerprints of sos Mutants to Drought Stress. Sensors, 19.10.3390/s19122649 ↗
15. Implications of the Dirichlet assumption for discretization of continuous variables in naive Bayesian classifiers10.1023/A:1026367023636 ↗
16. MLTSVM: a novel twin support vector machine to multi-label learning10.1016/j.patcog.2015.10.008 ↗
17. Wang, A., Ning, A., Chen, G., Lian, L., and Alterovitz, G. (2015, January 2–5). Accelerating Incremental Wrapper Based Gene Selection with K-Nearest-Neighbor. Proceedings of the IEEE International Conference on Bioinformatics & Biomedicine, Belfast, UK.10.1109/BIBM.2014.6999395 ↗
18. Xing, E.P., Jordan, M.I., and Karp, R.M. (July, January 28). Feature Selection for HighDimensional Genomic Microarray Data. Proceedings of the Eighteenth International Conference on Machine Learning, Williamstown, MA, USA.
19. Kim, H., and Jeong, Y.S. (2019). Sentiment Classification Using Convolutional Neural Networks. Appl. Sci., 9.10.3390/app9112347 ↗
20. Wen, Y., Zhang, W., Luo, R., and Wang, J. (2016). Learning text representation using recurrent convolutional neural network with highway layers. arXiv.
21. Kapočiūtė-Dzikienė, J., Damaševičius, R., and Woźniak, M. (2019). Sentiment analysis of Lithuanian texts using traditional and deep learning approaches. Computers, 8.10.3390/computers8010004 ↗
22. Design of Computational Intelligence-Based Language Interface for Human-Machine Secure Interaction
23. Deep Convolution Neural Network and Autoencoders-Based Unsupervised Feature Learning of EEG Signals10.1109/ACCESS.2018.2833746 ↗
24. A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection10.1016/j.neunet.2009.04.003 ↗
25. Johnson, R., and Zhang, T. (2015, January 7–12). Semi-Supervised Convolutional Neural Networks for Text Categorization via Region Embedding. Proceedings of the Advances in neural information processing systems, Montreal, QC, Canada.
26. A deep belief network with PLSR for nonlinear system modeling
27. Accurate Recognition of Words in Scenes without Character Segmentation using Recurrent Neural Network10.1016/j.patcog.2016.10.016 ↗
28. Chelba, C., Norouzi, M., and Bengio, S. (2017). N-gram Language Modeling using Recurrent Neural Network Estimation. arXiv.
29. Deep Convolution Neural Network for Accurate Diagnosis of Glaucoma Using Digital Fundus Images10.1016/j.ins.2018.01.051 ↗
30. Self-Taught convolutional neural networks for short text clustering10.1016/j.neunet.2016.12.008 ↗
31. A Deep Cascade of Convolutional Neural Networks for MR Image Reconstruction10.1109/TMI.2017.2760978 ↗
32. Visual Saliency Detection Based on Multiscale Deep CNN Features10.1109/TIP.2016.2602079 ↗
33. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising10.1109/TIP.2017.2662206 ↗
34. Learning Stylometric Representations for Authorship Analysis10.1109/TCYB.2017.2766189 ↗
35. Improving patch-based scene text script identification with ensembles of conjoined networks10.1016/j.patcog.2017.01.032 ↗
36. Severyn, A., and Moschitti, A. (2015, January 9–13). Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks. Proceedings of the International Acm Sigir Conference, Santiago, Chile.10.1145/2766462.2767738 ↗
37. Conneau, A., Schwenk, H., Barrault, L., and Lecun, Y. (2016). Very Deep Convolutional Networks for Text Classification. arXiv.10.18653/v1/E17-1104 ↗
38. Benchmark Dataset and Method for Depth Estimation from Light Field Images10.1109/TIP.2018.2814217 ↗
39. Considerable improvement of entanglement swapping by considering multiphoton transitions via cavity quantum electrodynamics method10.1142/S0217979218500935 ↗
40. Label Information Guided Graph Construction for Semi-Supervised Learning10.1109/TIP.2017.2703120 ↗
41. Semi-Supervised Text Classification With Universum Learning10.1109/TCYB.2015.2403573 ↗
42. Alom, M.Z., Taha, T.M., Yakopcic, C., Westberg, S., Hasan, M., Esesn, B.C.V., Awwal, A.A.S., and Asari, V.K. (2018). The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches. arXiv.
43. Facial Expression Recognition with Cross-connect LeNet-5 Network
44. Bi, N., Chen, J., and Tan, J. (2019). The Handwritten Chinese Character Recognition Uses Convolutional Neural Networks with the GoogLeNet. Int. J. Pattern Recognit. Artif. Intell., 1940016.10.1142/S0218001419400160 ↗
45. Deep Convolutional Neural Network for Inverse Problems in Imaging10.1109/TIP.2017.2713099 ↗
46. Fei, Y., Wang, K.C.P., Zhang, A., Chen, C., and Li, B. (2019). Pixel-Level Cracking Detection on 3D Asphalt Pavement Images Through Deep-Learning-Based CrackNet-V. IEEE Trans. Intell. Transp. Syst., 1–12.10.1109/TITS.2019.2891167 ↗
47. Hybrid CS-DMRI: Periodic Time-Variant Subsampling and Omnidirectional Total Variation Based Reconstruction10.1109/TMI.2017.2717502 ↗
48. LG-CNN: From local parts to global discrimination for fine-grained recognition10.1016/j.patcog.2017.06.002 ↗
49. Roth, W., and Pernkopf, F. (2018). Bayesian Neural Networks with Weight Sharing Using Dirichlet Processes. IEEE Trans. Pattern Anal. Mach. Intell.
50. Scene Text Detection and Segmentation based on Cascaded Convolution Neural Networks10.1109/TIP.2017.2656474 ↗
51. Liu, H. (2017). Sentiment Analysis of Citations Using Word2vec. arXiv.
52. Rao, A., and Spasojevic, N. (2016). Actionable and Political Text Classification using Word Embeddings and LSTM. arXiv.
53. Johnson, R., and Tong, Z. (2014). Effective Use of Word Order for Text Categorization with Convolutional Neural Networks. IEEE Trans. Pattern Anal. Mach. Intell.10.3115/v1/N15-1011 ↗
54. Kalchbrenner, N., Grefenstette, E., and Blunsom, P. (2014). A Convolutional Neural Network for Modelling Sentences. arXiv.10.3115/v1/P14-1062 ↗
55. Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification10.1016/j.neucom.2015.09.096 ↗
56. Shen, Y., He, X., Gao, J., Deng, L., and Mesnil, G. (2014, January 7–11). Learning Semantic Representations Using Convolutional Neural Networks for Web Search. Proceedings of the International Conference on World Wide Web, Seoul, Korea.10.1145/2567948.2577348 ↗
57. Elevated preoperative plasma fibrinogen level is an independent predictor of malignancy and advanced stage disease in patients with bladder urothelial tumors10.1016/j.ijsu.2016.11.010 ↗
58. Improving handwritten chinese text recognition using neural network language models and convolutional neural network shape models10.1016/j.patcog.2016.12.026 ↗
59. Yan, S., Hardmeier, C., Tiedemann, J., and Nivre, J. (2017). Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF. arXiv.
60. RCV1: A New Benchmark Collection for Text Categorization Research
61. Hierarchical Multi-Label Classification using Fully Associative Ensemble Learning10.1016/j.patcog.2017.05.007 ↗
62. Liu, J., Chang, W.C., Wu, Y., and Yang, Y. (2017, January 7–11). Deep Learning for Extreme Multi-label Text Classification. Proceedings of the International Acm Sigir Conference on Research & Development in Information Retrieval, Tokyo, Japan.10.1145/3077136.3080834 ↗
63. A method for multi-class sentiment classification based on an improved one-vs-one (OVO) strategy and the support vector machine (SVM) algorithm10.1016/j.ins.2017.02.016 ↗