Convolutional neural network with contextualized word embedding for text classification

Gaoyang Fan; Cui Zhu; Wenjun Zhu

doi:10.1117/12.2544614

27 November 2019 Convolutional neural network with contextualized word embedding for text classification

Gaoyang Fan, Cui Zhu, Wenjun Zhu

Proceedings Volume 11321, 2019 International Conference on Image and Video Processing, and Artificial Intelligence; 1132126 (2019) https://doi.org/10.1117/12.2544614
Event: The Second International Conference on Image, Video Processing and Artifical Intelligence, 2019, Shanghai, China

Abstract

Text classification is a fundamental task in natural language processing. This task is widely concerned and applied. However, previous methods mainly use traditional static word embedding, but static word embedding could not deal with the problem of polysemy. For this reason, we propose to utilize contextualized BERT word embedding to effectively encode the input sequence and then use the temporal convolutional module which simply computes a 1-D convolution to extract high-level features, finally, the max-pooling layer retains the most critical features for text classification. We conduct experiments on six commonly used large-scale text categorization datasets, including sentiment analysis, problem classification and topic classification tasks. Due to the limitation of BERT processing long text, we propose an effective truncation method. Experimental results show that our proposed method outperforms previous methods.

Citation Download Citation

Gaoyang Fan, Cui Zhu, and Wenjun Zhu "Convolutional neural network with contextualized word embedding for text classification", Proc. SPIE 11321, 2019 International Conference on Image and Video Processing, and Artificial Intelligence, 1132126 (27 November 2019); https://doi.org/10.1117/12.2544614

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available