Chinese Unknown Word Recognition for PCFG-LA Parsing

Abstract EN

This paper investigates the recognition of unknown words in Chinese parsing.

Two methods are proposed to handle this problem.

One is the modification of a character-based model.

We model the emission probability of an unknown word using the first and last characters in the word.

It aims to reduce the POS tag ambiguities of unknown words to improve the parsing performance.

In addition, a novel method, using graph-based semisupervised learning (SSL), is proposed to improve the syntax parsing of unknown words.

Its goal is to discover additional lexical knowledge from a large amount of unlabeled data to help the syntax parsing.

The method is mainly to propagate lexical emission probabilities to unknown words by building the similarity graphs over the words of labeled and unlabeled data.

The derived distributions are incorporated into the parsing process.

The proposed methods are effective in dealing with the unknown words to improve the parsing.

Empirical results for Penn Chinese Treebank and TCT Treebank revealed its effectiveness.