2.1

Coreference resolution

There are also three types of research on coreference disambiguation, rule-based research methods, machine learning-based research methods, and deep learning-based research methods. Hobbs’ plain algorithm adds semantic knowledge to the rules, which has a great improvement effect on the model¹. However, rule-based methods have some general problems, which are complicated to implement, prone to errors, poor generalization ability, and the model effect is limited by the formulation of rules. With the rise of machine learning algorithms, scholars have tried to use machine learning methods for the task of coreferential disambiguation, using machine learning methods to predict output information by using partial input information. soon et al. used partial semantic features as input to the model for learning to complete coreferential disambiguation²; Luo et al. applied Bell trees to obtain referential features in the representation of phrases³, but the machine learning methods also suffer from The quality of the input features suffers from the dependence on the set threshold, among other problems. Thus, researchers have applied the emerging deep learning techniques in coreference disambiguation by applying linear transformations to the intrinsic logic in the data using deep learning algorithms to abstract more implicit information, and then fitting and optimizing them to capture more over-dimensional features such as utterance information, semantic information, structural information, and dependency information in the dataset, reducing the dependence on manual input features, and also having strong generalization ability to make the models to be used in different tasks. Neural network models such as CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory Neural Network) have superb learning and characterization capabilities, which provide a good basis for the study of co-referential disambiguation. Wiseman et al. used RNN (Recurrent Neural Network) to learn clustering of entity pointing object words to solve the pronoun referential disambiguation problem⁴; Clark et al. used reinforcement learning to construct a co-referential disambiguation system⁵. Lee et al. proposed an end-to-end neural network model for denotational disambiguation without using artificial feature inputs, incorporating attention mechanisms and long- and short-term memory networks to achieve better results⁶.

2.2

Dependency analysis

Dependency analysis is divided into transfer-decoded dependency analysis and graph-decoded dependency analysis, and graph-decoded dependency analysis is an important method of dependency analysis, which can find the global optimal dependency syntax tree. The analysis algorithm finds the optimal dependency tree based on the score of each dependency tree. Deep graph decoding dependency analysis is the use of deep learning techniques for syntactic analysis. deep learning applications for syntactic analysis are relatively recent. Collobert et al. used deep learning for discriminative syntactic analysis, decomposed the analysis tree of the sentence using a deep convolutional model, and then analyzed each layer of the tree⁷; Durrett et al. combined field random fields and neural network models, and then used this method for syntactic analysis⁸; Zheng et al. proposed convolutional neural network for syntactic analysis, which was based on word for sentence analysis, and achieved good results⁹; Chen et al. proposed two recurrent neural networks and used them on transfer-based dependent syntactic analysis for dense feature learning to improve the performance of the model¹⁰.

3.1

Double-layer label

Traditional coreference elimination uses single-layer tags or does not use any tags for syntactic analysis and semantic analysis, and eliminates misreference and misassociation through simple syntactic information and partial semantic information, which can be applied to Chinese text information extraction in general fields. Moreover, the presence of a large number of long continuous entities in the power safety text will also have a direct impact on the extraction results.

In this paper, we use two layers of tags to label the relationship of sentence entities. The first layer of tags uses sentence component tags to label the components in a sentence, which focuses on the component information of the sentence; the second layer of tags labels the position of words, which focuses on the position information of words in the sentence.

For a sentence, tag one labels the entity and relation categories of the sentence, and tag two labels the position information of the entity relation in the sentence. For the second layer of labels, two labeled words are considered as adjacent in the position of the sentence when there is no valid adjacent label in the middle of the two valid adjacent labels.

3.2

Dependency analysis

Dependency syntactic analysis is different from the traditional compositional syntactic analysis in that the focus of dependency legal analysis is not on the components of the phrase, but on the direct use of the words themselves and the binary dependencies between them, enabling a more direct analysis of the subject-predicate and other components of the sentence, and the results obtained using dependency syntactic analysis show the relationships between words more directly.

Syntactic relations are the basis of dependency relations, which are binary relations between words, and each dependency relation is composed of the central word and its dependencies, and this kind of is good for distinguishing between long continuous entities and non-continuous entities with unknown referents that appear in the entity relations extracted from the power text.

Dependency syntax considers the verb in the predicate as the center of a sentence, and the words of other components are directly or indirectly related to the central word, and dependency syntax analysis analyzes the sentence into a dependency syntax tree, which describes the dependency relationship between each word and also points out the syntactic collocation relationship of the words associated with the semantics

Figure 1.

Model structure diagram.

In this paper, we use the set of candidate entity relations to be disambiguated by entity relation extraction, and then use the graph decoding dependency analysis based on the deep learning model to optimize and reason about the set of entity relations to be disambiguated by using the position information of the fused entity relations in the sentence, and then use the scoring function to rank and determine the disambiguation result.

The model processes the sentences with extracted entity relations with the input sequence x=(w₀,w₁,w₂,…,w_n) (w0 is the added word root ROOT), τ(x) represents the set of all possible dependency trees of the sentence, and the process of dependency analysis of sentences is shown in Equation 1.

The process of dependency analysis is to find the optimal dependency tree. Sore(x,y) is the scoring function for the dependency tree, which is usually obtained by decomposing the subgraphs c of the dependency tree and scoring them independently, and summing the independent scores.

The scoring model uses a multilayer perceptron (MLP), and the graph decomposition method uses a first-order decomposition strategy to decompose the dependency tree into dependency edges.

This Φ represents the characteristics of dependent edges, using a low-dimensional dense embedding representation, and θ is a network parameter model, the specific process is shown in Equation 4-8..

The and is the hidden layer parameter,the and is the output layer parameter, and the value of d is 0 or 1. When the value is 0, it means the left-dependent edge parameter, and when the value is 1, it is the right-dependent edge parameter. The computed value is a multidimensional vector, and the number of dimensions depends on the number of dependent labels. The value of the i dimension of the vector is the score of the corresponding dependent edge lb_i, as shown in Equation 9.

The long short-term memory network is introduced to learn the features using a bidirectional long short-term memory network, and the structure of the long short-term memory network is shown in Figure 2.

Figure 2.

Long short-term memory network model.

For a sequence of vectors X=(x₁,x₂,…,x_n), each moment corresponding input has a corresponding output vector h_i(i=1,2,…,n), recursively learning the sequence information as shown in Equation 10.

For the embedding vectors e_wi and e_ti of words and word categories in the sentences are spliced and transformed into input vectors by a single-layer ReLU network, and then the long-short time encoding vectors in both forward and reverse directions are summed up¹¹, and the output of the computationally shared long short-term memory model is obtained, and the output of the bidirectional long short-term memory network is represented by v_i, as shown in Equations 11-12.

By learning the features, we are able to use not only the information in the dependency library, but also the contextual information of the sentence to derive the dependency of entity relations in the sentence. When two entities have the same dependency relationship in the sentence adjacent to each other, it is determined that there is a coreference relationship between the two entities and the merging between entities is performed.