JP6702456B2 - Character shape feature extraction method, character shape feature extraction device, electronic device, and storage medium - Google Patents

Description

The present invention belongs to the field of character processing technology, and more particularly, to a method, apparatus, and electronic device for extracting character shape features.

In the prior art, character shape extraction is usually done by CNN (Convolutional
It is realized by models such as Neural Network) and LSTM (Long Short-Term Memory). However, during the implementation of the present invention, it was found that the character shape feature acquisition performance was low due to the high complexity of these models.

In view of the above problems, the present invention provides a character shape feature extraction method, apparatus, and electronic device for the purpose of improving the extraction performance of character shape features.

In order to solve the above-mentioned problem, in the embodiment of the present invention, first, a step of performing preprocessing on the processed data, a step of acquiring a character shape feature extraction window having a predetermined size, and the character shape feature Extracting the shape features of the character from the preprocessed processed data using the extraction window, wherein the size of the character shape feature extraction window is held constant during the character shape feature extraction. A method for extracting a character shape feature as a feature is provided.

Among them, the step of performing pre-processing on the processed data includes a step of removing noise from the processed data and a step of dividing the processed data from which noise is removed into one or more sentences, and the divided sentences. Is divided into one or more words, and an ID is assigned to the plurality of words.

Further, the step of assigning an ID to the plurality of words is a step of selecting V different words from the plurality of words and configuring a model library using the V different words, where V is a natural number. There is a step, among the plurality of words, a step of assigning a first ID to a first target word in the model library, wherein different first target words have different IDs, and Assigning a second ID different from the first ID to a second target word that is not in the model library, wherein the second target word includes a word that is not in the model library and all second The IDs are the same, including steps.

Further, the step of acquiring the character shape feature extraction window having the predetermined size, P characters are extracted as prefix information from the beginning of a preset word, and S characters are extracted from the end of the preset word. A step of extracting a character as suffix information, wherein P and S are natural numbers, and a step of configuring the character shape feature extraction window using the prefix information and the suffix information.

Further, using the character shape feature extraction window, the step of extracting the shape feature of the character from the preprocessed processing data, from the character set composed of uppercase and lowercase letters of the alphabet, C different characters Choosing as a known character, the step of imparting an N-dimensional display vector to each of the C different characters, N is a natural number, step, and in the third target word of the plurality of words, Assigning a display vector to a first target character that is not in the known characters, wherein the display vector assigned to the first target character is different from the display vector assigned to the C characters, step A step of obtaining a display vector of P characters from the beginning and a display vector of S characters from the end to form a first vector that is a (P+S)*N-dimensional vector; +S)*N rows, a step of obtaining a weighted matrix M including F columns, F is a natural number, step, and multiplying the first vector and the weighted matrix, F-dimensional character And acquiring the shape feature of.

Further, a step of merging the shape feature of the character and the vector feature of the acquired word, the step of inputting the merged vector to the learning model, and training the learning model, the weighting matrix M and/or the display Updating the vector weights.

Next, the embodiment of the present invention uses a preprocessing module for performing preprocessing on the processed data, an acquisition module for acquiring a character shape feature extraction window having a predetermined size, and the character shape feature extraction window. A character shape feature extraction window for extracting character shape features from the preprocessed processed data, wherein the size of the character shape feature extraction window is held constant during the character shape feature extraction. A shape feature extraction device is provided.

Among them, the extraction module acquires a vector feature of the word, a weighting matrix M and a display vector, and the character shape feature extraction device merges the character shape feature and the acquired word vector feature. And a training module that trains the learning model and updates the weighting matrix M and/or the weight of the display vector.

Further, in an embodiment of the present invention, in an electronic device including a processor and a memory, and the computer program instruction is stored in the memory, when the computer program instruction is executed by the processor, the processor processes the processed data. Performing a preliminary process by using the character shape feature extraction window having a predetermined size, and extracting the character shape feature from the preprocessed processed data using the character shape feature extraction window. , And the size of the character shape feature extraction window is held constant during character shape feature extraction.

Finally, an embodiment of the present invention is a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is stored in the processor when the computer program is executed by the processor. A step of performing preprocessing on the processed data, a step of obtaining a character shape feature extraction window having a predetermined size, and a character shape feature from the preprocessed processed data using the character shape feature extraction window. And a step of extracting the character shape feature extraction window, wherein the size of the character shape feature extraction window is held constant during the character shape feature extraction.

In the embodiment of the present invention, the character shape feature extraction window used in the process of extracting the character shape feature from the preprocessed processed data is kept unchanged. That is, the size of the character shape feature extraction window is constant. Therefore, according to the embodiment of the present invention, the performance of character shape feature extraction is improved.

6 is a flowchart showing a character shape feature extraction method according to an embodiment of the present invention. It is a figure which shows the system structure concerning the Example of this invention. It is a figure which shows the hardware concerning the Example of this invention. 6 is a flowchart illustrating a method for extracting character shape features according to an exemplary embodiment of the present invention. It is a figure which shows the character shape feature extraction window concerning the Example of this invention. It is a figure which shows the extraction device of the character shape feature concerning the Example of this invention. It is a block diagram which shows the structure of the pretreatment module concerning the Example of this invention. It is a block diagram which shows the structure of the extraction module concerning the Example of this invention. It is a figure which shows the structure of the extraction device of the character shape feature concerning the Example of this invention. It is a figure which shows the structure of the electronic device concerning the Example of this invention.

Hereinafter, specific embodiments of the present invention will be described in more detail with reference to the drawings and examples. The following examples are intended to illustrate the invention and not limit the scope of the invention.

As shown in FIG. 1, the method for extracting character shape features according to the embodiment of the present invention includes the following steps.

In step 101, preliminary processing is performed on the processed data.

Here, the processed data is arbitrary data, for example, a segment in text on a web page.

In the embodiment of the present invention, the preliminary processing performed on the processed data mainly includes the following steps.

(1) Remove noise from the processed data.

This step is also called data cleaning, and its main purpose is to remove noise from the processed data. Noise includes URLs (Uniform Resource Locator), email addresses, and web-introduced symbols such as "&lt;" and "&gt;" shapes. Here, XML (extensible Markup Language) tags such as "<html>", "<title>", and "<body>" introduced by the Web page are removed, and only the text between the tags is retained.

(2) The processed data from which noise has been removed is divided into one or more sentences, and the divided sentences are divided into one or more words.

Here, using the Python library, the processing data is divided into sentences by the NLTK (Natural Language Toolkit), and these sentences are further divided into words. The word is the minimum unit of data.

(3) Assign IDs to multiple words.

In this step, V different words are selected from the plurality of words, and V different words are used to construct a model library. Here, V is a natural number. Of the plurality of words, if the first target word is in the model library, assign the first ID to the first target word. Different IDs are assigned depending on the first target word. The second target word assigns a second ID to the second target word if it is not in the model library. The second ID is different from the first ID.

The first target word is any one of the obtained words.

In addition, the second target words include words that are not in the model library, and the second target words all have the same ID value. For example, the target word may be a word that is not included in the model library among a plurality of words obtained by dividing, or may be a word that is not included in another model library.

In a practical application, V different words are selected from the obtained plural words to construct a model library. The parameter V is specified by the user. After uniquely assigning IDs to different words, replace the obtained words with IDs. The following are specific methods.

(a) Replace the word in the model library with the corresponding unique ID.

(b) Specify an unused ID as an unknown word ID for a word that does not exist in the model library.

In step 102, a character shape feature extraction window whose size is preset is acquired. In the character shape feature extraction process, the size is kept constant without changing.

In this step, P characters can be extracted as prefix information from the beginning of a preset word, and S characters can be extracted as suffix information from the preset end of the word. Here, P and S are natural numbers. Further, the preset word may be an arbitrary word. Note that P and S can be designated by the user. Then, the character shape feature extraction window is constructed using the prefix information and the suffix information.

In step 103, the character shape feature is extracted from the preprocessed process data using the character shape feature extraction window.

This step mainly includes the following processes.

In step 1031, C different characters are selected from the character set composed of uppercase and lowercase letters of the alphabet to be known characters. Then, an N-dimensional display vector is given to each of these C different characters. Here, N is a natural number.

In actual application, there are 52 alphabetic characters in total, including upper and lower case letters. C characters different from 52 characters are arbitrarily selected as known characters, and an N-dimensional display vector is given to each character. Here, the values of C and N are specified by the user. In it, the display vector of C characters is initialized randomly, and is updated during the training to be performed later. Alternatively, the one-hot vector may be used as it is as the display vector, but the one-hot vector is not involved in the training described later.

In step 1032, for a third target word in the plurality of words, assign a display vector to the first target character if the first target character of the third target word is not a known character. .. Here, the display vector assigned to the first target character is different from the display vectors assigned to the C characters described above.

Here, the third target word is any one of the plurality of words. When the first target character included in the third target word (e.g., alphabets, numbers, symbols, etc.) is not a known character, the first target character is again the display vector assigned to the C characters. Assign different display vectors.

In step 1033, the display vector of P characters from the head and the display vector of S characters from the end are acquired to form a first vector that is a (P+S)*N-dimensional vector.

A display vector of P characters from the beginning and a display vector of S characters from the end are acquired from the assigned display vectors to form a first vector. Here, the first vector is a (P+S)*N-dimensional vector. The character shape feature is extracted from the first vector thus configured. If the word is not long enough to extract the characters, add all 0-dimensional N-dimensional vectors.

In step 1034, the weighting matrix M is acquired. Here, the weight matrix M includes (P+S)*N rows and F columns, where F is a natural number and represents the dimension of the character shape feature, which is a parameter specified by the user. In the weight matrix, the weight values are floating point numbers, obtained by random initialization, and constantly updated in subsequent training.

In step 1035, the first vector and the weighted matrix are multiplied to obtain a character shape feature. Here, the dimension of the character shape feature is F.

In the embodiment of the present invention, in the process of extracting the character shape feature from the preprocessed processed data, the character shape feature extraction window to be used is held unchanged. That is, in this process, the size of the character shape feature extraction window is constant for any word. Therefore, the performance and accuracy of character shape feature extraction can be improved as compared with the embodiment of the present invention. Also, the method according to the embodiment of the present invention can be applied to an even simpler network model. As a result, the difficulty level of character shape feature extraction is reduced, and the reliability is improved.

In order to further enhance the function of the network model and further improve the accuracy of the character shape feature extraction, the method according to the embodiment of the present invention is, in addition to the contents described above, the character shape feature and the acquired word The vector is merged, the merged vector is used as an input of the learning model, the learning model is trained, and the weight matrix M and/or the weight of the display vector is updated. Here, as for the method of acquiring the word vector, the conventional technique can be referred to.

FIG. 2 is a diagram showing a system configuration according to the embodiment of the present invention. Character shape features are used as input to the natural language processing model. The system-wide input is natural text gathered from the Internet, which at the same time contains the noise of web pages that have been substituted into the gathering process.

Here, the data preprocessing module 210 performs preprocessing on the processed data in order to remove noise in the data. The character shape feature extraction module 220 extracts the character shape, and the feature merging module 230 merges the character shape feature with other features and inputs them to the natural language processing system.

FIG. 3 is a diagram showing hardware according to the embodiment of the present invention. As shown in FIG. 3, the hardware according to the embodiment of the present invention includes a network interface 310 that is an interface for connecting to the Internet or another communication network, and an input device 320 that collects an input signal by a user of the system. , A hard disk 330 that stores information such as user logs, a central processing unit (CPU) 340 that executes programs, a storage unit 350 that stores temporary variables during program execution, and information related to the system user. And a display 360 for displaying.

Next, with reference to FIG. 4, extraction of character shape features according to the embodiment of the present invention will be described in detail. It mainly includes the following steps.

In step 401, preliminary processing is performed on the data. Specifically, the following steps are included.

(1) Data washing Remove noise from processed data. Noise includes URLs, email addresses, and symbols introduced by web pages such as "&lt;" or "&gt;" in form. Also, xml tags such as "<html>", "<title>" and "<body>" introduced by web pages are removed, and only the text between the tags is retained.

(2) Data division
Using python library, the processed data is divided into sentences by NLTK, and further divided into words.

(3) Data ID conversion
From the words divided in (2), 30,000 different words are selected as model words. Assign a unique ID to each word. Of the 30,000 words, the first word is assigned an ID of 1, the second word is assigned an ID of 2, and the following analogy is made. In addition, 0 is the ID of the unknown word. Then, it replaces with the ID corresponding to the divided word.

In step 402, character shape features are extracted.

As shown in FIG. 5, four alphabetic characters are extracted from the front part of the word information. Prefix character shape information is extracted from these four characters, four alphabetic characters are extracted from the end of the word, and suffix character shape information is extracted from these four characters. The extracted eight characters are connected in alphabetical order to form a character shape feature extraction window. By doing so, the character shape feature extraction window is infotion or tioninfo.

A set consisting of all uppercase and lowercase letters is selected as the known character set. It contains 52 elements. Assign a 53-dimensional display vector to each character in the known character set. These display vectors take values in one-hot form. That is, each weight of the vector is composed of 0 and 1, and only one element is 1, but the other elements are 0. A vector having 1 in the first element and 0 in the other elements is set as the display vector of the unknown character. In addition, the display vector of the first character in the known character set is configured with 1 in the second element and 0 in the other elements. This is an analogy.

Further, in the character shape extraction window, display vectors corresponding to characters are connected to form a (4+4)*53=42 four-dimensional vector. If the word is less than 4 characters and the length is insufficient, add it with 53-dimensional all 0 vector.

Construct a weighted matrix M of floating point numbers. The matrix has 424 rows and 256 columns. The weights in the matrix are obtained by random initialization and are constantly updated in subsequent training. Then, a vector of (4+4)*53=424 dimensions is multiplied by the weight matrix M to obtain a 256-dimensional vector, that is, a character shape feature vector.

In step 403, a sample vector is generated.

The obtained 256-dimensional character shape feature vector is connected to other features including the word vector and is input to the model to train the model. Update the weighting matrix M accordingly when training the model.

As described above, the embodiment of the present invention can improve the performance and accuracy of character shape feature extraction. Further, the method according to the embodiment of the present invention can be applied to a simple network model, which reduces the difficulty level of character shape feature extraction and improves reliability.

An apparatus 600 for extracting character shape features according to an embodiment of the present invention, as shown in FIG. 6, includes a preliminary processing module 601 for performing a preliminary process on processing data, and a character shape feature extraction unit having a preset size. An acquisition module 602 that acquires a character shape feature extraction window that is held constant at, and an extraction module 603 that extracts a character shape feature from preprocessed processed data using the extracted character shape feature extraction window, including.

Among them, as shown in FIG. 7, the pre-processing module 601, the removal sub-module 6011 for removing noise of the processing data, and the processing data from which noise has been removed is divided into one or a plurality of sentences. And a sub-module 6013 for assigning IDs to the plurality of words, and a sub-module 6012 for dividing the sentence into one or more words.

Specifically, the allocation sub-module 6013 selects V different words from the plurality of words to form a model library. Here, V is a natural number. If the first target word of the plurality of words is in the model library, a first ID is assigned to the first target word, and the ID is different depending on the first target word. A second ID is assigned to the second target word when the second target word is in the model library, and the second ID is different from the first ID and is not a word that is not in the model library. Including, the IDs of all the second target words are the same.

Specifically, the acquisition module 602 extracts P alphabetic characters from the beginning of a preset word as prefix information and extracts S alphabetic characters from the end of a preset word as suffix information. .. Among them, P and S are natural numbers. The character shape feature extraction window is configured using the prefix information and the suffix information.

Here, as shown in FIG. 8, the extraction module 603, specifically, the first allocation sub-module 6031, the second allocation sub-module 6032, the first acquisition sub-module 6033, the second acquisition sub-module 6034, and an extraction sub-module 6035.

The first allocation sub-module 6031 is a known character by selecting C different characters from the word set consisting of uppercase and lowercase letters of the alphabet, and specify the N-dimensional display vector to each of the C different characters, Here, N is a natural number.

A second assignment sub-module 6032 displays for a third target word in a plurality of words the first target character in the third target word if the first target character is not in the known characters. Assigning a vector, in which the display vector assigned to the first target character is different from the display vector of the C characters.

The first acquisition sub-module 6033 acquires a display vector of P characters from the head and acquires a display vector of S characters from the end to form a first vector. Among them, the first vector is a (P+S)*N-dimensional vector.

The second acquisition sub-module 6034 acquires the weighting matrix M. The weighting matrix M includes (P+S)*N rows and F columns therein, and F is a natural number.

The extraction sub-module 6035 multiplies the first vector and the weighted matrix to obtain a character shape feature. In that, the dimension of the character shape feature is F.

The operation principle of the device according to the embodiment of the present invention can be referred to the description of the above-described method embodiment.

In the embodiment of the present invention, in the process of extracting the character shape feature from the preprocessed processed data, the character shape feature extraction window used is held unchanged. That is, during that time, the size of the character shape feature extraction window is constant for any word. Therefore, the performance and accuracy of character shape feature extraction can be improved as compared with the embodiment of the present invention. Also, the method according to the embodiment of the present invention can be applied to an even simpler network model. As a result, the difficulty level of character shape feature extraction is reduced, and the reliability is improved.

In the device according to the embodiment of the present invention, in order to further enhance the function of the network model and further improve the accuracy of character shape feature extraction, in addition to the above description, as shown in FIG. The character shape feature acquired by the feature extraction module 603 and the word vector feature acquired by the extraction module 603 are merged, and the merged module 604 that inputs the merged vector to the learning model and the learning model are trained, A weighting matrix M and/or a training module 605 for updating the weights of the display vector.

The embodiment of the present invention, as shown in FIG. 10, is provided with an electronic device 1000 including a processor 1001 and a memory 1002, a computer program command is stored in the memory 1002, the electronic device 1000, a computer program command When executed by the processor 1001, the processor 1001 performs a preliminary process on the processed data, and a character shape feature extraction window whose size is preset and is held constant during character shape feature extraction. The step of acquiring and the step of extracting the character shape feature from the preprocessed process data using the character shape feature extraction window can be executed.

Further, as shown in FIG. 10, the electronic device 1000 further includes a network interface 1003, an input device 1004, a hard disk 1005, and a display device 1006.

The above interfaces are connected to the respective devices via the bus architecture. A bus architecture is a bus and bridge that can include any number of interconnects. Specifically, one or a plurality of central processing units (CPU) represented by the processing device 1001 and various circuits of one or a plurality of memories represented by the memory 1002 are connected. Also, various other circuits such as an external device, a regulator, and a power management circuit are connected from the bus architecture. In this way, these devices are communicatively connected by the bus architecture. In addition to the data bus, the bus architecture includes a power bus, a control bus and a status signal bus. These are known techniques in the field of the present invention, and detailed description thereof will be omitted.

The network interface 1003 is an interface that is connected to a network (for example, the Internet or LAN), receives related data from the network, and stores the related data in the hard disk 1005.

The input device 1004 is a unit that receives various commands input by the user, transmits the commands to the processor 1001, and causes the processor 1001 to execute the commands. Further, the input device 1004 includes a keyboard, a click means (for example, a mouse, a trackball, a touch board), a touch panel, a touch screen, or the like.

The display device 46 is means for displaying the result of execution of the command by the processor 42.

The memory 1002 is a memory for storing programs and data necessary for executing the operating system, and data such as intermediate results in the calculation process of the processor 1001.

The memory 1002 according to the embodiment of the present invention is a volatile memory or a non-volatile memory, or a memory including both volatile and non-volatile memory. Among them, the non-volatile memory is ROM, PROM, EPROM, EEPROM, flash memory. Volatile memory is RAM and it is used as an external cache. However, the memory 1002 used in the devices and methods described herein is not limited to these memories and may be any other suitable type of memory.

In some embodiments, memory 1002 stores operating system 10021 and application programs 10010, which are executable modules or data structures or their submodules or expansion modules.

The operating system 10021 includes various system programs, for example, a framework layer, a core library layer, and a driving layer, and is used to realize various core tasks and hardware-based tasks. The application program 10010 includes various application programs, for example, a web browser (Browser) and the like, and is for realizing various application tasks. A program for executing the method according to the present embodiment is included in the application program 10010.

When an application program stored in the memory 1002, for example, a program or an instruction of the application program 10010 is executed by the processor 1001, the processor 1001 performs a preliminary process on the processed data, and a character shape feature having a preset size. Performing a step of obtaining a character shape feature extraction window that is constant during extraction, and a step of extracting character shape features from preprocessed processed data using the extracted character shape feature extraction window ..

The method according to the embodiments of the present invention described above is applied to or implemented by the processor 1001. The processor 1001 is an integrated circuit board capable of processing signals. Each step of the above method is implemented by an integrated logic circuit which is hardware in the processor 1001 or a command in software form. The processor 1001 is a general-purpose processor, a digital signal processor (DSP), a dedicated integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, The methods, steps, and logic boxes disclosed in the embodiments of the present invention can be implemented or executed. A general-purpose processor can be a microprocessor or any conventional processor. Each step of the method according to the embodiment of the present invention may be realized by being executed by a decoder which is hardware, or may be realized by a combination of hardware and software which can be executed by the decoder. The software modules are stored in storage media mature in the field, such as random memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers. The processor 1001 reads the information from the memory 1002 having a storage medium in which this software is stored, and implements the steps of the above method according to the hardware.

The embodiments described above are implemented by hardware, software, firmware, middleware, microcode, or a combination thereof. Among them, in terms of hardware implementation, the processing unit is one or more dedicated integrated circuits (ASIC), digital signal processor (DSP), digital signal processor (DSPD), programmable logic circuit (PLD), field. It may be implemented by a programmable gate array (FPGA), a general purpose processor, a controller, a microcontroller, a microprocessor, another electronic unit that performs the functions of the present invention, or a combination thereof.

Regarding software implementation, the above technology is implemented by modules (for example, processes, functions, etc.) that implement the functions described above. The software code is stored in memory and executed by the processor. The memory is realized inside or outside the processor.

Specifically, the processor 1101 reads the computer program to remove noise from the processed data, divides the processed data from which noise has been removed into one or more sentences, and divides the divided sentence into one sentence. Alternatively, the steps of dividing into a plurality of words and assigning an ID to the plurality of words are executed.

Specifically, the processor 1101 reads a computer program, selects V different words from the plurality of words, and configures a model library using the V different words, where V is a natural number. Step, and among the plurality of words, if the first target word is in the model library, assign a first ID to the first target word, and give a different ID depending on the first target word. And a second ID is assigned to the second target word if the second target word is not in the model library, the second ID being different from the first ID and the second target word being The word is not in the model library and all the second IDs are the same.

Specifically, the processor 1101 reads a computer program, extracts P characters from the beginning of a preset word as prefix information, and S characters from the end of a preset word as suffix information. And extracting P and S are natural numbers, and constructing the character shape feature extraction window using the prefix information and the suffix information.

Further, specifically, the processor 1101 reads the computer program, selects C different characters from the character set composed of uppercase and lowercase letters of the alphabet, and makes them known characters, and these C different Each character is given an N-dimensional display vector, where N is a natural number, and for a third target word in a plurality of words, the first target character of the third target word is not a known character. In this case, the step of assigning a display vector different from the display vector assigned to the C characters to the first target character, the display vector of the P characters from the beginning and the display of the S characters from the end. Obtaining a vector to form a first vector that is a (P+S)*N dimensional vector, and obtaining a weighting matrix M containing (P+S)*N rows and F columns, where F is a natural number. Performing a step and a step of multiplying the first vector and the weight matrix to obtain an F-dimensional character shape feature.

Specifically, the processor 1101 reads the computer program, merges the character shape feature and the acquired vector of words, and inputs the learning vector to the merged vector, and training the learning model. And updating the weighting matrix M and/or the weight of the display vector.

The embodiments of the present invention also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor performs a preliminary process on the processed data, and a size is preset and a character shape is set. Performing a step of obtaining a character shape feature extraction window that is constant during feature extraction, and a step of extracting a character shape feature from preprocessed processed data using the extracted character shape feature extraction window To be made.

Here, the step of performing preprocessing on the processed data includes a step of removing noise from the processed data and a step of dividing the processed data from which noise has been removed to obtain one or more sentences, Is divided into one or more words, and an ID is assigned to the plurality of words.

Among them, the step of assigning an ID to the plurality of words configures a model library by selecting V different words from the plurality of words, where V is a natural number, and among the plurality of words, If the first target word is in the model library, the first target word is assigned a first ID, and the ID has different steps depending on the first target word, and the second target word is in the model library. A second ID is assigned to the second target word, wherein the second ID includes a word that is different from the first ID and is not in the model library, and all the second target words With the same ID.

Further, in the step of acquiring a character shape feature extraction window of which size is preset and constant during character shape feature extraction, P characters from the beginning of a preset word are extracted as prefix information. , Extracting S characters from the end of a preset word as suffix information, P and S being a natural number, and configuring the character shape feature extraction window using the prefix information and suffix information And, including.

Using the extracted character shape feature extraction window, the step of extracting character shape features from the preprocessed processing data, from the character set consisting of uppercase and lowercase letters of the alphabet, C different characters To a known character, and each of these C different characters is given an N-dimensional display vector, where N is a natural number, and for the third target word in a plurality of words, the third The first target character of the target word of, when not in a known character, assigns a display vector to the first target character, the display vector assigned to the first target character is the C characters described above. The first vector which is a step different from the assigned display vector and the display vector of the P characters from the beginning and the display vector of the S characters from the end is the (P+S)*N dimensional vector. To obtain a weighting matrix M including (P+S)*N rows and F columns, F being a natural number, and multiplying the first vector by the weighting matrix to obtain F Obtaining a dimensional character shape feature.

Further, the method, the step of merging the character shape features and the obtained vector of words, the merged vector as an input of the learning model, and training the learning model, weighted matrix M and / or display. Update vector weights.

It is readily conceivable that the method and apparatus disclosed by the embodiments of the present invention can be implemented in other forms. For example, the apparatus described above is only schematic. For example, the division of the unit described above is only an example of logical function allocation, and another division method may be adopted when actually realizing it. For example, multiple units or modules may be combined, integrated into another system, or some functions may be omitted or not performed. Note that the mutual connection, the direct connection, or the communicable connection shown or displayed above is a connection through an interface. Indirect or communicable connections between devices or units may be electrical or mechanical or other forms of connection.

The functional units according to the embodiments of the present invention may be integrated into one processing unit, may be physically independent, or may be integrated into two or more as one unit. The integrated means may be realized in the form of hardware or a functional unit composed of hardware and software.

The aggregation unit realized by the functional means by the software is stored in a computer-readable storage medium. The steps included in the method according to the embodiment of the present invention are executed by causing a computer (for example, a PC, a server, or a network device) to execute the software instructions stored in the storage medium. The storage medium is a medium such as a USB, a hard disk, a ROM (Read Only Memory), a RAM (Random Access Memory), a CD or a DVD that can store programming codes.

Finally, the preferred embodiment of the present invention described above can be further improved or changed by a person skilled in the art without departing from the spirit of the present invention. However, these improvements and modifications should be considered as the protection scope of the present invention.

Claims (10)

A method for extracting character shape features,
The processor
Preprocessing the processed data,
Obtaining a character shape feature extraction window having a predetermined size,
Using said character shape feature extraction window, perform the steps of extracting shape features of a character from the pretreated processed data, and
The size of the character shape feature extraction window is held constant during character shape feature extraction ,
The step of performing preprocessing on the processed data is
Removing noise from the processed data,
Dividing the processed data from which noise has been removed into one or more sentences, and dividing the divided sentences into one or more words;
Assigning IDs to the plurality of words,
The step of acquiring the character shape feature extraction window having the predetermined size includes
This is a step of extracting P characters from the beginning of a preset word as prefix information and extracting S characters from the end of the preset word as suffix information, where P and S are natural numbers. , Step,
A step of configuring the character shape feature extraction window using the prefix information and the suffix information, the method for extracting character shape features. Assigning IDs to the plurality of words,
Selecting V different words from the plurality of words and configuring a model library using the V different words, where V is a natural number, and
Of the plurality of words, assigning a first ID to a first target word in the model library, wherein different first target words have different IDs, and
Assigning a second ID different from the first ID to a second target word that is not in the model library, wherein the second target word includes a word that is not in the model library and all second The character shape feature extraction method according to claim 1 , further comprising: a step having the same ID. Using the character shape feature extraction window, the step of extracting the character shape feature from the preprocessed processing data,
From the character set composed of uppercase and lowercase letters of the alphabet, C different characters are selected as known characters, the step of assigning N-dimensional display vector to each of the C different characters, N Is a natural number, and
In a third target word of a plurality of words, a step of assigning a display vector to the first target character not in the known character, the display vector assigned to the first target character is the C Different from the display vector given to the character
Obtaining a display vector of P characters from the beginning and a display vector of S characters from the end, and forming a first vector that is a (P+S)*N dimensional vector,
(P+S)*N rows, a step of obtaining a weighting matrix M including F columns, where F is a natural number, and
The method for extracting character shape features according to claim 1 , further comprising: multiplying the first vector by the weight matrix to obtain shape features of an F-dimensional character. Merging the shape feature of the character and the vector feature of the acquired word, and using the merged vector as an input of the learning model,
Training the learning model and updating the weights of the weighting matrix M and/or the display vector. 4. The method of claim 3 , further comprising: A pre-processing module for pre-processing the processed data,
An acquisition module for acquiring a character shape feature extraction window having a predetermined size,
An extraction module for extracting character shape features from preprocessed processed data using the character shape feature extraction window;
The size of the character shape feature extraction window is held constant during character shape feature extraction ,
The pretreatment module is
Remove noise from the processed data,
Divide the processed data with noise removed into one or more sentences, divide the divided sentences into one or more words,
Assigning IDs to the plurality of words,
The acquisition module is
Extracting P characters from the beginning of the preset word as prefix information, extracting S characters from the end of the preset word as suffix information,
Using the prefix information and the suffix information, configure the character shape feature extraction window,
P and S are natural numbers . Character shape feature extraction device. The pre-processing module, when assigning IDs to the plurality of words,
Select V different words from the plurality of words, configure a model library using the V different words,
Of the plurality of words, assign a first ID to the first target word in the model library,
Assigning a second ID different from the first ID to a second target word that is not in the model library,
V is a natural number,
Different first target words have different IDs,
The character shape feature extracting apparatus according to claim 5, wherein the second target words include words that are not in the model library, and all the second IDs are the same. The extraction module is
From the character set consisting of uppercase and lowercase letters of the alphabet, select C different characters as known characters, and give each of the C different characters an N-dimensional display vector,
Assigning a display vector to a first target character that is not in the known characters in a third target word of the plurality of words,
Obtaining the display vector of P characters from the beginning and the display vector of S characters from the end, forming a first vector that is a (P+S)*N dimensional vector,
(P+S)* Get a weighted matrix M containing N rows and F columns,
Multiplying the first vector and the weighted matrix to obtain shape features of the F-dimensional character,
N is a natural number,
The display vector assigned to the first target character is different from the display vector assigned to the C characters,
The character shape feature extracting apparatus according to claim 5, wherein F is a natural number. The extraction module obtains vector features of words, a weighting matrix M and a display vector,
The character shape feature extraction device,
A merge module that merges the shape feature of the character and the vector feature of the acquired word, and uses the merged vector as the input of the learning model;
The apparatus for extracting character shape features according to claim 7, further comprising: a training module that trains the learning model and updates the weight matrix M and/or the weight of the display vector. In an electronic device comprising a processor and a memory, the computer program instructions being stored in the memory, in the processor, when the computer program instructions are executed by the processor,
Preprocessing the processed data,
Obtaining a character shape feature extraction window having a predetermined size,
Using the character shape feature extraction window, to extract the shape feature of the character from the preprocessed processing data,
The size of the character shape feature extraction window is held constant during character shape feature extraction ,
The step of performing preprocessing on the processed data is
Removing noise from the processed data,
Dividing the processed data from which noise has been removed into one or more sentences, and dividing the divided sentences into one or more words;
Assigning IDs to the plurality of words,
The step of acquiring the character shape feature extraction window having the predetermined size includes
This is a step of extracting P characters from the beginning of a preset word as prefix information and extracting S characters from the end of the preset word as suffix information, where P and S are natural numbers. , Step,
Configuring the character shape feature extraction window using the prefix information and the suffix information . A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor,
Preprocessing the processed data,
Obtaining a character shape feature extraction window having a predetermined size,
Using the character shape feature extraction window, to extract the shape feature of the character from the preprocessed processing data,
The size of the character shape feature extraction window is held constant during character shape feature extraction ,
The step of performing preprocessing on the processed data is
Removing noise from the processed data,
Dividing the processed data from which noise has been removed into one or more sentences, and dividing the divided sentences into one or more words;
Assigning IDs to the plurality of words,
The step of acquiring the character shape feature extraction window having the predetermined size includes
This is a step of extracting P characters from the beginning of a preset word as prefix information and extracting S characters from the end of the preset word as suffix information, where P and S are natural numbers. , Step,
A step of configuring the character shape feature extraction window using the prefix information and the suffix information, the storage medium.

Images