CN115731550A - Deep learning-based automatic drug specification identification method and system and storage medium - Google Patents

Abstract

The invention provides a method, system and storage medium for automatic recognition of drug instructions based on deep learning. The method of the present invention includes: acquiring the image of the drug instruction manual, preprocessing the acquired image based on the image processing method, and extracting the effective font area; based on the extracted effective font area, using a segmented recognition method to initially identify the text information; The text information of the frequent word training is used to optimize the recognition of the text information initially recognized, and obtain the optimized recognition result of the characters. Using the deep learning-based automatic recognition method for drug instructions provided by the present invention, it is possible to automatically detect whether the key feature information such as the drug name, approval number, approval and modification date uploaded by each pharmaceutical company to the electronic drug instructions is up to standard, so it can Establish a real-time query system for manuals in the hospital, allowing front-line medical workers to query complete and up-to-date scanned manuals at any time, reducing manpower and material resources while greatly improving work efficiency.

Description

A method, system and storage for automatic recognition of drug instructions based on deep learning medium

technical field

The present invention relates to the technical field of image recognition, in particular, to an automatic recognition method, system and storage medium for drug instructions based on deep learning.

Background technique

Optical Character Recognition (OCR) refers to the process in which electronic devices (such as scanners or digital cameras) check characters printed on paper, and then use character recognition methods to translate shapes into computer text. The wide application of digital cameras and flatbed scanners has greatly promoted the development of optical character recognition technology. The existing technologies are mainly divided into two categories: one is based on traditional image processing methods, and the other is based on deep neural network learning strategies. For the automatic management system of drug instructions, the existing methods have the following defects:

(1) Functional defects: the existing technology cannot automatically identify key feature information such as the title, approval number, approval date, and revision date in the instruction manual, and the management of the instruction manual needs to rely on manual review;

(2) Technical defect 1: The quality of scanned documents is uneven: Some scanned documents may have problems such as unclear or deformed text, document tilt, and uncertain scanning direction (horizontal, vertical), etc. The existing text recognition algorithm does not solve the above problems. targeted research;

(3) Technical defect 2: The common names or active ingredients of drugs are mostly transliterated from Western languages, and are often distinguished by uncommon words. These uncommon words are often complicated in structure, which increases the difficulty of correct identification. These uncommon words are often Combined with other words to express a fixed chemical structure name; in addition, drug names usually contain fixed vocabulary, such as injection, oxyfluoride, silicone oil, compound, etc. For these two types of names, if single-character recognition is used, the error rate is often high due to ignoring the fixed collocation between characters.

Contents of the invention

According to the technical problems raised above, a method, system and storage medium for automatic recognition of drug instructions based on deep learning are provided. The present invention mainly uses data-driven thought and deep learning technology to realize high-precision extraction and recognition of Chinese characters and numbers.

The technical means adopted in the present invention are as follows:

A method for automatic identification of drug instructions based on deep learning, including:

Obtain the image of the drug instructions, preprocess the acquired image based on the image processing method, and extract the effective font area;

Based on the extracted effective font area, the text information is initially recognized by the segmented recognition method;

Combined with the text information of the high-frequency word training, the text information of the preliminary recognition is optimized and recognized, and the optimized recognition result of the characters is obtained.

Further, the preprocessing of the acquired image based on the image processing method specifically includes: scanning image enhancement processing, scanning main direction correction processing, image tilt direction correction processing, text information area positioning processing, character area segmentation processing, independent Character segmentation processing, and font correction processing.

Further, the scanning image enhancement processing, the main direction correction processing of the scanned document, the image oblique direction correction processing, the image oblique direction correction processing, the text information area positioning processing, the character area segmentation processing, the independent character segmentation processing, and the font correction processing, Specifically include:

The scanning image enhancement process includes: using a weighted average method to grayscale the image; using mean filtering to linearly filter the image;

The main direction correction processing of the scanned part includes: extracting the length and width characteristics of the scanned part, and projecting the gray value of the image into two directions respectively to obtain the projection features, and judging the main direction of the scanned part in combination with the prior characteristics of the main direction;

The image inclination direction correction process includes: utilizing the Radon transform to estimate the inclination angle of the image, and projecting the image space into the polar coordinate space using the following formula:

A point in polar coordinates is equivalent to a straight line corresponding to two points in the image space. The corresponding line in the image space is determined by the cumulative peak value of the point set in the polar coordinate space. Since the polar coordinate itself contains the inclination angle θ, it is determined according to the cumulative peak value of the point set slope;

The text information area positioning processing includes: performing morphological expansion operation on the image to reduce the gap between adjacent strokes and adjacent characters; extracting the connected domain of the image and merging similar areas; using projection method to perform horizontal projection Histogram to obtain the projection features; for the characteristics of the drug name part of the drug instructions with the largest font and all in the dark background area, select the area with the largest character code and the largest color block projection value as the drug name image area; for the approval date and The date of modification is at the top of the file and the text is sparse. Select the color block projection value at the top of the image that is less than a certain threshold as the image area of the approval date and modification date; identify the brackets and the keywords in the brackets, so as to approve Locate the area where the document number is located;

The character area segmentation processing includes: making a horizontal projection histogram on the selected approval date or approval number image area, the line characters present a peak on the histogram, and the line interval presents an obvious trough on the histogram, according to Divide at the trough to obtain the divided approval number, approval date and modification date;

In the independent character segmentation process, a longitudinal projection histogram is made for the approval date and modification date, each line of the approval number and the drug name area, and each character lattice presents a peak on the histogram, and the character gap presents an obvious trough on the histogram The form is divided according to the trough to obtain the approval date, modification date, approval number, and the numbers, Chinese characters and symbols of the drug name;

Described glyph correction processing comprises: for the locality of font deformation, carry out glyph correction respectively to each row of characters; Utilize Hough transform to obtain the minimum circumscribed quadrilateral of each row of characters, calculate the affine matrix H of the transformation of quadrangle to rectangle, each Multiply the segmented independent characters with the affine matrix H to obtain the rectified character image.

Further, based on the extracted effective font area, the segmented recognition method is used to conduct preliminary recognition of the text information, including: using single-character training to obtain the preliminary recognition results of the approval and modification date, the approval number and the name of the drug, through the volume The product recurrent neural network model extracts the correlation and collocation relationship between words, and further optimizes the preliminary recognition results.

Further, the preliminary recognition results of the date of approval and modification, the approval number and the name of the drug are obtained by using single-character training, and the relevant collocation relationship between words is extracted through the convolutional cyclic neural network model, and the preliminary recognition results are further optimized, specifically including:

Build a character training library: According to the national drug catalog, extract the symbols, including Chinese characters, numbers, and percent signs, generate symbol images of commonly used fonts, and slightly perturb each image to increase noise;

Divide the training set and the verification set: the generated character training library is used to generate a training set and a verification set at a ratio of 5:1. The training set is used to train the optimal depth model, and the verification set is used to generate the optimal depth model hyperparameters ;

Construct a convolutional neural network model: input a character image with a dimension of 32×32, and use six convolution kernels with a size of 5×5 for convolution operation to obtain a convolutional feature map with a size of 6@28×28; = 2 Perform average pooling or downsampling to obtain a pooled feature map of 6@14×14; perform convolution operation with 16 convolution kernels with a size of 5×5 to obtain a convolution with a size of 16@10×10 Feature map; perform average pooling with stride=2, that is, downsampling, to obtain a pooled feature map of 16@5×5; use a convolution with a kernel of 5×5 and two kernels of 1×1 to scale the feature To obtain a rich combination of features, and finally determine the category output through nonlinear mapping;

Optimize the depth model: randomly select a reference sample, randomly select a sample from the character library of the same category as a positive sample, and randomly select a sample from a character library of a different category as a negative sample; use the twin mechanism, in one iteration, the reference sample Input branch 1, input positive samples and negative samples in turn to branch 2, and the two branches share network parameters; respectively classify the sample features of branch 1 and branch 2 with SoftMax, and use the cross-entropy loss function to constrain ; Combine branch 1 and branch 2 to constrain the comparison loss function, so that the characteristics of the reference sample and the positive sample are as similar as possible, and at the same time make the difference between the characteristics of the reference sample and the negative sample as large as possible; carry out backpropagation on the network, and update the network;

Model evaluation: update the network hyperparameters, and select the optimal network hyperparameters by monitoring the verification set;

Preliminary character discrimination: Input the image-processed character picture into the trained convolutional neural network to obtain the preliminary judgment result of each single character, and retain the single-character classification probability.

Further, the combined text information trained by high-frequency words optimizes and recognizes the initially recognized text information, and obtains an optimized recognition result of characters, which specifically includes:

Build a high-frequency thesaurus: use the JIEBA open source word segmentation system to automatically segment the names of drugs in the national drug catalog, and manually screen and correct some difficult phrases; count the occurrence probability of all phrases, select high-frequency phrases to build a high-frequency thesaurus, Generate high-frequency lexicon images of commonly used fonts, and slightly perturb each image to increase noise;

Divide the training set and the verification set: generate the training set and verification set according to the ratio of 5:1 from the generated high-frequency lexicon. The training set is used to train the optimal depth model, and the verification set is used to generate the optimal depth model. parameter;

Construct a convolutional cyclic memory model: use the convolutional neural sub-network to extract the features of each character of the high-frequency word X={x _t }, and obtain the character features F={f _t }; the cyclic neural sub-network is at time step t Take an input x _t , take a hidden state h _{t-1 at time step t-1} to calculate the hidden state h _t at time step t, and use Relu to obtain the nonlinear relationship between the output y _t and the input at time t:

h _t ＝tanh(w _hh h _t-1 +w _hx x _t )

y _t = Why _hy h _t

Among them, w _hh , w _hx , Why _are the network weights to be learned;

Deep model optimization: the loss function of each time step is the cross entropy loss function, the overall loss function is the sum of the loss functions of each time step, and the network is backpropagated to update the network;

High-frequency word correction: Multiply the probability of each character obtained by the convolutional cyclic neural network with the single-character classification probability reserved in the preliminary identification of the character to obtain the optimized recognition result of the character.

The present invention also provides an automatic recognition system for drug instructions based on deep learning based on the above-mentioned method for automatic recognition of drug instructions based on deep learning, including:

The text information extraction module is used to obtain the image of the drug instructions, preprocess the obtained image based on the image processing method, and extract the effective font area;

The text information preliminary recognition module is used for preliminary recognition of the text information based on the extracted effective font area by adopting a segmented recognition method;

The text information optimization recognition module is used to combine the text information of the high-frequency word training to optimize the recognition of the initially recognized text information, and obtain the optimized recognition result of the characters.

The present invention also provides a storage medium, which includes a stored program, wherein, when the program is running, the above-mentioned deep learning-based automatic recognition method for drug instructions is executed.

Compared with the prior art, the present invention has the following advantages:

1. The present invention proposes an integrated identification method for drug instructions, which can reduce labor costs and greatly improve the management efficiency and timeliness of drug instructions;

2. The present invention proposes a series of methods for image enhancement and text recognition, which can have better effects on problems such as unclear or deformed text, tilted documents, and uncertain scanning directions (horizontal and vertical);

3. The present invention proposes a segmented recognition method. Firstly, the convolutional neural network is used to initially recognize individual characters, and then the convolutional cyclic memory network is combined with high-frequency word training to further optimize the recognition of text information, thereby utilizing the The high-frequency words improve the accuracy of name detection.

Based on the above reasons, the present invention can be widely promoted in fields such as identification of drug insert sheets.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is a flow chart of the method of the present invention.

Fig. 2 is a schematic diagram of a convolutional neural network model provided by an embodiment of the present invention.

Fig. 3 is a schematic diagram of a convolutional cyclic memory model provided by an embodiment of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only It is an embodiment of a part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

It should be noted that the terms "first" and "second" in the description and claims of the present invention and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

As shown in Figure 1, the present invention provides a method for automatic recognition of drug instructions based on deep learning, including:

S1. Acquire the image of the drug instruction manual, preprocess the acquired image based on the image processing method, and extract the effective font area;

S2. Based on the extracted effective font area, a segmented recognition method is used to initially recognize the text information;

S3. Combining the text information of the high-frequency word training, optimize the recognition of the initially recognized text information, and obtain an optimized recognition result of characters.

During specific implementation, as a preferred embodiment of the present invention, in the step S1, the acquired image is preprocessed based on the image processing method, which specifically includes: scanning image enhancement processing, scanning part main direction correction processing, and image tilt direction correction processing , Text information area positioning processing, character area segmentation processing, independent character segmentation processing, and font correction processing. Its purpose is to orientate and locate the key areas to extract clear and correct effective font areas, laying the foundation for the improvement of the accuracy of subsequent text information recognition.

During specific implementation, as a preferred embodiment of the present invention, the scanned image enhancement processing, the main direction correction processing of the scanned document, the image oblique direction correction processing, the image oblique direction correction processing, the text information area positioning processing, the character area segmentation processing, and the independent Character segmentation processing, and font correction processing, including:

The scanning image enhancement process includes: graying the image by using a weighted average method; performing linear filtering on the image by using an average value filter; removing noise caused by poor performance of the scanning device;

The main direction correction processing of the scanned part includes: extracting the length and width characteristics of the scanned part, and projecting the gray value of the image into two directions respectively to obtain the projection features, and judging the main direction of the scanned part in combination with the prior characteristics of the main direction;

The image inclination direction correction process includes: utilizing the Radon transform to estimate the inclination angle of the image, and projecting the image space into the polar coordinate space using the following formula:

A point in polar coordinates is equivalent to a straight line corresponding to two points in the image space. The corresponding line in the image space is determined by the cumulative peak value of the point set in the polar coordinate space. Since the polar coordinate itself contains the inclination angle θ, it is determined according to the cumulative peak value of the point set slope;

The text information area positioning processing includes: performing morphological expansion operation on the image to reduce the gap between adjacent strokes and adjacent characters; extracting the connected domain of the image and merging similar areas; using projection method to perform horizontal projection Histogram to obtain the projection features; for the characteristics of the drug name part of the drug instructions with the largest font and all in the dark background area, select the area with the largest character code and the largest color block projection value as the drug name image area; for the approval date and The date of modification is at the top of the file and the text is sparse. Select the color block projection value at the top of the image that is less than a certain threshold as the image area of the approval date and modification date; identify the brackets and the keywords in the brackets, so as to approve Locate the area where the document number is located;

The character area segmentation processing includes: making a horizontal projection histogram on the selected approval date or approval number image area, the line characters present a peak on the histogram, and the line interval presents an obvious trough on the histogram, according to Divide at the trough to obtain the divided approval number, approval date and modification date;

In the independent character segmentation process, a longitudinal projection histogram is made for the approval date and modification date, each line of the approval number and the drug name area, and each character lattice presents a peak on the histogram, and the character gap presents an obvious trough on the histogram The form is divided according to the trough to obtain the approval date, modification date, approval number, and the numbers, Chinese characters and symbols of the drug name;

Described glyph correction processing comprises: for the locality of font deformation, carry out glyph correction respectively to each row of characters; Utilize Hough transform to obtain the minimum circumscribed quadrilateral of each row of characters, calculate the affine matrix H of the transformation of quadrangle to rectangle, each Multiply the segmented independent characters with the affine matrix H to obtain the rectified character image.

During specific implementation, as a preferred embodiment of the present invention, in the step S2, based on the extracted effective font area, a segmented recognition method is used to perform preliminary recognition of the text information, including: using single-character training to obtain the approval and modification date, For the initial recognition results of the approval number and drug name, the relevant collocation relationship between words is extracted through the convolutional cyclic neural network model, and the preliminary recognition results are further optimized, including:

Build a character training library: According to the national drug catalog, extract the symbols, including Chinese characters, numbers, and percent signs, generate symbol images of commonly used fonts, and slightly perturb each image to increase noise, thereby enhancing the recognition robustness of the depth model The perturbation operations include clipping and angle deflection.

Divide the training set and the verification set: the generated character training library is used to generate a training set and a verification set at a ratio of 5:1. The training set is used to train the optimal depth model, and the verification set is used to generate the optimal depth model hyperparameters , such as batch size, training step size, etc.

Construct a convolutional neural network model: input a character image with a dimension of 32×32, and use six convolution kernels with a size of 5×5 for convolution operation to obtain a convolutional feature map with a size of 6@28×28; = 2 Perform average pooling or downsampling to obtain a pooled feature map of 6@14×14; perform convolution operation with 16 convolution kernels with a size of 5×5 to obtain a convolution with a size of 16@10×10 Feature map; perform average pooling with stride=2, that is, downsampling, to obtain a pooled feature map of 16@5×5; use a convolution with a kernel of 5×5 and two kernels of 1×1 to scale the feature To obtain a rich combination of features, and finally determine the category output through nonlinear mapping; the constructed convolutional neural network model is shown in Figure 2.

Optimize the deep model: the purpose is to train the network so that it can extract the discriminative features that can represent each character. The network is jointly trained in the form of multi-classification and similarity-difference classification to improve the accuracy of the model. The specific steps are:

A. Randomly select a reference sample, randomly select a sample from the character library of the same category as a positive sample, and randomly select a sample from a character library of a different category as a negative sample;

B. Using the twin mechanism, in one iteration, the reference sample is input into branch 1, and the positive samples and negative samples are input into branch 2 in turn, and the two branches share network parameters;

C. Use SoftMax to classify the sample features of branch 1 and branch 2 respectively, and use the cross-entropy loss function to constrain;

D. Combine branch 1 and branch 2 to constrain the comparison loss function, so that the characteristics of the reference sample and the positive sample are as similar as possible, and at the same time, the difference between the characteristics of the reference sample and the negative sample is as large as possible;

E. Backpropagating the network and updating the network;

Model evaluation: update the network hyperparameters, and select the optimal network hyperparameters by monitoring the verification set;

Preliminary character discrimination: Input the image-processed character picture into the trained convolutional neural network to obtain the preliminary judgment result of each single character, and retain the single-character classification probability. Here, the single-character classification probability should be retained. For some error-prone words with similar fonts, the probability will be updated through high-frequency word training in the next step.

During specific implementation, as a preferred embodiment of the present invention, in the step S3, combined with the text information of the high-frequency word training, the text information of the preliminary recognition is optimized and recognized, and the optimized recognition result of the characters is obtained, which specifically includes:

Build a high-frequency thesaurus: use the JIEBA open source word segmentation system to automatically segment the names of drugs in the national drug catalog, and manually screen and correct some difficult phrases; count the occurrence probability of all phrases, select high-frequency phrases to build a high-frequency thesaurus, Generate high-frequency lexicon images of commonly used fonts, and slightly perturb each image to increase noise; thereby enhancing the robustness of high-frequency vocabulary recognition, perturbation operations include cropping, angle deflection, etc.

Divide the training set and the verification set: generate the training set and verification set according to the ratio of 5:1 from the generated high-frequency lexicon. The training set is used to train the optimal depth model, and the verification set is used to generate the optimal depth model. parameter;

Constructing a convolutional cyclic memory model: The convolutional cyclic memory model is used in the recognition of high-frequency words, that is, the spatial information of high-frequency words is extracted by using the convolutional neural sub-network, and the inter-character information of high-frequency words is extracted by using the cyclic memory sub-network. The settings are as follows:

a. Use the convolutional neural sub-network to perform feature extraction for each character of the high-frequency word X={x _t }, and obtain each character feature F={f _t };

b. The recurrent neural sub-network takes an input x _t at time step t, takes a hidden state h _{t-1 at time step t-1} to calculate the hidden state h _t at time step t, and uses Relu to obtain t The nonlinear relationship between the output y _t and the input at any time:

h _t ＝tanh(w _hh h _t-1 +w _hx x _t )

y _t = Why _hy h _t

Among them, w _hh , w _hx , Why are the network weights to be learned; the constructed convolutional _cycle memory model is shown in Figure 3.

Deep model optimization: the loss function of each time step is the cross entropy loss function, the overall loss function is the sum of the loss functions of each time step, and the network is backpropagated to update the network;

High-frequency word correction: Multiply the probability of each character obtained by the convolutional cyclic neural network with the single-character classification probability reserved in the preliminary identification of the character to obtain the optimized recognition result of the character.

Corresponding to the deep learning-based automatic recognition method for drug instructions in this application, this application also provides an automatic recognition system for drug instructions based on deep learning, including: a text information extraction module, a text information preliminary recognition module, and a text information optimization recognition module ,in:

The text information extraction module is used to obtain the image of the drug instructions, preprocess the obtained image based on the image processing method, and extract the effective font area;

The text information preliminary recognition module is used for preliminary recognition of the text information based on the extracted effective font area by adopting the segmented recognition method; in this embodiment, the function of the text information preliminary recognition module is to target the extracted target block, Based on data-driven thinking and deep learning technology, high-precision recognition of Chinese characters and numbers is realized. The core principle is to form a high-precision discriminator by mining the potential features and mapping laws hidden in the Chinese character database. It mainly includes training database construction, deep neural network model construction, deep model optimization and other sub-modules. Most of the chemical elements in medicines are transliterated from Western languages, and they are often distinguished by rare words. Such rare words often have a complex structure, which increases the difficulty of correct identification. These rare words are often combined with other words to express a certain fixed element In addition, drug names usually contain fixed vocabulary, such as injection, oxyfluoride, silicone oil, compound, etc. For these two types of names, if single-character recognition is used, the error rate is often high due to ignoring the fixed collocation between characters. Therefore, this method proposes a segmented recognition method. First, use single-character training to obtain the approval and revision date, approval number, etc. And the preliminary recognition results of drug names, and then use the convolutional cyclic neural network model to extract the correlation and collocation relationship between words, and further optimize the preliminary recognition results.

The text information optimization recognition module is used to combine the text information of the high-frequency word training to optimize the recognition of the initially recognized text information, and obtain the optimized recognition result of the characters.

For the embodiment of the present invention, because it corresponds to the above embodiment, the description is relatively simple. For related similarities, please refer to the part of the description in the above embodiment, and will not be described in detail here.

The embodiment of the present application also discloses a computer-readable storage medium. The computer-readable storage medium stores a computer instruction set. When the computer instruction set is executed by a processor, the deep learning-based Automatic Recognition Method of Drug Instructions

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be realized in other ways. Wherein, the device embodiments described above are only illustrative. For example, the division of the units may be a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or may be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of units or modules may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes. .

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (8)

1. A medicine specification automatic identification method based on deep learning is characterized by comprising the following steps:

acquiring a medicine specification image, preprocessing the acquired image based on an image processing method, and extracting an effective font area;

based on the extracted effective font area, adopting a sectional type identification method to carry out preliminary identification on the character information;

and combining the character information of the high-frequency word training to perform optimized recognition on the preliminarily recognized character information to obtain an optimized recognition result of the characters.

2. The method for automatically identifying the drug instruction manual based on deep learning of claim 1, wherein the image processing method is used for preprocessing the acquired image, and specifically comprises: the processing method includes a scanned image enhancement process, a scanned object main direction correction process, an image tilt direction correction process, a character information region positioning process, a character region division process, an independent character division process, and a font correction process.

3. The method for automatically identifying a drug order based on deep learning according to claim 2, wherein the scan image enhancement processing, the scan item main direction correction processing, the image tilt direction correction processing, the text information region positioning processing, the character region segmentation processing, the independent character segmentation processing, and the font correction processing specifically include:

the scan image enhancement processing includes: graying the image by adopting a weighted average method; performing linear filtering on the image by adopting mean filtering;

the main direction correction treatment of the scanning piece comprises the following steps: extracting the length and width characteristics of the scanning piece, projecting the gray value of the image to two directions respectively to obtain projection characteristics, and judging the main direction of the scanning piece by combining the prior characteristics of the main direction;

the image tilt direction correction processing includes: estimating the tilt angle of the image by using Radon transformation, and projecting the image space to the polar coordinate space by using the following formula:

the point in the polar coordinate is equivalent to a straight line of two corresponding points in the image space, the corresponding line of the image space is determined through the accumulated peak value of the point set in the polar coordinate space, and the inclination angle is determined according to the accumulated peak value of the point set because the polar coordinate itself comprises the inclination angle theta;

the character information area positioning processing comprises the following steps: performing morphological dilation operation on the image to reduce gaps between adjacent strokes of the character and adjacent characters; extracting connected domains of the images, and merging the regions of the same type; adopting a projection method to make a transverse projection histogram to obtain projection characteristics; aiming at the characteristics that partial characters of the medicine names in the medicine specification are maximum and are all positioned in a deep-color background region, selecting a region with the maximum character code and the maximum color block projection value as a medicine name image region; aiming at the characteristics that the approval date and the modification date of the medicine specification are on the top of the file and characters are sparse, selecting an image area with the approval date and the modification date, wherein the projection value of a color block on the top of the image is smaller than that in a certain threshold value; identifying the bracketed sign and the keywords in the bracket so as to position the area where the approved character number is located;

the character region segmentation process includes: making a horizontal projection histogram for the selected approved date or approved character image area, displaying a peak on the histogram by line characters, displaying an obvious trough on the histogram by line intervals, and segmenting according to the trough to obtain the divided approved date and modified date;

the independent character segmentation processing is to perform longitudinal projection column diagrams on each row of the approved date, the modified date, the approved letter number and the medicine name area, wherein each character dot matrix presents a peak on the column diagrams, and the character gaps present obvious trough shapes on the column diagrams, and are segmented according to the trough positions to obtain numbers, chinese characters and symbols of the approved date, the modified date, the approved letter number and the medicine name;

the font correction processing includes: aiming at the locality of font deformation, respectively performing font correction on each line of characters; and obtaining the minimum external quadrangle of each line of characters by utilizing Hough transformation, calculating an affine matrix H for transforming the quadrangle into the rectangle, and multiplying each segmented independent character by the affine matrix H to obtain a corrected character image.

4. The method for automatically recognizing the deep learning-based medicine specification according to claim 2, wherein the preliminary recognition of the text information by using a segmented recognition method based on the extracted effective font area comprises: and obtaining preliminary recognition results of approval and modification dates, approval character numbers and medicine names by utilizing single character training, and extracting related collocation relationships among words through a convolution cyclic neural network model to further optimize the preliminary recognition results.

5. The method as claimed in claim 4, wherein the deep learning-based automatic drug specification recognition method comprises obtaining preliminary recognition results of approved and modified dates, approved characters and drug names by single character training, extracting related collocation relationships among words through a convolutional recurrent neural network model, and further optimizing the preliminary recognition results, specifically comprising:

constructing a character training library: extracting symbols including Chinese characters, numbers and percentiles from the national medicine catalog, generating symbol pictures with common fonts, and slightly disturbing each picture to increase noise;

dividing a training set and a verification set: and (3) the generated character training library is processed according to the following steps of 5:1, generating a training set and a verification set, wherein the training set is used for training to obtain an optimal depth model, and the verification set is used for generating an optimal depth model hyperparameter;

constructing a convolutional neural network model: inputting a character picture, wherein the dimensionality is 32 multiplied by 32, carrying out convolution operation by using 6 convolution kernels with the size of 5 multiplied by 5 to obtain a convolution characteristic diagram with the size of 6@28 multiplied by 28; performing average pooling, namely downsampling, by stride =2 to obtain a pooling characteristic diagram of 6@14 × 14; performing convolution operation by 16 convolution kernels with the size of 5 multiplied by 5 to obtain a convolution feature map with the size of 16@10 multiplied by 10; performing average pooling, namely downsampling, by stride =2 to obtain a pooling feature map of 16@5 × 5; respectively utilizing convolution with one kernel of 5 multiplied by 5 and two kernels of 1 multiplied by 1 to scale the features so as to obtain rich feature combinations, and finally judging category output through nonlinear mapping;

optimizing a depth model: randomly selecting a reference sample, randomly selecting a sample from character libraries of different types as a positive sample, and randomly selecting a sample from character libraries of different types as a negative sample; a twin mechanism is adopted, in one iteration, a reference sample is input into a branch 1, a positive sample and a negative sample are sequentially input into a branch 2 in turn, and the two branches share network parameters; classifying the sample characteristics of the branch 1 and the branch 2 by SoftMax respectively, and constraining by adopting a cross entropy loss function; the combined branch 1 and the branch 2 are restricted by a contrast loss function, so that the characteristics of the reference sample and the positive sample are similar as much as possible, and the characteristic difference between the reference sample and the negative sample is large as much as possible; carrying out backward propagation on the network and updating the network;

and (3) model evaluation: updating the network super parameters, and selecting the optimal network super parameters through a monitoring verification set;

and (3) character preliminary discrimination: and inputting the character picture after image processing into a convolutional neural network obtained by training to obtain a primary judgment result of each single character and reserve the single character classification probability.

6. The method for automatically recognizing the medicine specification based on the deep learning of claim 1, wherein the step of optimally recognizing the preliminarily recognized text information in combination with the text information trained by the high-frequency words to obtain the optimal recognition result of the characters specifically comprises the steps of:

constructing a high-frequency word bank: automatically segmenting the medicine names in the national medicine catalogue by adopting a JIEBA open source segmentation system, and manually screening and correcting partial difficult phrases; counting the occurrence probability of all phrases, selecting high-frequency phrases to construct a high-frequency word stock, generating high-frequency word stock pictures with common fonts, and slightly disturbing each picture to increase noise;

dividing a training set and a verification set: and (4) enabling the generated high-frequency word bank to be as follows: 1, generating a training set and a verification set according to the proportion, wherein the training set is used for training to obtain an optimal depth model, and the verification set is used for generating an optimal depth model hyper-parameter;

constructing a convolution cyclic memory model: using convolutional neural sub-networks to pair high-frequency words X = { X _t Performing feature extraction on each character to obtain each character feature F = { F = { F } _t }; the recurrent neural subnetwork takes an input x at time step t _t Taking a hidden state h at time step t-1 _t-1 To calculate the hidden state h at time step t _t And using Relu to calculate the output y at t moment _t Nonlinear relationship to input:

h _t ＝tanh(w _hh h _t-1 +w _hx x _t )

y _t ＝W _hy h _t

wherein, w _hh ,w _hx ,W _hy The weights are network weights to be learned;

optimizing a depth model: each time step loss function is a cross entropy loss function, the total loss function is the sum of each time step loss function, the network is reversely propagated, and the network is updated;

high-frequency word correction: and multiplying the probability of each character obtained by the convolution cyclic neural network with the single character classification probability reserved in the preliminary character discrimination to obtain the optimal character recognition result.

7. An automatic deep learning-based drug instruction identification system based on the automatic deep learning-based drug instruction identification method according to any one of claims 1 to 6, comprising:

the character information extraction module is used for acquiring a medicine specification image, preprocessing the acquired image based on an image processing method and extracting an effective font area;

the character information preliminary identification module is used for preliminarily identifying the character information by adopting a sectional identification method based on the extracted effective font area;

and the character information optimization and recognition module is used for optimizing and recognizing the preliminarily recognized character information in combination with the character information of the high-frequency word training to obtain an optimized recognition result of the characters.

8. A storage medium comprising a stored program, wherein the program when executed performs the method for automatic deep learning based drug instruction manual identification of any one of claims 1 to 6.

Images