WO2023103894A1 - Nameplate recognition model training method, nameplate recognition method, and related apparatuses - Google Patents

Abstract

Provided in the present application are a nameplate recognition model training method, a nameplate recognition method, and related apparatuses. The nameplate recognition model training method comprises: acquiring sample image data, which is collected from a nameplate that is mounted on a power device, and device parameters, which are recorded in the nameplate; writing the device parameters, which are located in boxes, into an imprint in sample area data, and taking same as label area data; sampling a label point from the device parameters in the label area data; inputting the sample image data into an encoder and extracting feature data; inputting the feature data into a regression network, and sampling a reference point from the imprint in the sample area data; inputting the feature data into a decoder, reconstructing the imprint in the sample area data into characters, and taking the characters as reference area data; and training the encoder, the regression network and the decoder according to the difference between the reference point and the label point and the difference between the reference area data and the label area data, until the reference point is aligned with the reference area data.

Description

Nameplate recognition model training, nameplate recognition method and related devices

This disclosure claims priority to a Chinese patent application with application number 202111479203.X filed with the China Patent Office on December 7, 2021, the entire contents of which are incorporated herein by reference.

technical field

The present application relates to the technical field of electric power, for example, it relates to a training of a nameplate recognition model, a nameplate recognition method and related devices.

Background technique

In the power industry, there are many types and quantities of power equipment, such as electro-hydraulic drum brakes, power cables, power transformers, integrated distribution boxes, electric energy metering boxes, etc. The equipment parameters of these power equipment are usually recorded on the nameplate, the nameplate Mounted on electrical equipment.

When promoting the process of digital management, technicians will collect image data on the nameplate, and use Optical Character Recognition (OCR) technology to identify the content of the nameplate on the image data.

However, since the power equipment is mostly deployed outdoors, the aging of the nameplate is relatively obvious, and some of the contents are peeled off, resulting in errors in the results of optical character recognition. At this time, technical personnel are often required to manually proofread these contents and enter them into the database, which takes a long time , the efficiency is low.

Contents of the invention

This application proposes a nameplate recognition model training, a nameplate recognition method and a related device, in order to solve the problem that the content of the nameplate is peeled off and the result of optical character recognition is wrong.

In the first aspect, the embodiment of the present application provides a training method for a nameplate recognition model, the nameplate recognition model includes an encoder and a regression network, and the method includes:

Acquiring sample image data collected on nameplates installed on electrical equipment and equipment parameters recorded on the nameplates, wherein the sample image data includes sample area data where multiple boxes are located;

Writing the device parameters located in the box on the imprint in the sample area data as label area data;

Sampling label points for the device parameters in the label area data;

inputting the sample image data into the encoder to extract feature data;

inputting the feature data into the regression network, and sampling reference points for the imprints in the sample area data;

inputting the feature data into a decoder, reconstructing the imprint in the sample area data into a font as reference area data;

According to the difference between the reference point and the label point, the difference between the reference area data and the label area data, the encoder, the regression network and the decoder are trained until The reference points are aligned with the reference region data, which the decoder discards when training is complete.

In the second aspect, the embodiment of the present application also provides a method for identifying a nameplate, including:

Load the nameplate recognition model trained according to the method described in the first aspect;

collecting target image data for nameplates installed on electrical equipment, where the target image data includes target area data where multiple boxes are located;

Input the target image data into the encoder to extract feature data;

inputting the feature data into the regression network, and sampling target points for the imprints in the target area data;

writing the target point on the footprint in the target area data in the target image data to obtain reconstructed image data;

Performing optical character recognition on the reconstructed image data to obtain device parameters recorded in the nameplate.

In the third aspect, the embodiment of the present application also provides a training device for a nameplate recognition model, the nameplate recognition model includes an encoder and a regression network, and the device includes:

The sample acquisition module is configured to acquire the sample image data collected on the nameplate installed on the electric equipment, the equipment parameters recorded in the nameplate, and the sample image data includes the sample area data where multiple boxes are located;

The label area data generation module is configured to write the device parameters located in the box on the imprint in the sample area data as label area data;

A label point sampling module is configured to sample label points for the device parameters in the label area data;

A feature data extraction module, configured to input the sample image data into the encoder to extract feature data;

A reference point sampling module, configured to input the characteristic data into the regression network, and sample reference points for the imprints in the sample area data;

A reference area data reconstruction module, configured to input the feature data into a decoder, and reconstruct the imprint in the sample area data into fonts as reference area data;

An auxiliary training module, configured to perform training on the encoder, the regression network, and the The decoder trains until the reference point is aligned with the reference region data, the decoder discards when training is complete.

In the fourth aspect, the embodiment of the present application also provides a nameplate identification device, including:

The nameplate recognition model loading module is configured to load the nameplate recognition model trained according to the method described in the first aspect;

The target image data collection module is configured to collect target image data for nameplates installed on electric equipment, and the target image data includes target area data where multiple boxes are located;

A feature data extraction module, configured to input the target image data into the encoder to extract feature data;

The target point sampling module is configured to input the characteristic data into the regression network, and sample target points for the imprints in the target area data;

A reconstructed image data generation module, configured to write the target point in the target image data on the imprint in the target area data to obtain reconstructed image data;

The optical character recognition module is configured to perform optical character recognition on the reconstructed image data to obtain the device parameters recorded in the nameplate.

In the fifth aspect, the embodiment of the present application also provides a computer device, the computer device comprising:

processor;

memory, set to store program,

When the program is executed by the processor, the processor implements the nameplate recognition model training method according to the first aspect or the nameplate recognition method according to the second aspect.

In the sixth aspect, the embodiment of the present application also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the nameplate identification as described in the first aspect is realized The training method of the model or the nameplate recognition method as described in the second aspect.

Description of drawings

FIG. 1 is a flowchart of a training method for a nameplate recognition model provided in Embodiment 1 of the present application;

FIG. 2 is an example diagram of a nameplate provided in Embodiment 1 of the present application;

Fig. 3 is a flowchart of a method for identifying a nameplate provided in Embodiment 2 of the present application;

4 is a schematic structural diagram of a training device for a nameplate recognition model provided in Embodiment 3 of the present application;

FIG. 5 is a schematic structural diagram of a nameplate identification device provided in Embodiment 4 of the present application;

FIG. 6 is a schematic structural diagram of a computer device provided in Embodiment 5 of the present application.

Detailed ways

The application will be described below in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments described here are only for explaining the present application. In addition, it should be noted that, for ease of description, only parts relevant to the present application are shown in the drawings.

Embodiment one

Figure 1 is a flow chart of a training method for a nameplate recognition model provided in Embodiment 1 of the present application. This embodiment is applicable to the situation of training a nameplate recognition model for auxiliary optical character recognition, and the method can be implemented by a training device for a nameplate recognition model. To perform, the training device of the nameplate recognition model can be implemented by software and/or hardware, and can be configured in computer equipment, such as servers, workstations, personal computers, etc., the method includes the following steps:

Step 101. Obtain sample image data collected from nameplates installed on electrical equipment and equipment parameters recorded on the nameplates.

When promoting digital management of electrical equipment, technicians will collect image data on the nameplate installed on the electrical equipment, use optical character recognition technology to identify the equipment parameters recorded in the nameplate on the image data, and manually check the equipment through technical personnel. The parameters are proofread and entered into the database.

In this embodiment, the image data can be extracted from the database, recorded as sample image data, and the device parameters corresponding to the image data can be extracted, and the sample image data and device parameters can be reused as the data for training the nameplate recognition model, which can reduce labeling data workload.

Step 102, write the device parameters located in the box on the imprint in the sample area data as the label area data.

In practical applications, as shown in FIG. 2 , the information recorded on the nameplate generally includes the type of electric equipment, the equipment parameters recorded on the nameplate, the manufacturer of the electric equipment, and the like.

For the type of power equipment ("box-type substation" in Figure 2), the manufacturer of the power equipment ("XXXX Electric Co., Ltd." in Figure 2) is generally located at the top and bottom of the nameplate and other specific positions.

The equipment parameters of electric equipment include parameter names and parameter values. For different types of electric equipment, parameter names and parameter values are different. Some parameter values can be divided into numerical values and units, and some parameter values cannot be divided into numerical values and units, such as model .

In general, the parameter name and parameter value are generally located on the same line, and the parameter name is generally located before the parameter value.

Exemplarily, as shown in Figure 2, for a box-type substation, the parameter name of one of the equipment parameters is "rated frequency" and the parameter value is "50Hz", where "50" is a value, and "Hz" is a unit of Hertz, The parameter name of another device parameter is "high voltage rated voltage" and the parameter value is "12KV", where "12" is the value and "KV" is the kilovolt unit.

The nameplate is produced for the same type of electrical equipment, not for a specific type of electrical equipment. Therefore, before leaving the factory, the nameplate will be printed with some information common to different types of electrical equipment in the equipment parameters, such as the parameter name, while for the equipment parameters Some information that is not common to different types of electrical equipment is left blank, such as parameter values.

In different situations, the unit of the parameter value can be included as common information and pre-recorded before leaving the factory, or it can be included as non-common information.

For the part of the information that is left blank, it is usually printed with a box (that is, a rectangular box, the background may be the same as or different from other areas), and the model of the electrical equipment is determined after the nameplate leaves the factory, so that the model can be identified using a marking machine. The parameter values in are engraved into the boxes.

After the nameplate leaves the factory, limited by the performance of the engraving machine, part of the information in the box is prone to paint off during the outdoor aging process, leaving imprints, which are prone to errors in the process of optical character recognition.

Then, in this embodiment, the nameplate recognition model for assisted optical character recognition can be trained for imprints. During the training process, the box can be used as the target, and the target detection algorithm can be used to detect and crop the box in the sample image data. The area of , is recorded as the sample area data, that is, the sample image data contains the sample area data where multiple boxes are located, so that the device parameters located in the box are written on the imprint of the corresponding sample area data, which is recorded as Label area data to achieve nameplate restoration.

In one embodiment, the sample area data can be divided into first sample area data with fonts (i.e., device parameters) and second sample area data with prints, that is, the device parameters in the first sample area data are not lost. lacquer, presents relatively clear, colored (usually black, red, etc.) fonts, and does not show imprints, and the original equipment parameters in the data of the second sample area are lacquered, and cannot present clear, colored (usually Black, red, etc.) fonts show imprints instead.

Considering that the equipment parameters stored in the database are not consistent with the font style engraved on the nameplate, therefore, use algorithms such as Multi-Content GAN to compare the equipment located in the box of the second sample area data according to the font on the first sample area data. Perform style migration on the parameters to obtain style parameters, that is, the style parameters are consistent with the style of the font (ie, device parameters) on the first sample area data.

At this time, the style parameter is written on the imprint of the second sample area data as the label area data, thereby improving the authenticity of the nameplate.

Step 103, sampling label points for the device parameters in the label area data.

In this embodiment, the device parameters in the label area data belong to visible fonts, and the device parameters in the label area data are down-sampled to obtain a plurality of points constituting the device parameters (ie, fonts), which are marked as label points.

In the case of relatively dense label points, the label points can be considered as the trend of the strokes of the device parameters (ie font).

Step 104. Input the sample image data into the encoder to extract feature data.

In this embodiment, the regression network and the decoder Decoder share the low-latitude encoder Encoder, and the decoder Decoder can be used to enhance the ability of the encoder Encoder to extract the strokes of the font, so that the strokes of the encoder Encoder font assist in training the regression network.

Conversely, if the Decoder is ignored, the Encoder does not enhance the ability to extract the strokes of the font, and the points extracted by the regression network do not fall on the strokes of the font. The reason is that the Encoder and the regression network are optimal during training. Yes, it is not the Encoder, the regression network, and the Decoder that are optimal.

When the encoder Encoder, the regression network, and the decoder Decoder are jointly trained, the information learned by the encoder Encoder is more mixed information, that is, the points of the font and the strokes of the font are mixed, and the influence of the strokes of the font on the points of the font is strengthened. Therefore, the points of the font are more sensitive to the strokes of the font, and the points of the font are placed on the strokes of the font.

Of course, you can’t just rely on the strokes of the font to make the font point, because there are many types of fonts and strokes are also diverse, especially in the details of the font, it is easy to cause overfitting. These details are very important for OCR. It is not very meaningful, and the strokes of some fonts are still blocked. At this time, the dots of mixed fonts and the strokes of fonts can learn from each other, so that the dots of fonts can perform better in terms of trends.

At the time of design, the role of the Encoder is to transform an input sequence of variable length into a background variable of fixed length, and encode the input sequence information in the background variable.

In this embodiment, the basic module of encoder Encoder uses multiple convolutional layers, pooling layer Polling (such as average pooling), mainly realizes the function of feature extraction, that is, extracts feature data on the trend from sample image data , to extract the feature data on the texture. The strength of this part of feature data extraction will affect the response of the regression network to the high-dimensional convolution input, thereby affecting the accuracy of the regression network.

Step 105, input the characteristic data into the regression network, and sample the reference points of the imprints in the sample area data.

In one embodiment, the regression network includes ShufflenetV2, MobileNet, ShuffleNetV1, Sception, etc., for identifying points in traces in image data.

Taking ShufflenetV2 as an example, ShufflenetV2 divides the input feature data into two branches in the channel dimension, and concatenates the outputs of the two branches into one feature element. Moreover, the ShuffleNetv2 network is a lightweight neural network, a neural network model with a small number of parameters and a low computational cost. Using the ShuffleNetv2 network for high-dimensional feature extraction can reduce the computing resource consumption of the regression network and improve the recognition efficiency of points.

In this embodiment, the sample area data is input into the regression network, and the regression network samples a plurality of points from the imprint in the sample area data, which are recorded as reference points.

Step 106: Input the feature data into the decoder, and reconstruct the imprints in the sample area data into fonts as reference area data.

The initial time step input of the Decoder comes from a specific symbol. For an output sequence, when the Decoder searches out the symbol in a time step, the output sequence is completed.

The background variable output by the encoder Encoder encodes the information of the entire input sequence. Given the output sequence in the training sample, for each time step, the conditional probability output by the decoder Decoder will be calculated based on the previous output sequence and background variables.

The decoder Decoder is usually a multi-layer RNN. For the time step of the output sequence, the decoder Decoder takes the output of the previous time step and the background variable as input, and transforms them and the hidden state of the previous time step into the current time step hidden state.

In this embodiment, the feature data is input into the Decoder, and the Decoder reconstructs the imprints in the sample area data into fonts, which are recorded as reference area data.

Step 107: According to the difference between the reference point and the label point, and the difference between the reference area data and the label area data, train the encoder, regression network and decoder until the reference point and the reference area data are aligned.

In this embodiment, the difference between the reference point and the label point, and the difference between the reference area data and the label area data can be calculated respectively, so as to perform backpropagation on the encoder and backpropagation on the regression network respectively. Backpropagation and backpropagation to the decoder, update the weights in the encoder, the weights in the regression network and the weights in the decoder, respectively, until the reference point is aligned with the reference area data.

The so-called alignment can mean that the trend of the reference point and the reference area data are consistent. When the reference point and the reference area data are superimposed together, the reference point and the reference area data fit together. At this time, it can be considered that the encoder, regression network and decoder training Complete, store the encoder, regression network, including storing the structure and parameters of the encoder, the structure and parameters of the regression network, in addition, the decoder is discarded when the training is completed.

In one embodiment of the present application, step 107 may include the following steps:

Step 1071. Calculate the difference between the reference point and the label point as the first loss value.

In this embodiment, the reference point and the label point are substituted into the preset first loss function, and the difference between the reference point and the label point is calculated to obtain the first loss value, that is, the first loss value is used to evaluate the reference point The overall positional deviation between the (predicted value) and the label point (true value), which can be used to update the regression network.

For example, when generating reference points and label points, you can configure numbers for the reference points and label points. The reference points and label points with the same number have the same position in theory. Therefore, for the reference points and label points with the same number, The norm distance L2 between the reference point and the label point may be calculated, and an average value of all norm distances L2 may be calculated as the first loss value.

Step 1072. Calculate the difference between the reference area data and the label area data as a second loss value.

In this embodiment, the reference area data and the label area data are substituted into the preset second loss function, and the difference between the reference area data and the label area data is calculated to obtain the second loss value, that is, the second loss value is used The overall writing deviation between the evaluation reference point (reference area data) and label point (label area data) can be used to update the decoder.

Exemplarily, the reference area data may be converted into a first matrix, the label area data may be converted into a second matrix, and the Euclidean distance between the first matrix and the second matrix may be calculated as the second loss value.

Step 1073. Combine the first loss value and the second loss value into a third loss value.

In this embodiment, the first loss value and the second loss value can be fused to obtain a third loss value, and the third loss value can be used to update the encoder by integrating the position deviation and the stroke deviation.

Exemplarily, the first loss value and the second loss value may be linearly fused to obtain the third loss value.

In this example, on the one hand, the product of the first loss value and the first weight is calculated as the first weight adjustment value; on the other hand, the product of the second loss value and the second weight is calculated as the second For the weight adjustment value, the first weight is greater than the second weight, and the sum of the first weight adjustment value and the second weight adjustment value is calculated as the third loss value.

Wherein, the first weight is greater than the second weight.

Step 1074, respectively use the first loss value to update the regression network, use the second loss value to update the decoder, and use the third loss value to update the encoder.

Backpropagating the regression network, updating the weights in the regression network based on the first loss value, backpropagating the decoder, updating the weights in the decoder based on the second loss value, and backpropagating the encoder, based on the first loss value Three loss values update the weights in the encoder.

In one embodiment, the first loss value is substituted into optimization algorithms such as stochastic gradient descent (SGD) and adaptive momentum (Adaptive momentum, Adam) to calculate the update range of weights in the regression network, so that according to the update Magnitude updates the weights in the regression network.

Substitute the second loss value into optimization algorithms such as SGD and Adam to calculate the update range of the weights in the decoder, so as to update the weights in the decoder according to the update range.

Substitute the third loss value into optimization algorithms such as SGD and Adam to calculate the update range of the weight in the encoder, so as to update the weight in the encoder according to the update range.

Step 1075, judge whether the number of current iterations reaches the preset threshold, based on the judgment result that the number of current iterations reaches the preset threshold, execute step 1076, and return to execute based on the judgment result that the number of current iterations does not reach the preset threshold Step 104.

Step 1076, determine that the training of the encoder, regression network and decoder is completed, and discard the decoder.

In this embodiment, a threshold value can be set in advance for the number of iterations as a stop condition. In each round of iterative training, the number of current iterations is counted, so as to determine whether the number of times of training the encoder, regression network, and decoder in the current iteration reaches the threshold. threshold.

If the threshold is reached, it can be considered that the training of the encoder, regression network and decoder is completed. At this time, the weights in the encoder and regression network are recorded respectively, and the decoder is discarded.

If the threshold is not reached, the next round of iterative training can be entered, and the iterative training is repeated in this way until the training of the encoder, regression network and decoder is completed.

In this embodiment, the encoder, regression network and decoder are trained offline, the structure and weight of the encoder and regression network are recorded, and distributed to the detection device in various ways. The detection device can load the encoder and regression network, Detect the equipment parameters recorded on the nameplate on the electrical equipment.

In this embodiment, the nameplate recognition model includes an encoder and a regression network to obtain sample image data collected from nameplates installed on electrical equipment and equipment parameters recorded in the nameplate. The sample image data includes samples where multiple boxes are located. Area data; write the device parameters located in the box into the imprint in the sample area data as label area data; sample label points for the device parameters in the label area data; input the sample image data into the encoder to extract feature data; The characteristic data is input into the regression network, and the reference point is sampled for the imprint in the sample area data; the feature data is input into the decoder, and the imprint in the sample area data is reconstructed into a font as the reference area data; according to the relationship between the reference point and the label point The difference between the reference point and the label region data, the encoder, the regression network and the decoder are trained until the reference point is aligned with the reference region data, and the decoder is discarded when the training is completed. This embodiment uses written strokes as supervision to help the low-dimensional features of the regression network focus on the information extraction of strokes. When the information on the nameplate is aged and painted, the points that help regression can fall on the written strokes, that is, Help the regression point to fall on the engraved trace, reorganize the written strokes instead of falling on the experience value, avoid overfitting and make the written strokes unable to be composed, and the regression point can fall on the written strokes, which can be more accurate With the help of optical character recognition, it can improve the accuracy of identifying the equipment parameters recorded in the nameplate, reduce the cost of manual proofreading and entering the database, reduce the time spent, and greatly improve the efficiency.

Embodiment two

Fig. 3 is a flow chart of a nameplate recognition method provided in Embodiment 2 of the present application. This embodiment is applicable to the situation where a nameplate recognition model is used to assist in the recognition of a nameplate of an electric device, and the method can be executed by a nameplate recognition device. The identification device of the nameplate can be implemented by software and/or hardware, and can be configured in computer equipment, such as servers, workstations, personal computers, mobile terminals (such as mobile phones, tablet computers, etc.), and the method includes the following steps:

Step 301, load the nameplate recognition model.

In this embodiment, a nameplate recognition model can be trained in advance, and the nameplate recognition model is used to recognize equipment parameters recorded in the nameplate (image data).

Among them, the nameplate recognition model includes an encoder and a regression network. The training method of the nameplate recognition model is as follows:

Obtain the sample image data collected on the nameplate installed on the electrical equipment, the equipment parameters recorded in the nameplate, and the sample image data includes the sample area data where multiple boxes are located;

Write the device parameters located in the box on the imprint in the sample area data as the label area data;

Sampling label points for device parameters in label area data;

Input the sample image data into the encoder to extract the feature data;

Input the feature data into the regression network, and sample the reference points for the imprints in the sample area data;

Input the feature data into the decoder, reconstruct the imprint in the sample area data into a font, and use it as the reference area data;

According to the difference between the reference point and the label point, the difference between the reference area data and the label area data, the encoder, the regression network and the decoder are trained until the reference point and the reference area data are aligned. When the training is completed, the decoder throw away.

In the embodiment of the present application, since the training method of the nameplate recognition model is basically similar to the application of the first embodiment, the description is relatively simple, and for relevant details, please refer to the part of the description of the first embodiment.

Load the encoder and regression network (structure and its parameters) in the nameplate recognition model into the memory for operation, and the device parameters recorded in the nameplate (image data) are to be recognized.

Step 302, collecting target image data of nameplates installed on electric equipment.

In this embodiment, the user can collect image data facing the nameplate installed on the electrical equipment, which is recorded as target image data. Generally, the target image data includes areas where multiple boxes are located, which is recorded as target area data.

Step 303: Input the target image data into the encoder to extract feature data.

The target image data is input into the encoder, and the encoder processes the target image data to extract low-latitude feature data.

Step 304, input the feature data into the regression network, and sample the target points for the footprints in the target area data.

The feature data is input into the regression network, and the regression network samples multiple points from the imprint in the target area data, which are recorded as target points.

Step 305. Write the target point on the imprint in the target area data in the target image data to obtain reconstructed image data.

Record the coordinates of the target point on the target area data, and record it as the relative position. In the target image data, write the target point into the relative position in the corresponding target area data, so that the target point can be written into the corresponding target area data. On the imprint, the reconstructed image data is obtained.

Step 306, perform optical character recognition on the reconstructed image data, and obtain the equipment parameters recorded in the nameplate.

In this embodiment, deep learning technology can be applied to perform optical character recognition on the reconstructed image data to obtain information recorded on the nameplate, for example, end-to-end text recognition algorithm (End-to-End Text Spotting), end-to-end text spotting Detection and recognition algorithm (FOTS), text box recognition algorithm (TextBoxes), text detection algorithm (PSENet), etc.

In the case of relatively dense target points, the target point can be considered as the trend of the strokes of the device parameters (fonts), and the compatibility of OCR is strong. The impact of the trend of strokes on OCR is significantly greater than the details of the strokes. In the process of optical character recognition, the target points in the reconstructed image data will be recognized as fonts, and in the case of a good trend, the success rate of recognition can be improved.

In practical applications, the information recorded on the nameplate generally includes the type of the electric equipment, the equipment parameters recorded on the nameplate, the manufacturer of the electric equipment, and the like.

The type of power equipment and the manufacturer of the power equipment are generally located at specific positions such as the top and bottom of the nameplate, and the type of power equipment and the manufacturer of the power equipment are relatively fixed. Therefore, the power can be identified by location or keywords. Type of equipment, manufacturer of electrical equipment.

In addition, the device parameters include parameter names and parameter values. The parameter names and parameter values are generally located in the same line, and the parameter names are generally located before the parameter values. Then, optical character recognition can be performed on the reconstructed image data to obtain text information.

Look for the text information located in the box as the parameter value recorded on the nameplate.

Look for the text information preceding the box as the parameter name recorded on the nameplate.

In one embodiment, the parameter value can be divided into numerical value and unit. In some cases, the unit may be printed behind the box before the nameplate leaves the factory, that is, the box is used to record the numerical value in the parameter value, and the unit may also be in the The nameplate is engraved in the box after leaving the factory, that is, the box is used to record the value and unit of the parameter value, then, if the text information in the box is the unit, the text information is determined to be the parameter value recorded in the nameplate .

For the information recorded on the identified nameplate (such as the type of electrical equipment, the equipment parameters recorded in the nameplate, the manufacturer of the electrical equipment, etc.), it can be stored in the database according to the established format.

In this embodiment, the nameplate recognition model is loaded; the target image data is collected for the nameplate installed on the electric equipment, and the target image data includes the target area data where multiple boxes are located; the target image data is input into the encoder to extract feature data ;Input the feature data into the regression network, and sample the target point on the imprint in the target area data; write the target point in the target image data on the imprint in the target area data to obtain the reconstructed image data; execute on the reconstructed image data Optical character recognition to obtain equipment parameters recorded in the nameplate. This embodiment uses written strokes as supervision to help the low-dimensional features of the regression network focus on the information extraction of strokes. When the information on the nameplate is aged and painted, the points that help regression can fall on the written strokes, that is, Help the regression point to fall on the engraved trace, reorganize the written strokes instead of falling on the experience value, avoid overfitting and make the written strokes unable to be composed, and the regression point can fall on the written strokes, which can be more accurate With the help of optical character recognition, it can improve the accuracy of identifying the equipment parameters recorded in the nameplate, reduce the cost of manual proofreading and entering the database, reduce the time spent, and greatly improve the efficiency.

It should be noted that, for the method embodiment, for the sake of simple description, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiment of the present application is not limited by the described action sequence, because According to the embodiment of the present application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

Embodiment three

Fig. 4 is a structural block diagram of a training device for a nameplate recognition model provided in Embodiment 3 of the present application. The nameplate recognition model includes an encoder and a regression network. The device may include the following modules:

The sample acquisition module 401 is configured to acquire the sample image data collected on the nameplate installed on the electric equipment, the equipment parameters recorded in the nameplate, and the sample image data includes the sample area data where multiple boxes are located;

The label area data generation module 402 is configured to write the device parameters located in the box on the imprint in the sample area data as label area data;

The label point sampling module 403 is configured to sample label points for the device parameters in the label area data;

A feature data extraction module 404, configured to input the sample image data into the encoder to extract feature data;

The reference point sampling module 405 is configured to input the feature data into the regression network, and sample reference points for the imprints in the sample area data;

The reference area data reconstruction module 406 is configured to input the characteristic data into the decoder, and reconstruct the imprint in the sample area data into a font as the reference area data;

Auxiliary training module 407, configured to train the encoder, the regression network and the The decoder is trained until the reference point is aligned with the reference region data, and the decoder is discarded when training is complete.

In one embodiment of the present application, the label area data generation module 402 is also set to:

distinguishing the sample area data into first sample area data having fonts and second sample area data having prints;

performing style migration on the device parameters located in the box to which the second sample area data belongs according to the font on the first sample area data, to obtain style parameters;

Writing the style parameter into the imprint of the second sample area data as label area data.

In one embodiment of the present application, the auxiliary training module 407 is also set to:

calculating the difference between the reference point and the label point as a first loss value;

calculating the difference between the reference area data and the label area data as a second loss value;

combining the first loss value and the second loss value into a third loss value;

updating the regression network using the first loss value;

updating the decoder using the second loss value;

updating the encoder using the third loss value;

Judging whether the number of current iterations reaches a preset threshold, based on the judgment result that the number of current iterations reaches a preset threshold, it is determined that the training of the encoder, the regression network and the decoder is completed, and the decoder is discarded ; Based on the judging result that the number of current iterations does not reach the preset threshold, return to execute the step of inputting the sample image data into the encoder to extract feature data.

In one embodiment of the present application, the auxiliary training module 407 is also set to:

For the reference point and the label point with the same number, calculate the norm distance between the reference point and the label point;

An average value is calculated for the norm distance as a first loss value.

In one embodiment of the present application, the auxiliary training module 407 is also set to:

converting the reference area data into a first matrix;

converting the label area data into a second matrix;

Calculate the Euclidean distance between the first matrix and the second matrix as a second loss value.

In one embodiment of the present application, the auxiliary training module 407 is also set to:

calculating the product of the first loss value and the first weight as a first weight adjustment value;

Calculate the product between the second loss value and the second weight, as the second weight adjustment value, the first weight is greater than the second weight;

calculating the sum of the first weight adjustment value and the second weight adjustment value as a third loss value;

Wherein, the first weight is greater than the second weight.

The nameplate recognition model training device provided in the embodiment of the present application can execute the nameplate recognition model training method provided in any embodiment of the present application, and has corresponding functional modules and beneficial effects for executing the method.

Embodiment four

Fig. 5 is a structural block diagram of a nameplate identification device provided in Embodiment 4 of the present application, and the device may include the following modules:

The nameplate recognition model loading module 501 is configured to load the nameplate recognition model;

The target image data collection module 502 is configured to collect target image data for the nameplate installed on the electrical equipment, and the target image data includes target area data where multiple boxes are located;

The feature data extraction module 503 is configured to input the target image data into the encoder to extract feature data;

The target point sampling module 504 is configured to input the feature data into the regression network, and sample target points for the imprints in the target area data;

The reconstructed image data generation module 505 is configured to write the target point in the target image data on the imprint in the target area data to obtain reconstructed image data;

The optical character recognition module 506 is configured to perform optical character recognition on the reconstructed image data to obtain the device parameters recorded in the nameplate.

Wherein, the nameplate recognition model includes an encoder and a regression network, and the training method of the nameplate recognition model is as follows:

Acquiring sample image data collected on nameplates installed on electrical equipment and equipment parameters recorded on the nameplates, wherein the sample image data includes sample area data where multiple boxes are located;

Writing the device parameters located in the box on the imprint in the sample area data as label area data;

Sampling label points for the device parameters in the label area data;

inputting the sample image data into the encoder to extract feature data;

inputting the feature data into the regression network, and sampling reference points for the imprints in the sample area data;

inputting the feature data into a decoder, reconstructing the imprint in the sample area data into a font as reference area data;

According to the difference between the reference point and the label point, the difference between the reference area data and the label area data, the encoder, the regression network and the decoder are trained until The reference points are aligned with the reference region data, which the decoder discards when training is complete.

In one embodiment of the present application, the device parameters include parameter names and parameter values;

The optical character recognition module 506 is also set to:

performing optical character recognition on the reconstructed image data to obtain text information;

Find the text information located in the box as the parameter value recorded in the nameplate;

Find the text information before the box as the parameter name recorded in the nameplate;

If the text information in the box is a unit, it is determined that the text information is the parameter value recorded in the nameplate.

The nameplate recognition device provided in the embodiment of the present application can execute the nameplate recognition method provided in any embodiment of the present application, and has corresponding functional modules and beneficial effects for executing the method.

Embodiment five

FIG. 6 is a schematic structural diagram of a computer device provided in Embodiment 5 of the present application. FIG. 6 shows a block diagram of an exemplary computer device 12 suitable for implementing embodiments of the present application. The computer device 12 shown in FIG. 6 is only one example.

As shown in FIG. 6, computer device 12 takes the form of a general-purpose computing device. Components of computer device 12 may include at least one processor or processing unit 16 , memory 28 , and bus 18 connecting various system components including memory 28 and processing unit 16 .

Bus 18 represents at least one of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus structures. These architectures include, for example, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA bus, the Video Electronics Standard Association (VESA) ) Local bus and Peripheral Component Interconnect (PCI) bus.

Computer device 12 may include a variety of computer system readable media. Such media can be all available media that can be accessed by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32 . Computer device 12 may include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"). Disk drives for reading and writing to removable non-volatile disks (such as "floppy disks") and for removable non-volatile optical disks (such as Compact Disc-Read Only Memory (CD) -ROM), Digital Video Disc-Read Only Memory (Digital Video Disc-Read Only Memory, DVD-ROM) or other optical media) CD-ROM drive. In these cases, each drive may be connected to bus 18 via at least one data medium interface. Memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present application.

A program/utility tool 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including an operating system, at least one application program, other program modules, and program data, in these examples Each or a combination may include implementations of network environments. The program modules 42 generally perform the functions and/or methods of the embodiments described herein.

Computer device 12 may also communicate with at least one external device 14 (e.g., a keyboard, pointing device, display 24, etc.), and at least one device that enables a user to interact with 12 A device (eg, network card, modem, etc.) capable of communicating with at least one other computing device. This communication can be performed through an input/output (Input/Output, I/O) interface 22 . Moreover, the computer device 12 can also communicate with at least one network (such as a local area network (Local Area Network, LAN), a wide area network (Wide Area Network, WAN) and/or a public network, such as the Internet) through the network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18 . It should be appreciated that other hardware and/or software modules may be used in conjunction with computer device 12, including: microcode, device drivers, redundant processing units, external disk drive arrays, Redundant Array of Inexpensive Disks (RAID) systems , tape drives, and data backup storage systems.

The processing unit 16 executes various functional applications and data processing by running the programs stored in the memory 28 , for example, implementing the nameplate recognition model training method or the nameplate recognition method provided in the embodiment of the present application.

Embodiment six

Embodiment 6 of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned nameplate recognition model training method or nameplate recognition method is realized. , and can achieve the same technical effect, in order to avoid repetition, it will not be repeated here.

Wherein, the computer-readable storage medium may include, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination thereof. Examples of computer readable storage media include: an electrical connection having at least one lead, a portable computer disk, a hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (such as electronic programmable read-only memory (Electronic Programable Read Only Memory, EPROM) or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or a suitable combination of the above. In this document, a computer-readable storage medium may be a tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

Claims (10)

1. A training method for a nameplate recognition model, the nameplate recognition model comprising an encoder and a regression network, the method comprising:

   Acquiring sample image data collected on nameplates installed on electrical equipment and equipment parameters recorded on the nameplates, wherein the sample image data includes sample area data where multiple boxes are located;

   Writing the device parameters located in the box on the imprint in the sample area data as label area data;

   Sampling label points for the device parameters in the label area data;

   inputting the sample image data into the encoder to extract feature data;

   inputting the feature data into the regression network, and sampling reference points for the imprints in the sample area data;

   inputting the feature data into a decoder, reconstructing the imprint in the sample area data into a font as reference area data;

   According to the difference between the reference point and the label point, the difference between the reference area data and the label area data, the encoder, the regression network and the decoder are trained until The reference points are aligned with the reference region data, which the decoder discards when training is complete.
2. The method according to claim 1, wherein said writing said device parameters located in said box on the footprint in said sample area data as label area data comprises:

   distinguishing the sample area data into first sample area data having fonts and second sample area data having prints;

   performing style migration on the device parameters located in the box to which the second sample area data belongs according to the font on the first sample area data, to obtain style parameters;

   Writing the style parameter into the imprint of the second sample area data as label area data.
3. The method according to claim 1 or 2, wherein the encoding is performed according to the difference between the reference point and the label point, the difference between the reference area data and the label area data The device, the regression network and the decoder are trained until the reference point is aligned with the reference area data, including:

   calculating the difference between the reference point and the label point as a first loss value;

   calculating the difference between the reference area data and the label area data as a second loss value;

   combining the first loss value and the second loss value into a third loss value;

   updating the regression network using the first loss value;

   updating the decoder using the second loss value;

   updating the encoder using the third loss value;

   Judging whether the number of current iterations reaches a preset threshold, based on the judgment result that the number of current iterations reaches a preset threshold, determining that the encoder, the regression network, and the decoder have been trained, and discarding the decoder; Based on the judging result that the number of current iterations does not reach the preset threshold, return to performing the step of inputting the sample image data into the encoder to extract feature data.
4. The method according to claim 3, wherein said calculating the difference between said reference point and said label point, as a first loss value, comprises:

   For the reference point and the label point with the same number, calculate the norm distance between the reference point and the label point;

   An average value is calculated for the norm distance as a first loss value.
5. The method according to claim 3, wherein said calculating the difference between said reference area data and said label area data, as a second loss value, comprises:

   converting the reference area data into a first matrix;

   converting the label area data into a second matrix;

   Calculate the Euclidean distance between the first matrix and the second matrix as a second loss value.
6. The method according to claim 3, wherein said combining the first loss value and the second loss value into a third loss value comprises:

   calculating the product of the first loss value and the first weight as a first weight adjustment value;

   calculating the product of the second loss value and the second weight, as a second weight adjustment value, the first weight is greater than the second weight;

   calculating the sum of the first weight adjustment value and the second weight adjustment value as a third loss value;

   Wherein, the first weight is greater than the second weight.
7. A method for identifying a nameplate, comprising:

   Loading the nameplate recognition model trained according to the method according to any one of claims 1-6;

   collecting target image data for nameplates installed on electrical equipment, where the target image data includes target area data where multiple boxes are located;

   Input the target image data into the encoder to extract feature data;

   inputting the feature data into the regression network, and sampling target points for the imprints in the target area data;

   writing the target point on the footprint in the target area data in the target image data to obtain reconstructed image data;

   Performing optical character recognition on the reconstructed image data to obtain device parameters recorded in the nameplate.
8. The method according to claim 7, wherein the device parameters include parameter names and parameter values;

   The performing optical character recognition on the reconstructed image data to obtain the equipment parameters recorded in the nameplate includes:

   performing optical character recognition on the reconstructed image data to obtain text information;

   Find the text information located in the box as the parameter value recorded in the nameplate;

   Find the text information before the box as the parameter name recorded on the nameplate;

   If the text information in the box is a unit, it is determined that the text information is the parameter value recorded in the nameplate.
9. A computer device comprising:

   processor;

   memory, set to store program,

   When the program is executed by the processor, the processor implements the nameplate recognition model training method according to any one of claims 1-6 or the method described in any one of claims 7-8. Nameplate identification method.
10. A computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the nameplate recognition model training method according to any one of claims 1-6 or The nameplate identification method according to any one of claims 7-8.

Images