Intelligent character recognition method and device
Technical Field
The invention relates to the field of metal material image recognition, in particular to an intelligent character recognition method and device.
Background
The steel plate number is a numerical sequence for identifying important characteristic information such as product process, production batch and the like in the steel manufacturing process. A standardized design and classification method taking the steel plate number as a key word is established, and the tracking and management of steel products are finished according to the steel plate number, so that the key link of process automation is realized.
In a general machine vision system solution, a light source controller controls the work of a camera and a light source, an industrial camera transmits shot data back to a server for information extraction, a user is connected to the server through a terminal PC, a character intelligent recognition system is operated on the PC to complete character recognition, and a recognition result is stored in the server or uploaded to other field systems, as shown in FIG. 1.
At present, deep learning is vigorously developed in the field of character recognition. In deep learning recognition models represented by deep convolutional neural networks, a Non-Maximum Suppression (NMS) method is used, which is a method for eliminating redundant detection results and avoids repeated detection of the same character or object. Grouping detection results, and reserving the detection result with the highest recognition confidence coefficient of the detection result in each group; the grouping is based on the intersection-to-union ratio, namely the ratio of the intersection and union of the two detection areas, when the intersection-to-union ratio of the two detection areas is greater than a certain threshold value, the two detection areas are determined to comprise a redundant detection area, and the two detection areas are placed in the same group. The method is simple in calculation, but the steel plate characters are recognized according to the highest recognition confidence coefficient, and the recognition accuracy is low.
Disclosure of Invention
The invention aims to solve the technical problem of providing an intelligent character recognition method and a recognition device, and aims to solve the problems that in the prior art, steel plate characters are recognized according to the highest recognition confidence coefficient, and the recognition accuracy is low.
In order to solve the above technical problem, an embodiment of the present invention provides an intelligent character recognition method, including:
acquiring a target picture containing steel plate characters to be identified;
identifying the target picture to obtain the label, the position coordinate and the identification confidence of each character to be detected in the target picture;
determining the position confidence of each character to be detected in the target picture according to the obtained position coordinates and the recognition confidence of each character to be detected in the target picture;
and screening the obtained labels of the characters to be detected according to the obtained position confidence coefficient to obtain the steel plate characters in the target picture.
Further, the identifying the target picture to obtain the label, the position coordinate and the identification confidence of each character to be detected in the target picture comprises:
and identifying the target picture by utilizing a predetermined identification model using non-maximum suppression to obtain the label, the position coordinate and the identification confidence of each character to be detected in the target picture.
Further, the identifying model includes: a deep convolutional neural network.
Further, the position confidence is expressed as:
wherein, LOC CONF
xFor the position of character xDegree of confidence, CONF
xFor recognition confidence, α is an adjustment factor, N is the number of characters, L
xW is the total width of the character, omega is the character size coefficient, i is the number of the character sequence,
is a reference point spatial position coordinate, L
offsetIs a coordinate offset.
Further, the step of screening the obtained labels of the characters to be detected according to the obtained position confidence coefficient to obtain the steel plate characters in the target picture comprises the following steps:
and performing non-maximum value inhibition according to the position confidence coefficient, and sequencing each group of results of which the cross-over ratio is greater than a preset cross-over ratio threshold value according to the position confidence coefficient, wherein each group of results comprises: detecting the sequence number of the area, the group sequence number, the label and the position confidence of the character to be detected;
and screening out the labels of the characters to be detected with the position confidence coefficient larger than the preset position confidence coefficient to obtain the steel plate characters in the target picture.
The embodiment of the present invention further provides an intelligent character recognition device, which is characterized by comprising:
the acquisition module is used for acquiring a target picture containing the steel plate characters to be recognized;
the identification module is used for identifying the target picture to obtain the label, the position coordinate and the identification confidence of each character to be detected in the target picture;
the determining module is used for determining the position confidence coefficient of each character to be detected in the target picture according to the obtained position coordinates and the recognition confidence coefficient of each character to be detected in the target picture;
and the screening module is used for screening the obtained labels of the characters to be detected according to the obtained position confidence coefficient to obtain the steel plate characters in the target picture.
Further, the identification module is configured to identify the target picture by using a predetermined identification model using non-maximum suppression, so as to obtain a tag, a position coordinate, and an identification confidence of each to-be-detected character in the target picture.
Further, the identifying model includes: a deep convolutional neural network.
Further, the position confidence is expressed as:
wherein, LOC CONF
xFor positional confidence of character x, CONF
xFor recognition confidence, α is an adjustment factor, N is the number of characters, L
xW is the total width of the character, omega is the character size coefficient, i is the number of the character sequence,
is a reference point spatial position coordinate, L
offsetIs a coordinate offset.
Further, the screening module includes:
the sorting unit is used for carrying out non-maximum value suppression according to the position confidence coefficient, and sorting each group of results of which the intersection ratio is greater than a preset intersection ratio threshold value according to the position confidence coefficient, wherein each group of results comprises: detecting the sequence number of the area, the group sequence number, the label and the position confidence of the character to be detected;
and the screening unit is used for screening out the labels of the characters to be detected, with the position confidence coefficient larger than the preset position confidence coefficient, so as to obtain the steel plate characters in the target picture.
The technical scheme of the invention has the following beneficial effects:
in the scheme, a target picture containing the steel plate characters to be identified is obtained; identifying the target picture to obtain the label, the position coordinate and the identification confidence of each character to be detected in the target picture; determining the position confidence of each character to be detected in the target picture according to the obtained position coordinates and the recognition confidence of each character to be detected in the target picture; screening the obtained labels of the characters to be detected according to the obtained position confidence coefficient to obtain steel plate characters in the target picture; therefore, the intelligent character recognition method based on the recognition confidence coefficient and the position confidence coefficient can improve the recognition accuracy of the steel plate characters so as to complete the tracking and management of steel products according to the steel plate number.
Drawings
FIG. 1 is a schematic diagram of the results of a machine vision system;
fig. 2 is a schematic flow chart of an intelligent character recognition method according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating unscreened raw results of character recognition according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a recognition result of performing non-maximum suppression with recognition confidence according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a recognition result of performing non-maximum suppression with position confidence according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an intelligent character recognition device according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments.
The invention provides an intelligent character recognition method and device, aiming at the problems that the existing method for recognizing steel plate characters according to the highest recognition confidence coefficient is low in recognition accuracy.
Example one
As shown in fig. 2, the method for intelligently recognizing characters provided in the embodiment of the present invention includes:
s101, acquiring a target picture containing steel plate characters to be recognized;
s102, identifying the target picture to obtain the label, the position coordinate and the identification confidence of each character to be detected in the target picture;
s103, determining the position confidence of each character to be detected in the target picture according to the obtained position coordinates and the recognition confidence of each character to be detected in the target picture;
and S104, screening the obtained labels of the characters to be detected according to the obtained position confidence coefficient to obtain the steel plate characters in the target picture.
The intelligent character recognition method of the embodiment of the invention obtains a target picture containing the steel plate character to be recognized; identifying the target picture to obtain the label, the position coordinate and the identification confidence of each character to be detected in the target picture; determining the position confidence of each character to be detected in the target picture according to the obtained position coordinates and the recognition confidence of each character to be detected in the target picture; screening the obtained labels of the characters to be detected according to the obtained position confidence coefficient to obtain steel plate characters in the target picture; therefore, the intelligent character recognition method based on the recognition confidence coefficient and the position confidence coefficient can improve the recognition accuracy of the steel plate characters so as to complete the tracking and management of steel products according to the steel plate number.
In a specific implementation manner of the intelligent character recognition method, further, the recognizing the target picture to obtain a label, a position coordinate, and a recognition confidence of each character to be detected in the target picture includes:
and identifying the target picture by utilizing a predetermined identification model using non-maximum suppression to obtain the label, the position coordinate and the identification confidence of each character to be detected in the target picture.
In this embodiment, the recognition model may be a deep convolutional neural network. Identifying the steel plate characters by adopting a deep convolution neural network inhibited by using a non-maximum value to obtain the label, the position coordinate and the identification confidence coefficient CONF of each character to be detected in the target picturexThe results are shown in FIG. 3. Therefore, the steel plate characters are automatically identified in a deep learning mode, labor cost and errors can be reduced, and identification efficiency is improved.
In this embodiment, it is assumed that the set intersection ratio IoU is 0.5, and the calculation of the non-maximum suppression is performed. First, the results of the tests with the cross-over ratio of more than 0.5 were grouped into the same group, and the recognition results shown in Table 1 were obtained. In table 1, the identification results of the same group number are the same group, 1 and 2 are the same group, 3 and 4 are the same group, 5 and 6 are the same group, 7 and 8 are the same group, 9, 10, 11 and 12 are the same group, and 13 and 14 are the same group.
As can be seen from Table 1, when only the recognition confidence CONF is adoptedxWhen screening the recognition results, assuming that the results of each group of recognition confidence coefficient not less than 0.900 are reserved, the final recognition result of the obtained steel plate characters is as follows: '310229'.
TABLE 1 deep convolutional neural network recognition results
As can be seen from fig. 3, the correct steel plate characters are: '310279', it follows that only the recognition confidence CONF is usedxThe intelligent character recognition method in (2) has recognition errors, because in actual detection, due to the existence of factors such as contamination, errors between the detection result and the detection space position may occur, as shown in fig. 3, there are errors in the space positions of the detection areas No. 9 and No. 10, and the number '7' is not completely in the detection area, so that the detection result is erroneous. Therefore, in this embodiment, the recognition confidence is converted into a position confidence LOC CONFxBy using LOC CONFxAnd screening the recognition result.
In a specific embodiment of the foregoing character intelligent recognition method, further, the position confidence is expressed as:
wherein, LOC CONF
xFor positional confidence of character x, CONF
xFor recognition confidence, α is an adjustment factor, N is the number of characters, L
xW is the total width of the character, omega is the character size coefficient, i is the number of the character sequence,
as a reference pointSpatial position coordinate, L
offsetIs a coordinate offset.
In this embodiment, a picture with size 540 × 300 is taken as an example, and it is assumed that the character size coefficient ω is 75, the adjustment factor α is 0.5, the character reference point position coordinates are (30,90), and the coordinate offset (i.e., the character space position offset) is L
offset30. Fig. 3 shows the overall recognition result to be eliminated, with the coordinate positions (with the coordinate origin at the top left, the positive y-axis direction downward, and the positive x-axis direction rightward). Taking the first filter box in fig. 3 as an example for calculation, since a 6-bit character is known, i is 0,1,2,3,4,5, as shown in table 2, the first row is the value of i, and for different values i, an intermediate result is calculated
Listed in the second row, the minimum value is 0.075 according to the calculation, so it is substituted for LOCCONF
1Calculation formula assuming recognition confidence CONF
10.880, the position confidence LOC CONF is obtained
10.852. By analogy, the position confidence LOC CONF of each recognition result
xCalculation, results are set forth in LOC CONF of Table 1
xAnd (4) columns.
TABLE 2 intermediate results
i
|
0
|
1
|
2
|
3
|
4
|
5
|
Intermediate results
|
0.075
|
1.055
|
2.054
|
3.054
|
4.054
|
5.054 |
In a specific implementation manner of the intelligent character recognition method, further, the screening the obtained tags of the characters to be detected according to the obtained position confidence coefficient, and obtaining the steel plate characters in the target picture includes:
and performing non-maximum value inhibition according to the position confidence coefficient, and sequencing each group of results of which the cross-over ratio is greater than a preset cross-over ratio threshold value according to the position confidence coefficient, wherein each group of results comprises: detecting the sequence number of the area, the group sequence number, the label and the position confidence of the character to be detected;
and screening out the labels of the characters to be detected with the position confidence coefficient larger than the preset position confidence coefficient to obtain the steel plate characters in the target picture.
In this embodiment, the position confidence LOC CONFxAfter the calculation is finished, non-maximum suppression is carried out, and results of each group with the intersection ratio larger than 0.5 are subjected to LOC CONFxSorting the values, and reserving the result that the position confidence coefficient is not less than 0.850, wherein the obtained steel plate characters are as follows: '310279', correct results.
In this embodiment, with reference to fig. 4 and 5, a recognition result of an intelligent recognition method based only on a recognition confidence and a recognition result of an intelligent character recognition method based on a recognition confidence and a position confidence are described:
fig. 4 shows the recognition result of the intelligent recognition method based only on the recognition confidence, and fig. 5 shows the recognition result of the intelligent character recognition method based on the recognition confidence and the position confidence. By CONFxTo carry outThe final result of non-maxima suppression is '310229', with LOC CONFxNon-maxima suppression was performed to give a final result of '310279'. Due to the addition of spatial distribution factors, the spatial position is taken as an influence factor to be counted in, and the LOC CONF is influencedx. Screening of characters using the original NMS, due to CONF of the number' 7xThe value is less than the number '2', therefore the final result should be '310229', not in line with the true result. Calculating LOC CONF when spatial distribution is taken into accountxIn value, the '7' spatial position of the number is substantially accurate without having a drastic effect on the result, while the '2' spatial position of the redundant number has a greater deviation from the reference position, which reduces the LOC CONFxThe value is obtained. For other numbers, the spatial position is substantially accurate, LOC CONFxThe value size is slightly reduced and is retained as the final result.
Example two
The present invention also provides a specific embodiment of an intelligent character recognition device, and since the intelligent character recognition device provided by the present invention corresponds to the specific embodiment of the intelligent character recognition method, the intelligent character recognition device can achieve the purpose of the present invention by executing the flow steps in the specific embodiment of the method, so that the explanation in the specific embodiment of the intelligent character recognition method is also applicable to the specific embodiment of the intelligent character recognition device provided by the present invention, and will not be described in detail in the following specific embodiment of the present invention.
As shown in fig. 6, an embodiment of the present invention further provides an intelligent character recognition apparatus, including:
the acquisition module 11 is used for acquiring a target picture containing the steel plate characters to be identified;
the identification module 12 is configured to identify the target picture to obtain a label, a position coordinate, and an identification confidence of each to-be-detected character in the target picture;
the determining module 13 is configured to determine a position confidence of each to-be-detected character in the target picture according to the obtained position coordinate and the obtained recognition confidence of each to-be-detected character in the target picture;
and the screening module 14 is configured to screen the obtained tag of the character to be detected according to the obtained position confidence, so as to obtain a steel plate character in the target picture.
The intelligent character recognition device of the embodiment of the invention obtains a target picture containing the steel plate character to be recognized; identifying the target picture to obtain the label, the position coordinate and the identification confidence of each character to be detected in the target picture; determining the position confidence of each character to be detected in the target picture according to the obtained position coordinates and the recognition confidence of each character to be detected in the target picture; screening the obtained labels of the characters to be detected according to the obtained position confidence coefficient to obtain steel plate characters in the target picture; therefore, the intelligent character recognition method based on the recognition confidence coefficient and the position confidence coefficient can improve the recognition accuracy of the steel plate characters so as to complete the tracking and management of steel products according to the steel plate number.
In a specific implementation manner of the foregoing character intelligent recognition apparatus, further, the recognition module is configured to recognize the target picture by using a predetermined recognition model using non-maximum suppression, so as to obtain a tag, a position coordinate, and a recognition confidence of each character to be detected in the target picture.
In an embodiment of the foregoing character intelligent recognition apparatus, further, the recognition model includes: a deep convolutional neural network.
In an embodiment of the foregoing character intelligent recognition apparatus, further, the position confidence is expressed as:
wherein, LOC CONF
xFor positional confidence of character x, CONF
xFor recognition confidence, α is an adjustment factor, N is the number of characters, L
xW is the total width of the character, omega is the character size coefficient, i is the number of the character sequence,
is a reference point spatial position coordinate, L
offsetIs a coordinate offset.
In an embodiment of the foregoing character intelligent recognition apparatus, further, the filtering module includes:
the sorting unit is used for carrying out non-maximum value suppression according to the position confidence coefficient, and sorting each group of results of which the intersection ratio is greater than a preset intersection ratio threshold value according to the position confidence coefficient, wherein each group of results comprises: detecting the sequence number of the area, the group sequence number, the label and the position confidence of the character to be detected;
and the screening unit is used for screening out the labels of the characters to be detected, with the position confidence coefficient larger than the preset position confidence coefficient, so as to obtain the steel plate characters in the target picture.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.