CN112699883B - Identification method and identification system for plate spray code - Google Patents

Abstract

The invention discloses a plate code-spraying identification method which is characterized in that the plate code-spraying comprises P rows of code-spraying characters, wherein P is more than or equal to 3 and is a positive integer; the identification method comprises the following steps: acquiring an image of a plate code spraying; carrying out light reflection treatment on the image of the plate sprayed code to obtain a light-reflected image; binarizing the image after the light reflection treatment to obtain a binarized image; determining a region of interest (ROI) image from the binarized image; extracting character lines of the ROI image to obtain M line character images, wherein M is more than or equal to P; dividing and corroding each row of character image to obtain N single character images included in the plate spray code; predicting N single character images by using a neural network to obtain recognition results comprising N predicted characters; the method realizes the image recognition of the spray codes of the steel coil plates in complex arrangement, and obviously improves the recognition efficiency of the spray code characters.

Description

Identification method and identification system for plate spray code

Technical Field

The application relates to the technical field of ferrous metallurgy, in particular to a plate code spraying identification method and system.

Background

In the hot rolling or cold rolling production line of steel, after the rolled plate/coil is produced, a number spraying device is arranged on the production line to spray marks (the mark characters comprise steel coil numbers, specifications and steel types) on the surface of the plate. When spraying equipment goes wrong, can arrange the manual work and carry out the spraying, and the manual spraying appears because of manual misoperation spraying error message easily, consequently needs the quality inspection personnel scene to carry out the material information and compares, confirms the sign of manual spraying to carry out quality surface judgement, block deblocking, quality inspection operations such as off-line flaw detection according to the material information. At this time, the on-site quality inspector needs to take a picture of the spraying mark on each sheet, and take the picture back to the office and confirm the picture one by one; the manual confirmation mode is adopted, so that the workload of quality inspection personnel is obviously increased, the working efficiency is low, and the personnel identification result is lagged.

Disclosure of Invention

The invention provides a plate code-spraying identification method and a plate code-spraying identification system, which aim to solve or partially solve the technical problems of low identification efficiency and lagging result caused by manual identification of the plate code-spraying of the existing steel coil.

In order to solve the technical problems, the invention provides a plate code-spraying identification method, wherein the plate code-spraying comprises P rows of code-spraying characters, and P is more than or equal to 3 and is a positive integer; the identification method comprises the following steps:

acquiring an image of a plate code spraying;

carrying out light reflection treatment on the image of the plate sprayed code to obtain a light-reflected image;

binarizing the image after the light reflection treatment to obtain a binarized image;

determining a region of interest (ROI) image from the binarized image;

extracting character lines of the ROI image to obtain M line character images, wherein M is more than or equal to P;

dividing and corroding each row of character image to obtain N single character images included in the plate spray code;

and predicting the N single-character images by using the neural network to obtain a recognition result comprising N predicted characters.

Optionally, determining the ROI image from the binarized image specifically includes:

performing a first closing operation on the binarized image to obtain a first closed-processing image;

Performing a first opening operation on the first closed-process image to obtain a first open-process image;

performing first expansion on the first opening processing image to obtain a first expansion image;

determining a maximum contour of the first dilated image;

determining a minimum fit rectangle corresponding to a maximum contour of the first dilated image;

according to the vertex coordinates of the minimum fitting rectangle corresponding to the maximum outline of the first expansion image, a first image is intercepted from the image after the light reflection treatment;

rotating the first image to obtain a rotated first image;

performing a second closing operation on the rotated first image to obtain a second closed-processing image;

performing a second opening operation on the second closed processed image to obtain a second open processed image;

and denoising the second processing image to obtain an ROI image of the region of interest.

Further, after rotating the first image to obtain the rotated first image, the identification method further includes:

judging whether the rotated first image only comprises one row of code spraying characters or not;

if not, performing a second closing operation on the rotated first image to obtain a second closing processing image;

if yes, increasing the expansion coefficient, and performing second expansion on the first processed image to obtain a second expansion image;

Determining a maximum contour of the second dilation image;

determining a minimum fitting rectangle corresponding to the maximum outline of the second expansion image;

according to the vertex coordinates of the minimum fitting rectangle corresponding to the maximum outline of the second expansion image, the second image is intercepted from the image after the light reflection treatment;

rotating the second image to obtain a rotated second image;

scaling the rotated second image to obtain a scaled second image;

performing a second closing operation on the rotated first image to obtain a second closed-processing image, including:

and performing a second closing operation on the reduced second image to obtain a second closed-processing image.

According to the technical scheme, character line extraction is performed on the ROI image to obtain M line character images, and the method specifically comprises the following steps:

horizontally expanding the ROI image to obtain a horizontally expanded ROI image;

determining P line character outlines from the horizontally inflated ROI image;

determining a minimum fitting rectangle corresponding to each row character outline;

according to the minimum fitting rectangle corresponding to the line character outline, P line character images are intercepted from the ROI image after horizontal expansion;

acquiring the character spacing of all adjacent characters in each line character image, and judging whether the adjacent characters with the character spacing larger than or equal to a preset spacing exist in P line character images;

If so, dividing the corresponding line character image according to adjacent characters with the character spacing being greater than or equal to the preset spacing, and obtaining a divided line character image;

and obtaining M line character images according to the line character images with the character spacing between adjacent characters smaller than the preset spacing and the segmented line character images.

Further, after obtaining the M line character images, the recognition method further includes:

respectively carrying out vertical projection on each line character image, determining a left boundary and a right boundary of each line character image, and starting from the left boundary to the left, a first projection value in each pixel width and a second projection value in each pixel width from the right boundary to the right;

detecting whether the sum of the first projection values is larger than a pixel threshold value within a preset pixel width from the left boundary to the left; if the sum of the first projection values is greater than or equal to a pixel threshold value, moving a left boundary leftwards according to a first preset pixel width; if the sum of the first projection values is smaller than the pixel threshold value, deleting the left image with the left boundary;

detecting whether the sum of the second projection values is larger than a pixel threshold value or not within a preset pixel width from the right boundary to the right; if the sum of the second projection values is greater than or equal to the pixel threshold value, moving the right boundary to the right according to the preset pixel width; if the sum of the second projection values is smaller than the pixel threshold value, deleting the image right below the right boundary.

According to the technical scheme, each row character image is segmented and corroded to obtain N single character images included in the plate code spraying, and the method specifically comprises the following steps:

determining a plurality of first division points in each row of character images by adopting a vertical projection method;

determining a maximum character width and a minimum character width of the single character;

detecting whether the distance between adjacent first division points is larger than the maximum character width, corroding character images between the adjacent first division points with the distance larger than the maximum character width, and then vertically projecting to obtain second division points;

detecting whether the distance between the second division point and the adjacent second division point and/or the adjacent first division point is smaller than the minimum character width;

if the distance between the second division point and the adjacent second division point and/or the adjacent first division point is smaller than the minimum character width, deleting the corresponding second division point;

and dividing the line character image according to the first dividing point and the rest of the second dividing points to obtain N single character images included in the plate code spraying.

According to the technical scheme, the N single character images are predicted by using the neural network to obtain the recognition result comprising N predicted characters, and the method specifically comprises the following steps:

Carrying out pixel normalization on the N single character images to obtain normalized N single character images;

taking the normalized N single character images as input, and predicting by using a preset convolutional neural network model to obtain N predicted characters corresponding to the normalized N single character images;

and arranging N predicted characters sequentially to obtain a recognition result comprising N predicted characters.

According to the technical scheme, the light reflection treatment is carried out on the image of the plate code spraying to obtain the image after the light reflection treatment, and the method specifically comprises the following steps:

and (3) carrying out top hat transformation on the image of the plate sprayed code to obtain the image after the light reflection treatment.

According to the technical scheme, after the image of the plate spray code is acquired, the identification method further comprises the following steps:

determining the inclination angle of the plate code-spraying image, and performing inclination correction on the plate code-spraying image according to the inclination angle to obtain a corrected plate code-spraying image;

carrying out light reflection treatment on the image of the plate sprayed code to obtain a light-reflected image, wherein the method specifically comprises the following steps of:

and carrying out reflection treatment on the corrected plate code-spraying image to obtain a reflection treated image.

Based on the same invention conception as the technical scheme, the invention also provides a plate code-spraying identification system, wherein the plate code-spraying comprises P rows of code-spraying characters, and P is more than or equal to 3 and is a positive integer; the identification system comprises:

The acquisition module is used for acquiring an image of the plate spray code;

the light reflection processing module is used for carrying out light reflection processing on the plate code-spraying image to obtain a light-reflected image;

the binarization module is used for binarizing the image after the light reflection treatment to obtain a binarized image;

an ROI determining module for determining a region of interest ROI image from the binarized image;

the line character extraction module is used for extracting character lines of the ROI image to obtain M line character images, wherein M is more than or equal to P;

the single character segmentation module is used for segmenting and corroding each row of character image to obtain N single character images included in the plate code spraying;

and the neural network module is used for predicting the N single character images by using the neural network to obtain a recognition result comprising N predicted characters.

Through one or more technical schemes of the invention, the invention has the following beneficial effects or advantages:

the invention provides a character image recognition method suitable for plate code spraying in the field of steel production, which is characterized in that firstly, reflection treatment is carried out on an obtained plate code spraying image so as to eliminate adverse effect of a field illuminating lamp on image recognition due to uneven illumination; then binarizing the image after the reflection treatment, determining an ROI image from the binarized image, and then extracting M line character images in the ROI image by combining the distribution of character fragments; then, each row of character image is segmented and corroded to obtain a single character image, and the operation of segmentation and corrosion combination is adopted, because character adhesion possibly exists in manual code spraying, and accurate single character images can be segmented after the character adhesion is corroded; finally, predicting the single character image by using a neural network to obtain a character recognition result of the plate code spraying; by the method, the image recognition of the spray codes of the steel coil plates in complex arrangement is realized, and the recognition efficiency of the spray code characters is remarkably improved.

The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:

FIG. 1 shows a flow chart of a method for identifying a board code spray according to one embodiment of the invention;

FIG. 2 shows a schematic representation of an image of an ROI according to an embodiment of the present invention;

FIG. 3 illustrates a line character image extracted from an ROI image according to an embodiment of the present invention;

FIG. 4 illustrates a single character image after segmentation of a line character image in accordance with one embodiment of the present invention;

fig. 5 shows a schematic diagram of a board code-spraying identification system according to an embodiment of the invention.

Detailed Description

In order to make the technical solution more clearly understood by those skilled in the art, the following detailed description is made with reference to the accompanying drawings. Throughout the specification, unless specifically indicated otherwise, the terms used herein should be understood as meaning as commonly used in the art. Accordingly, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. In case of conflict, the present specification will control. The various devices and the like used in the present invention are commercially available or can be prepared by existing methods unless otherwise specifically indicated.

At present, character images are identified by adopting an image processing and neural network method, and the character images are already applied to the fields of automobile license plate number identification, bank card number identification and the like. However, the plate code spraying of the steel plate has the following difficulties in image recognition: the code spraying character is complex and comprises a trademark and a plurality of rows of characters, wherein the trademark is positioned at the upper left corner, and each row of characters comprises letters and numbers; a code-spraying character is formed by: the first row is the steel coil mark, the second row is the execution standard, the third row is the material number and the furnace number, and the fourth row is the specification and the team of the steel coil; the third line and the fourth line have different kinds of code spraying, so that one line of code spraying characters is divided into a plurality of character fragments. The characteristics of the plate code spraying lead to the fact that common image recognition and neural network methods cannot well accurately recognize code spraying characters.

Therefore, in an alternative embodiment, as shown in fig. 1, a method for identifying a plate code is provided, wherein the plate code includes P rows of code characters, and P is greater than or equal to 3 and is a positive integer; the whole idea is as follows:

s1: acquiring an image of a plate code spraying;

s2: carrying out light reflection treatment on the image of the plate sprayed code to obtain a light-reflected image;

s3: binarizing the image after the light reflection treatment to obtain a binarized image;

s4: determining a region of interest (ROI) image from the binarized image;

s5: extracting character lines of the ROI image to obtain M line character images, wherein M is more than or equal to P;

s6: dividing and corroding each row of character image to obtain N single character images included in the plate spray code;

s7: and predicting the N single-character images by using the neural network to obtain a recognition result comprising N predicted characters.

In general, the method provides a character image recognition method suitable for plate code spraying in the field of steel production, and as the light of a steel production workshop is dim enough and a lighting lamp is continuously started for 24 hours, compared with natural light, the non-uniform light of the lighting lamp is easier to reflect light on the surface of a steel plate, the obtained plate code spraying image is subjected to light reflection treatment so as to eliminate the adverse effect of non-uniform illumination on image recognition; then binarizing the image after the reflection treatment, determining an ROI image from the binarized image, and extracting a line character image from the ROI image; because some rows in the plate code spraying comprise more than two types of character fragments which are far apart, extracting code spraying characters according to the character fragments to obtain M row character images; then, each row of character image is segmented and corroded to obtain a single character image, and the operation of segmentation and corrosion combination is adopted, because character adhesion possibly exists in manual code spraying, and accurate single character images can be segmented after the character adhesion is corroded; finally, predicting the single character image by using a neural network to obtain a character recognition result of the plate code spraying; by the method, character image recognition of the code spraying of the steel coil plates in complex arrangement can be realized, and character recognition efficiency is remarkably improved.

Next, taking python language implementation as an example, the above scheme will be fully described:

first S1: acquiring an image of a plate code spraying; the image acquisition mode can be manual mobile phone photographing or camera photographing, or an industrial camera can be fixedly installed on site, and a photo comprising a plate code spraying is used as an image to be processed.

When an image is photographed and acquired, code-spraying characters in the image inevitably incline to a certain extent, and the inclined characters influence image processing in the subsequent process. Thus, tilt correction can be performed on the image: the inclination correction scheme is specifically as follows:

after the image of the plate code is obtained, determining the inclination angle of the image of the plate code, and carrying out inclination correction on the image of the plate code according to the inclination angle to obtain a corrected image of the plate code; and then carrying out reflection treatment on the corrected plate code-spraying image to obtain a reflection treated image.

When the inclination correction is carried out, the inclination angle comprises an initial angle deviation, a vertical angle deviation and a horizontal angle deviation, and according to one of the three angle deviations, the image is rotated by using a rotateRect function, and the inclination image obtained by photographing is corrected.

Next, step S2 is performed to perform a light reflection process on the corrected image, and optionally, top hat transformation is performed on the image to obtain a light-reflected image. Various reflection points on the steel plate can be eliminated through top cap conversion.

Next, according to step S3, the image after the light reflection processing is binarized. The binarization is to set the gray scale of all pixel points in the color image to 0 or 255, and OSTU binarization can be performed on the image after top hat transformation through a threshold function.

Next, according to S4, a region of interest ROI image including the code-sprayed character is determined from the binarized image. An alternative method for determining an ROI image provided in this embodiment is as follows:

s411: performing a first closing operation on the binarized image to obtain a first closed-processing image;

s412: performing a first opening operation on the first closed-process image to obtain a first open-process image;

s413: performing first expansion on the first opening processing image to obtain a first expansion image;

s414: determining a maximum contour of the first dilated image;

s415: determining a minimum fit rectangle corresponding to a maximum contour of the first dilated image;

s416: according to the vertex coordinates of the minimum fitting rectangle corresponding to the maximum outline of the first expansion image, the first image is intercepted from the image after the light reflection treatment;

S417: rotating the first image to obtain a rotated first image;

s418: performing a second closing operation on the rotated first image to obtain a second closed-processing image;

s419: performing a second opening operation on the second closed processed image to obtain a second open processed image;

s420: and denoising the second processing image to obtain an ROI image of the region of interest.

The scheme includes that firstly, closing operation is carried out on a binary image, and then opening operation is carried out; the closing operation is to expand the binary image by using structural elements and then to perform the corrosion operation; the opening operation is to use structural elements to erode the binarized image first, and then to expand the eroded image. The closing operation is firstly carried out to eliminate the narrow discontinuities and small holes on the pictures; the picture is then opened again to slide the object outline, break the narrow discontinuities and eliminate the fine protrusions. Through the combined operation of opening before closing, tiny impurities left in the image after the light reflection treatment can be reduced, so that the binarized image is clearer. Alternatively, the closing operation is implemented by the function MORPH_CLOSE and the opening operation is implemented by the MORPH_OPEN function.

And then performing expansion operation on the image after the opening operation to enable the code-spraying character area in the binarized image to form a connected area, wherein the expansion operation can be realized through a function dilate. The character outline is then looked up. Since the binarized image comprises a plurality of rows of code-spraying characters, a plurality of character outlines are searched, and the outline with the largest area is selected from the plurality of character outlines to be used as a possible ROI area. In order to realize interception of the ROI area, a minimum fitting rectangle (minimum circumscribing rectangle) is fitted to the contour with the largest area, and then an ROI area image is intercepted from the image after the reflection treatment according to the vertex coordinates of the minimum circumscribing rectangle; since the image is obtained by clipping from the image after the reflection processing, the ROI area image is binarized again and then rotated and aligned, and then the closing operation, the opening operation and the image noise reduction are sequentially performed, thereby obtaining the final ROI image, as shown in fig. 2.

Alternatively, finding the outline can be achieved by using a findContours function, selecting the outline with the largest area can be achieved by using a contourArea function, and acquiring the minimum circumscribed rectangle from the plurality of outlines can be achieved by using a minueatact function.

In the foregoing, since the board code-spraying includes a plurality of rows of characters, in order to avoid the error in the step of searching the maximum outline of the characters, the intercepted ROI image includes only one row of characters, and the first rotated image needs to be judged, an alternative method is as follows:

At S417: the first image is rotated, and after the rotated first image is obtained, the identification method further comprises the following steps:

s4171: judging whether the rotated first image only comprises one row of code spraying characters or not;

if not, execution S41: performing a closing operation on the rotated first image to obtain a second closed-processing image;

if yes, executing S413 after increasing the expansion coefficient, and performing second expansion on the first processed image to obtain a second expansion image;

starting from S414, the subsequent steps are changed to:

s414: determining a maximum contour of the second dilation image;

s415: determining a minimum fitting rectangle corresponding to the maximum outline of the second expansion image;

s416: the vertex coordinates of the minimum fitting rectangle corresponding to the maximum outline of the second expansion image are obtained, and the second image is intercepted from the image after the light reflection treatment;

s417: rotating the second image to obtain a rotated second image;

s418: scaling the rotated second image to obtain a scaled second image;

s419: performing a second closing operation on the rotated first image to obtain a second closed-processing image, including:

s420: and performing a second closing operation on the reduced second image to obtain a second closed-processing image.

In the scheme, judging whether the ROI image only comprises one row of characters or not can be realized by detecting the aspect ratio of the ROI image; through actual statistical analysis of the pixel height-to-width ratios of a large number of plate code-spraying characters, when the pixel height/pixel width of the ROI image is less than 0.2, the fact that only one row of characters are included in the ROI image at the moment can be confirmed; the plate code spraying at least comprises 3 rows of characters, so that the selection of the ROI area is unreasonable; at this time, the expansion coefficient of step S413 should be increased, the maximum contour should be redetermined, the minimum fitting rectangle corresponding to the maximum contour should be redefined, and the ROI area image should be redetected according to the minimum fitting rectangle. Since the expansion coefficient is further increased here, in order to avoid how large the ROI area image is, the effect of the subsequent image processing such as character segmentation is deteriorated, the ROI area image is appropriately reduced, and then the subsequent second closing operation, second opening operation, and noise reduction processing are performed. Scaling the image may be accomplished using a resize function.

After determining the ROI image, the next step is to extract the line character image from the ROI image according to the character lines, so as to facilitate the subsequent segmentation and erosion according to the line characters. A method for segmenting line characters from an ROI image is as follows:

S51: horizontally expanding the ROI image to obtain a horizontally expanded ROI image;

s52: determining P line character outlines from the horizontally inflated ROI image;

s53: determining a minimum fitting rectangle corresponding to each row character outline;

s54: according to the minimum fitting rectangle corresponding to the line character outline, P line character images are intercepted from the ROI image after horizontal expansion;

s55: acquiring the character spacing of all adjacent characters in each line character image, and judging whether the adjacent characters with the character spacing larger than or equal to a preset spacing exist in P line character images; if so, dividing the corresponding line character image according to adjacent characters with the character spacing being greater than or equal to the preset spacing, and obtaining a divided line character image;

s56: and obtaining M line character images according to the line character images with the character spacing between adjacent characters smaller than the preset spacing and the segmented line character images.

In general, the above scheme performs cluster merging according to the character distribution of the code-spraying characters and the character line distribution, so that each line of characters forms a line character set.

Specifically, the ROI image is firstly expanded horizontally, character outlines are determined according to rows in the image after horizontal expansion, a corresponding minimum circumscribed rectangle is determined according to the row character outlines, and then the row character image is intercepted from the binary image according to the minimum circumscribed rectangle. Optionally, after determining the minimum bounding rectangle, in order to determine whether the line character is valid, adding a row selection of the minimum bounding rectangle, and when detecting that the row of the minimum bounding rectangle is smaller than a preset pixel value, for example, 50 pixels, indicating that no valid code-spraying character exists in the minimum bounding rectangle at this time, deleting the minimum bounding rectangle with the row of the row character smaller than the preset pixel value.

Because of the specificity of the code-spraying characters of the plate, for the information of more than two character fragments included in some rows, obvious gaps exist between adjacent character fragments, such as the third row in fig. 2, including the material number (P220802222100) and the heat number (202H 02063) of the steel coil. The line character image cut out according to the character outline and the minimum circumscribed rectangle method is an entire line image including the two character segments. However, in the subsequent single character segmentation, the apparent spacing between character segments will affect the accuracy of the single character segmentation, and thus in this embodiment, a line character image including two or more character segments is further segmented. And taking adjacent characters with the spacing larger than the preset spacing as reference points, performing internal division on character fragments far apart in each row of character images to obtain M rows of character images in total, wherein one row of character image comprises one character fragment, as shown in fig. 3.

After the line character image is acquired, since noise may exist on the left and right sides of the line character image, the left and right boundaries of the line character image that meets the condition need to be determined, and then threshold judgment needs to be performed on the left and right boundaries respectively. An alternative scheme is as follows:

After obtaining the M line character images, the recognition method further includes:

respectively carrying out vertical projection on each line character image, determining a left boundary and a right boundary of each line character image, and starting from the left boundary to the left, a first projection value in each pixel width and a second projection value in each pixel width from the right boundary to the right;

detecting whether the sum of the first projection values is larger than a pixel threshold value within a preset pixel width from the left boundary to the left; if the sum of the first projection values is greater than or equal to a pixel threshold value, moving a left boundary leftwards according to a first preset pixel width; if the sum of the first projection values is smaller than the pixel threshold value, deleting the left image with the left boundary;

detecting whether the sum of the second projection values is larger than a pixel threshold value or not within a preset pixel width from the right boundary to the right; if the sum of the second projection values is greater than or equal to the pixel threshold value, moving the right boundary to the right according to the preset pixel width; if the sum of the second projection values is smaller than the pixel threshold value, deleting the image right below the right boundary.

In the above-described range, the first projection value and the second projection value are calculated for each pixel width range, and the pixel point having a gray value of 255 (white) is counted in the vertical direction. According to the characteristic of the plate code spraying, the range of the preset pixel width can be 20-50 pixel widths, and the selectable range of the pixel threshold value is 200-400. Taking a preset pixel width as 35 and a pixel threshold value as 300 as an example, when the sum of white pixel points in 35 pixel widths is larger than 300 according to vertical projection in a range of 35 pixel widths from the right boundary to the right, indicating that a code spraying character exists in the range of 35 pixel widths from the right boundary to the right, and moving the right boundary of the line character to be calculated in the line character; if the number is less than 300, it indicates that no character or noise exists in the width range of 35 pixels from the right boundary to the right, and the image should be shaved to avoid influencing the segmentation of single characters in the subsequent steps.

Optionally, after obtaining M line character images, it is further determined whether the first line character image includes a logo image (product logo) of the blank, and if the logo is included, the logo needs to be removed. In the field of character recognition, mixing logo images in characters has been a difficulty in recognition. As shown in fig. 2, the upper left corner of the code-spraying character is a logo image (logo) of the product, and according to the number of occupied lines, the code-spraying character can be divided into a first logo image occupying two lines and a second logo image occupying one line; when dividing the character, the method adopted is to horizontally expand and select the connected area, so two situations can occur: the second mark logo and the second line of characters form a communication area; the first mark image forms a communication area with the second mark image, the first line character and the second line character. Therefore, when removing logo, it is necessary to consider the two cases to be processed separately. Because of the difference of shooting angles and distances of each photo, a fixed length cannot be set to directly cut out logo; thus, the first or second marker image may be segmented as follows:

after obtaining the M line character images, the recognition method further includes:

performing vertical projection on the first line character image to obtain a plurality of right boundaries of the first line character image;

Judging whether the left character image of the first right boundary is broken or not and is adhered to the right character of the first right boundary or not;

when the character image of the left of the first right boundary is not broken and is not adhered to the character of the right of the first right boundary, determining the first right boundary as a dividing point of the mark image;

when the left character image of the first right boundary is not broken but is adhered to the right character of the first right boundary, performing corrosion-vertical projection recursion operation on the first right boundary and the images in the first right boundary, and determining the first right boundary meeting the preset condition after recursion as a dividing point of the mark image;

when the first right boundary is broken by the left character image and is not adhered with the right character of the first right boundary, traversing all right boundaries, determining a target right boundary meeting preset conditions, and determining the target right boundary as a dividing point of the mark image;

when the first right boundary is broken by the left character image and is adhered with the character from the first right boundary to the right, the character image from the first right boundary to the left and the character image from the first right boundary to the right are segmented, and the segmentation point is determined as the segmentation point of the mark image;

Deleting the character image left of the dividing point of the mark image to obtain the first line character image with the mark image removed.

When judging whether the character image with the left at the first right boundary is broken or stuck or not, the scheme can carry out numerical judgment according to the pixel width of the character image with the left at the first right boundary and the respective width ranges of the first mark image and the second mark image. For example, the width of the first logo image ranges from 140 to 160; when the first right boundary reaches 180, indicating that the character adhesion exists; when the pixel width is less than the minimum width of the first logo image, such as 140, it is indicated that there is a break.

When the first right boundary is determined, the judging modes of the first mark images and the second mark images with different row occupation numbers are different, and the thought is processed according to the characteristics of the junction of the logo and the character, and the method specifically comprises the following steps:

for the first mark image, since the initial positions of the two rows of characters behind the mark image are the same, and the upper left area of the initial position of the first row of characters is blank; therefore, the height is one sixth of the row of the first row character image, the width is the width of a single character, the window is left-slid from the upper direction of the middle position of the first row character image, when the sum of white pixels in the window is not 0, the right boundary of the window is indicated as the character starting position, and the position of the current window is determined as the first right boundary;

For the second logo image, the first line character image occupies the entire line because it occupies the upper half or the lower half of the height of the line; according to this feature, windows of a preset pixel (4 to 8, preferably 5 pixels) width, with two third of the first line character image in height, slide synchronously from left to right at the upper third and lower third, respectively; when the sum of the white pixels in one window is 0, the judgment of the position where neither of the white pixels in the two windows is 0 is started, and the position is the character starting position, and the character starting position is determined as the first right boundary.

After the logo is removed, if the first line character image comprises two lines of characters, the first line character image can be divided into two lines of character images so as to facilitate the subsequent single character cutting.

Optionally, after obtaining M line character images, determining whether M is less than 3, if so, indicating that the line character segmentation fails, and executing steps S51 to S56 again is required.

After the line character image is obtained, the single character segmentation step of S6 may be performed. Because the manual code spraying is easy to produce character adhesion, the key characteristics in the characters are required to be reinforced. An alternative method is thus as follows:

S61: determining a plurality of first division points in each row of character images by adopting a vertical projection method;

s62: determining a maximum character width and a minimum character width of the single character;

s63: detecting whether the distance between adjacent first division points is larger than the maximum character width, corroding character images between the adjacent first division points with the distance larger than the maximum character width, and then vertically projecting to obtain second division points;

s64: detecting whether the distance between the second division point and the adjacent second division point and/or the adjacent first division point is smaller than the minimum character width;

s65: if the distance between the second division point and the adjacent second division point and/or the adjacent first division point is smaller than the minimum character width, deleting the corresponding second division point;

s66: and dividing the line character image according to the first dividing point and the rest of the second dividing points to obtain N single character images included in the plate code spraying.

Firstly, determining a plurality of first segmentation points for segmenting characters in a line character image by adopting vertical projection, then fixing the first character segments of the third line to 13 characters according to priori knowledge, and calculating the possible range of single character width according to the width range of the first character segments of the third line. After determining that the characters are adhered, obtaining image copies of adhered character areas, then continuously using small structural elements to perform corrosion operation, and then determining second segmentation points between adjacent first segmentation points by using vertical projection to serve as segmentation points for performing secondary segmentation on the adhered character areas; and judging whether the pixel width between two adjacent dividing points is larger than the maximum character width or not again, and if the pixel width between two adjacent dividing points is smaller than the maximum character width, indicating that the adhered characters are completely separated.

Because the single character may be broken during corrosion, the obtained second division points are judged according to the minimum character width of the single character, and if the interval between a certain second division point and the adjacent first division point or the second division point is smaller than the minimum character width, the character breaking caused by corrosion is described, and the second division point is deleted. Finally, the line character image is divided into single character images according to the first division point and the undeleted second division point, as shown in fig. 4.

After the single character image is obtained, the neural network prediction in S7 is performed as follows:

s71: carrying out pixel normalization on the N single character images to obtain normalized N single character images;

s72: taking the normalized N single character images as input, and predicting by using a preset convolutional neural network model to obtain N predicted characters corresponding to the normalized N single character images;

s73: and arranging N predicted characters sequentially to obtain a recognition result comprising N predicted characters.

The preset roll diameter neural network model in the scheme can be obtained through training according to the following 3 steps:

(a) Collecting a large number of segmented single character image samples, constructing a training database after pixel normalization, and classifying according to character characteristics; then, dividing the training set, the verification set and the test set in the training database according to character characteristics, for example, 80% of samples can be divided into the training data set, 10% of samples can be divided into the verification data set, and 10% of samples can be divided into the test data set;

(b) Constructing a convolutional neural network CNN model, and setting a CNN network structure, a loss function, an optimizer and a training strategy; training the model parameters of the CNN by using the training data set, continuously adjusting the model according to the verification data set and the test data set, and selecting the optimal model structure and super parameters;

(c) And training by applying all data samples based on the optimal model, and storing model parameters with optimal generalization performance for predicting the newly input character image.

Based on the same inventive concept as the previous embodiment, in yet another alternative embodiment, as shown in fig. 5, a board code-spraying identification system is provided, where the board code-spraying includes P rows of code-spraying characters, and P is greater than or equal to 3 and is a positive integer; the identification system comprises:

the acquisition module 10 is used for acquiring an image of the plate code spraying;

the light reflection processing module 20 is used for carrying out light reflection processing on the plate code-spraying image to obtain a light-reflected image;

a binarization module 30 for binarizing the image after the light reflection processing to obtain a binarized image;

a ROI determination module 40 for determining a region of interest ROI image from the binarized image;

the line character extraction module 50 is used for carrying out character line extraction on the ROI image to obtain M line character images, wherein M is more than or equal to P;

The single character segmentation module 60 is configured to segment and erode each line character image to obtain N single character images included in the board code;

the neural network module 70 is configured to predict the N single-character images by using the neural network, and obtain a recognition result including N predicted characters.

Optionally, the light reflection processing module 20 is specifically configured to:

and (3) carrying out top hat transformation on the image of the plate sprayed code to obtain the image after the light reflection treatment.

Optionally, the identification system further comprises a tilt correction module, specifically configured to:

after the image of the plate code is obtained, determining the inclination angle of the image of the plate code, and carrying out inclination correction on the image of the plate code according to the inclination angle to obtain a corrected image of the plate code;

carrying out light reflection treatment on the image of the plate sprayed code to obtain a light-reflected image, wherein the method specifically comprises the following steps of:

and carrying out reflection treatment on the corrected plate code-spraying image to obtain a reflection treated image.

Optionally, the ROI determination module 40 is specifically configured to:

performing a first closing operation on the binarized image to obtain a first closed-processing image;

performing a first opening operation on the first closed-process image to obtain a first open-process image;

Performing first expansion on the first opening processing image to obtain a first expansion image;

determining a maximum contour of the first dilated image;

determining a minimum fit rectangle corresponding to a maximum contour of the first dilated image;

according to the vertex coordinates of the minimum fitting rectangle corresponding to the maximum outline of the first expansion image, a first image is intercepted from the image after the light reflection treatment;

rotating the first image to obtain a rotated first image;

performing a second closing operation on the rotated first image to obtain a second closed-processing image;

performing a second opening operation on the second closed processed image to obtain a second open processed image;

and denoising the second processing image to obtain an ROI image of the region of interest.

Further, the identification system further comprises:

the judging module is used for judging whether the rotated first image only comprises one row of code spraying characters;

if not, the ROI determination module 40 is specifically configured to:

performing a second closing operation on the rotated first image to obtain a second closed-processing image;

if yes, the ROI determination module 40 specifically is configured to:

increasing the expansion coefficient, and performing second expansion on the first processed image to obtain a second expansion image;

determining a maximum contour of the second dilation image;

Determining a minimum fitting rectangle corresponding to the maximum outline of the second expansion image;

according to the vertex coordinates of the minimum fitting rectangle corresponding to the maximum outline of the second expansion image, the second image is intercepted from the image after the light reflection treatment;

rotating the second image to obtain a rotated second image;

scaling the rotated second image to obtain a scaled second image;

performing a second closing operation on the rotated first image to obtain a second closed-processing image, including:

and performing a second closing operation on the reduced second image to obtain a second closed-processing image.

Optionally, the line character extraction module 50 is specifically configured to:

horizontally expanding the ROI image to obtain a horizontally expanded ROI image;

determining P line character outlines from the horizontally inflated ROI image;

determining a minimum fitting rectangle corresponding to each row character outline;

according to the minimum fitting rectangle corresponding to the line character outline, P line character images are intercepted from the ROI image after horizontal expansion;

acquiring the character spacing of all adjacent characters in each line character image, and judging whether the adjacent characters with the character spacing larger than or equal to a preset spacing exist in P line character images;

If so, dividing the corresponding line character image according to adjacent characters with the character spacing being greater than or equal to the preset spacing, and obtaining a divided line character image;

and obtaining M line character images according to the line character images with the character spacing between adjacent characters smaller than the preset spacing and the segmented line character images.

Further, the line character extraction module 50 is further configured to:

respectively carrying out vertical projection on each line character image, determining a left boundary and a right boundary of each line character image, and starting from the left boundary to the left, a first projection value in each pixel width and a second projection value in each pixel width from the right boundary to the right;

detecting whether the sum of the first projection values is larger than a pixel threshold value within a preset pixel width from the left boundary to the left; if the sum of the first projection values is greater than or equal to a pixel threshold value, moving a left boundary leftwards according to a first preset pixel width; if the sum of the first projection values is smaller than the pixel threshold value, deleting the left image with the left boundary;

detecting whether the sum of the second projection values is larger than a pixel threshold value or not within a preset pixel width from the right boundary to the right; if the sum of the second projection values is greater than or equal to the pixel threshold value, moving the right boundary to the right according to the preset pixel width; if the sum of the second projection values is smaller than the pixel threshold value, deleting the image right below the right boundary.

Optionally, the single character segmentation module 60 is specifically configured to:

determining a plurality of first division points in each row of character images by adopting a vertical projection method;

determining a maximum character width and a minimum character width of the single character;

detecting whether the distance between adjacent first division points is larger than the maximum character width, corroding character images between the adjacent first division points with the distance larger than the maximum character width, and then vertically projecting to obtain second division points;

detecting whether the distance between the second division point and the adjacent second division point and/or the adjacent first division point is smaller than the minimum character width;

if the distance between the second division point and the adjacent second division point and/or the adjacent first division point is smaller than the minimum character width, deleting the corresponding second division point;

and dividing the line character image according to the first dividing point and the rest of the second dividing points to obtain N single character images included in the plate code spraying.

Optionally, the neural network module 70 is specifically configured to:

carrying out pixel normalization on the N single character images to obtain normalized N single character images;

taking the normalized N single character images as input, and predicting by using a preset convolutional neural network model to obtain N predicted characters corresponding to the normalized N single character images;

And arranging N predicted characters sequentially to obtain a recognition result comprising N predicted characters.

Through one or more embodiments of the present invention, the present invention has the following benefits or advantages:

the invention provides a character image recognition method suitable for plate code spraying in the field of steel production, which is characterized in that firstly, reflection treatment is carried out on an obtained plate code spraying image so as to eliminate adverse effect of a field illuminating lamp on image recognition due to uneven illumination; then binarizing the image after the reflection treatment, determining an ROI image from the binarized image, and then extracting M line character images in the ROI image by combining the distribution of character fragments; then, each row of character image is segmented and corroded to obtain a single character image, and the operation of segmentation and corrosion combination is adopted, because character adhesion possibly exists in manual code spraying, and accurate single character images can be segmented after the character adhesion is corroded; finally, predicting the single character image by using a neural network to obtain a character recognition result of the plate code spraying; by the method, the image recognition of the spray codes of the steel coil plates in complex arrangement is realized, and the character recognition efficiency is remarkably improved;

The key of the scheme is that the ROI image of the code spraying character is confirmed, a line character image is extracted from the ROI image, and then a single character image is obtained by character segmentation from the line character image; in order to further improve the precision of character processing and the accuracy of neural network prediction, firstly, the combined form operation of opening before closing is carried out on the ROI image so as to reduce tiny impurities on the picture and make the picture clearer; then judging whether the ROI only comprises one row of characters, if so, increasing the expansion structural elements to redetermine the ROI, and then sequentially performing zooming, closing operation, opening operation and noise reduction to obtain a more accurate ROI image; when extracting the row character image of the ROI image, determining whether the row character image only comprises one row of code spraying characters according to the aspect ratio of the row characters; for a row comprising a plurality of character fragments, dividing the character fragments with far intervals in the row according to the character spacing, so that the obtained row character image is more beneficial to the recognition of a neural network; after the line character image is determined, judging according to a threshold value of a vertical projection value at the boundary, and eliminating noise points at the left side and the right side of the line character image; and finally, when single character division is carried out on the line character image, separating the adhered characters in the line character image by combining vertical projection segmentation with corrosion operation, so as to obtain an accurate single character image. Through the series of means, the accuracy of single character segmentation can be remarkably improved, so that the recognition efficiency and accuracy of the follow-up neural network prediction are ensured.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.

Claims (10)

1. The plate code spraying identification method is characterized in that the plate code spraying comprises P rows of code spraying characters, wherein P is more than or equal to 3 and is a positive integer; the identification method comprises the following steps:

acquiring an image of the plate spray code;

carrying out light reflection treatment on the plate code-spraying image to obtain a light-reflected image;

binarizing the image after the light reflection treatment to obtain a binarized image;

determining a region of interest (ROI) image from the binarized image;

extracting character lines of the ROI image to obtain M line character images, wherein M is more than or equal to P;

Dividing and corroding each row character image to obtain N single character images included in the plate code spraying;

predicting the N single character images by using a neural network to obtain a recognition result comprising N predicted characters;

wherein after the M line character images are obtained, the recognition method further includes:

performing vertical projection on the first line character image to obtain a plurality of right boundaries of the first line character image;

judging whether the left character image of the first right boundary is broken or not and is adhered to the right character of the first right boundary or not;

when the character image of the left of the first right boundary is not broken and is not adhered to the character of the right of the first right boundary, determining the first right boundary as a dividing point of the mark image;

when the left character image of the first right boundary is not broken but is adhered to the right character of the first right boundary, performing corrosion-vertical projection recursion operation on the first right boundary and the images in the first right boundary, and determining the first right boundary meeting the preset condition after recursion as a dividing point of the mark image;

when the first right boundary is broken by the left character image and is not adhered with the right character of the first right boundary, traversing all right boundaries, determining a target right boundary meeting preset conditions, and determining the target right boundary as a dividing point of the mark image;

When the first right boundary is broken by the left character image and is adhered with the character from the first right boundary to the right, the character image from the first right boundary to the left and the character image from the first right boundary to the right are segmented, and the segmentation point is determined as the segmentation point of the mark image;

deleting the character image left of the dividing point of the mark image to obtain the first line character image with the mark image removed.

2. The identification method according to claim 1, wherein said determining a region of interest ROI image from said binarized image, in particular comprises:

performing a first closing operation on the binarized image to obtain a first closed-processing image;

performing a first opening operation on the first closed-process image to obtain a first open-process image;

performing first expansion on the first opening processing image to obtain a first expansion image;

determining a maximum contour of the first dilation image;

determining a minimum fit rectangle corresponding to a maximum contour of the first dilated image;

according to the vertex coordinates of the minimum fitting rectangle corresponding to the maximum outline of the first expansion image, a first image is intercepted from the image after the light reflection treatment;

rotating the first image to obtain a rotated first image;

Performing a second closing operation on the rotated first image to obtain a second closed-processing image;

performing a second opening operation on the second closed-process image to obtain a second open-process image;

and denoising the second processing image to obtain the region of interest (ROI) image.

3. The identification method of claim 2, wherein after rotating the first image to obtain a rotated first image, the identification method further comprises:

judging whether the rotated first image only comprises one row of code spraying characters or not;

if not, performing a second closing operation on the rotated first image to obtain a second closing processing image;

if yes, increasing the expansion coefficient, and performing second expansion on the first processed image to obtain a second expansion image;

determining a maximum contour of the second dilation image;

determining a minimum fitting rectangle corresponding to the maximum outline of the second expansion image;

according to the vertex coordinates of the minimum fitting rectangle corresponding to the maximum outline of the second expansion image, a second image is intercepted from the image after the light reflection treatment;

rotating the second image to obtain a rotated second image;

Scaling the rotated second image to obtain a scaled second image;

and performing a second closing operation on the rotated first image to obtain a second closed-processing image, wherein the method specifically comprises the following steps of:

and performing a second closing operation on the reduced second image to obtain the second closed-processing image.

4. The method of claim 1, wherein the extracting the character lines from the ROI image to obtain M line character images specifically includes:

horizontally expanding the ROI image to obtain a horizontally expanded ROI image;

determining P line character outlines from the horizontally inflated ROI image;

determining a minimum fitting rectangle corresponding to each line character outline;

according to the minimum fitting rectangle corresponding to the line character outline, P line character images are intercepted from the ROI image after horizontal expansion;

acquiring the character spacing of all adjacent characters in each row of character images, and judging whether the adjacent characters with the character spacing larger than or equal to a preset spacing exist in the P row of character images;

if so, dividing the corresponding line character image according to the adjacent characters with the character spacing being greater than or equal to the preset spacing, and obtaining a divided line character image;

And obtaining the M line character images according to the line character images with the character spacing of adjacent characters smaller than the preset spacing and the segmented line character images.

5. The recognition method according to claim 4, wherein after the obtaining of the M line character images, the recognition method further comprises:

respectively carrying out vertical projection on each row character image, determining a left boundary and a right boundary of each row character image, and starting from the left boundary to the left, a first projection value in each pixel width and a second projection value in each pixel width from the right boundary to the right;

detecting whether the sum of the first projection values is larger than a pixel threshold value within a preset pixel width from the left boundary to the left; if the sum of the first projection values is greater than or equal to a pixel threshold value, moving the left boundary leftwards according to the preset pixel width; if the sum of the first projection values is smaller than a pixel threshold value, deleting the left image of the left boundary;

detecting whether the sum of the second projection values is larger than a pixel threshold value or not within a preset pixel width from the right boundary to the right; if the sum of the second projection values is greater than or equal to a pixel threshold value, moving the right boundary to the right according to the preset pixel width; and deleting the image right by the right boundary if the sum of the second projection values is smaller than a pixel threshold value.

6. The method for identifying a plate according to claim 1, wherein the step of dividing and corroding each of the line character images to obtain N single character images included in the plate code-spraying specifically includes:

determining a plurality of first division points in each row of character images by adopting a vertical projection method;

determining a maximum character width and a minimum character width of the single character;

detecting whether the distance between adjacent first division points is larger than the maximum character width, corroding character images between the adjacent first division points with the distance larger than the maximum character width, and then vertically projecting to obtain second division points;

detecting whether the distance between the second division point and the adjacent second division point and/or the adjacent first division point is smaller than the minimum character width;

if the distance between the second division point and the adjacent second division point and/or the adjacent first division point is smaller than the minimum character width, deleting the corresponding second division point;

and dividing the line character image according to the first dividing points and the rest second dividing points to obtain N single character images included in the plate code spraying.

7. The recognition method according to claim 1, wherein predicting the N single character images using a neural network to obtain a recognition result including N predicted characters, specifically includes:

Carrying out pixel normalization on the N single character images to obtain normalized N single character images;

using the normalized N single character images as input, and predicting by using a preset convolutional neural network model to obtain N predicted characters corresponding to the normalized N single character images;

and arranging the N predicted characters sequentially to obtain the recognition result comprising the N predicted characters.

8. The method for identifying a plate according to claim 1, wherein the step of performing a light reflection treatment on the image of the plate sprayed code to obtain a light-reflected image comprises the following steps:

and carrying out top hat transformation on the image of the plate sprayed code to obtain the image after the light reflection treatment.

9. The method of claim 1, wherein after said capturing the image of the board spray code, the method further comprises:

determining the inclination angle of the plate code-spraying image, and performing inclination correction on the plate code-spraying image according to the inclination angle to obtain a corrected plate code-spraying image;

the method for carrying out the light reflection treatment on the plate code-spraying image to obtain the light-reflected image specifically comprises the following steps:

And carrying out reflection treatment on the corrected plate code-spraying image to obtain the reflection treated image.

10. The plate code-spraying identification system is characterized in that the plate code-spraying comprises P rows of code-spraying characters, wherein P is more than or equal to 3 and is a positive integer; the identification system comprises:

the acquisition module is used for acquiring the image of the plate spray code;

the light reflection processing module is used for carrying out light reflection processing on the image of the plate sprayed code to obtain a light reflection processed image;

the binarization module is used for binarizing the image after the light reflection treatment to obtain a binarized image;

an ROI determining module for determining a region of interest ROI image from the binarized image;

the line character extraction module is used for extracting character lines of the ROI image to obtain M line character images, wherein M is more than or equal to P;

the single character segmentation module is used for segmenting and corroding each row character image to obtain N single character images included in the plate spray code;

the neural network module is used for predicting the N single character images by using a neural network to obtain a recognition result comprising N predicted characters;

wherein, the line character extraction module is further configured to:

Performing vertical projection on the first line character image to obtain a plurality of right boundaries of the first line character image;

judging whether the left character image of the first right boundary is broken or not and is adhered to the right character of the first right boundary or not;

when the character image of the left of the first right boundary is not broken and is not adhered to the character of the right of the first right boundary, determining the first right boundary as a dividing point of the mark image;

when the left character image of the first right boundary is not broken but is adhered to the right character of the first right boundary, performing corrosion-vertical projection recursion operation on the first right boundary and the images in the first right boundary, and determining the first right boundary meeting the preset condition after recursion as a dividing point of the mark image;

when the first right boundary is broken by the left character image and is not adhered with the right character of the first right boundary, traversing all right boundaries, determining a target right boundary meeting preset conditions, and determining the target right boundary as a dividing point of the mark image;

when the first right boundary is broken by the left character image and is adhered with the character from the first right boundary to the right, the character image from the first right boundary to the left and the character image from the first right boundary to the right are segmented, and the segmentation point is determined as the segmentation point of the mark image;

Deleting the character image left of the dividing point of the mark image to obtain the first line character image with the mark image removed.

Images