CN114612427A

CN114612427A - Nameplate defect detection method and device, electronic equipment and storage medium

Info

Publication number: CN114612427A
Application number: CN202210233522.0A
Authority: CN
Inventors: 王春芳; 陆华章; 徐强; 王晓琳; 梁治华
Original assignee: Gree Electric Appliances Inc of Zhuhai
Current assignee: Gree Electric Appliances Inc of Zhuhai
Priority date: 2022-03-10
Filing date: 2022-03-10
Publication date: 2022-06-10

Abstract

The application relates to a nameplate defect detection method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a first picture obtained by photographing a nameplate; identifying the first picture, and determining a template picture corresponding to the first picture and a date area in the first picture; erasing the date in the date area in the first picture to obtain a second picture; and comparing the second picture with the template picture to determine the defect area of the nameplate. The scheme provided by the invention can realize the detection of the nameplate defect.

Description

Nameplate defect detection method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of defect detection technologies, and in particular, to a method and an apparatus for detecting a defect of a nameplate, an electronic device and a storage medium.

Background

The laser marking instruments in the market have wide application range, before marking, a user needs to introduce patterns or characters, adjust parameters, fill and the like (such as CAD drawing generally), and during the period, the operation and artificial emotion of the user can cause the defect of the nameplate and influence the subsequent production.

At present, in an actual production environment, the nameplate is inspected by means of manual first inspection, the number of characters of the nameplate is large, fatigue and other factors exist in manual review, and production efficiency is reduced.

Disclosure of Invention

The application provides a nameplate defect detection method and device, electronic equipment and a storage medium, and aims to solve the technical problem of low production efficiency caused by manual detection of nameplate defects.

In a first aspect, the present application provides a nameplate defect detection method, including:

acquiring a first picture obtained by photographing a nameplate;

identifying the first picture, and determining a template picture corresponding to the first picture and a date area in the first picture;

erasing the date in the date area in the first picture to obtain a second picture;

and comparing the second picture with the template picture to determine the defect area of the nameplate.

In an embodiment, the identifying the first picture and the determining the template picture corresponding to the first picture and the date area in the first picture includes:

identifying the first picture, determining a bar code area in the first picture, and acquiring bar code characters in the bar code area;

based on the bar code characters, searching a template picture corresponding to the bar code characters in a preset template picture set;

and determining a date area in the first picture based on the corresponding relation between the barcode area and the date area by using the barcode area.

In an embodiment, before erasing the date in the date area in the first picture, the method further comprises:

acquiring a date in the date area;

judging whether the date is consistent with a preset date or not;

and when the date is inconsistent with a preset date, judging that the defect area of the nameplate comprises the date area.

In an embodiment, the searching, based on the barcode character, a template drawing corresponding to the barcode character in a preset template drawing set includes:

judging whether a template picture corresponding to the bar code character is found in a preset template picture set or not based on the bar code character;

when the template picture corresponding to the bar code character is found in a preset template picture set, acquiring the template picture;

and prompting to input a correct bar code when the template picture corresponding to the bar code character is not found in the preset template picture set.

In an embodiment, before comparing the second picture with the template picture, the method further comprises:

identifying the template drawing and determining a date area in the template drawing;

erasing the date in the date area in the template picture.

In an embodiment, the comparing the second picture with the template picture to determine the defect area of the nameplate includes:

and carrying out global matching and sectional screenshot matching on the second picture and the template picture, and determining the defect area of the nameplate.

In an embodiment, the step of performing screenshot matching on the second picture and the template picture in different areas and determining the defect area of the nameplate includes:

respectively intercepting the second picture and the template picture in regions to obtain a first region picture and a second region picture;

continuously moving the coordinates of the four vertex angles in the first region picture, and matching the picture subjected to perspective transformation of the four vertex angles of the first region picture with the second region picture to obtain a matching result with the minimum difference;

and determining a defect area of the nameplate based on the matching result.

In a second aspect, the present application provides a nameplate defect detecting device, comprising:

the acquisition module is used for acquiring a first picture obtained by photographing a nameplate;

the identification module is used for identifying the first picture and determining a template picture corresponding to the first picture and a date area in the first picture;

the erasing module is used for erasing the date in the date area in the first picture to obtain a second picture;

and the comparison module is used for comparing the second picture with the template picture to determine the defect area of the nameplate.

In a third aspect, the present application provides an electronic device, comprising: a processor and a memory for storing a computer program capable of running on the processor; wherein,

the processor is configured to perform the steps of the method according to any of the embodiments of the first aspect when running the computer program.

In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, where the computer program is executed by a processor to implement the steps of the method according to any one of the embodiments of the first aspect.

Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages:

the method provided by the embodiment of the application can identify characters in the nameplate and icons in the nameplate, and can realize accurate detection of nameplate defects.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a schematic flow chart illustrating a nameplate defect detecting method according to an embodiment of the present disclosure;

FIG. 2 is a schematic view of an overall process for detecting defect of a nameplate according to an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a nameplate defect detecting apparatus according to an embodiment of the present application;

FIG. 4 is a schematic diagram illustrating a preset template map set according to an embodiment of the present application;

FIG. 5 is a schematic diagram of 2 template images in a template image set preset in the application example of the present application;

FIG. 6 is a schematic diagram illustrating a positive effect of an embodiment of the present application;

FIG. 7 is a schematic diagram of a barcode character recognition process according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a date location detection in an embodiment of the present application;

FIG. 9 is a schematic diagram of template matching according to an embodiment of the present application;

FIG. 10 is a diagram illustrating a test result according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Fig. 1 is a schematic flow chart of a nameplate defect detection method according to an embodiment of the present application. As shown in fig. 1, the method includes:

step 101: acquiring a first picture obtained by photographing a nameplate;

step 102: identifying the first picture, and determining a template picture corresponding to the first picture and a date area in the first picture;

step 103: erasing the date in the date area in the first picture to obtain a second picture;

step 104: and comparing the second picture with the template picture to determine the defect area of the nameplate.

Specifically, the template drawing in this embodiment may be a CAD drawing according to which the nameplate is manufactured. In practical application, the template drawing can be commanded by the bar code characters in the drawing, so that the searching is convenient.

Because the date of the date area in the CAD image can be changed based on the date in the manufacturing process of the nameplate, the date in the date area in the first image needs to be erased to obtain the second image in order to realize accurate identification during identification. And comparing the second picture with the template picture so as to detect the defect area in the nameplate.

Further, before the first picture is identified, the first picture may be corrected. The method specifically comprises the following steps: acquiring a maximum connected domain of a first picture, acquiring a peripheral frame outline and a minimum external rectangle of the first picture, taking 1/4 on the shortest side of the external rectangle as the length, taking 5 pixels as a step length, traversing points in the maximum connected domain by using the step length, acquiring a line segment by each traversal, and calculating to acquire a slope and an offset according to a starting point; and dividing the data into four groups of collinear data according to the slope angles, and respectively taking the mean values of the slope and the offset as optimal values to perform straight line fitting to obtain four frame fitted straight lines. And obtaining four intersection points of the straight line to perform perspective transformation on the picture to obtain the corrected picture.

Since the barcode region is generally located at the lower right corner of the nameplate, the 1/3 region at the bottom of the first picture can be captured, and the 1/3 region at the bottom can be processed to determine the barcode region in the first picture. Here, the 1/3 area at the bottom is processed, which may specifically be: and (3) calculating gradients in the x direction and the y direction in the 1/3 area, subtracting the gradient in the y direction from the gradient in the x direction to obtain an absolute value, and performing expansion and corrosion operations on the effect map for multiple times to obtain a maximum profile in the 1/3 area as a bar code. Further, 1/2 of the height of the bar code can be taken to extend downwards to obtain bar code numbers, and the deep learning convolutional neural network is adopted to carry out character recognition to obtain bar code characters.

acquiring a date in the date area;

judging whether the date is consistent with a preset date or not;

In practical application, before the date is erased, the date can be judged, if the date is not consistent with the preset date, the date area is judged to be a defect area, and if the date is consistent with the preset date, the date in the date area is erased, and the nameplate defect detection is continued.

Here, it should be noted that in the present embodiment, all the areas of the nameplate are identified, and all the defective areas are identified, so that there are many defective areas during identification. Here, when it is recognized that the date is not consistent with the preset date, the date area of the nameplate may be determined to be one of the defective areas.

Specifically, after the date area is determined, the date within the date area may be determined using character recognition.

In order to avoid the situation that the defect of the nameplate cannot be detected, the correct bar code can be prompted to be input when the template picture corresponding to the bar code character is not found in the preset template picture set. So as to continue to perform the detection of the nameplate defect.

and erasing the date in the date area in the template picture.

Specifically, a date area is also present in the template map, and the content in the date area is generally in the yyy. In order to successfully detect the defect of the nameplate, the date in the date area in the template map is erased.

Further, the completion process of this embodiment may be:

erasing the date in the date area in the template picture.

Acquiring a first picture obtained by photographing a nameplate;

Due to the fact that the second picture is different from the template picture greatly (for example, characters are thick and thin and the like), detection precision can be improved and false detection rate can be reduced through the global matching and partition screenshot matching modes.

and determining a defect area of the nameplate based on the matching result.

Specifically, continuously moving four vertex angle coordinates aiming at the intercepted picture, matching the picture subjected to perspective transformation by the four vertex angles with a template picture, obtaining the picture with the minimum difference value as a matching result, and determining a white area in the matching result as a defect; and positioning the defect area by binarizing the matching result.

In addition, for the fine defect detection, the difference between the template picture and the second picture is large, and the number of false detections is large. Therefore, the binary image can be optimized; expanding the binary image, and amplifying a white difference area; and intercepting the difference region by searching the outline, and further comparing the difference region with the region corresponding to the template picture to obtain the finally determined defect region.

The nameplate defect detection method provided by the embodiment of the application acquires a first picture obtained by photographing a nameplate; identifying the first picture, and determining a template picture corresponding to the first picture and a date area in the first picture; erasing the date in the date area in the first picture to obtain a second picture; and comparing the second picture with the template picture to determine the defect area of the nameplate. The scheme that this application embodiment provided can discern the character in the data plate, can also discern the icon in the data plate, can realize the accurate detection of data plate defect.

Hereinafter, the present application will be described in detail with reference to application examples.

The embodiment provides a nameplate defect detection method which can replace first-check manual review and improve the overall efficiency; defects can be identified according to the template graph; automatic detection can be performed without manually establishing a template.

Specifically, referring to fig. 2, the overall process of the nameplate defect detecting method in the present embodiment is as follows:

step 201: the nameplate is arranged in the device; then, step 202 is executed:

here, a schematic view of the structure of the apparatus may be as shown in fig. 3. The device comprises a lens, a camera, a bar-shaped light source, a notch for placing a nameplate, a support, a shading square structure, a red alarm lamp, a green signal lamp and a display.

Step 202: taking a picture by a camera; then, step 203 is executed:

step 203: positioning a bar code; then, step 204 is executed:

step 204: identifying the bar code characters; then, step 205 is executed:

step 205: searching a CAD graph according to the bar code;

here, the CAD drawing refers to each template drawing in a preset template drawing set. And searching the CAD graph in the preset template graph set corresponding to the bar code according to the bar code. Specifically, referring to fig. 4, fig. 4 is a diagram of each template map in a preset template map set. Each template map is a CAD format map. Because the current nameplate is manufactured according to the corresponding CAD drawings, in order to achieve the effects of reducing staff and manual modeling operation, the CAD drawings corresponding to all products can be exported to a folder in the early stage, and each picture is named by adopting a bar code. Such derived pictures are used as template pictures, and can be subsequently searched through the bar code names. FIG. 5 is a drawing of two of the templates in the template storage collection of FIG. 4.

If the CAD drawing is not found, then step 206 is executed; if the CAD drawing is found, step 208 is executed;

step 206: alarming and prompting; then, go to step 207;

step 207: inputting a correct bar code;

if the user's input of the correct barcode is detected, go to step 205;

step 208: identifying and comparing dates; then, go to step 209;

step 209: whether the date was successfully located;

if the date is not successfully located, go to step 210; if the date is successfully located, go to step 211;

step 210: a date defect; then, go to step 212;

step 211: erasing the date area, and then performing step 212;

step 212: defect detection, then step 213 is performed;

step 213: judging whether the defect exists;

if there is a defect, go to step 214; if there is no defect, go to step 215;

step 214: alarming, and lighting a red light; then, go to step 216;

step 215: green light amount, then go to step 216;

step 216: and displaying the result.

The detailed description will be made based on the above steps.

(1) Taking the left nameplate example in fig. 5, placing the nameplate in the device in fig. 2 to obtain a picture taken by the camera; obtaining a maximum connected domain of the picture, obtaining a peripheral frame outline and a minimum external rectangle of the picture, taking 1/4 on the shortest side of the external rectangle as the length, taking 5 pixels as a step length, traversing points in the maximum connected domain by using the step length, obtaining a line segment by each traversal, and calculating to obtain a slope and an offset according to a starting point; and dividing the data into four groups of collinear data according to the slope angles, and respectively taking the mean values of the slope and the offset as optimal values to perform straight line fitting to obtain four frame fitted straight lines. And obtaining four intersection points of the straight line to perform perspective transformation on the picture to obtain the corrected picture. Here, the specific positive effect can be seen in fig. 6.

(2) Performing barcode positioning digital identification, and intercepting the bottom 1/3 area for processing because the barcodes are positioned at the lower right corner of the nameplate; calculating gradients in the x direction and the y direction in the image, subtracting the gradient in the y direction from the gradient in the x direction to obtain an absolute value, referring to fig. 7, performing expansion and corrosion operations on the effect image for multiple times to obtain a maximum outline in the image as a bar code; taking 1/2 bar code height to extend downwards to obtain bar code number; the deep learning convolutional neural network can be used for identifying the characters to obtain the bar code numbers.

(3) Acquiring a CAD template drawing by using a bar code, and manually inputting if the bar code is not found; date identification and positioning are carried out after the template drawing is obtained (the date format of the CAD drawing is YYYY. MM format, and independent detection is needed); firstly, acquiring the current year and month, then intercepting a date local area for character recognition (the recognized character strings may have multiple types, and filtering to acquire the character strings in a date format), referring to fig. 8, if the recognition characters of the picture conform to the current year and month, erasing the area on the detected picture without detection, and otherwise, indicating that the current picture date area has defects (manual modeling is not needed in the method, and labor force for manual marking is saved).

(4) Matching the processed picture with the template picture, wherein the CAD picture is used as the template picture and has larger difference (such as fat and thin characters) with the actual photographed picture, and if only global matching is adopted and the false detection is higher, the pictures are intercepted in different areas and compared (such as a left picture and a left picture in figure 9); continuously moving the coordinates of the four vertex angles of the intercepted picture, matching the picture subjected to perspective transformation of the four vertex angles with the template picture, and obtaining the picture with the minimum difference value as a matching result, wherein a white area is a defect (such as the left three pictures in fig. 9); and (4) binarizing the matching result (such as the left four graphs in fig. 9) and positioning the defect area.

(5) Aiming at the detection of the tiny defects, considering that the difference between a CAD graph and an actual picture is large, the number of false detections is large, and the binary picture is optimized; expanding the binary image obtained by the processing in the step (4), and amplifying a white difference area; searching the outline, and intercepting the difference area to further compare with the area corresponding to the template picture to obtain the best effect; the resulting effect is shown in fig. 10.

The embodiment can realize the recognition of characters or icons. Compared with the single character recognition, the method can improve the detection range of the nameplate, does not need to manufacture a large number of data sets, and improves the detection precision. The embodiment can replace the manual first-check, and the overall efficiency is improved; defects can be identified according to the template graph; automatic detection can be performed without manually establishing a template.

In order to realize the method of the embodiment of the invention, the embodiment of the invention also provides a nameplate defect detection device. This data plate defect detecting device includes:

It should be noted that: the above-mentioned apparatus provided by the above-mentioned embodiment belongs to the same concept as the above-mentioned method embodiment, and the specific implementation process thereof is described in the method embodiment, which is not described herein again.

As shown in fig. 11, an electronic device according to an embodiment of the present application includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, where the processor 111, the communication interface 112, and the memory 113 complete mutual communication via the communication bus 114,

a memory 113 for storing a computer program;

in one embodiment of the present application, the processor 111, when executing the program stored in the memory 113, is configured to implement the steps of the method provided in any of the foregoing method embodiments.

Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

The device provided by the embodiment of the present invention includes a processor, a memory, and a program stored in the memory and capable of running on the processor, and when the processor executes the program, the method according to any one of the embodiments described above is implemented.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program object. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program object embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program objects according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the method as provided in any of the foregoing method embodiments.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include transitory computer readable media (transmyedia) such as modulated data signals and carrier waves.

It will be appreciated that the memory of embodiments of the invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. Among them, the nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a magnetic random access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical disk, or a Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Enhanced Synchronous Dynamic Random Access Memory (Enhanced DRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory (DRmb Access), and Random Access Memory (DRAM). The described memory for embodiments of the present invention is intended to comprise, without being limited to, these and any other suitable types of memory.

It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A nameplate defect detection method, comprising:

acquiring a first picture obtained by photographing a nameplate;

2. The method of claim 1, wherein the identifying the first picture and the determining the template picture corresponding to the first picture and the date area in the first picture comprises:

and determining a date area in the first picture by utilizing the bar code area based on the corresponding relation between the bar code area and the date area.

3. The method of claim 2, wherein before erasing the date in the date area in the first picture, the method further comprises:

acquiring a date in the date area;

judging whether the date is consistent with a preset date or not;

4. The method of claim 2, wherein the searching for the template drawing corresponding to the barcode character in a preset template drawing set based on the barcode character comprises:

5. The method of claim 1, wherein before comparing the second picture to the template picture, the method further comprises:

erasing the date in the date area in the template picture.

6. The method of claim 1, wherein comparing the second picture to the template picture to determine the defective area of the nameplate comprises:

7. The method of claim 6, wherein the matching of the second picture with the template picture in a sectional screenshot, the determining the defect region of the nameplate comprises:

and determining a defect area of the nameplate based on the matching result.

8. A nameplate defect detecting device, comprising:

9. An electronic device, comprising: a processor and a memory for storing a computer program capable of running on the processor; wherein,

the processor is adapted to perform the steps of the method of any one of claims 1 to 7 when running the computer program.

10. A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the steps of the method of any one of claims 1 to 7.