TW202207078A - Image processing method, electronic device and storage device - Google Patents

Abstract

The present application provides an image processing method. The method includes: acquiring a first text area in an image to be recognized; acquiring a second text area in a standard image, and extracting a text window based on the second text area, wherein the text window includes multiple sub-windows; acquiring a target text area in the image to be recognized; and obtaining a first text sub-areas set by dividing the target text area according to the multiple sub-windows; and obtaining a second text sub-areas set by dividing the second text area according to the multiple sub-windows. The method further includes marking the image to be recognized as a qualified image when each of the first text sub-area is the same as each of the second text sub-area. The application also provides an electronic device and a storage medium, which can improve production efficiency.

Description

Image processing method, electronic device and storage medium

The present application relates to the field of image technology, and in particular, to an image processing method, an electronic device, and a storage medium.

Automated Optical Inspection (AOI) is an automated visual inspection of printed circuit board manufacturing. It is a high-speed and high-precision optical image inspection system, which uses machine vision to compare whether the object to be tested is different from the standard image to determine whether the object to be tested meets the standard. AOI machines are widely used in SMT (Surface Mount Technology) assembly lines to inspect the quality of components on circuit boards after solder assembly (PCB Assembly), or to check whether solder paste meets standards after printing. Generally speaking, the AOI machine engineer will set the detection standard for each object to be tested. If the inspection standard is set too strict, the false alarm rate will be too high; if the inspection standard is set too loosely, product defects may be missed. At present, the products that are detected as defective by AOI need to be further judged by manual visual inspection (Visual Inspection) to determine whether there is really a problem with the object to be tested (such as circuit boards), and the results of manual visual inspection shall prevail.

For example, when the current AOI machine detects IC components, the text on the components is used as the judgment basis to confirm whether the components are qualified. Usually, the detection standard of the text on the detection element is stricter. For the same IC components, the engraved characters are the same, but because the components are supplied by different manufacturers, the fonts of the same characters may be different, resulting in the AOI machine inspection being judged as defective. At this time, engineers need to reconfirm the inspection standards or add standard images to adjust the misjudgment of the AOI machine caused by different fonts, resulting in a decrease in the overall production line efficiency.

In view of this, it is necessary to provide an image processing method, an electronic device and a storage medium, which can improve the production efficiency of the production line.

A first aspect of the present application provides an image processing method, the method includes: acquiring a to-be-recognized image and a standard image; acquiring a first text area in the to-be-recognized image; acquiring a first text area in the standard image Second text area, extract a text window according to the second text area, wherein the text window includes a plurality of sub-windows; based on the first text area and the text window, obtain the target text in the image to be recognized dividing the target text area according to the plurality of sub-windows to obtain a first text sub-area set, and dividing the second text area according to the plurality of sub-windows to obtain a second text sub-area set; Whether all the first text sub-regions in a text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set; and when all the first text sub-regions in the first text sub-region set are the same as the The corresponding second text sub-regions in the two text sub-region sets are all the same, and the to-be-recognized image is marked as a qualified image.

In some embodiments of the present application, the obtaining the target text area in the to-be-recognized image based on the first text area and the text window includes: intercepting the second text area in the standard image; using The second text area is matched with the first text area, and the area with the highest similarity with the first text area in the second text area is found; and the first text area is displayed in the first text area by using the text window The middle box selects the region with the highest similarity to obtain the target text region.

In some embodiments of the present application, determining whether all the first text sub-regions in the first text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set includes: calculating the first text sub-regions The similarity between the first text sub-region in the set and the second text sub-region in the set corresponding to the second text sub-region, to obtain a similarity set; determine whether each similarity in the similarity set is greater than or equal to a predetermined set value; when each similarity in the similarity set is greater than or equal to a preset value, confirm that all the first text sub-regions in the first text sub-region set and the first text sub-region set corresponding to the second text sub-region set The two text sub-regions are the same; when the similarity in the similarity set is less than the preset value, confirm that the first text sub-region and the second text sub-region exist in the first text sub-region set. The corresponding second text sub-regions are not the same.

In some embodiments of the present application, the method for obtaining the similarity set includes: extracting the first feature value of each first text sub-region in the first text sub-region set by using a preset classifier, and extracting the first feature value of each first text sub-region in the first text sub-region Suppose the classifier extracts the second feature value of each second text subregion in the second text subregion set; and calculates the similarity between the first feature value and the second feature value to obtain a similarity set .

In some embodiments of the present application, the method for obtaining the similarity set further comprises: inputting each first text sub-region in the first text sub-region set into a preset classifier to identify each first text sub-region A text subarea.

In some embodiments of the present application, the method further includes: when the first text sub-region in the first text sub-region set is different from the second text sub-region corresponding to the second text sub-region set, marking the The image to be identified is a defect image.

In some embodiments of the present application, the method further includes preprocessing the to-be-identified image, and the preprocessing of the to-be-identified image includes: filtering out noise in the to-be-recognized image through a filter; Image enhancement technology enhances the first text area; and binarizes the to-be-recognized image.

In some embodiments of the present application, the method further includes: after marking the to-be-recognized image as a qualified image, outputting prompt information indicating that the to-be-recognized image is a qualified image; or marking the to-be-recognized image as a qualified image After the defective image is generated, prompt information is output indicating that the to-be-identified image is a defective image.

A second aspect of the present application provides an electronic device, the electronic device includes: a processor; and a memory, where a plurality of program modules are stored in the memory, and the plurality of program modules are carried by the processor Enter and execute the image processing method described above.

A third aspect of the present application provides a storage medium on which at least one computer instruction is stored, the instruction is loaded by a processor and executes the image processing method as described above.

Compared with the prior art, an image processing method, electronic device, and storage medium provided by the embodiments of the present application extract the features of the image to be recognized by using the image processing method and the classifier, so that the same text has different fonts. Similar features can be extracted from all types, and then compared with the text features of the standard image to determine whether the image to be recognized is a qualified image. This application can greatly reduce the misjudgment rate of the machine and greatly improve the overall production line efficiency.

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments.

Please refer to FIG. 1 , the image processing system 20 runs in the electronic device 1 . The electronic device 1 includes, but is not limited to, a communication unit 10 , a memory 11 and at least one processor 12 . The communication unit 10 , the memory 11 and at least one processor 12 are electrically connected.

In this embodiment, the communication unit 10 is used to provide network communication for the electronic device 1 . The network may be a wired network or a wireless network, such as radio, wireless fidelity (Wireless Fidelity, WIFI), cellular, satellite, broadcast, and the like.

The electronic device 1 is connected in communication with an AOI machine (not shown in the figure) through the communication unit 10 .

In one embodiment, the electronic device 1 may be an electronic device installed with an image processing program, such as a computer, a smart phone, a personal computer, a server, and the like.

Those skilled in the art should understand that the structure of the electronic device 1 shown in FIG. 1 does not constitute a limitation of the embodiments of the present invention, and the electronic device 1 may also include more or less other hardware or software than that shown in FIG. 1 , Or a different component arrangement.

It should be noted that the electronic device 1 is only an example, and other existing or future electronic devices that can be adapted to the present invention should also be included within the protection scope of the present invention, and are incorporated herein by reference.

Please refer to FIG. 2 , which is a flowchart of an image processing method according to an embodiment of the present application. The image processing method is applied in the electronic device 1 . According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

Step S11: Acquire the image to be recognized and the standard image.

In this embodiment, when an AOI machine is used to detect IC-type components, the text on the IC-type components is used as the basis for judging whether the IC products are of the same type. It can be understood that the text in each component of the same type of IC type components will be pasted in the same preset position. The AOI machine can acquire the preset position information when detecting IC-type components, so as to obtain the to-be-recognized image and the standard image including the text area. The to-be-identified image and the standard image are images obtained from the same type of product. The to-be-recognized image is an image obtained from the AOI machine. It should be noted that each text on the standard image is clear and complete without damage, the text is not skewed or greatly shifted, the light source is normal and the picture is clear.

It can be understood that both the to-be-identified image and the standard image include multiple primitive points. The primitive point refers to dividing an image into several small squares, and each small square is called a primitive point. The electronic device can represent the entire image by representing the information such as the position, color and brightness of these primitive points.

Step S12: Preprocessing the to-be-recognized image, and acquiring a first text area in the to-be-recognized image. In this implementation manner, in order to highlight the text area in the to-be-recognized image, the to-be-recognized image is preprocessed first.

Specifically, the preprocessing of the to-be-recognized image includes:

(1) Filter out the noise in the to-be-recognized image through a filter. In this implementation manner, while retaining the main features of the to-be-recognized image as much as possible, useless noise information that affects subsequent processing is removed.

(2) Enhancing the first text area through image enhancement technology.

In this implementation manner, the text in the to-be-recognized image after the noise is removed by the filter may become relatively blurred, and an image enhancement technology needs to be used to enhance the first text area, so that the first text area The text in is more obvious. For example, if the high-frequency components in the to-be-recognized image are enhanced, the outline of the first text area in the image can be clear and the details are obvious.

(3) Binarizing the to-be-recognized image. In this embodiment, the to-be-recognized image can be converted into a black and white image by binarizing the to-be-recognized image, so as to facilitate distinguishing the first text area and the background area.

(4) Acquire the first text area in the to-be-recognized image. In this embodiment, the to-be-recognized image is identified by an 8-connected component method, a plurality of connected objects are obtained, the area of the plurality of connected objects is calculated, and the area less than A line object with a preset area cuts out the text on the to-be-recognized image, and uses the smallest circumscribed rectangle to frame all the text, which is the first text area in the to-be-recognized image. It can be understood that the method for acquiring the first text area in the image to be recognized is not limited to the above method.

In this embodiment, in order to ensure that each image to be recognized can remove noise and enhance the text area, the above steps (1) and (2) may be used alternately.

Step S13: Acquire a second text area in the standard image, and extract a text window according to the second text area, wherein the text window includes a plurality of sub-windows.

In this embodiment, the method for acquiring the second text area in the standard image is the same as the method for acquiring the first text area in the to-be-recognized image, and details are not described herein again.

It should be noted that the text window is a circumscribed rectangle with the smallest area including all the primitive points of the first text area extracted from the image. The text window includes a plurality of sub-windows, and each sub-window is a circumscribed rectangle with the smallest area of each character area primitive point in the first text area. For example, as shown in FIG. 3 , the second text area in the standard image includes the text "SW3", that is, the standard image includes all the area corresponding to the second text "SW3" in the area of the primitive point with the smallest circumscribed rectangle . The text window 30 includes three sub-windows 301, and each sub-window 301 corresponds to the letter "S", "W" and the number "3" respectively.

Step S14: Based on the first text area and the text window, obtain a target text area in the to-be-recognized image.

In this embodiment, the first text area may include more text information than the second text area in the standard image, in order to find the same text as in the standard image from the first text area information, it is necessary to use the text window 30 to slide in the first text area in the to-be-recognized image to find the same text information as in the standard image.

Specifically, based on the first text area and the text window 30, obtaining the target text area in the to-be-recognized image includes:

(1) Intercept the second text area in the standard image.

(2) Matching the second text area with the first text area to find the area with the highest similarity with the first text area in the second text area. In this implementation manner, each graphic element point in the second text area is used to match each graphic element point of the first text area from left to right, from top to bottom, to find the first text area. The area with the highest similarity with the first text area in the second text area.

The similarity index defined in this application includes similarity measures commonly used to calculate different samples, such as the reciprocal distance (including Euclidean distance, Manhattan distance, Hamming distance, etc.), correlation coefficient (Correlation coefficient), structural similarity (SSIM, Structural Similarity), Complex Wavelet Structural Similarity (CW-SSIM, Complex Wavelet SSIM) and Cosine similarity (Cosine similarity), etc. Different similarity indicators will be used according to different situations to facilitate subsequent text detection and comparison.

(3) Using the text window 30 to frame the region with the highest similarity in the first text region to obtain the target text region.

For example, as shown in FIG. 4 , the first text area RA in the to-be-recognized image _A includes the character "GSW30", the first text area RB in the to-be-recognized image _B includes the character "LBJ23", and the to-be-recognized image B includes the character "LBJ23". The first text area R _C in the image C is identified, and the second text area R in the standard image includes the character "SW3". Intercept the second text area in the standard image to obtain the area R including the text "SW3", use the area _R to match the first text area RA, and find the first text area _RA . For the character area "SW3", use the text window to frame the character area "SW3" in the first text area _RA to obtain the target character area including the character "SW3". Use the area _R to match the first text area _RB , find the character area "BJ23" in the first text area RB, and use the text window 30 in the first text area _RB Select the character area "BJ23" in the middle box to obtain the target character area including the character "BJ23". Use the area _{R 0} to match the first text area RC, find that the character area in the first text area RC includes part of the letter "E" and the character " _Oti3 ", use the text window 30 Select a character area including part of the letter "E" and the character " _Oti3 " in the first text area RC, and obtain the target character area including part of the letter "E" and the character "Oti3".

It should be noted that the target text area does not necessarily include complete characters, but the first text area is framed according to the size of the text window 30 to obtain the target text area.

Step S15: Divide the target text region according to the multiple sub-windows to obtain a first text sub-region set, and divide the second text region according to the multiple sub-windows to obtain a second text sub-region set.

In this implementation manner, in order to more accurately compare whether the target text area is consistent with the second text area in the standard image, the target text area needs to be segmented and then compared one by one. Specifically, the target text area is divided according to a plurality of sub-windows in the text window to obtain a first text sub-area set. For example, as shown in FIG. 4 , the text window of the standard image includes three sub-windows, and the three sub-windows are used to divide the target text area in the image A to be recognized, and a first set of text sub-areas can be obtained. The first text sub-area set includes a letter "S" framed by a sub-window, a letter "W" framed by a sub-window, and a digit "3" framed by a sub-window; the three sub-windows are used to segment the image to be identified. Like the target text area in B, the first text sub-area set can be obtained, and the first text sub-area set includes the letter "B" framed by a sub-window, the letter and digit "J2" framed by a sub-window and a digit "3" selected by a sub-window; utilize the three sub-windows to segment the target text region in the image C to be recognized, and can obtain the first text sub-region set, and the first text sub-region set includes a sub-region set. Part of the letter "E" and part of the letter "O" framed by a window, part of the letter "O" and letter "ti" framed by a sub-window, and digit "3" framed by a sub-window.

In this embodiment, a second text sub-region set can be obtained by dividing the second text region in the standard image by using the three sub-windows. The second set of text sub-regions includes a letter "S" framed by a sub-window, a letter "W" framed by a sub-window, and a digit "3" framed by a sub-window.

It should be noted that, the first text sub-region set and the second text sub-region set do not necessarily include complete characters. The size of the sub-regions in the first text sub-region set and the second text sub-region set is determined by the size of the sub-windows in the text window 30 .

Step S16: Determine whether all the first text sub-regions in the first text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set.

In this embodiment, whether the image to be recognized is confirmed by comparing whether the first text sub-region in the first text sub-region set is the same as the second text sub-region in the corresponding second text sub-region set is a qualified image. When all the first text sub-regions in the first text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set, the flow goes to step S17; When a text sub-region is different from the corresponding second text sub-region in the second text sub-region set, the flow goes to step S18.

In this embodiment, determining whether all the first text sub-regions in the first text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set includes:

(a) Calculate the similarity between the first text sub-region in the first text sub-region set and the corresponding second text sub-region in the second text sub-region set to obtain a similarity set. Specifically, the method for obtaining the similarity set includes:

(1) Input each first text sub-region in the first text sub-region set into a preset classifier to identify each first text sub-region. In this embodiment, the preset classifier is a classifier obtained by training based on a single character data set obtained after the preprocessing of the image to be processed, and additionally collecting various English alphabet and digital data sets. .

(2) extracting the first feature value of each first text sub-region in the first text sub-region set by the preset classifier, and extracting the second text sub-region set by using the preset classifier A second feature value for each second text subregion. For example, the first feature value T ₁₀ of the first text sub-region including the character "S" in the image A to be recognized is extracted by the preset classifier, and the to-be-recognized image A is extracted by the preset classifier. The first feature value T 11 of the first text sub-region including the character "W" in the image A, and the first feature value T ₁₁ including the character "3" in the image A to be recognized is extracted by the preset classifier. The first feature value T ₁₂ of the text sub-region; the first feature value T ₂₀ of the first text sub-region including the character "B" in the image B to be recognized is extracted by the preset classifier, and the The preset classifier extracts the first feature value T ₂₁ of the first text sub-region including the character "J2" in the image B to be recognized, and extracts the image B to be recognized through the preset classifier. The first feature value T ₂₂ of the first text sub-region including the character "3"; and extracting the image C to be recognized that includes part of the letter "E" and part of the letter "O" by the preset classifier The first feature value T ₃₀ of the first text sub-region, the first feature of the first text sub-region including part of the letter "O" and the letter "ti" in the to-be-recognized image C is extracted by the preset classifier The value T ₃₁ , and the first feature value T ₃₂ of the first text sub-region including the character “3” in the image C to be recognized is extracted by the preset classifier.

The second feature value T 00 of the second text sub-region including the character "S" in the standard image is extracted by the preset classifier, and the second feature value T ₀₀ of the second text sub-region including the character "S" in the standard image is extracted by the preset classifier. the second feature value T ₀₁ of the second text sub-region of the element "W", and the second feature value of the second text sub-region including the character "3" in the standard image extracted by the preset classifier T ₀₂ .

(3) Calculate the similarity between the first eigenvalue and the second eigenvalue to obtain a similarity set. For example, calculating the similarity between the first text sub-region in the first text sub-region set in the to-be-recognized image A and the second text sub-region in the second text sub-region set corresponding to the standard image, to obtain Similarity set. Specifically, the similarity between the first feature value T ₁₀ and the second feature value T ₀₀ is calculated to obtain the similarity S ₀₀ , and the first feature value T ₁₁ and the second feature value T are calculated ₀₁ to obtain the similarity S ₀₁ , and the similarity between the first feature value T ₁₂ and the second feature value T ₀₂ to obtain the similarity S ₀₂ . The similarity set is {S ₀₀ , S ₀₁ , S ₀₂ }. Or calculate the similarity between the first text sub-region in the first text sub-region set in the to-be-recognized image B and the second text sub-region in the second text sub-region set corresponding to the standard image, to obtain similarity degree set. Specifically, the similarity between the to-be-identified image B and the standard image is calculated, such as calculating the similarity between the first feature value T ₂₀ and the second feature value T ₀₀ to obtain the similarity S _10. Calculate the similarity between the first feature value T ₂₁ and the second feature value T ₀₁ to obtain the similarity S ₁₁ , and the first feature value T ₂₂ and the second feature value T ₀₂ The similarity between them is obtained to obtain the similarity S ₁₂ . The similarity set is {S ₁₀ , S ₁₁ , S ₁₂ }. Similarly, the similarity between the first text sub-region in the first text sub-region set in the to-be-recognized image C and the second text sub-region in the second text sub-region set corresponding to the standard image can be calculated to obtain Similarity set.

(b) Determine whether each similarity in the similarity set is greater than or equal to a preset value. When each similarity in the similarity set is greater than or equal to a preset value, confirm that all the first text sub-regions in the first text sub-region set and the corresponding second text sub-regions in the second text sub-region set The regions are all the same, that is, it is confirmed that the first text region in the to-be-recognized image is the same as the second text region in the standard image, and the process goes to step S17; , confirm that the first text sub-region in the first text sub-region set is different from the corresponding second text sub-region in the second text sub-region set, that is, confirm that the first text region in the image to be recognized is the same as the standard The second text areas in the image are not the same, and the flow goes to step S18.

For example, if the similarity S ₀₀ , the similarity S ₀₁ and the similarity S ₀₂ in the similarity set are all greater than or equal to the preset value, the process goes to step S17 ; if the similarity S ₀₀ in the similarity set is similar to If the degree S ₀₁ , or the similarity degree S ₀₂ is smaller than the preset value, the flow goes to step S18 .

Step S17: marking the to-be-recognized image as a qualified image.

In this embodiment, when each similarity in the similarity set is greater than or equal to a preset value, it is determined that all characters in the target text area in the image to be recognized are determined to be the same as those in the standard image. All characters in the second text area are the same, and the to-be-recognized image is marked as a qualified image. For example, if all characters in the target text area in the image A to be recognized are determined to be the same as all characters in the second text area in the standard image, the image A to be recognized can be marked as a qualified image.

In one embodiment, the image processing method may also output prompt information to indicate that the to-be-recognized image is a qualified image. For example, outputting "Pass" information indicates that the to-be-recognized image is a qualified image. That is to say, the object to be tested corresponding to the image A to be recognized is a qualified object to be tested.

Step S18: marking the to-be-identified image as a defective image.

In this embodiment, when the similarity in the similarity set is smaller than the preset value, that is, there is text in the target text area of the image to be recognized that is determined to be the same as the second in the standard image. If the characters in the text area are different, it means that the character features of the to-be-recognized image are different from those of the standard image, and the to-be-recognized image is marked as a defective image. For example, if the text in the target text area in the image to be recognized B is different from the text in the second text area in the standard image, the image to be recognized B may be marked as a defective image. That is to say, the object to be tested corresponding to the to-be-recognized image B is an unqualified object to be tested.

In one embodiment, the image to be identified is marked as a defective image including a number of possible situations. For example, the text on the to-be-recognized image is different from the text on the standard image, as shown in FIG. 5A . At this time, it can be confirmed that the test object corresponding to the to-be-recognized image and the test object corresponding to the standard image are not of the same type . For example, the object to be tested corresponding to the image to be recognized and the object to be tested corresponding to the standard image are IC-type components produced by different manufacturers; the text on the image to be recognized is greatly shifted, as shown in FIG. 5B , this It can be confirmed that the object to be tested corresponding to the image to be recognized is displaced, and errors are prone to occur in subsequent use. For example, when the object to be detected corresponding to the image to be recognized is a square element, the four vertices of the square element need to be welded on the circuit board. If the image to be recognized corresponding to the square element appears as shown in FIG. 5B , it cannot be soldered on the circuit board in preparation; the text on the image to be recognized is missing, as shown in Figure 5C, at this time, it cannot be confirmed whether the object to be tested corresponding to the image to be recognized is the one to be tested corresponding to the standard image. The objects are of the same type; the text on the to-be-recognized image is blurred, as shown in Figure 5D, at this time it is impossible to confirm whether the DUT corresponding to the to-be-recognized image is of the same type as the DUT corresponding to the standard image; The text on the image to be recognized is covered by foreign objects or the light source is abnormal, as shown in Figure 5E, at this time, it is confirmed that there may be foreign objects on the object to be tested corresponding to the image to be recognized, which may affect the performance of the object to be tested; and the When the image to be recognized is skewed, as shown in FIG. 5F , it is confirmed that the object to be tested corresponding to the image to be recognized may be skewed.

In one embodiment, the image processing method may further output prompt information indicating that the to-be-identified image is a defective image. For example, outputting "Fail" information indicates that the to-be-identified image is a defective image.

To sum up, the image processing method provided in this application uses the image processing method and the classifier to extract the features of the image to be recognized, so that the same text can be extracted with similar features even though it has different font types. The text features of the images are compared to determine whether the to-be-recognized image is a qualified image. This application can greatly reduce the misjudgment rate of the machine and greatly improve the overall production line efficiency.

Referring to FIG. 6 , in this embodiment, the image processing system 20 can be divided into one or more modules, and the one or more modules can be stored in the processor 12 and processed by the The processor 12 executes the image processing method of the embodiment of the present application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the image processing system 20 in the electronic device 1 . For example, the image processing system 20 can be divided into an acquisition module 201 , a processing module 202 , a judgment module 203 and a marking module 204 in FIG. 6 .

The acquisition module 201 is used to acquire the image to be recognized and the standard image; the acquisition module 201 is also used to acquire the first text area in the image to be recognized; the acquisition module 201 is also used to Acquiring the second text area in the standard image, and extracting a text window according to the second text area, wherein the text window includes a plurality of sub-windows; the processing module 202 is configured to based on the first text area and the text window to obtain the target text area in the to-be-recognized image; the processing module 202 is further configured to divide the target text area according to the plurality of sub-windows to obtain a first set of text sub-areas, and The second text area is divided according to the plurality of sub-windows to obtain a second text sub-area set; the judgment module 203 is configured to judge whether all the first text sub-areas in the first text sub-area set are the same as the first text sub-area set. The corresponding second text sub-regions in the two text sub-region sets are the same; and the marking module 204 is used for when all the first text sub-regions in the first text sub-region set and the corresponding first text sub-regions in the second text sub-region set The two text sub-regions are the same, and the to-be-recognized image is marked as a qualified image.

Due to the variability of production line data, the present application also develops a system update mechanism, through which the image processing system 20 can be automatically updated through continuously accumulated data, so as to ensure the accuracy of the model and achieve the effect of adapting to various products.

In one embodiment, the results of the image to be identified through the system's judgment can be compared with the results of manual processing and marking, and detection indicators such as the accuracy rate, the missed detection rate, and the overkill rate can be calculated. When the detection index reaches the standard set by the production line, it means that the overall system is stable, and the new data (ie, the image to be recognized) of the production line will continue to be judged by the image processing system 20 of the present application for defects.

If the detection index does not meet the standard set by the production line, the system update mechanism is activated, and the preset classifier is retrained for the production line data (ie, the image to be recognized), and the adaptability of the preset classifier to the production line data is strengthened. , the retrained result is compared with the result marked by the manual data, and the detection index is calculated. Repeat this until the detection index meets the requirements set by the production line, and the system update is completed.

For example, this application collects a total of 699 images to be recognized of IC-type text components judged to be defective by AOI machines on the SMT production line, of which 386 are training data for developing the image processing system, and 313 are The verification data is used for verification and testing after the system development is completed. According to the confusion matrix of the image processing system, the results are shown in Figure 7. The confusion matrix on the left in Figure 7 records the results of the training data when developing the image processing system. The real results through manual detection are that 252 images are qualified and marked as "PASS", and 134 images are unqualified and marked as "FAIL". However, the prediction result of the image processing system is that 246 images are qualified and marked as "PASS", and 140 images are unqualified and marked as "FAIL", of which 6 images are processed by the image. The system misjudges the image as an unqualified image. From this, it can be calculated that the missed detection rate in the training data is 0/(0+134)=0%, and the overkill rate is 6/(246+6)=2.3%.

The confusion matrix on the right in Figure 7 records the results of the validation data when developing the image processing system. The real results through manual detection are that 183 images are qualified and marked as "PASS", and 130 images are unqualified and marked as "FAIL". The result predicted by the image processing system is that 179 images are qualified, of which 1 image is unqualified and misjudged as qualified, 134 images are unqualified, and 5 images are unqualified and misjudged as qualified qualified. From this, it can be calculated that the missed detection rate in the verification data is 1/(1+129)=0.7%, and the overkill rate is 5/(178+5)=2.7%. It can be seen that in the verification data, the image processing system of this application has an accuracy rate of 98%. The application of this system to the SMT production line can greatly reduce the time required for manual visual inspection, and can reduce the production line engineers to adjust the AOI The frequency of machine parameters greatly improves the efficiency and overall production line stability.

In one embodiment, the processor 12 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), dedicated integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor 12 may be any other conventional processor or the like.

If the modules in the image processing system 20 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, so When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of original code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory); Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium Electric carrier signals and telecommunication signals are not included.

It can be understood that the module division described above is a logical function division, and other division methods may be used in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated in the same processing unit, or each module may exist physically alone, or two or more modules may be integrated in the same unit. The above-mentioned integrated modules can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.

In another embodiment, the electronic device 1 may further include a memory (not shown), and the one or more modules may also be stored in the memory and executed by the processor 12 . The memory may be an internal memory of the electronic device 1 , that is, a memory built in the electronic device 1 . In other embodiments, the memory may also be an external memory of the electronic device 1 , that is, a memory externally connected to the electronic device 1 .

In some embodiments, the memory is used to store program codes and various data, for example, to store the program codes of the image processing system 20 installed in the electronic device 1 , and to achieve high speed during the operation of the electronic device 1 . , Automatically complete access to programs or data.

The memory may include random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (Secure) Digital, SD) card, flash memory card (Flash Card), at least one disk memory device, flash memory device, or other volatile solid state memory device.

It will be apparent to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, but that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the above-described embodiments of the present application should be considered in all respects as exemplary and not restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and it is therefore intended to All changes that come within the meaning and range of equivalents of the claims are included in this application.

1: electronic device 10: communication unit 11: memory 12: processor 30: text window 301: sub-windows _RA , _RB , RC: first text area _R : area 20: image processing system 201: acquisition module 202: Processing module 203: Judging module 204: Marking module

FIG. 1 is a schematic diagram of an electronic device according to an embodiment of the present application.

FIG. 2 is a flowchart of an image processing method according to an embodiment of the present application.

FIG. 3 is a schematic diagram of an image according to the standard of the present application.

FIG. 4 is a schematic diagram of obtaining a second text sub-region by matching an image to be recognized with a standard image according to the present application.

5A to 5F are schematic diagrams of images to be recognized in this application.

FIG. 6 is a functional module diagram of an image processing system according to an embodiment of the present application.

FIG. 7 is a schematic diagram of a confusion matrix according to an embodiment of the present application.

Claims (10)

An image processing method, the method comprising: Obtain the image to be recognized and the standard image; obtaining the first text area in the to-be-recognized image; acquiring a second text area in the standard image, and extracting a text window according to the second text area, wherein the text window includes a plurality of sub-windows; Based on the first text area and the text window, obtain the target text area in the to-be-recognized image; The target text region is divided according to the plurality of sub-windows to obtain a first text sub-region set, and the second text region is divided according to the plurality of sub-windows to obtain a second text sub-region set; determining whether all the first text sub-regions in the first text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set; and When all the first text sub-regions in the first text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set, the to-be-recognized image is marked as a qualified image. According to the image processing method described in claim 1, the obtaining the target text area in the to-be-recognized image based on the first text area and the text window includes: intercepting the second text area in the standard image; Using the second text area to match the first text area, find the area with the highest similarity to the first text area in the second text area; and The area with the highest similarity is selected in the first text area by using the text window to obtain the target text area. According to the image processing method described in claim 1, determining whether all the first text sub-regions in the first text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set includes: calculating the similarity between the first text sub-region in the first text sub-region set and the second text sub-region corresponding to the second text sub-region set, to obtain a similarity set; Judging whether each similarity in the similarity set is greater than or equal to a preset value; When each similarity in the similarity set is greater than or equal to a preset value, confirm that all the first text sub-regions in the first text sub-region set and the corresponding second text sub-regions in the second text sub-region set the regions are the same; When the similarity in the similarity set is smaller than the preset value, confirm that there is a second text sub-region corresponding to the first text sub-region and the second text sub-region set in the first text sub-region set Are not the same. According to the image processing method described in claim 3, the method for obtaining the similarity set includes: The first feature value of each first text sub-region in the first text sub-region set is extracted by a preset classifier, and each second text in the second text sub-region set is extracted by the preset classifier the second eigenvalue of the subregion; and Calculate the similarity between the first eigenvalue and the second eigenvalue to obtain a similarity set. According to the image processing method described in claim 4, the method for obtaining the similarity set further comprises: Each first text sub-region in the set of first text sub-regions is input into the preset classifier to identify each first text sub-region. The image processing method according to claim 1, further comprising: When the first text sub-region in the first text sub-region set is different from the corresponding second text sub-region in the second text sub-region set, the to-be-recognized image is marked as a defective image. The image processing method according to claim 6, further comprising preprocessing the to-be-recognized image, and the pre-processing of the to-be-recognized image includes: Filter out the noise in the to-be-recognized image through a filter; Enhance the first text area by image enhancement techniques; and The to-be-recognized image is binarized. The image processing method according to claim 7, further comprising: After marking the to-be-recognized image as a qualified image, output prompt information indicating that the to-be-identified image is a qualified image; or After marking the to-be-recognized image as a defective image, prompt information is output to prompt that the to-be-recognized image is a defective image. An electronic device comprising: processor; and A memory, wherein a plurality of program modules are stored in the memory, and the plurality of program modules are loaded by the processor to execute the image processing method according to any one of the request item 1 to the request item 8. A storage medium having at least one computer instruction stored thereon, the instruction being loaded by a processor and executing the image processing method according to any one of claim 1 to claim 8.

Images