TWI737447B - Image processing method, electronic device and storage device - Google Patents

Abstract

The present application provides an image processing method. The method includes: acquiring a first text area in an image to be recognized; acquiring a second text area in a standard image, and extracting a text window based on the second text area, wherein the text window includes multiple sub-windows; acquiring a target text area in the image to be recognized; and obtaining a first text sub-areas set by dividing the target text area according to the multiple sub-windows; and obtaining a second text sub-areas set by dividing the second text area according to the multiple sub-windows. The method further includes marking the image to be recognized as a qualified image when each of the first text sub-area is the same as each of the second text sub-area. The application also provides an electronic device and a storage medium, which can improve production efficiency.

Description

Image processing method, electronic device and storage medium

This application relates to the field of image technology, and in particular to an image processing method, electronic device and storage medium.

Automated Optical Inspection (AOI) is an automatic visual inspection of printed circuit board manufacturing. It is a high-speed and high-precision optical image detection system that uses machine vision to compare whether there is a difference between the object to be tested and the standard image to determine whether the object to be tested meets the standard. AOI machines are commonly used in SMT (Surface Mount Technology) assembly lines to check the quality of parts on the circuit board after PCB assembly, or to check whether the solder paste meets the standards after printing. Generally speaking, the AOI machine engineer will set the inspection standards for each object to be tested. If the detection standard is set too strict, the false alarm rate is too high; if the detection standard is set too loose, the product defect may be missed. At present, products that are detected as defective by AOI need to be further judged by manual visual inspection (Visual Inspection) to determine whether the object under test (such as circuit board) is really defective, and the result of manual visual inspection shall prevail.

For example, when the current AOI machine inspects IC components, it will use the text on the component as a judgment basis to confirm whether the component is qualified. Usually, the text on the detection element has a stricter detection standard. For the same IC components, the engraved text is the same, but it may be that the components are supplied by different manufacturers, and the font type of the same text may be different, which will result in being judged as defective after being inspected by the AOI machine. At this time, engineers need to reconfirm the inspection standards or add standard images to adjust the misjudgment of the AOI machine caused by different fonts, resulting in a decrease in the overall production line efficiency.

In view of this, it is necessary to provide an image processing method, an electronic device, and a storage medium, which can improve the production efficiency of the production line.

The first aspect of the application provides an image processing method, the method includes: acquiring a to-be-recognized image and a standard image; acquiring a first text area in the to-be-recognized image; acquiring a first text area in the standard image Two text areas, extracting a text window based on the second text area, wherein the text window includes a plurality of sub-windows; based on the first text area and the text window, the target text in the image to be recognized is obtained Region; segment the target text region according to the multiple sub-windows to obtain a first text subregion set, and segment the second text region according to the multiple sub-windows to obtain a second text subregion set; determine the first Whether all first text subregions in a text subregion set are the same as the corresponding second text subregions in the second text subregion set; and when all first texts in the first text subregion set The subregion is the same as the corresponding second text subregion in the second text subregion set, and the image to be recognized is marked as a qualified image.

In some embodiments of the present application, the obtaining the target text area in the image to be recognized based on the first text area and the text window includes: intercepting the second text area in the standard image; using The second text area is matched with the first text area to find the area in the second text area with the highest similarity to the first text area's primitive point; and the text window is used in the first text area In a text area, frame the area with the highest similarity of the graphic element points to obtain the target text area.

In some implementation manners of the present application, determining whether all first text subregions in the first text subregion set are the same as the corresponding second text subregions in the second text subregion set includes: calculating the first text subregion The similarity of graphic element points between a first text subarea in a text subarea set and a second text subarea in the corresponding second text subarea set, to obtain a similarity set of graphic element points; judging the said Whether the similarity of each primitive point in the primitive point similarity set is greater than or equal to the preset value; when the similarity of each primitive point in the primitive point similarity set is greater than or equal to the preset value, confirm All the first text sub-areas in the first text sub-areas set and the corresponding first text sub-areas in the second text sub-areas set The two text subregions are the same; when there is a primitive point similarity in the primitive point similarity set that is less than the preset value, it is confirmed that the first text subregion and the first text subregion in the first text subregion set exist The corresponding second text subregions in the second text subregion set are different.

In some implementation manners of the present application, the method for obtaining the set of similarity of primitive points includes: extracting the first feature value of each first text subregion in the first text subregion set through a preset classifier, And extracting the second feature value of each second text subregion in the second text subregion set by the preset classifier; and calculating the difference between the first feature value and the second feature value The similarity of primitive points is obtained, and the similarity set of primitive points is obtained.

In some implementation manners of the present application, the method for obtaining the similarity set of graphic element points further includes: inputting each first text subregion in the first text subregion set into a preset classifier, To identify each first text subregion.

In some implementation manners of the present application, the method further includes: when the first text subregion in the first text subregion set is different from the corresponding second text subregion in the second text subregion set When, mark the image to be recognized as a flawed image.

In some embodiments of the present application, the method further includes preprocessing the image to be recognized, and the preprocessing the image to be recognized includes: filtering noise in the image to be recognized through a filter; Image enhancement technology enhances the first text area; and binarizes the image to be recognized.

In some embodiments of the present application, the method further includes: after marking the image to be recognized as a qualified image, outputting prompt information to prompt the image to be recognized as a qualified image; or marking the image to be recognized as a qualified image After the defective image, prompt information is output to prompt that the image to be recognized is a defective image.

A second aspect of the present application provides an electronic device, the electronic device includes: a processor; and a memory, the memory stores a plurality of program modules, the plurality of program modules are carried by the processor Incorporate and execute the image processing method described above.

A third aspect of the present application provides a storage medium on which at least one computer instruction is stored, and the instruction is loaded by a processor and executes the image processing method described above.

Compared with the conventional technology, an image processing method, an electronic device, and a storage medium provided by the embodiments of the present application extract the features of the image to be recognized by using an image processing method and a classifier, so that the same text has different fonts. Similar features can be extracted from all types, and then compared with the text features of the standard image to determine whether the image to be recognized is a qualified image. This application can greatly reduce the machine misjudgment rate and greatly improve the overall production line efficiency.

1: Electronic device

10: Communication unit

11: Memory

12: processor

30: text window

301: sub-window

R _A , R _B , R _C : the first text area

R: area

20: Image processing system

201: Get Mods

202: Processing Module

203: Judgment Module

204: Marking Module

Fig. 1 is a schematic diagram of an electronic device according to an embodiment of the present application.

Fig. 2 is a flowchart of an image processing method according to an embodiment of the present application.

Figure 3 is a schematic diagram of a standard image according to this application.

FIG. 4 is a schematic diagram of a second text subregion obtained by matching the image to be recognized with the standard image according to the present application.

5A to 5F are schematic diagrams of images to be recognized in this application.

FIG. 6 is a functional module diagram of an image processing system according to an embodiment of the present application.

Fig. 7 is a schematic diagram of a confusion matrix according to an embodiment of the present application.

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them.

Please refer to FIG. 1, the image processing system 20 runs in the electronic device 1. The electronic device 1 includes, but is not limited to, a communication unit 10, a memory 11 and at least one processor 12. The communication unit 10, the memory 11 and at least one processor 12 are electrically connected.

In this embodiment, the communication unit 10 is used to provide network communication for the electronic device 1. The network may be a wired network or a wireless network, such as radio, wireless fidelity (WIFI), cellular, satellite, broadcast, etc.

The electronic device 1 is communicatively connected with an AOI machine (not shown in the figure) through the communication unit 10.

In one embodiment, the electronic device 1 may be an electronic device installed with an image processing program, such as a computer, a smart phone, a personal computer, a server, and so on.

Those skilled in the art should understand that the structure of the electronic device 1 shown in FIG. 1 does not constitute a limitation of the embodiment of the present invention. The electronic device 1 may also include more or less other hardware or software than in FIG. Or different component arrangements.

It should be noted that the electronic device 1 is only an example, and other existing or future electronic devices that can be adapted to the present invention should also be included in the protection scope of the present invention and included here by reference.

Please refer to FIG. 2, which is a flowchart of an image processing method according to an embodiment of the present application. The image processing method is applied in the electronic device 1. According to different needs, the order of the steps in the flowchart can be changed, and some steps can be omitted.

Step S11: Obtain the image to be recognized and the standard image.

In this embodiment, when an AOI machine is used to detect IC components, the text on the IC components will be used as a basis for judging whether the IC products are of the same type. It is understandable that the text in each of the IC components of the same type will be pasted at the same preset position. The AOI machine can obtain the preset position information when detecting IC components, so that the to-be-recognized image and the standard image including the text area can be obtained. The image to be recognized and the standard image are images obtained from products of the same type. The image to be recognized is an image obtained from the AOI machine. It should be noted that each text on the standard image is clear and complete without damage, the text is not skewed or greatly shifted, the light source is normal, and the picture is clear.

It can be understood that both the to-be-recognized image and the standard image include a plurality of image element points. The picture element point refers to dividing an image into a number of small squares, and each small square is called a picture element point. The electronic device can represent the entire image by representing the location, color, and brightness of these pixel points.

Step S12: preprocess the image to be recognized, and obtain a first text area in the image to be recognized. In this embodiment, in order to highlight the text area in the image to be recognized, the image to be recognized is preprocessed first.

Specifically, the preprocessing the image to be recognized includes:

(1) Filter out the noise in the image to be identified through a filter. In this embodiment, while preserving the main features of the image to be recognized as much as possible, the useless noise information that affects the subsequent processing is removed.

(2) Enhancing the first text area through image enhancement technology.

In this embodiment, the text in the image to be recognized after the noise is removed by the filter may become relatively blurred. It is necessary to use image enhancement technology to strengthen the first text area so that the first text area The text in is more obvious. For example, if the high-frequency components in the image to be recognized are enhanced, the outline of the first text area in the image can be clear and the details can be obvious.

(3) Binarize the image to be recognized.

In this embodiment, by binarizing the image to be recognized, the image to be recognized can be converted into a black and white image, so as to distinguish the first text area from the background area.

(4) Obtain the first text area in the image to be recognized.

In this embodiment, the 8-connected component method is used to identify the image to be recognized to obtain multiple connected objects, calculate the area of the multiple connected objects, and delete the area less than The connecting object with a preset area is divided into the text on the image to be recognized, and all text is framed by the smallest circumscribed rectangle, which is the first text area in the image to be recognized. It can be understood that the method of obtaining the first text area in the image to be recognized is not limited to the above method.

In this embodiment, in order to ensure that each image to be recognized can remove noise and enhance the text area, the above steps (1) and (2) can be used alternately.

Step S13: Obtain a second text area in the standard image, and extract a text window based on the second text area, wherein the text window includes a plurality of sub-windows.

In this embodiment, the method of obtaining the second text area in the standard image, The method is the same as the method for obtaining the first text area in the image to be recognized, and will not be repeated here.

It should be noted that the text window is a circumscribed rectangle with the smallest area extracted from the image and containing all the primitive points of the first text area. The text window includes a plurality of sub-windows, and each sub-window is a circumscribed rectangle with the smallest area of the primitive point of each character area in the first text area. For example, as shown in FIG. 3, the second text area in the standard image contains the text "SW3", that is, the circumscribed rectangle with the smallest area of the pixel points in the area corresponding to all the second text "SW3" in the standard image. . The text window 30 includes three sub-windows 301, and each sub-window 301 corresponds to the letter "S", "W" and the number "3" respectively.

Step S14: Obtain a target text area in the image to be recognized based on the first text area and the text window.

In this embodiment, the first text area may include more text information than the second text area in the standard image, in order to find the same text in the standard image from the first text area Information, it is necessary to use the text window 30 to slide in the first text area in the image to be recognized to find the same text information as in the standard image.

Specifically, based on the first text area and the text window 30, obtaining the target text area in the image to be recognized includes:

(1) Intercept the second text area in the standard image.

(2) Use the second text area to match the first text area to find the area in the second text area with the highest similarity to the primitive point of the first text area. In this embodiment, each image element point in the second text area is used to match each image element point in the first text area from left to right, from top to bottom, to find the first text area. The second text area is the area with the highest similarity of the primitive point of the first text area.

The similarity index of image element points defined in this application includes similarity measures commonly used to calculate the similarity between different samples, such as the reciprocal distance (including Euclidean distance, Manhattan distance, Hamming distance, etc.), correlation coefficient, Structural similarity (SSIM, Structural Similarity), complex wavelet structural similarity (CW-SSIM, Complex Wavelet SSIM) and cosine similarity (Cosine similarity), etc., root According to different situations, different primitive point similarity indicators are used to facilitate subsequent text detection and comparison.

(3) Using the text window 30 to select the area with the highest similarity of the graphic element points in the first text area to obtain the target text area.

For example, as shown in FIG. 4, the first text area RA in the image to be recognized _A includes the character "GSW30", and the first text area R _B in the image B to be recognized includes the character "LBJ23". identifying a first image C _C R text area, the standard image in the second region R includes text characters "SW3". Intercepting the standard image obtained in the second text area including the text region R "SW3", and the region R using the first matching text region R _A, find the first text area R _A of character region "SW3", with the marquee text window in the region of the first text character region R _A in "SW3", to give the title character region includes the character "SW3". The region R using the first matching text region R _B, to find the first character of the text area in the region R _B "BJ23", using the text window 30 in the first text region R _B Select the character area "BJ23" in the middle frame to obtain the target character area including the character "BJ23". Use the region R _{0 to} match the first text region R _C , find that the character region in the first text region R _C includes part of the letter "E" and the character "Oti3", and use the text window 30. In the first text area R _C , frame-select the character area including part of the letter "E" and the character "Oti3", and obtain the target character area including the part of the letter "E" and the character "Oti3".

It should be noted that the target text area does not necessarily include complete characters, but the first text area is selected according to the size of the text window 30 to obtain the target text area.

Step S15: Divide the target text area according to the multiple sub-windows to obtain a first text sub-areas set, and divide the second text area according to the multiple sub-windows to obtain a second text sub-areas set.

In this embodiment, in order to more accurately compare whether the target text area is consistent with the second text area in the standard image, the target text area needs to be segmented and then compared one by one. Specifically, the target text area is divided according to the multiple sub-windows in the text window to obtain the first text sub-areas set. For example, as shown in FIG. 4, the text window of the standard image includes three sub-windows, and the three sub-windows are used to segment the target text area in the image A to be recognized, and the first A set of text subregions, the first set of text subregions includes a letter "S" selected by a subwindow, a letter "W" selected by a subwindow, and a digit "3" selected by a subwindow; The three sub-windows divide the target text area in the image B to be recognized, and a first text sub-areas set can be obtained. The first text sub-areas set includes a sub-window frame selection letter "B" and a sub-window frame selection The letters and digits "J2" and the digit "3" selected by a sub-window; using the three sub-windows to segment the target text area in the image C to be recognized, the first set of text sub-areas can be obtained. A set of text subregions includes a part of the letter "E" and part of the letter "O" selected by a subwindow, a part of the letter "O" and the letter "ti" selected by a subwindow, and a digit "selected by the subwindow" 3".

In this embodiment, the second text area in the standard image is divided by using the three sub-windows to obtain the second text sub-areas set. The second text subregion set includes a letter "S" selected by a sub-window, a letter "W" selected by a sub-window, and a digit "3" selected by a sub-window.

It should be noted that the first text subregion set and the second text subregion set do not necessarily include complete characters. The size of the sub-areas in the first text sub-areas set and the second text sub-areas set is determined by the size of the sub-window in the text window 30.

Step S16: Determine whether all first text subregions in the first text subregion set are the same as the corresponding second text subregions in the second text subregion set.

In this embodiment, the first text subregion in the first text subregion set is compared with the second text subregion in the corresponding second text subregion set to confirm the waiting Identify whether the image is a qualified image. When all the first text subregions in the first text subregion set are the same as the corresponding second text subregions in the second text subregion set, the process goes to step S17; when the first text subregion set When the first text subregion in the region set is different from the corresponding second text subregion in the second text subregion set, the process goes to step S18.

In this embodiment, determining whether all the first text subregions in the first text subregion set are the same as the corresponding second text subregions in the second text subregion set includes:

(a) Calculate the similarity of the graphic element points between the first text subarea in the first text subarea set and the corresponding second text subarea in the second text subarea set to obtain the graphic element points Mutually Similarity set.

Specifically, the method for obtaining the similarity set of the graphic element points includes:

(1) Input each first text subregion in the first text subregion set into a preset classifier to identify each first text subregion. In this embodiment, the preset classifier is a single-character data set obtained after preprocessing of the image to be processed, and additional collections of various English letters and digital data sets for training. .

(2) Extracting the first feature value of each first text subregion in the first text subregion set through the preset classifier, and extracting the second text subregion through the preset classifier The second feature value of each second text subregion in the region set. _{For example, the first feature value T 10} of the first text subregion including the character "S" in the image A to be recognized is extracted by the preset classifier, and the first feature value T 10 is extracted by the preset classifier. _{The first feature value T 11} of the first text subregion including the character "W" in the image A, and the first feature value T 11 of the first text subregion including the character "3" in the image A to be recognized is extracted by the preset classifier. The first feature value T _{12 of the text subregion; the first feature value T 20} of the first text subregion including the character "B" in the image B to be recognized is extracted by the preset classifier, and the _{The preset classifier extracts the first feature value T 21} of the first text subregion including the character "J2" in the image B to be recognized, and extracts the image B from the image B to be recognized by the preset classifier _{The first feature value T 22} of the first text subregion including the character "3"; and extracting the part of the letter "E" and the part of the letter "O" from the image C to be recognized by the preset classifier The first feature value T ₃₀ of the first text subregion, the first feature of the first text subregion including part of the letter "O" and the letter "ti" in the image C to be recognized is extracted by the preset classifier The value T _{31 , and the first feature value T 32} of the first text subregion including the character "3" in the image C to be recognized is extracted by the preset classifier.

_{The second feature value T 00} of the second text sub-region including the character "S" in the standard image is extracted by the preset classifier, and the character included in the standard image is extracted by the preset classifier _{The second feature value T 01} of the second text subregion of the element "W", and the second feature value of the second text subregion including the character "3" in the standard image is extracted by the preset classifier T ₀₂ .

(3) Calculate the similarity of graphic element points between the first eigenvalue and the second eigenvalue to obtain a set of similarity of graphic element points. For example, calculate the difference between the first text subregion in the first text subregion set in the image A to be recognized and the second text subregion in the second text subregion set corresponding to the standard image The similarity of primitive points is obtained, and the similarity set of primitive points is obtained. Specifically, the primitive point similarity between the first characteristic value T ₁₀ and the second characteristic value T ₀₀ is calculated to obtain the primitive point similarity S ₀₀ , and the first characteristic value T _{11 is} calculated to be The graphic element point similarity between the second feature value T _{01 is obtained, and the graphic element point similarity S 01 is obtained} , and the graphic element point between the first feature value T ₁₂ and the second feature value T ₀₂ is similar , Get the similarity S _{02 of the} primitive point. The set of similarity of the graphic element points is {S ₀₀ , S ₀₁ , S ₀₂ }. Or calculate the image between the first text subregion in the first text subregion set in the image B to be recognized and the second text subregion in the second text subregion set corresponding to the standard image The similarity of the primitive point is obtained, and the similarity set of the primitive point is obtained. Specifically, calculating the similarity of the image element points between the image B to be recognized and the standard image, such as calculating the similarity of the image element points between the first feature value T ₂₀ and the second feature value T ₀₀ , Obtain the primitive point similarity S ₁₀ , calculate the primitive point similarity between the first characteristic value T ₂₁ and the second characteristic value T ₀₁ , obtain the primitive point similarity S ₁₁ , and The graph element point similarity between the first feature value T ₂₂ and the second feature value T ₀₂ is obtained to obtain the graph element point similarity S ₁₂ . The similarity set of the graphic element points is {S ₁₀ , S ₁₁ , S ₁₂ }. It is also possible to calculate the difference between the first text subregion in the first text subregion set in the image C to be recognized and the second text subregion in the second text subregion set corresponding to the standard image. The similarity of primitive points is obtained, and the similarity set of primitive points is obtained.

(b) Determine whether the similarity of each graphic element point in the graphic element point similarity set is greater than or equal to a preset value. When the similarity of each graphic element point in the graphic element point similarity set is greater than or equal to the preset value, it is confirmed that all the first text subregions and the second text subregion in the first text subregion set are The corresponding second text subregions in the region set are all the same, that is, it is confirmed that the first text region in the image to be recognized is the same as the second text region in the standard image, and the process goes to step S17; when the image element points are similar When the similarity of primitive points in the degree set is less than the preset value, it is confirmed that there is a corresponding second text subregion in the first text subregion set and the second text subregion set in the first text subregion set. The text sub-areas are not the same, that is, it is confirmed that the first text area in the image to be recognized is different from the second text area in the standard image, and the process goes to step S18.

For example, if the graphic element point similarity S ₀₀ in the graphic element point similarity set, the graphic element point similarity S ₀₁ and the graphic element point similarity S ₀₂ are both greater than or equal to the preset value, the process goes to step S17; if FIG similarity points of the element set point primitive or primitives similarity S ₀₀ point similarity S _01, or S ₀₂ of FIG membered point similarity is smaller than the predetermined value, the flow proceeds to step S18.

Step S17: Mark the image to be recognized as a qualified image.

In this embodiment, when the similarity of each of the primitive points in the primitive point similarity set is greater than or equal to the preset value, it is determined that all the characters in the target text area in the image to be recognized are determined to be If it is the same as all the characters in the second text area in the standard image, the image to be recognized is marked as a qualified image. For example, all characters in the target text area in the image A to be recognized are determined to be the same as all characters in the second text area in the standard image, and the image A to be recognized can be marked as a qualified image.

In one embodiment, the image processing method may also output prompt information to prompt that the image to be recognized is a qualified image. For example, outputting "Pass" information prompts that the image to be recognized is a qualified image. That is, the test object corresponding to the image A to be recognized is a qualified test object.

Step S18: Mark the image to be recognized as a defective image.

In this embodiment, when the similarity of the primitive points in the set of primitive point similarities is less than the preset value, that is, the existence of characters in the target text area of the image to be recognized is determined and the standard The text in the second text area in the image is different, which means that the text feature of the image to be recognized is different from the text feature of the standard image, and the image to be recognized is marked as a defective image. For example, if there is a text in the target text area in the image B to be recognized that is different from the text in the second text area in the standard image, the image B to be recognized can be marked as a defective image. That is to say, the object to be tested corresponding to the image B to be recognized is an unqualified object to be tested.

In one embodiment, the image to be recognized is marked as a defect image including multiple possible situations. For example, the text on the image to be recognized is different from the text on the standard image, as shown in Figure 5A. At this time, it can be confirmed that the object under test corresponding to the image to be recognized is not the same type as the object under test corresponding to the standard image. . For example, the test object corresponding to the image to be recognized and the test object corresponding to the standard image are from different manufacturers IC components produced; the text on the image to be recognized is greatly shifted, as shown in Figure 5B. At this time, it can be confirmed that the object to be tested corresponding to the image to be recognized is displaced, which is prone to errors in subsequent use. For example, when the object under test corresponding to the image to be recognized is a square component, the four vertices of the square component need to be soldered on the circuit board. If the image to be recognized corresponding to the square component appears as shown in Figure 5B , It cannot be soldered on the circuit board in preparation; the text on the image to be recognized is missing, as shown in Figure 5C, at this time it is impossible to confirm whether the object to be tested corresponding to the image to be recognized corresponds to the standard image to be tested The objects are of the same type; the text on the image to be recognized is blurry, as shown in Figure 5D, at this time, it is also impossible to confirm whether the object to be tested corresponding to the image to be recognized is of the same type as the object to be tested corresponding to the standard image; The text on the image to be recognized is covered by a foreign object or the light source is abnormal, as shown in FIG. 5E. At this time, it is confirmed that there may be a foreign object on the object to be detected corresponding to the image to be recognized, which may affect the performance of the object to be detected; and The image to be recognized is skewed, as shown in FIG. 5F, at this time, it is confirmed that the object to be tested corresponding to the image to be recognized may be skewed.

In one embodiment, the image processing method may also output prompt information to prompt that the image to be recognized is a defective image. For example, outputting "Fail" information prompts that the image to be recognized is a defective image.

In summary, the image processing method provided in this application uses image processing methods and classifiers to extract the features of the image to be recognized, so that the same text can extract similar features even though it has different font types. The text characteristics of the image are compared to determine whether the image to be recognized is a qualified image. This application can greatly reduce the machine misjudgment rate and greatly improve the overall production line efficiency.

Please refer to FIG. 6, in this embodiment, the image processing system 20 can be divided into one or more modules, and the one or more modules can be stored in the processor 12 and used by the The processor 12 executes the image processing method of the embodiment of the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the image processing system 20 in the electronic device 1. For example, the image processing system 20 can be divided into the acquisition module 201, the processing module 202, the judgment module 203, and the marking module 204 in FIG. 6.

The acquisition module 201 is used to acquire the image to be recognized and the standard image; the acquisition module 201 is also used to acquire the first text area in the image to be recognized; the acquisition module 201 is also used to Get Take a second text area in the standard image, and extract a text window based on the second text area, where the text window includes a plurality of sub-windows; the processing module 202 is configured to be based on the first text area And the text window to obtain the target text area in the image to be recognized; the processing module 202 is further configured to segment the target text area according to the multiple sub-windows to obtain a first set of text sub-areas, and The second text area is divided according to the multiple sub-windows to obtain a second text sub-areas set; the judging module 203 is used to determine whether all the first text sub-areas in the first text sub-areas set are consistent with all the first text sub-areas in the first text sub-areas set. The corresponding second text subregions in the second text subregion set are the same; and the marking module 204 is used for when all the first text subregions in the first text subregion set are the same as the second text The corresponding second text subregions in the subregion set are all the same, and the image to be recognized is marked as a qualified image.

Due to the variability of production line data, this application has also formulated a system update mechanism to enable the image processing system 20 to be automatically updated through continuous accumulation of data to ensure the accuracy of the model to achieve the effect of adapting to various products.

In one embodiment, the result of the judgment of the image to be recognized by the system can be compared with the result of manual processing of the mark, and the detection index such as the accuracy rate, the missed detection rate, and the overkill rate can be calculated. When the detection index reaches the standard set by the production line, it represents that the overall system is stable, and the new data of the production line (ie, the image to be recognized) will continue to judge whether there is a defect through the image processing system 20 of the present application.

If the detection index does not meet the standards set by the production line, the system update mechanism is activated to retrain the preset classifier for the production line data (that is, the image to be recognized) to strengthen the adaptability of the preset classifier to the production line data , The retrained results are compared with the results marked by manual data, and the detection index is calculated. Repeat this process until the detection index meets the requirements set by the production line, then the system update is completed.

For example, this application collects a total of 699 images to be recognized for IC-type text components judged to be defective by AOI machines on the SMT production line, of which 386 training data are used to develop the image processing system, and 313 are The verification data is used for verification and testing after the development of the system, and the result of the confusion matrix according to the image processing system is shown in Figure 7. The confusion matrix on the left in Fig. 7 records the results of training data during the development of the image processing system. The real result of manual inspection is that 252 images are qualified and marked as "PASS", and 134 images are unqualified and marked as "FAIL". And through the image processing The prediction result of the system is that 246 images are qualified and marked as "PASS", and 140 images are unqualified and marked as "FAIL", of which 6 pictures are misjudged as unqualified by the image processing system image. It can be calculated that the missed detection rate in the training data is 0/(0+134)=0%, and the overkill rate is 6/(246+6)=2.3%.

The confusion matrix on the right in Figure 7 records the results of the verification data during the development of the image processing system. The real result of manual inspection is that 183 images are qualified and marked as "PASS", and 130 images are unqualified and marked as "FAIL". The result predicted by the image processing system is that 179 images are qualified, of which 1 image is unqualified and misjudged as qualified, 134 images are unqualified, and 5 of them are unqualified and misjudged as qualified. qualified. It can be calculated that the missed detection rate in the verification data is 1/(1+129)=0.7%, and the overkill rate is 5/(178+5)=2.7%. It can be seen that in the verification data, the accuracy rate of the image processing system of this application is as high as 98%. The application of this system to the SMT production line can greatly reduce the time required for manual visual inspection, and can reduce the adjustment of AOI by production line engineers. The frequency of machine parameters greatly improves efficiency and overall production line stability.

In one embodiment, the processor 12 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), and dedicated integrated circuits (Applications). Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor 12 may also be any other conventional processor or the like.

If the modules in the image processing system 20 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, this application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the foregoing method embodiments can be realized. Wherein, the computer program includes computer program code, and the computer program code may be in the form of original program code, object code, executable file, or some intermediate forms. The computer-readable medium may include Including: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, floppy disk, optical disk, computer memory, ROM, Read-Only Memory, random access RAM (Random Access Memory), electric carrier signal, telecommunications signal, software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

It is understandable that the module division described above is a logical function division, and there may be other division methods in actual implementation. In addition, the functional modules in the various embodiments of the present application may be integrated in the same processing unit, or each module may exist alone physically, or two or more modules may be integrated in the same unit. The above-mentioned integrated modules can be implemented either in the form of hardware or in the form of hardware plus software functional modules.

In another embodiment, the electronic device 1 may further include a memory (not shown), and the one or more modules may also be stored in the memory and executed by the processor 12. The memory may be an internal memory of the electronic device 1, that is, a memory built into the electronic device 1. In other embodiments, the memory may also be an external memory of the electronic device 1, that is, a memory external to the electronic device 1.

In some embodiments, the memory is used to store program codes and various data, for example, to store the program codes of the image processing system 20 installed in the electronic device 1, and achieve high speed during the operation of the electronic device 1. , Automatically complete program or data access.

The memory may include random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), at least one magnetic disk memory device, flash memory device, or other volatile solid-state memory device.

For those skilled in the art, it is obvious that the present application is not limited to the details of the foregoing exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the application. Therefore, no matter from which point of view, the above-mentioned embodiments of this application should be regarded as It is exemplary and non-restrictive. The scope of this application is defined by the appended claims rather than the above description. Therefore, it is intended that all changes falling within the meaning and scope of equivalent elements of the claims are included in this application Inside.

Claims (10)

An image processing method, the method comprising: acquiring a to-be-recognized image and a standard image; acquiring a first text area in the to-be-recognized image; acquiring a second text area in the standard image, according to the The second text area extracts a text window, wherein the text window includes a plurality of sub-windows; based on the first text area and the text window, the target text area in the image to be recognized is obtained; according to the multiple sub-windows The window divides the target text area to obtain a first text sub-areas set, and divides the second text area according to the plurality of sub-windows to obtain a second text sub-areas set; judging among the first text sub-areas set Whether all the first text subregions are the same as the corresponding second text subregions in the second text subregion set; and when all the first text subregions in the first text subregion set are the same as the second text subregion The corresponding second text sub-areas in the text sub-areas set are all the same, and the image to be recognized is marked as a qualified image. According to the image processing method of claim 1, said obtaining the target text area in the image to be recognized based on the first text area and the text window includes: intercepting the second in the standard image Text area; use the second text area to match the first text area to find the area in the second text area with the highest similarity to the first text area's primitive points; and use the text window In the first text area, the area with the highest similarity of the graphic element points is selected by frame selection to obtain the target text area. According to the image processing method of claim 1, determining whether all the first text subareas in the first text subareas set are the same as the corresponding second text subareas in the second text subareas set includes: Calculate the similarity of graphic element points between the first text subregion in the first text subregion set and the second text subregion in the corresponding second text subregion set to obtain a set of graphic element point similarities ; Determine whether the similarity of each image element point in the image element point similarity set is greater than or equal to a preset value; when the similarity of each image element point in the image element point similarity set is greater than or equal to the preset value Value, confirm that all the first text subregions in the first text subregion set are the same as the corresponding second text subregions in the second text subregion set; when the primitive point similarity is concentrated When the similarity of the existing graphic element points is less than the preset value, it is confirmed that there is a corresponding second text sub-region in the first text sub-region set and the second text sub-region set in the first text sub-region set The regions are not the same. According to the image processing method described in claim 3, the method for obtaining the similarity set of image element points includes: extracting characters in each first text subregion in the first text subregion set by a preset classifier And extract the second feature value of the character in each second text subregion in the second text subregion set by the preset classifier; and calculate the first feature value and the The graphic element point similarity between the second eigenvalues is described, and the graphic element point similarity set is obtained. According to the image processing method described in claim 4, the method for obtaining the set of similarity of primitive points further includes: inputting each first text subregion in the first text subregion set into the The classifier is preset to identify each first text subregion. According to the image processing method of claim 1, the method further includes: when there is a first text subregion in the first text subregion set and a corresponding second text in the second text subregion set When the sub-regions are not the same, mark the image to be recognized as a flawed image. According to the image processing method of claim 6, the method further includes preprocessing the image to be recognized, and the preprocessing the image to be recognized includes: filtering out the image in the image to be recognized through a filter. Noise; enhancing the first text area through image enhancement technology; and binarizing the image to be recognized. According to the image processing method of claim 7, the method further includes: after marking the image to be recognized as a qualified image, outputting prompt information to prompt the image to be recognized as a qualified image; or marking the image to be recognized as a qualified image; After the recognized image is a defective image, prompt information is output to prompt that the to-be-recognized image is a defective image. An electronic device, the electronic device comprising: a processor; and a memory in which a plurality of program modules are stored, and the plurality of program modules are loaded by the processor and executed as in claim 1 To the image processing method described in any one of Claim 8. A storage medium storing at least one computer instruction, the instruction being loaded by a processor and executing the image processing method as described in any one of request item 1 to request item 8.

Images