TWI836187B - Image processing method, program, and image processing system - Google Patents

Abstract

The purpose of the present disclosure is to speed up the processing. In the k+1th image search, the preliminary candidates for the k+1th image search are limited from the target images with the k+1th compression rate less than the kth compression rate; the k+1th candidate image is explored from the preliminary candidates for the k+1th image search, and the k+1th candidate image is added to the new candidate of the extracted image. The image processing method includes: search processing, acquisition processing, and sub-exploration processing. The search processing is to perform multiple image searches. The acquisition processing is to obtain relevant information indicating the correlation between the Nth candidate image and the sub-candidate image. The sub-exploration process is to obtain a sub-candidate image based on the N-th candidate image obtained in the N-th image search and the related information obtained in the acquisition process, and add the sub-candidate image to the reserve candidate for the M-th image search.

Description

Image processing methods, programs and image processing systems

The present disclosure relates to general image processing methods, programs and image processing systems, and more specifically to image processing methods, programs and image processing systems for discovering images containing the characteristics of model images from object images.

The image inspection device described in document 1 (JP2012-178106A) detects which part of the object image the inspection image (model image) is similar to by using image matching using a pyramid algorithm. In the pyramid algorithm, image matching is performed between a compressed object image and a compressed inspection image to identify a portion with a high image similarity (hereinafter referred to as a "candidate domain"). Subsequently, image matching is performed with respect to the peripheral areas of the candidate domain for an image with a lower compression rate. In the pyramid algorithm, since image matching does not need to be performed with respect to the entire image with a lower compression rate, processing speed can be increased.

In the imaging inspection apparatus described in Document 1, it is desired to further speed up the processing.

The purpose of this disclosure is to provide image processing methods, programs and image processing systems that can speed up processing.

The image processing method described in one aspect of the present disclosure performs a plurality of image searches to find an image containing the characteristics of the model image from the object image, that is, to extract the image. Let k be a variable of natural numbers. Let N and M be definite numbers of natural numbers and M is greater than N. In the first search for the first image, the first candidate image is found from the first target image with the first compression rate, and the first candidate image is added to the candidates for the extracted image. The first candidate image mentioned above contains the characteristics of the model image mentioned above with the first compression rate. In the k+1th preamble image search, the k+1th compression rate preamble target image whose compression rate is smaller than the kth compression rate is limited to the k+1th preamble image search preparation. Alternate. The prescript preparation candidates for the k+1th prescript image search include candidates for the prescript extraction image obtained in the k-th prescript image search. Then, in the k+1th foreword image search, the k+1th candidate image is searched out from among the foreword preliminary candidates used for the k+1th foreshadowing image search, and the k+1th candidate image is searched for , added to the preface to extract new candidates for the image. The k+1th candidate image mentioned above contains the characteristics of the model image with the k+1th compression rate mentioned above. The aforementioned image processing methods include: search processing, acquisition processing, and sub-exploration processing. The foreword search processing is to perform a plurality of foreword image searches. The aforementioned acquisition process is to acquire relevant information indicating the correlation between the Nth candidate image and the sub-candidate image. The prefix sub-search process is to obtain the prefix sub-candidate image based on the N-th candidate image obtained in the N-th prefix image search and the prefix sub-candidate information obtained in the prefix acquisition process. The image is added to the preamble candidate for the Mth preamble image search.

The program described in one aspect of the present disclosure is used to enable one or more processors to execute the aforementioned image processing method.

The image processing system described in one aspect of the present disclosure performs multiple image searches to search for images containing features of a model image from an object image, that is, to extract an image. Let k be a natural number variable. Let N and M be natural number constants and M be greater than N. In the first pre-record image search, the first candidate image is searched from the pre-record object image at the first compression rate, and the pre-record first candidate image is added to the candidate of the pre-record extracted image. The pre-record first candidate image contains features of the pre-record model image at the first compression rate. In the k+1th pre-record image search, the reserve candidate for the k+1th pre-record image search is narrowed down from the pre-record object image at the k+1th compression rate whose compression rate is less than the kth compression rate. The preceding reserve candidate for the k+1th preceding image search includes the candidate of the preceding extracted image found in the kth preceding image search. Then, in the k+1th preceding image search, the k+1th candidate image is searched out from the preceding reserve candidates for the k+1th preceding image search, and the preceding k+1th candidate image is added to the new candidate of the preceding extracted image. The preceding k+1th candidate image includes the characteristics of the preceding model image at the preceding k+1th compression rate. The preceding image processing system includes: a search processing unit, an acquisition unit, and a sub-exploration unit. The preceding search processing unit performs multiple preceding image searches. The preceding acquisition unit acquires relevant information indicating the correlation between the Nth candidate image and the sub-candidate image. The previous record sub-exploration unit obtains a previous record sub-candidate image based on the Nth candidate image obtained in the Nth previous record image search and the previous record related information obtained by the previous record acquisition unit, and adds the previous record sub-candidate image to the previous record reserve candidate for the Mth previous record image search.

The following uses drawings to illustrate the image processing method, program, and image processing system X1 described in the implementation form. However, the implementation form described below is only one of the various implementation forms disclosed herein. The implementation form described below can be modified in various ways according to the design, etc. as long as the purpose of the disclosure can be achieved. In addition, the various drawings described in the implementation forms described below are schematic drawings, and the size and thickness of each component in the drawing and even the proportion of each component do not necessarily reflect the actual size.

(1) Summary In the image processing method, an image search is performed. Image search is a process of comparing an object image generated by photographing an inspection object with a model image prepared in advance. By performing image search, the shape, position, and direction of the inspection object can be determined. As an example, the image processing method is executed for the purpose of inspecting the appearance of components (inspection objects) such as products in a factory. As another example, image processing methods are performed to identify certain components from other components.

Furthermore, the image search is performed multiple times. The number of image searches is assumed to be F (F is a natural number constant). In at least a portion of the multiple image searches, compressed images of the object image and the model image are used to speed up the processing. Each time the image search is repeated, the compression rate of each of the object image and the model image decreases. In other words, each time the consistent image search is repeated, the resolution of each of the object image and the model image increases.

Through the first image search, among the target images, the images that contain the characteristics of the model image, that is, the candidates for extracting the position and direction of the image, will be restricted to a certain extent. The extracted image is the object finally found through multiple (F) image searches. In other words, through the F-th (final) image search, the extracted image will be found. In the k+1th image search (k is a natural number variable. That is, k=1, 2, 3,...,F), the image search will be limited to the value narrowed in the k-th image search. The surrounding area of the position and the direction similar to the direction restricted in the k-th image search are further restricted. This makes it possible to shorten the time required for processing until the extracted image is specified, compared to a case where an image search is performed on all uncompressed (high-resolution) target images.

FIG. 1 is a flow chart of the image processing method of this embodiment. The image processing method of this embodiment performs a plurality of image searches to find an image containing the characteristics of the model image from the target image, that is, to extract the image. Let k be a variable of natural numbers. Let N and M be definite numbers of natural numbers and M is greater than N. In the first image search (step ST4), the first candidate image is found from the target image with the first compression rate, and the first candidate image is added to the candidates for the extracted image. The first candidate image contains the characteristics of the model image with the first compression rate. In the k+1-th image search, preliminary candidates for the k+1-th image search are narrowed out from target images with a compression rate of k+1 that is smaller than the k-th compression rate. The preliminary candidates for the k+1-th image search include candidates for the extracted images found in the k-th image search. Then, in the k+1th image search, the k+1th candidate image is discovered from among the preliminary candidates for the k+1th image search, and the k+1th candidate image is added to the extracted A new candidate for the image. The k+1th candidate image contains the characteristics of the model image with the k+1th compression rate. The image processing method includes: search processing, acquisition processing, and sub-exploration processing. Search processing involves performing multiple image searches. The acquisition process is to acquire related information indicating the correlation between the N-th candidate image and the sub-candidate image (step ST1). The sub-exploration process is to obtain the sub-candidate image based on the N-th candidate image obtained in the N-th image search and the related information obtained in the acquisition process, and add the sub-candidate image to the M-th candidate image. Preliminary candidates for image search (step ST6). The relevant information includes, for example, information about the rotation angle difference between the Nth candidate image and the sub-candidate image. In this case, the so-called sub-candidate image is an image obtained by rotating the N-th candidate image by the above-mentioned rotation angle difference.

The key point is that through the k-th image search, the k-th candidate image (the main preliminary candidate) will be obtained, and the k-th candidate image is added to the preliminary candidate for the k+1-th image search. When k+1=M, the sub-candidate images will also be added to the preliminary candidates for the k+1th image search. In the k+1th image search, the k+1th candidate image is obtained from the preliminary candidates for the k+1th image search. When the k-th image search is the final image search (that is, when k=F), the k-th candidate image is output as the final search result (extracted image).

According to this embodiment, compared with the case where an image search is performed on all uncompressed target images, the time required for processing until the extracted image is specified can be shortened.

Furthermore, in the Nth image search, since the compression rate of the object image and the model image is relatively large, the difference between the uncompressed object image and the uncompressed model image may lack the necessary data. Therefore, an image that does not match the model image in the uncompressed state may be obtained as the Nth candidate image. In this case, the Nth candidate image becomes one of the backup candidates for the N+1th image search (the search range of the N+1th image search), and the Nth candidate image is judged to be inconsistent with the model image in the image search after the N+1th. However, in the Mth image search which is later than the Nth image search, not only the candidate image obtained in the previous image search but also the sub-candidate image is used as a reserve candidate for the Mth image search, so the sub-candidate image is specified as an extracted image. That is, even if the Nth candidate image does not match the model image in a non-compressed state, there is still a possibility that the sub-candidate image and the model image match in a non-compressed state. In the image processing method of this embodiment, compared with the case where the sub-candidate image is not added to the reserve candidate, the time required to find the sub-candidate image can be shortened, and therefore the time required to specify the extracted image can be shortened.

In addition, N=1 is ideal. In the present embodiment, N=1 is used for explanation. That is, a sub-candidate supplementary image is obtained based on the 1st candidate image obtained in the 1st image search and related information. The sub-candidate supplementary image is added to the reserve candidate used in the Mth image search. In the Mth image search, a new candidate (Mth candidate supplementary image) for extracting an image is explored from the M-1th candidate image obtained in the M-1th image search and the reserve candidate. In the present embodiment, M=4 is used for explanation. Furthermore, in the present embodiment, the Mth image search is the last image search among multiple image searches. That is, in the present embodiment, F=M.

The flowchart shown in FIG. 1 is only an example of the image processing method, and the processing order may be changed as appropriate, and processing may be added or omitted as appropriate.

The image processing method is executed by the image processing system X1. As shown in FIG. 2 , the image processing system X1 includes a search processing unit S1, an acquisition unit 51, and a sub-search unit 52. The search processing unit S1 performs a plurality of image searches. The acquisition unit 51 acquires relevant information indicating the correlation between the N-th candidate image and the sub-candidate image. The sub-exploration unit 52 obtains sub-candidate images based on the N-th candidate image obtained in the N-th image search and the related information obtained by the acquisition unit 51, and adds the sub-candidate images to A candidate for the Mth image search.

By using the image processing system X1, the time required to process the extracted images to a specific level can be shortened.

(2)Image processing system The image processing system X1 includes: a computer system with more than one processor and memory. The program recorded in the memory of the computer system is executed by the processor of the computer system to realize at least part of the functions of the image processing system X1. The program may also be recorded in a memory, may be provided through an electrical communication line such as the Internet, or may be recorded on a non-transitory recording medium such as a memory card and provided.

The image processing system X1 includes a search processing unit S1, an acquisition unit 51, a sub-search unit 52, a generation processing unit 53, a storage unit 54, a control unit 55, and an image processing unit 56. The search processing unit S1 includes a first search unit 1, a second search unit 2, a third search unit 3, and a fourth search unit 4. In addition, these are merely examples of functions implemented by the image processing system X1, and do not necessarily represent actual structures.

In FIG. 2 , as external components of the image processing system X1, an imaging unit 61, a drive mechanism 62, and an operation unit 63 are provided. In addition, at least one of the imaging unit 61, the driving mechanism 62, and the operating unit 63 may be a component of the image processing system X1.

(3) Imaging unit The imaging unit 61 is a two-dimensional image sensor such as a CCD (Charge Coupled Devices) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. The imaging unit 61 photographs the inspection object (component). In this way, the imaging unit 61 generates an image of the inspection object, that is, an object image. The inspection object is transported to the imaging range of the imaging unit 61 by a moving device. The moving device is, for example, a conveyor belt. The timing of the imaging unit 61 performing imaging is controlled by the control unit 55.

(4)Driving mechanism The driving mechanism 62 includes, for example, a robot arm or an air generating device. The drive mechanism 62 moves the inspection object. For example, once an inspection object is determined to be a suitable product by the image processing system X1, the robot arm holds (picks up) the inspection object and moves it to a location where the next process is performed. For example, once a certain inspection object is determined to be unsuitable by the image processing system X1, the air generating device removes the inspection object by blowing air. The operation of the driving mechanism 62 is controlled by the control unit 55 .

(5)Memory department The memory unit 54 is, for example, ROM (Read Only Memory), RAM (Random Access Memory), or EEPROM (Electrically Erasable Programmable Read Only Memory). The memory unit 54 stores model images. In addition, the storage unit 54 stores various parameters used in the image processing method.

(6)Image processing department The image processing unit 56 compresses (compresses) the target image generated by the imaging unit 61 . Furthermore, the image processing unit 56 obtains the model image from the storage unit 54 and compresses the model image (compression processing). More specifically, the image processing unit 56 converts the object image and the model image into images with a smaller number of pixels (compressed images).

The object image and the model image are two-dimensional images. In addition, the object image and the model image are digital images. In this embodiment, the description is made based on the assumption that the object image and the model image are black and white images. However, the object image and the model image may also be grayscale images or color images.

The image processing unit 56 generates, for example, a 1/2i compressed image by multiplying the number of pixels in the vertical and horizontal directions of the original image (the object image and the model image) by 1/2i (i is a variable of a natural number). In the present embodiment, the image processing unit 56 generates a 1/2 compressed image, a 1/4 compressed image, and a 1/8 compressed image.

The image processing unit 56 extracts contours from the compressed image and the uncompressed image of each of the object image and the model image. For example, the Sobel method or the Laplace method of Gauss can be used as an algorithm for extracting contours. In addition, the model image can be provided as an image consisting of only contours from the beginning.

(7)Operation Department The operation unit 63 is a user interface that accepts user operations. The operation unit 63 includes, for example, at least one of a button, a switch, a DIP switch, a touch panel, a touch display, and the like.

The operation unit 63 accepts the user's operations required for determining relevant information.

Furthermore, the operation unit 63 receives a user operation required to determine a setting angle (a rotation angle for rotating the k-th candidate image or the sub-candidate image) described later.

Then, the operation unit 63 accepts the user's operation required to determine the number of image searches.

Furthermore, the operation unit 63 only needs to be able to receive at least an operation from the user, and the subject who actually operates the operation unit 63 is not limited to the user. The operation unit 63 may also receive an operation from someone other than the user.

(8) Acquisition Department The acquisition unit 51 acquires relevant information from the operation unit 63 . That is, once the operation required to determine the relevant information is performed on the operation unit 63, the operation unit 63 outputs a signal corresponding to the operation content, and the acquisition unit 51 obtains the signal as the relevant information.

That is, the image processing method of this embodiment includes the relevant information determination processing executed by the acquisition unit 51, and the relevant information determination processing is a processing for determining relevant information in response to the user's operation.

Furthermore, the acquisition unit 51 acquires information about the setting angle from the operation unit 63. That is, once the operation required to determine the setting angle is performed on the operation unit 63, the operation unit 63 outputs a signal corresponding to the operation content, and the acquisition unit 51 acquires the signal as information about the setting angle.

That is, the image processing method of this embodiment includes a setting process executed by the acquisition unit 51, and the setting process is a process of acquiring information on the setting angle. In addition, the image processing method of this embodiment includes a set angle determination process executed by the acquisition unit 51. The set angle determination process is a process of determining the set angle in accordance with the user's operation. The set angle is, for example, the angle itself designated by the user operating the operation unit 63 .

(9) Search processing unit The search processing unit S1 includes a plurality of search units. In the present embodiment, the number of search units is 4. That is, the search processing unit S1 includes, as a plurality of search units: a first search unit 1, a second search unit 2, a third search unit 3, and a fourth search unit 4.

Each of the plurality of search units performs an image search. Image search is a process of comparing an object image generated by photographing the inspection object with a pre-prepared model image. For one model image, the plurality of search units each perform one image search. The first search unit 1, the second search unit 2, the third search unit 3, and the fourth search unit 4 perform image searches in this order. The number of image searches is ideally three or more. In other words, in the search process (processing of performing multiple image searches), it is ideal to perform the image search three or more times. The number of image searches is four in the present embodiment, but is not limited to four, and may be two or more.

A plurality of search units use object images and model images with different compression rates to perform image search. In the image processing method of this embodiment, the compression rate of the object image and the model image is reduced each time the image search is repeated. That is, the first search unit 1 uses the object image and the model image with the first compression rate to perform the first image search, and the second search unit 2 uses the object image and the model image with the second compression rate to perform the second image search. The third search unit 3 uses the object image and the model image with the third compression rate to perform the third image search, and the fourth search unit 4 uses the object image and the model image with the fourth compression rate to perform the fourth image search. The k+1th compression rate is a compression rate less than the kth compression rate. For example, a 1/8 compressed image is used in the first image search, a 1/4 compressed image is used in the second image search, a 1/2 compressed image is used in the third image search, and a non-compressed image is used in the fourth (last) image search. That is, the compression rate may also include 0 (non-compressed).

(10) Sub-exploration unit The sub-exploration unit 52 obtains a sub-candidate supplementary image based on the first (Nth) candidate image obtained in the first (Nth) image search and the related information obtained by the acquisition unit 51. As an example, the related information includes information about the rotation angle difference between the first (Nth) candidate image and the sub-candidate supplementary image. In this case, the sub-exploration unit 52 generates an image in which the first (Nth) candidate image is rotated by the above-mentioned rotation angle difference, and regards the generated image as the sub-candidate supplementary image. The above-mentioned information about the rotation angle difference is generated, for example, by the generation processing unit 53. The above-mentioned information about the rotation angle difference can also be provided by the user's operation on the operation unit 63. For example, when only one type of component is considered as the inspection object, since the above information on the rotation angle difference does not need to be changed later, it is not a problem even if the user provides the above information on the rotation angle difference in advance.

As the rotation angle difference, only one angle difference may be used, or a plurality of angle differences may be used. For example, three angle differences of 90 degrees, 180 degrees, and 270 degrees may be used as the rotation angle difference. In this case, the sub-searching unit 52 rotates the first candidate image by 90 degrees, 180 degrees, and 270 degrees, respectively, to generate three sub-candidate images.

(11)Generation processing department The image processing method of this embodiment includes a generation process executed by the generation processing unit 53. The generation process is a process of generating the above-mentioned information on the rotation angle difference based on the rotational symmetry of the subject in the model image. . For example, when the subject system of the model image has L-order rotational symmetry (L is a fixed number of natural numbers), the generation processing unit 53 determines the above-mentioned rotation angle difference to 360/L [degree].

For example, the object (inspection object Ob1) of the model image shown in FIG. 3 is twice rotationally symmetric, so the generation processing unit 53 determines the above-mentioned rotation angle difference to be 180 degrees.

The information on the rotational symmetry of the object of the model image may also be provided by the user's operation on the operation unit 63. Alternatively, the generation processing unit 53 may generate the information on the rotational symmetry of the object of the model image by analyzing the model image.

For example, the timing at which a certain inspection object Ob1 is transported into the imaging range of the imaging unit 61 and the timing at which an inspection object of another shape is transported into the imaging range of the imaging unit 61 are determined in advance. Therefore, the generation processing unit 53 changes the above-mentioned information about the rotation angle difference according to time.

In addition, when the object is not rotationally symmetrical, information indicating this is output by the generation processing unit 53 .

(12)Example 1 A first example of a program for inspecting an inspection object using an image processing method will be described with reference to FIGS. 1, 3 to 6B.

First, the user operates the operation unit 63 to input relevant information. The acquisition unit 51 acquires relevant information from the operation unit 63 (step ST1). The imaging unit 61 captures the inspection object Ob1 to generate an object image (step ST2).

Figure 3 shows a model image of the first compression ratio. FIG. 4 shows the target image of the first compression rate. Figure 5 shows a model image at the fourth compression ratio (uncompressed). 6A and 6B show target images at the fourth compression rate (uncompressed). However, in any of Figures 3 to 6B, the model image and the object image are images composed only of contour lines (dot portions in the figure). The image processing unit 56 compresses the target image as necessary and extracts the contour lines to generate the target image (step ST3). In addition, the model image is a rectangular or square image, which is obtained by removing rows and columns without outlines in the model image in advance.

In addition, it is ideal that the resolution of the model image and the object image is higher than the resolution shown in FIG. 3 to FIG. 6B , but in order to simplify the description, the images are shown in FIG. 3 to FIG. 6B as images with lower resolution.

As shown in Fig. 5, the two-dimensional shape of the inspection object Ob1 is a cross shape. The inspection object Ob1 has a center portion Ob10 and four protruding portions Ob11 to Ob14. The four protruding portions Ob11 to Ob14 protrude from the center portion Ob10. The protruding portions Ob11 and Ob12 are provided on opposite sides of the center portion Ob10. The protruding portions Ob13 and Ob14 are provided on opposite sides of the central portion Ob10. The protruding directions of the protruding portions Ob11 and Ob12 are orthogonal to the protruding directions of the protruding portions Ob13 and Ob14.

In Figure 5, the lengths of the four protruding parts Ob11~Ob14 are 8[pix], 9[pix], 6[pix], and 6[pix] respectively. If the model image in Figure 5 is compressed to the first compression ratio, as shown in Figure 3, the lengths of the four protruding parts Ob11~Ob14 become 4[pix], 4[pix], 3[pix], and 3 respectively. [pix]. That is, in FIG. 3 , the lengths of the protruding portions Ob11 and Ob12 are equal. In Fig. 3, the two-dimensional shape of the inspection object Ob1 is twice rotationally symmetric.

FIG. 6B is a view obtained by rotating the inspection object Ob1 by 180 degrees with respect to FIG. 6A . When any of the target images in FIG. 6A and FIG. 6B is compressed to the first compression rate, it will become the image shown in FIG. 4 . That is, in the model image and the object image of the first compression rate shown in FIGS. 3 and 4 , the protruding portion Ob11 is located on the paper surface with respect to the center portion Ob10 and is located below the paper surface. difference.

In image search, the model image is compared with the object image by scanning the object image.

(12-1) The first image search In the first image search (step ST4), the first search unit 1 compares the model image of FIG. 3 with the object image of FIG. 4. First, the first search unit 1 regards the area in the upper left corner of the object image as the comparison object (comparison area R1). The size of the comparison area R1 is equal to the size of the model image. The first search unit 1 compares each pixel of the image of the comparison area R1 with each pixel of the model image. If the consistency rate between the image of the comparison area R1 and the model image is above the threshold, the first search unit 1 determines that the image of the comparison area R1 is the first candidate image containing the features of the model image. That is, in this case, the first search unit 1 adds the image of the comparison area R1 to the candidate of the extracted image. On the other hand, if the consistency rate between the image of the comparison area R1 and the model image is less than the threshold, the first search unit 1 determines that the image of the comparison area R1 is not the first candidate image.

The threshold is pre-stored in the memory unit 54. In this embodiment, the threshold is assumed to be 100% for explanation. That is, in this embodiment, only when the image of the comparison area R1 is completely consistent with the model image, the first search unit 1 will determine that the image of the comparison area R1 is the first candidate image.

The first search unit 1 translates the comparison area R1 by 1 [pix] at a time, and each time it translates by 1 [pix], it compares the image of the comparison area R1 with the model image. For example, the comparison area R1 is translated by 1 [pix] in the column direction (left-right direction of the paper of FIG. 4 ), and once the comparison area R1 reaches one end (right end or left end) of the object image, the comparison area R1 is translated by 1 [pix] in the row direction, and this process is regarded as 1 set. By repeating this process for multiple sets, the first search unit 1 compares all areas of the object image with the model image. In FIG. 4 , when area R11 is selected as the comparison area R1, the first search unit 1 determines that the image of the comparison area R1 is the first candidate image.

Furthermore, the first search unit 1 compares a plurality of rotated images with the object image. The plurality of rotated images are images obtained by performing a phased rotation of the model image. For example, by rotating the model image 360 degrees clockwise or counterclockwise in units of 1 degree, a plurality of rotated images are generated. By repeating the above process for each rotated image a plurality of times, the first search unit 1 compares all areas of the object image with the rotated image and searches for the first candidate image.

Here, when the 1st (Nth) candidate image is found at a certain position, the rotated image at that position can also be excluded from the search range in the 1st (Nth) image search. In other words, it does not matter if the rotated image at that position is not selected as the 1st (Nth) candidate image. This can shorten the time required for the image search. The rotated image at that position is compared with the model image in the 4th image search to determine whether it is equivalent to the extracted image.

Once at least one first candidate image is found by the first image search, the memory unit 54 stores information about the first candidate image. The information about the first candidate image includes at least information about the position and direction of the first candidate image. The information about the first candidate image may also include the consistency rate between the first candidate image and the model image.

As described above, in the model image and the object image of the first compression ratio shown in FIGS. 3 and 4 , the protruding portion Ob11 is located on the upper side of the paper with respect to the center portion Ob10 , and the state is located on the lower side of the paper. , there is no difference. In either case, when the area R11 is selected as the comparison area R1, the first search unit 1 determines that the image in the comparison area R1 is the first candidate image.

Once it is determined that the image of at least one region is the first candidate image (step ST5: YES), the sub-exploration unit 52 generates a sub-candidate image (step ST6: sub-exploration processing). Specifically, the sub-exploration unit 52 uses the image obtained by rotating the first candidate image by 180 degrees as the sub-candidate image.

(12-2) Image search after the second time Next, the second search unit 2 performs a second image search using the target image with the second compression rate and the model image with the second compression rate (step ST7). In the second image search, first, the preliminary candidates for the second image search are narrowed down. The preliminary candidates for the second image search include the first candidate images obtained in the first image search. In this example, through the first image search, the image corresponding to the area R11 (see FIG. 4 ) is obtained and used as the first candidate image. The memory unit 54 stores information on the position and direction of the first candidate image. Therefore, using this information, the second search unit 2 specifies the area corresponding to the area R11 among the target images of the second compression rate, and treats the image of the area corresponding to the area R11 as the second A candidate for image search. The second search unit 2 searches for a second candidate image containing the characteristics of the model image at a second compression rate from among the preliminary candidates for the second image search.

In the same manner, the third and fourth image searches (steps ST9 and ST11) search for candidate images from the images (preparatory candidates) in the area found in the previous image search. That is, in the k+1th image search (here, k=1, 2, 3), the k+1th image is compressed from the target image with the k+1th compression rate. Preparatory candidates for image search (search range of the k+1th image search). Then, from the preliminary candidates for the k+1th image search, the k+1th candidate image containing the characteristics of the model image with the k+1th compression rate is searched for.

The preliminary candidates for the k+1th image search include, in addition to the candidates for the extracted images found in the k-th image search (k-th candidate image), they also include the candidates used in the k-th image search. Find the surrounding images of the extracted image candidates. For example, in FIG. 6A , let the area corresponding to the extracted image candidate found in the third image search be area R41. The preliminary candidates for the fourth image search include images of area R41 and images surrounding area R41. In the fourth image search, the fourth candidate image was discovered from the images of area R41 and the images surrounding area R41.

The image surrounding the area R41 is an image of the area moved by a predetermined amount relative to the area R41. More specifically, the image surrounding the area R41 is generated by performing at least one of the following: parallel movement of the area R41 by an amount equal to or less than a predetermined number of pixels, and rotation of the area R41 by an amount equal to or less than a predetermined rotation angle. The image of the formed field.

When k=1 or 2, the preliminary candidates for the k+1th image search only include the extracted image candidates found in the k-th image search and their surrounding images. When k=3, the preliminary candidates for the k+1th (Mth) image search include, in addition to the extracted image candidates found in the k-th image search, their surrounding images, and Sub-candidate images. In this example, the sub-candidate image is an image obtained by rotating the first candidate image by 180 degrees. That is, the preliminary candidates for the fourth image search include at least the image of the area R41 in Figure 6A and the image obtained by rotating the image of the area R41 by 180 degrees, that is, the image of the area R42 of Figure 6B.

In addition, when k=3, the preliminary candidates for the k+1th (Mth) image search also include a plurality of rotated images. The plurality of rotated images are images obtained by rotating the k-th candidate image or sub-candidate image in stages from 0 degrees to a range below a set angle (first set value). That is, the image processing method of this embodiment includes range expansion processing. The range expansion processing is a process of adding a plurality of rotated images to the preliminary candidates for the k+1th image search. The range expansion process is executed by, for example, the fourth search unit 4 . A plurality of rotated images are generated based on each of the k-th candidate image and the sub-candidate image.

For example, the third candidate image (kth candidate image) or the sub-candidate image is rotated clockwise and counterclockwise in units of the second setting value (value representing an angle) to generate a plurality of rotated images. In addition, the upper limit of the rotation angle difference between the third candidate image or the sub-candidate image and the rotated image, that is, the first setting value, is pre-specified. The first setting value and the second setting value are determined in response to the operation of the operating unit 63. That is, once the operation required to determine at least one of the first setting value and the second setting value is performed on the operating unit 63, the operating unit 63 outputs a signal corresponding to the operation content, and in response to the signal, the fourth search unit 4 determines the first setting value and the second setting value.

The first setting value is less than 1/2 times the difference in rotation angle between the first (Nth) candidate image and the sub-candidate image. The first set value can be selected from a plurality of values, for example. The first setting value can be selected from, for example, 5 degrees, 15 degrees, and 45 degrees.

The second setting value is smaller than the first setting value. The second setting value is, for example, 1 degree.

If a (4th) candidate image is found in the last (4th) image search (step ST12: YES), the search processing unit S1 treats the candidate image as the final search result. That is, the search processing unit S1 regards the candidate image as an image containing the characteristics of the model image, that is, as an extracted image. The search processing unit S1 outputs information about the extracted images. For example, the search processing unit S1 outputs information about the extracted image to the control unit 55 . The information about the extracted image includes, for example, information about the position and direction of the extracted image. Based on the information about the extracted image, the position and direction of the imaging object of the target image, that is, the inspection object Ob1, etc. can be specified. Information about the extracted image may also include the consistency rate between the extracted image and the model image.

The control unit 55 controls the drive mechanism 62 based on the information about the extracted image. The drive mechanism 62 holds (picks up) the inspection object Ob1 and moves it to a place where the next process is performed (step ST13). Furthermore, the drive mechanism 62 corrects the direction of the inspection object Ob1 based on the information indicating the direction of the inspection object Ob1.

Also, when no candidate image is found in the image search (the second and third times) between the first image search and the last (fourth) image search (step ST8 or ST10: NO) , then at least the sub-candidate images are included in the preliminary candidates for the fourth image search, and the fourth image search is performed (step ST14). That is, the sub-candidate image is compared with the model image to find the fourth candidate image. Here, in addition to the sub-candidate images, a rotated image in which the sub-candidate images are rotated may also be included in the preliminary candidates for the fourth image search.

(12-3) Advantages In the first example, in the second and third image searches, only the candidate for the extracted image found in the previous image search and the images surrounding it are used as the search range (reserve candidates). Therefore, compared with the case where the entire target image is used as the search range, the time required for the second and third image searches can be shortened. In addition, in the second and third image searches, only the candidate for the extracted image found in the previous image search can be used as the search range, and the images surrounding the candidate for the extracted image are excluded from the search range.

Furthermore, in the first example, in the fourth image search, the sub-candidate image is also added to the search range (preliminary candidate). The sub-candidate image is not directly obtained by the image search, but is obtained from the first candidate image, so the time required to find the sub-candidate image can be shortened. Therefore, when the sub-candidate image is output as the final search result (extracted image), the time required to identify the extracted image can be shortened.

Furthermore, the image processing method of this embodiment searches for extracted images based on one model image. That is, one model image is transformed to generate a plurality of rotated images. Therefore, compared with the case of preparing a plurality of rotated images in advance, the memory capacity occupied by the model image can be reduced.

(13)Example 2 A second example of a program for inspecting an inspection object using an image processing method will be described with reference to FIGS. 7 and 8 . Figure 7 shows a model image of a certain compression ratio. Figure 8 shows an object image with a certain compression ratio.

In the second example, the two-dimensional shape of the inspection object Ob2 is the shape of a gear. The inspection object Ob2 has a disc-shaped disc portion Ob20 and a plurality (18 in FIG. 7 ) of teeth Ob21. A plurality of teeth Ob21 are arranged at equal intervals on the outer edge of the disc portion Ob20. One tooth Ob210 among the plurality of teeth Ob21 is shorter than the other teeth Ob21.

In FIG. 8 , area R8 is an area corresponding to the entire object image. The image that is correct as the extracted image being searched for in the image search is the image of domain R81.

However, in the first image search, the target image with the first compression rate and the model image with the first compression rate are used, and the first compression rate is a relatively high compression rate. Therefore, the difference in shape between the shorter tooth Ob210 and the other teeth Ob21 among the plurality of teeth Ob21 may become smaller. As a result, in the first image search, the image in the area R82 may be output as a candidate for the extracted image (the first candidate image). The image in area R82 is an image rotated by a predetermined angle relative to the image in area R81.

Therefore, the sub-exploration unit 52 generates a plurality of images obtained by rotating the first candidate image (the image of the area R81), and divides the generated plurality of images into sub-candidate images. The plurality of sub-candidate images are, for example, images obtained by rotating the first candidate image by 360/NA [degrees]. NA is the number of teeth Ob21. That is, the plurality of sub-candidate images are images obtained by rotating the first candidate image by 20 degrees, 40 degrees, 60 degrees, ..., 340 degrees.

Here, the number NA of teeth Ob21 is information about the rotational symmetry of the object (inspection object Ob2) of the model image. The information of the number NA of teeth Ob21 can also be provided by the user's operation on the operation unit 63. Alternatively, the generation processing unit 53 can also obtain the number NA of teeth Ob21 by analyzing the model image of the first (Nth) compression rate.

One of the plurality of sub-candidate supplementary images is an image of the area R81. Therefore, in the fourth image search including the plurality of sub-candidate supplementary images in the search range (preliminary candidates), the image of the area R81 is identified as an extracted image.

The same applies to the second example. Like the first example, the time required for image search can be shortened.

(Variations of the implementation form) The following are variations of the implementation form. The following variations can also be implemented by combining them as appropriate.

In the implementation form, the sub-candidate image is added to the preliminary candidate only in the last (4th) image search among multiple (F) image searches. In contrast, the sub-candidate image may be added to the preliminary candidate in an image search that is later than the Nth (in the implementation form, N=1) image search and earlier than the last image search (the second or third time). Furthermore, the sub-candidate image may be added to the preliminary candidate in two or more image searches among multiple image searches. For example, the sub-candidate image may be added to the preliminary candidate in all image searches (the second to fourth times) that are later than the Nth (in the implementation form, N=1) image search.

The image used for image search is not limited to the image from which the contour has been extracted. The image used for image search may be the original data generated by the imaging unit 61 or may be data after appropriate conversion of the original data.

The extracted image is obtained by the image processing method. However, the information finally output by the image processing method may not be the extracted image, for example, it may only contain the position (coordinate) information and direction information of the extracted image.

In the case where the subject of the model image is not rotationally symmetric, the process of generating the candidate image can be invalidated.

The image processing method can also be specifically implemented by a (computer) program or a non-transitory recording medium that records the program. That is, the program described in one aspect is a program required to cause one or more processors to execute the image processing method.

The image processing system X1 disclosed herein includes a computer system. The computer system is mainly composed of a processor and a memory as hardware. The functions of the image processing system X1 disclosed herein can be realized by executing the program recorded in the memory of the computer system by the processor. The program can be pre-recorded in the memory of the computer system, provided through an electrical communication line, or recorded in a non-temporary recording medium such as a memory card, optical disk, hard disk drive, etc. that can be read by the computer system. The processor of the computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). The integrated circuits such as IC or LSI mentioned here have different names depending on the degree of integration, including integrated circuits called system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). Furthermore, FPGA (Field-Programmable Gate Array) that can be programmed after the manufacture of LSI, or logical devices that can reconfigure the connection relationship within LSI or reconfigure the circuit blocks within LSI can also be used as processors. Multiple electronic circuits can be integrated into one chip or dispersed into multiple chips. Multiple chips can be integrated into one device or dispersed into multiple devices. The computer system mentioned here includes a microcontroller having one or more processors and one or more memories. Therefore, the microcontroller is also composed of one or more electronic circuits including semiconductor integrated circuits or large-scale integrated circuits.

In addition, the fact that a plurality of functions in the image processing system X1 are concentrated in one device is not an essential configuration of the image processing system X1. The constituent elements of the image processing system X1 may be distributed in a plurality of devices. Furthermore, at least part of the functions of the image processing system X1, such as at least part of the functions of the search processing unit S1, can also be realized through the cloud (cloud computing) or the like.

On the contrary, in the embodiment, at least a part of the functions of the image processing system X1 and the like which are distributed to a plurality of devices may be integrated into one device. For example, the functions which are distributed to the image processing system X1 and the drive mechanism 62 may be integrated into one device.

(Summary) Based on the above-described embodiments and the like, the following aspects are disclosed.

The image processing method described in the first aspect is to perform multiple image searches to find an image containing the features of the model image from the object image, that is, to extract the image. Let k be a natural number variable. Let N and M be natural number constants and M be greater than N. In the first image search, the first candidate image is found from the object image of the first compression rate, and the first candidate image is added to the candidate of the extracted image. The first candidate image contains the features of the model image of the first compression rate. In the k+1th image search, the reserve candidate for the k+1th image search is narrowed down from the object image of the k+1th compression rate whose compression rate is less than the kth compression rate. The preliminary candidate for the k+1th image search includes the candidate for the extracted image found in the kth image search. Then, in the k+1th image search, the k+1th candidate image is explored from the preliminary candidate for the k+1th image search, and the k+1th candidate image is added to the new candidate for the extracted image. The k+1th candidate image includes the characteristics of the model image of the k+1th compression rate. The image processing method includes: search processing, acquisition processing, and sub-exploration processing. The search processing is to perform multiple image searches. The acquisition processing is to obtain relevant information indicating the correlation between the Nth candidate image and the sub-candidate image. The sub-exploration process is to obtain a sub-candidate image based on the N-th candidate image obtained in the N-th image search and the related information obtained in the acquisition process, and add the sub-candidate image to the reserve candidate for the M-th image search.

According to the above-mentioned structure, the processing required for extracting a specific image can be accelerated.

Furthermore, in the image processing method described in the second aspect, in the first aspect, the relevant information includes information on the rotation angle difference between the Nth candidate image and the sub-candidate image.

According to the above-mentioned structure, an extracted image containing a rotationally symmetric subject can be obtained at high speed.

Furthermore, the image processing method according to the third aspect, in the second aspect, further includes a generation process. In the generation process, information on the rotation angle difference is generated based on the rotation symmetry of the object photographed in the model image.

If the above structure is followed, the trouble of inputting the rotation angle difference can be reduced.

Furthermore, the image processing method according to the fourth aspect is, in any one of the first to third aspects, further comprising a related information determination process. In the related information determination process, related information is determined in response to a user's operation.

Based on the above structure, the relevant information may be changed according to the circumstances.

Furthermore, the image processing method described in the fifth aspect is in any one of the first to fourth aspects, and further includes range expansion processing and setting processing. In the range expansion process, a plurality of rotated images are added to the preliminary candidates for the k+1th image search. The plurality of rotated images are images obtained by rotating the k-th candidate image or sub-candidate image in stages within a range from 0 degrees to a set angle. In the setting process, information about the setting angle is obtained.

If the above composition is followed, the accuracy of image search can be improved.

Furthermore, the image processing method described in the sixth aspect is, in the fifth aspect, further comprising a setting angle determination process. In the setting angle determination process, the setting angle is determined in response to the user's operation.

If the above configuration is followed, the setting angle can be changed according to the situation.

Furthermore, in the image processing method described in the seventh aspect, in any one of the first to sixth aspects, N=1.

According to the above configuration, as a result of the first image search, the first candidate image and the sub-candidate image are added to the reserve candidate. Therefore, the possibility of shortage of reserve candidates in the first image search can be reduced, and the accuracy of the image search can be improved.

Furthermore, in the image processing method described in aspect 8, in any one of aspects 1 to 7, the image search is performed three or more times in the search process.

If the above structure is followed, the accuracy of image search can be improved.

Furthermore, the image processing method described in the ninth aspect is, in any one of the first to eighth aspects, a search for an extracted image based on a model image.

If the above composition is followed, the memory capacity occupied by the model image can be reduced.

The components other than the first aspect are not necessary for the image processing method and may be omitted as appropriate.

Furthermore, the program described in the tenth aspect is a program required to cause one or more processors to execute the image processing method described in any one of the first to ninth aspects.

According to the above-mentioned structure, the processing required for extracting a specific image can be accelerated.

In addition, the image processing system (X1) described in the eleventh aspect performs a plurality of image searches to find an image containing the characteristics of the model image from the target image, that is, to extract the image. Let k be a variable of natural numbers. Let N and M be definite numbers of natural numbers and M is greater than N. In the first image search, the first candidate image is found from the target image with the first compression rate, and the first candidate image is added to the candidates for the extracted image. The first candidate image contains the characteristics of the model image with the first compression rate. In the k+1-th image search, preliminary candidates for the k+1-th image search are narrowed out from target images with a compression rate of k+1 that is smaller than the k-th compression rate. The preliminary candidates for the k+1-th image search include candidates for the extracted images found in the k-th image search. Then, in the k+1th image search, the k+1th candidate image is discovered from the preliminary candidates for the k+1th image search, and the k+1th candidate image is added to the extracted A new candidate for the image. The k+1th candidate image contains the characteristics of the model image with the k+1th compression rate. The image processing system (X1) includes a search processing unit (S1), an acquisition unit (51), and a sub-exploration unit (52). The search processing unit (S1) performs multiple image searches. The acquisition unit (51) acquires relevant information indicating the correlation between the Nth candidate image and the sub-candidate image. The sub-exploration part (52) obtains the sub-candidate images based on the N-th candidate image obtained in the N-th image search and the related information obtained by the acquisition part (51), and converts the sub-candidate images into The image is added to the preliminary candidates for the M-th image search.

According to the above-mentioned structure, the processing required for extracting a specific image can be accelerated.

Not limited to the above aspects, various structures (including variations) of the image processing system (X1) described in the implementation aspects can be specifically implemented by image processing methods and programs.

S1:Search processing department X1:Image processing system 1:Search Department 1 2:Second Search Department 3:Search Department 3 4:Search Department 4 51: Acquisition Department 52: Sub-exploration department 53:Generation processing department 54:Memory Department 55:Control Department 56:Image processing department 61:Camera Department 62:Driving mechanism 63:Operation Department

[Figure 1] Figure 1 is a flow chart of an image processing method according to an embodiment. [Figure 2] Figure 2 is a block diagram of an image processing system for implementing the above image processing method. [Figure 3] Figure 3 is a model image of the first compression rate used in the first example of the above image processing method. [Figure 4] Figure 4 is an object image of the first compression rate used in the first example of the above image processing method. [Figure 5] Figure 5 is a model image of the fourth compression rate used in the first example of the above image processing method. [Figure 6A] Figure 6A is an object image of the fourth compression rate used in the first example of the above image processing method. [Figure 6B] Figure 6B is an object image of the fourth compression rate used in the first example of the above image processing method. [Figure 7] Figure 7 is a model image used in the second example of the above image processing method. [Figure 8] Figure 8 is an object image used in the second example of the above image processing method.

Claims (11)

An image processing method is an image processing method that performs multiple image searches to find images containing the characteristics of the model image from the target image, that is, to extract the image. Among them, let k be a natural number variable, let N, M is a definite number of natural numbers and M is greater than N; in the first search of the prefix image, the characteristics of the prefix model image containing the first compression rate are found from the prefix target image of the first compression rate. For the first candidate image, add the first candidate image mentioned above to the candidates for the extracted image; in the k+1th foregoing image search, the first candidate image is selected from the k+1th compression rate object whose compression rate is smaller than the kth compression rate. Among the images, the preliminary candidates for the k+1th foreword image search including candidates for the preamble extraction image found in the kth foreword image search are narrowed down, and the preliminary candidates for the k+1th foreword image are Among the preliminary candidates for searching, find the k+1th candidate image that contains the characteristics of the preceding model image with the k+1th compression rate mentioned above, and add the k+1th candidate image mentioned above to the new extracted image of the preceding paragraph. candidate; the aforementioned image processing method includes: search processing, which performs a plurality of searches for the aforementioned image; and acquisition processing, which obtains relevant information indicating the correlation between the N-th candidate image and the sub-candidate image; and sub-exploration processing, which is based on the N-th candidate image The Nth candidate image obtained in the search for the prefix image and the prefix related information obtained in the prefix acquisition process are used to obtain the prefix sub-candidate image, and the prefix sub-candidate image is obtained. The first candidate is added to the Mth previous image search; the first sub-candidate image is an image recognized as rotationally symmetrical with respect to the Nth candidate image. As described in claim 1, the image processing method, wherein the aforementioned related information includes: information about the rotation angle difference between the aforementioned Nth candidate image and the aforementioned sub-candidate image. The image processing method described in claim 2 further includes: generating processing to generate information about the rotation angle difference based on the rotational symmetry of the object in the aforementioned model image. The image processing method described in any one of claim items 1 to 3 further includes: related information determination processing, which determines the aforementioned related information in response to the user's operation. The image processing method as described in any one of claims 1 to 3, which further includes: range expansion processing, which is to rotate a plurality of rotated images, which are respectively the aforementioned k-th candidate image or the aforementioned sub-candidate image at 0 degrees. The image formed by performing stepwise rotation within the range below the set angle is added to the foregoing candidate for the k+1th foregoing image search; and the setting process is to obtain information about the foregoing set angle. The image processing method described in Claim 5 further includes: a setting angle determination process that determines the setting angle in response to the user's operation. An image processing method as described in any one of claim items 1 to 3, wherein N=1. The image processing method described in any one of Claims 1 to 3, wherein in the preamble search processing, the preamble image search is performed three or more times. An image processing method as described in any one of claim items 1 to 3, wherein the aforementioned image processing method is to explore the aforementioned extracted image based on a aforementioned model image. An image processing program is used to cause more than one processor to execute the image processing method described in any one of claims 1 to 9. An image processing system is an image processing system that performs multiple image searches to find images containing the characteristics of model images from target images, that is, to extract images. Among them, let k be a natural number variable, let N, M is a definite number of natural numbers and M is greater than N; in the first preamble image search, the preamble pair of the first compression rate is Among the image images, the first candidate image containing the characteristics of the aforementioned model image with the first compression rate is discovered, and the first candidate image is added to the candidates of the aforementioned extracted images; in the k+1th search for the aforementioned image , from the k+1th compression target image with a compression rate smaller than the k-th compression rate, limit the k+th candidate image containing the prefix extraction image found in the k-th preceding image search. From the preliminary candidates for the k+1th preamble image search, the k+1 th preamble model image containing the characteristics of the preamble model image with the k+1th compression rate is searched out. The candidate image adds the k+1th candidate image mentioned above to the new candidate of the extracted image mentioned above; the image processing system mentioned above includes: a search processing unit that performs a plurality of searches for the aforementioned image; and an acquisition unit that obtains the Nth candidate image. The relevant information related to the candidate image and the sub-candidate image; and the sub-exploration part is based on the N-th candidate image found in the N-th foreshadowing image search and the foregoing relevant information obtained by the foregoing acquisition part, and Obtain the prefix sub-candidate image, and add the prefix sub-candidate image to the prefix sub-candidates for the Mth prefix image search; the prefix sub-candidate image is an image recognized as rotationally symmetrical with respect to the N-th candidate image of the prefix .

Images