JP6890101B2 - An image identification device and an article manufacturing device equipped with an image identification device - Google Patents

Description

The present invention relates to an image identification device for identifying an image type, and an article manufacturing device including the image identification device.

In recent years, an image identification device for identifying an image type has been used when visually inspecting an article. As a kind of such an image identification device, for example, an image classification device for inspecting the appearance of a semiconductor substrate as an article (hereinafter, simply referred to as a substrate) as disclosed in Patent Document 1 is known.

This image classification device is an image pickup device that images an inspection target area on a substrate to create image data, and an image data created by the image pickup device in which a defect is reflected, depending on the type of defect. It is equipped with an inspection / classification device for sorting.

The inspection / classification device includes a defect detection unit that detects defects from image data, and a classification control unit that executes a classification process in which the defect detection unit classifies the image data in which defects are detected according to the type of defect. The classification control unit is configured to classify (identify) the image data each time the defect detection unit detects a defect from the image data.

Therefore, the image classification device can confirm the presence or absence of defects on the substrate, and further prepares a state in which defect analysis can be performed efficiently by aggregating the image data in which the defects are reflected by classifying them according to the type of defects. be able to.

Japanese Unexamined Patent Publication No. 2016-115115

By the way, depending on the state of the substrate to be inspected, a large amount of image data in which defects are reflected may be generated, but in the above-mentioned conventional image classification device, the classification control unit performs image data in which defects are reflected. Is configured to be sequentially classified according to the order in which they were generated (the order in which defects were detected), so while the classification process for one image data is being executed, the classification process for other image data is in a standby state. It becomes.

Therefore, in the above-mentioned conventional image classification device, there is a problem that a large number of classification processes for image data are accumulated and it takes time to finish classifying all the image data in which defects are reflected. Further, when the conventional image classification device is incorporated in the article manufacturing apparatus, there is a problem that the flow of the article manufacturing process is delayed.

It should be noted that such a problem is not limited to a device for detecting defects from image data, but is a problem that applies to all devices having a function of identifying an image.

Therefore, in view of such circumstances, it is an object of the present invention to provide an image identification device capable of speeding up an identification process for a large number of captured images, and an article manufacturing apparatus provided with the image identification device.

The image identification device of the present invention
A storage means for storing a plurality of captured images obtained by capturing an article as stock images, and
An identification means for executing the identification process for identifying the type of the stock image, and
An input means for executing an input process for collectively inputting a plurality of the stock images stored in the storage means into the identification means is provided.
The identification means is configured to execute a plurality of identification processes for each of the plurality of stock images collectively input by the input means in parallel.

According to such an image identification device, a plurality of images to be identified as a type are collectively stored as stock images in a storage means, and the plurality of stock images stored in the storage means are collectively identified by an input means. Is entered in.

Then, in the identification means, a plurality of identification processes for each of the plurality of stock images collectively input by the input means are performed in parallel, so that the identification process for each stock image is waiting to be executed (that is, another stock image). It is possible to suppress the situation (waiting for the end of the identification process for the image).

Therefore, in the image identification device, it is possible to eliminate the time waiting for the execution of the identification process for each stock image.

Further, the image identification device of the present invention is
An image acquisition means that continuously repeats an acquisition process for acquiring a plurality of the captured images, and an image acquisition means.
An extraction process is performed in which a defect candidate image showing a defect candidate point as a defect candidate is extracted from a plurality of captured images captured by the image acquisition means, and the defect candidate image is stored as the stock image in the storage means. Equipped with defect candidate extraction means to be executed,
The identification means is configured to identify the correctness of the defect candidate point of the defect candidate image stored in the storage means as the stock image in the identification process.
Before the completion of each of the extraction process, the input process, and the identification process for the captured image captured in the previously executed capture process, after the capture process executed earlier. The import process after execution may be configured to complete execution.

According to the image identification device having the above configuration, the extraction process, the input process, and the identification process for the plurality of captured images captured in the previously executed capture process are performed before the respective processes are completed. Since the capture process executed after the executed capture process is configured to complete the execution, the extraction process and the input process for the plurality of captured images captured in the previously executed capture process. From the time when each process of the identification process is completed to the start of each process of the extraction process, the input process, and the identification process for a plurality of captured images captured by the capture process executed later. You can have some time to spare.

Further, in the image identification device of the present invention,
The identification processing file for executing the identification means may be a processing file different from the processing file for executing other processing.

According to the image identification device having the above configuration, since the identification processing file for executing the identification processing is independent of the processing file for executing other processing, the existing identification processing file is replaced with a new identification processing file. By simply rewriting to, the processing content by the identification means can be changed.

The article manufacturing apparatus of the present invention
An article transport device that transports articles and
An imaging device that captures the article being transported by the article transporting device and creates a plurality of captured images.
With any of the above image identification devices
A marking device installed on the article transport path downstream of the article transport device in the transport direction by the article transport device, depending on the identification result of the stock image by the identification means of the image identification device. A marking device for performing a marking process for marking an article appearing in a stock image obtained from the identification result.
The image identification device is such that after the image pickup device captures an article and creates the captured image, the identification process for the captured image is completed before the marking device executes the mark process on the article. It is composed.

As for the captured image captured by the imaging device, the identification process is completed before the mark processing, which is a subsequent process, is executed for the article captured by the imaging device, so that the captured image (article) without delaying the manufacturing process of the article. ) Can be completed.

As described above, according to the image identification device of the present invention and the article manufacturing apparatus provided with the image identification device, it is possible to obtain an excellent effect that the identification process for a large number of captured images can be speeded up.

FIG. 1 is a schematic view of an article manufacturing apparatus according to an embodiment of the present invention. FIG. 2 is an explanatory diagram of an image pickup apparatus according to the same embodiment. FIG. 3 is a block diagram of the image identification device according to the same embodiment. FIG. 4 is an operation explanatory view of the article manufacturing apparatus according to the same embodiment. FIG. 5 is an operation explanatory view of the image identification device according to the same embodiment.

Hereinafter, the image identification device according to the embodiment of the present invention will be described with reference to the accompanying drawings.

The image identification device according to the present embodiment is configured to identify the type of captured image of an article or the like, and is used, for example, for inspection of a manufactured article. In this embodiment, the article manufacturing apparatus incorporating the image identification apparatus will be described.

The article manufacturing apparatus according to the present embodiment is an apparatus for manufacturing a long sheet-shaped article (hereinafter, referred to as a sheet) such as a polarizing film.

As shown in FIG. 1, the article manufacturing apparatus 1 captures an image of the article transporting device 2 for transporting the article and the article being transported by the article transporting device 2, and captures an image (reference numeral “P1” in FIG. 2). From the image pickup device 3 that creates a plurality of attached images), the image identification device 4 that identifies the type of the captured image created by the image pickup device 3, and the article transport device 2 on the article transport path by the article transport device 2. Is also a marking device 5 installed on the downstream side in the transport direction, and a mark process for marking an article appearing in the stock image obtained with the identification result according to the identification result of the stock image by the identification means of the image identification device 4. The marking device 5 for executing the above is provided.

The article transport device 2 is configured to perform roll transport in which the sheet is transported while being fed from the original fabric.

Further, the article transport device 2 continues to transport the sheet while the image pickup device 3 captures the article and creates the captured image, the image identification device 4 identifies the captured image, and the marking device 5 performs the mark processing. It is configured.

Therefore, the article manufacturing apparatus 1 according to the present embodiment captures a sheet in a transported state (a state in which the sheet is fed from the upstream side to the downstream side in the transport direction) with the imaging device 3 to create an captured image, and also The marking device 5 is configured to perform marking processing on the sheet in the transported state.

Further, with respect to the captured image created by the image pickup device 3, the identification by the image identification device 4 is completed after the captured image is created and before the mark processing is executed on the portion of the sheet that appears in the captured image. It has become.

As shown in FIG. 2, the image pickup apparatus 3 has an image pickup unit 30 that continuously performs an operation (hereinafter, referred to as an image pickup operation) of taking an image of an article (sheet) and creating an image capture image.

Further, in the imaging device 3, a plurality of imaging units 30 are installed side by side in a row in the width direction of the sheet (direction orthogonal to the feeding direction of the sheet). The plurality of imaging units 30 are configured to continuously perform imaging operations at the same or substantially the same timing.

Therefore, the imaging device 3 is configured to continuously output a plurality of captured images created by imaging the sheet at the same or substantially the same timing.

As shown in FIG. 3, the image identification device 4 has a defect in the image capturing means 40 that continuously and repeatedly executes the capturing process for capturing the captured image, and a plurality of captured images captured by the image capturing means 40. The defect candidate extraction means 41 that executes the extraction process for extracting the defect candidate image (the image with the reference numeral "P2" in FIG. 2) in which the candidate is captured, and the defect candidate image extracted by the defect candidate extraction means 41 are stocked. A storage means 42 that executes a storage process for storing a plurality of images as an image, an identification means 43 that executes an identification process for identifying the type of stock image, and a plurality of stock images stored in the storage means 42 are collectively stored in the identification means 43. An input means 44 for executing an input process for inputting an image, and a buffer means 45 for executing a storage process for temporarily storing instruction information for executing each of the extraction process, the input process, and the identification process. And have. The identification device may be a device including a CPU, a memory, a hard disk, and the like.

The image capturing means 40 is configured to capture a plurality of captured images (a plurality of captured images created by the imaging unit 30 at the same or substantially the same timing) output from the imaging device 3. The image capturing means 40 may be configured to directly capture the captured image from the imaging device 3, or may be configured to indirectly capture the captured image from the imaging device 3 (that is,. It may be configured to capture the captured image output from the imaging device 3 via another device).

The defect candidate extraction means 41 is configured to create (extract) a defect candidate image in which defect candidate points that may be defects are captured from the captured images captured by the image acquisition means 40.

Specifically, the defect candidate extraction means 41 includes a detection process for detecting a defect candidate point from the captured image, a pixel in which the defect candidate point detected in the detection process is captured, and a predetermined number of pixels around the pixel. It is configured to execute a candidate image creation process for extracting a region including the above and creating a defect candidate image, and to store the created defect candidate image as a stock image in the storage means 42.

In order to detect defect candidate points in the captured image by the detection process, for example, binarization processing and particle analysis may be performed.

In the storage means 42, a plurality of defect candidate images extracted from the captured images captured by the defect candidate extraction means 41 at the same time are collectively stored (stored) as stock images. When only one defect candidate image is extracted from the defect candidate extraction means 41, only this one defect candidate image may be stored in the storage means 42.

In the input process of the input means 44, the defect candidate images stored as stock images in the storage means 42, more specifically, a plurality of defect candidate images created by the imaging unit 30 at the same or substantially the same timing are collectively collected. It is configured to be input to the identification means 43.

The identification means 43 executes identification processing for each of the receiving means 430 that receives the plurality of defect candidate images collectively input by the input means 44 and the plurality of defect candidate images received by the receiving means 430 in parallel. It has identification execution means 431 and.

The identification execution means 431 is configured to identify the type of the defect candidate image by the identification process for the defect candidate image. The identification execution means 431 according to the present embodiment is configured to identify whether the defect candidate points appear in the defect candidate image are correct or incorrect, that is, whether the defect candidate points are defects or normal parts. As described above, the identification executing means 431 is configured to identify whether the defect candidate image is an image in which defects are reflected or an image of a normal portion having no defects.

The identification executing means 431 has a configuration corresponding to a so-called discriminator. Further, the identification execution means 431 according to the present embodiment is an identification processing file obtained by converting a classifier created (learned) by a device different from the image identification device 4 into a processing file (dynamic link library in the present embodiment). It is configured. Further, the identification processing file for executing the identification execution means 431 executes other processing (for example, processing performed by the image acquisition means 40, the defect candidate extraction means 41, the storage means 42, the input means 44, etc.). It is preferable that the data is different from the processing file for the purpose. In this way, since the identification processing file is independent of the processing file for executing other processing, the processing content of the identification means 43 can be obtained simply by rewriting the existing identification processing file with a new identification processing file. Can be modified.

The identification means 43 may be configured to execute all of the identification processes of the plurality of defect candidate images received by the receiving means 430 in parallel, or identify each of the plurality of defect candidate images. It may be configured to execute a predetermined number of processes in parallel.

Further, the identification means 43 may be configured to execute each identification process of a plurality of defect candidate images in parallel in the identification execution means 431, or one defect candidate image in the identification execution means 431. It may be configured to execute a plurality of tasks for identifying the above in parallel.

As described above, the identification means 43 is configured to identify a plurality of (two or more) defect candidate images in parallel.

The instruction information written in the buffer means 45 includes a plurality of captured images created by the imaging device 3 at the same or substantially the same timing, and an extraction process, an input process, and an identification process related to the plurality of captured images. The instruction is included.

The image identification device 4 of the present embodiment includes two buffer means 45, and while the extraction process, the input process, and the identification process are being executed based on the instruction information written in the one buffer means 45. In the other buffer means 45, instruction information different from the instruction information written in the one buffer means 45 is written.

Then, when the extraction process, the input process, and the identification process based on the instruction information written in the one buffer means 45 are completed, the extraction process and the input process are performed based on the instruction information written in the other buffer means 45. , The identification process is executed in order.

Then, when the extraction process, the input process, and the identification process based on the instruction information written in the other buffer means 45 are completed, yet another instruction information is newly written in the one buffer means 45.

As described above, in each buffer means 45, the reading of the written instruction information (that is, the execution of each process based on the instruction information) and the writing of the instruction information are alternately executed at different timings.

Further, the one buffer means 45 and the other buffer means 45 are instructed to include an image captured by the image pickup operation and a processing command related to the image capture image each time the image pickup operation is performed by the image pickup device 3. Information is written alternately.

Since the identification means 43 of the image identification device 4 outputs as the identification result of the defect candidate image, information indicating whether the defect candidate image is an image in which the defect is reflected (defective image) is output as the identification result. The marking device 5 is configured to give a mark indicating that a defect exists in the defective article among the articles transported by the article transporting device 2 in the marking process.

The article according to the present embodiment is a long sheet, but the marking device 5 is defective when, for example, information indicating that the defect candidate image is a defect image is output as an identification result from the identification means 43. It suffices that the position where the captured image, which is the source of extracting the candidate image, is captured is marked. Further, when the identification means 43 outputs information indicating that the defect candidate image is a normal image as an identification result, the article may be passed without being marked.

For the mark attached to the article by the marking device 5, for example, a sticker, printing, engraving, or the like may be adopted.

The configuration of the article manufacturing apparatus 1 according to the present embodiment is as described above. Subsequently, the operation of the article manufacturing apparatus 1 according to the present embodiment will be described.

In the article manufacturing apparatus 1, as shown in FIG. 4, a sheet-shaped article is constantly conveyed by the article conveying device 2 (S1), and the imaging device 3 continuously executes an imaging operation on the conveyed article. (S3, S8, S12).

In the article manufacturing apparatus 1 according to the present embodiment, as described above, while the process based on the instruction information written in the one buffer means 45 is being executed (S2, S10), the other buffer means 45 The instruction information is written in (S5, S13), and the instruction information is written in the one buffer means 45 while the process based on the instruction information written in the other buffer means 45 is being executed (S6). (S9).

As described above, the image identification device 4 prepares to execute the processing for the captured image created by the imaging device 3 in the next imaging operation while the imaging device 3 is executing the processing for the captured image created in the imaging operation. To complete. Therefore, the image identification device 4 can start the next new process immediately after the process being executed is completed, whereby the process after the identification process (in the present embodiment, the mark process by the marking device 5 (in the present embodiment) It is also possible to start the next identification process during the execution of S3, S7)) (S6, S10).

More specifically, the processing of the image identification device 4 will be described more specifically. As shown in FIG. 5, the image pickup device 3 outputs a plurality of captured images created in the image pickup operation (S14) toward the image identification device 4 (S15). ).

The plurality of captured images output to the image identification device 4 are captured by the image capturing means 40 (S16), included in the command information, and written to the one buffer means 45 (S17).

Then, various processes related to the captured image are started based on the instruction information written in the one buffer means 45. Specifically, first, the defect candidate extraction means 41 extracts a defect candidate image in which the defect candidate appears from the plurality of captured images (S18), and stores the defect candidate image as a stock image in the storage means 42. (S19).

A plurality of defect candidate images stored in the storage means 42 as stock images are collectively input to the identification means 43 by an input process (S20). Subsequently, the identification process for each of the plurality of defect candidate images is executed in parallel by the identification means 43 (S21).

Then, when the identification result is output from the identification means 43, the marking device 5 executes the marking process according to the identification result (see FIG. 4). As a result, a series of processes for the article in the article manufacturing apparatus 1 is completed.

While executing the process based on the instruction information written in the one buffer means 45, the image pickup apparatus 3 is executing the next image pickup operation (S22), and a plurality of images created in the next image pickup operation. The captured image is output to the image identification device 4 (S23) before the identification process (S21) for the captured image created in the previous imaging operation (S14) is completed, and is captured by the image capturing means 40 (S24). Further, it is written in the other buffer means 45 in a state of being included in the instruction information (S25).

Therefore, before the mark processing for the article shown in the captured image created in the previous imaging operation is completed, various processes related to the captured image created in the next (later) imaging operation, specifically, the defect candidate extraction means Extraction of defect candidate images by 41 (S26), storage of the defect candidate images in the storage means 42 (S27), batch input of a plurality of defect candidate images to the identification means 43 by input processing (S28), by the identification means 43. The parallel execution (S29) of the identification processing for each of the plurality of defect candidate images is executed in order.

Further, since new instruction information is written to the one buffer means 45 during execution of the process based on the instruction information written in the other buffer means 45 (S30), the other buffer means 45 is further executed. Immediately after the process based on the instruction information written in 45 is completed, the process based on the new instruction information is started.

As described above, according to the image identification device 4 according to the present embodiment, the identification means 43 performs a plurality of identification processes for each of the plurality of stock images collectively input by the input means 44 in parallel. , It is possible to prevent the identification process for each stock image from being waiting for execution (that is, waiting for the end of the identification process for another stock image).

Therefore, in the image identification device 4, it is possible to eliminate the time waiting for the execution of the identification process for each stock image, thereby speeding up the identification process for a large number of captured images, and the image identification device. The same effect can be obtained in the article manufacturing apparatus 1 provided with 4.

Further, the image identification device 4 is executed first before each of the extraction process, the input process, and the identification process for the plurality of captured images captured in the previously executed capture process is completed. Since the capture process and the storage process following (after) the capture process are configured to be executed, the extraction process and the storage process for a plurality of captured images captured in the previously executed capture process are performed. , Identification processing, and input processing are completed, and then each processing of extraction processing, storage processing, identification processing, and input processing for a plurality of captured images captured in the later execution processing is executed in order. You can give yourself plenty of time to get started.

Further, after the captured image is created by the imaging operation of the imaging device 3, the article (the portion of the sheet that appears in the captured image) imaged by the imaging device 3 is before the marking process by the marking device 5 is executed. Since each process (extraction process, storage process, identification process, input process) of the image identification device 4 for the captured image is completed, the image captured by the image identification device 4 is displayed in real time, that is, in the flow of the manufacturing process of the article. Identification (confirmation of the presence or absence of defects in this embodiment) can be performed.

Further, in the article manufacturing apparatus 1 according to the present embodiment, with respect to the captured image captured by the image pickup device 3, the identification process is completed before the mark process, which is a subsequent step, is executed for the article captured by the image pickup device 3. Therefore, the identification of the captured image (article) can be completed without delaying the manufacturing process of the article, and the identification of the captured image is executed and completed in-line (in the flow of the manufacturing process of the article). You can also.

The image identification device and the article manufacturing device of the present invention are not limited to the above-described embodiment, and it goes without saying that various modifications can be made without departing from the gist of the present invention.

The article manufacturing apparatus 1 of the above embodiment is for manufacturing a sheet, but is not limited to this configuration. For example, the article manufacturing apparatus 1 may manufacture a single-wafer-shaped article or the like.

In the article manufacturing apparatus 1 of the above embodiment, a plurality of image pickup units 30 are arranged so as to be arranged in a row in the width direction of the sheet. It may be changed as appropriate according to the article to be used.

Although not particularly mentioned in the above embodiment, the image capturing means 40 captures each of a plurality of captured images (a plurality of captured images created at the same time) output from the imaging device 3 as separate images. Alternatively, one image obtained by synthesizing the plurality of captured images may be captured.

In the above embodiment, the defect candidate extraction means 41 is configured to use a region including a pixel in which a defect candidate point detected by the detection process is captured and a predetermined number of pixels around the pixel as a defect candidate image. However, it is not limited to this configuration. For example, the defect candidate extraction means 41 may use the image itself in which the defect candidate points are captured as the defect candidate image.

In the above embodiment, the image identification device 4 includes two buffer means 45, but is not limited to this configuration. For example, the image identification device 4 may include one buffer means 45, or may include three or more buffer means 45.

1 ... Article manufacturing device, 2 ... Article transport device, 3 ... Imaging device, 4 ... Image identification device, 5 ... Marking device, 30 ... Imaging unit, 40 ... Image capturing means, 41 ... Defect candidate extraction means, 42 ... Storage Means, 43 ... Identification means, 44 ... Input means, 45 ... Buffer means, 430 ... Receiving means, 431 ... Identification executing means

Claims (3)

An image capturing means that continuously and repeatedly executes a capturing process for capturing a plurality of captured images captured by the imaging device at the same or substantially the same timing.
An extraction process is executed in which a defect candidate image showing a defect candidate point as a defect candidate is stored in the storage means as a stock image from a plurality of captured images captured at the same or substantially the same timing by the image acquisition means. Defect candidate extraction means and
An identification means for executing an identification process for identifying the correctness of the defect candidate point of the stock image, and
An input means for executing an input process of reading out a plurality of the stock images with the same or substantially the same timing captured by the image capturing means from the storage means and collectively inputting them to the identification means.
A storage process that temporarily stores a plurality of captured images captured by the imaging device at the same or substantially the same timing in a state in which command information for executing the extraction process, the input process, and the identification process is associated with each other. With multiple buffering means to execute ,
While the extraction process, the input process, and the identification process based on the instruction information stored in the buffer means are being executed, the import process and the storage process by another buffer means are executed. Configured to complete ,
Image identification device. The identification processing file for executing the identification means is a processing file different from the processing file for executing other processing.
The image identification device according to claim 1. An article transport device that transports articles and
An imaging device that captures the article being transported by the article transporting device and creates a plurality of captured images.
The image identification device according to claim 1 or 2,
A marking device installed on the article transport path downstream of the article transport device in the transport direction by the article transport device, depending on the identification result of the stock image by the identification means of the image identification device. A marking device for performing a marking process for marking an article appearing in a stock image obtained from the identification result.
The image identification device is such that after the image pickup device captures an article and creates the captured image, the identification process for the captured image is completed before the marking device executes the mark process on the article. Composed,
Goods manufacturing equipment.

Images