JP2014209310A - Labeling method, labeling device and defect inspection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accelerate labeling processing.SOLUTION: A labeling method includes: an initial label setting process for setting mutually different labels to respective runs; an update process for defining the run as a processing object run for each run, determining whether or not a first connection run connecting to the processing object run exists, and performing individual update processing in which the label of one of the processing object run and the first connection run is updated to the label of the other when the first connection run exists; and a search process for searching for the mutually-connected runs among the plurality of runs on the basis of the plurality of labels after the update process to apply the same label. In the update process, the plurality of runs are divided into a plurality of sets, the individual update processing executed in each set is defined as an update thread, and the plurality of update threads are executed in parallel while permitting rewriting of the label by one update thread among the plurality of update threads and inhibiting rewriting of the label rewritten by the one update thread by other update threads on the basis of exclusion control.

Description

  The present invention relates to a labeling technique for assigning labels to a plurality of runs created by converting binary image data into run lengths, and a defect inspection apparatus to which the labeling technique is applied.

  In the field of manufacturing technology such as semiconductor substrates and printed circuit boards, in order to detect defects contained in products and analyze / evaluate them, the evaluation object is imaged through a microscope or the like, and defective portions are included from the obtained images. An image, a so-called defect image, is extracted. Based on the binary image data of the defect image, the number of defective portions, the area, the center of gravity, and the like are measured. At this time, in order to automatically perform the measurement, labeling (connected region extraction) processing is widely used. For example, in Patent Document 1, the labeling process is speeded up by executing a two-pass labeling process on a run generated by converting the binary image data into run lengths for each row.

JP 2008-186123 A

  However, the above-described invention described in Patent Document 1 (hereinafter referred to as “conventional invention”) is premised on the investigation starting from the higher rank run data, and this is one of the factors that hinder further speeding up. In the conventional invention, a label table that records whether or not run data given different labels belong to the same connected component, and a run number table that associates the number of run data corresponding to the label for each label. Must be provided separately. For this reason, the number of elements in the label table and the run number table increases in accordance with the increase in the number of runs and the complexity of the connected state, which is another factor that hinders the speeding up of the labeling process.

  The present invention has been made in view of the above problems, and an object thereof is to provide a technique capable of speeding up the labeling processing of binary image data and a technique for efficiently performing defect inspection using the technique. To do.

  A labeling method according to the present invention is a labeling method in which a labeling process is performed on a plurality of runs created by run-lengthing binary image data, and different initial labels are set for each run. For each step, it is determined whether or not there is a first linked run that is to be treated as a run to be processed and connected to the run to be processed, and when there is a first linked run, An update process for performing an individual update process for updating one label of the first linked runs to the other label, and after the update process, a search is made for runs connected to each other among a plurality of runs based on a plurality of labels. And a search process for assigning a label, and in the update process, a plurality of runs are divided into a plurality of sets and an individual update process executed in each set is set as an update thread, Under a different control, allow rewriting of a label by one update thread among a plurality of update threads, and prohibit a rewrite by another update thread of a label that is rewritten by one update thread in parallel with a plurality of update threads. It is characterized by executing.

  The labeling device according to the present invention is a labeling device that performs a labeling process on a plurality of runs created by converting the binary image data into run lengths, a storage unit that stores the labels of each run, and an arithmetic operation A processor unit having a plurality of processor cores that perform processing, and a first connection that, after initializing different values for a plurality of labels, sets each run as a processing target run and connects to the processing target run It is determined whether or not a run exists, and when a first linked run exists, an individual update process is performed to update one label of the processing target run and the first linked run to the other label, and A control unit that searches for a run connected to each other among a plurality of runs based on the label and assigns the same label, and the plurality of runs corresponds to the number of processor cores. And each processor core executes individual update processing as an update thread for one or more of the divided runs, and the control unit uses one update thread among the plurality of update threads under exclusive control. It is characterized by performing individual update processing for each run by allowing multiple update threads to run in parallel while permitting rewriting of labels and prohibiting rewriting by other update threads of labels that are rewritten by one update thread Yes.

  Furthermore, the defect inspection apparatus according to the present invention includes an image acquisition unit that acquires an inspection target image, an image extraction unit that inspects the inspection target image and extracts a defect image including a defective portion, and binarizes the defect image. A binarization processing unit that generates binary image data by processing, a run generation unit that generates a plurality of runs by run lengthing the binary image data, and has the same configuration as the labeling device, And labeling means for assigning the same label to runs connected to each other.

  In the invention thus configured, labels having different values are initialized for each run. Then, for each run, an individual update process is executed, and a label update is executed depending on whether or not there is a linked run connected to the process target run. Thereafter, based on the plurality of labels, a search is made for the runs that are connected to each other among the plurality of runs, and the same label is assigned thereto. In this way, the two-pass labeling process of the update process and the search process is common to the conventional invention. However, in the present invention, the binary image data is labeled using the same number of labels as the number of runs. Thus, the label table and the run number table used in the conventional invention are unnecessary, and the labeling process can be speeded up.

  Moreover, in the update process of the present invention, a plurality of runs are divided into a plurality of sets, and individual update processing executed in each set is set as an update thread, and these update threads are executed in parallel. That is, the individual update process for each run is executed in parallel. Therefore, although the label update is exclusively controlled between the update threads, the total time required for the labeling process can be greatly reduced as compared with the conventional invention in which labels are sequentially assigned from the upper run.

Here, in the update process, rewriting by other update threads of the label that is rewritten by one update thread is prohibited, but rewriting by other update threads is permitted for labels other than the label, that is, labels that do not conflict. May be. As a result, the processing efficiency can be increased and the time required for the labeling process can be further shortened.
In the individual update process, when the label of the first linked run is updated, it is determined whether there is a second linked run other than the processing target run connected to the first linked run, and the second linked run exists. When doing so, you may comprise so that one label of a 1st connection run and a 2nd connection run may be updated to the other label. Thus, not only the first linked run but also the second linked run connected to the processing target run via the first linked run can be updated after the individual update process for the processing target run by updating the label. The time required for the individual update process for the second linked run executed in step S1 can be reduced. As a result, the total processing time can be shortened.

  In addition, a run having the same label as the pre-update label before updating the label of the first connected run may be used as the second connected run, and the second connected run can be easily and reliably made using the pre-update label. It can be found and is preferred.

  In addition, the binary image data may be run-length for each row, and in this case, the run connected to the lower row of the processing target run can be determined as the first connected run.

  Moreover, you may perform an individual search process for every run in the search process (The process which searches the run connected mutually among several runs and provides the same label) among 2 paths. This “individual search process” is a process in which one run is set as a process target run and a linked run connected to the process target run is found based on a plurality of labels, and the process target run and the label of the linked run are rewritten to the same Means. Thus, a search process can be rationally performed by the individual search process for each run. In addition, as in the individual update process for each run described above, a plurality of runs are divided into a plurality of sets, and the individual search process executed in each set is set as a search thread, and one of the plurality of search threads is controlled under exclusive control. A plurality of search threads may be executed in parallel while permitting rewriting of labels by the search thread and prohibiting rewriting of labels rewritten by one search thread by other search threads. Thereby, it is possible to further increase the speed of the labeling process by increasing the processing speed in the search process.

  According to the present invention, part of the process for executing the labeling process can be executed in parallel, and the total time required for the labeling process of the binary image data can be shortened.

It is a figure which shows schematic structure of the inspection system using the defect inspection apparatus equipped with one Embodiment of the labeling apparatus concerning this invention. It is a block diagram which shows schematic structure of an image process part. It is a block diagram which shows schematic structure of the labeling part corresponded to one Embodiment of the labeling apparatus concerning this invention. It is a flowchart which shows the labeling operation | movement by a labeling part. It is a figure which shows an example of the binary image data given to a labeling part. It is a figure which shows the initial value of the run connection data which show the run data of a run obtained by performing a run-length-ized process with respect to the binary image data of FIG. 5, and the connection relationship between runs. It is a flowchart which shows the individual update process performed with each processor core. It is a schematic diagram which shows an example of the individual update process performed with each processor core. It is a flowchart which shows the individual search process performed with each processor core. It is a schematic diagram which shows an example of the individual search process performed with each processor core.

  FIG. 1 is a diagram showing a schematic configuration of an inspection system using a defect inspection apparatus equipped with an embodiment of a labeling apparatus according to the present invention. The inspection system 1 is an inspection system for inspecting defects such as pinholes and foreign matters appearing on the appearance of a semiconductor substrate (hereinafter referred to as “substrate”) S to be inspected. The inspection system 1 includes an imaging device 2 that images an inspection target area on the substrate S, and a control device 3 that performs defect inspection based on image data from the imaging device 2.

  When a defect is found in the substrate S in the substrate detection apparatus M provided in the production line of the substrate S separately from the present system, the position coordinates of the defect are given to the inspection system 1. The substrate detection apparatus M incorporated in the production line inspects the entire substrate S by a predetermined processing algorithm, and acquires and outputs the position coordinates if there is a region on the substrate surface that satisfies the requirement as a defect. Therefore, the imaging unit included in the substrate detection apparatus M has a relatively low resolution, and the processing algorithm is fixed.

  On the other hand, the inspection system 1 is connected to the substrate detection apparatus M through an interface (not shown), and images an area in which position coordinates are reported as a defect from the substrate detection apparatus M by the imaging apparatus 2 having a higher resolution. At the same time, the control device 3 examines the image to determine in detail the presence or absence and type of the defect, and displays the image of the defective portion on the display unit.

  The imaging device 2 moves the stage 22 relative to the imaging unit 21 that acquires image data by imaging the inspection target region on the substrate S, the stage 22 that holds the substrate S, and the imaging unit 21. A stage driving unit 23 is provided. In addition, the imaging unit 21 includes an illumination unit 211 that emits illumination light, an optical system 212 that guides illumination light to the substrate S and receives light from the substrate S, and an image of the substrate S formed by the optical system 212. An imaging device 213 for converting the signal into an electrical signal. The stage drive unit 23 includes a ball screw, a guide rail, and a motor. The device control unit 4 provided in the control device 3 controls the stage drive unit 23 and the imaging unit 21, so that the inspection target region on the substrate S is changed. Imaged.

  The control device 3 has a device control unit 4, and when the device control unit 4 executes a control program read in advance, each unit of the control device shown in FIG. 1 is operated as follows. The control device 3 includes an image acquisition unit 5 and an image processing unit 6 in addition to the device control unit 4 described above. The image acquisition unit 5 converts the electrical signal output from the imaging unit 21 into data, and acquires image data corresponding to the captured image. The image processing unit 6 performs appropriate image processing on the image data acquired by the image acquisition unit 5 to detect a defect included in the image and create an image of a defective portion (hereinafter referred to as “defect image”). The image processing unit 6 includes a labeling unit that is an embodiment of a labeling apparatus according to the present invention, and can perform a labeling process on a defective image at high speed. The configuration and operation of the image processing unit 6, particularly the labeling unit, will be described in detail later.

  Further, the control device 3 displays a storage unit 7 for storing various data, an input receiving unit 8 such as a keyboard and a mouse for receiving an operation input from the user, and a visual information for the user such as an operation procedure and a processing result. Part 9 and the like. Although not shown in the figure, a reading device for reading information from a computer-readable recording medium such as an optical disk, a magnetic disk, a magneto-optical disk, etc. Are suitably connected via an interface (I / F) or the like.

  FIG. 2 is a block diagram illustrating a schematic configuration of the image processing unit. The image processing unit 6 includes a filtering unit 61, a difference extraction unit 62, a binarization processing unit 63, a labeling unit 64, and an isolated removal processing unit 65. A captured image is sent from the image acquisition unit 5 to the filtering unit 61, and a reference image is sent from the storage unit 7. Of these two images, the picked-up image is an image of the substrate S picked up by the image pickup device 2 and corresponds to an inspection target image to be subjected to defect detection inspection. Further, the reference image is an image corresponding to an ideal substrate having no defect, and in this embodiment, as will be described below, defect detection is performed from the inspection target image by comparing the inspection target image with the reference image. . These defect images and reference images are stored in the storage unit 7 and referred to as necessary. However, image data stored in an external storage medium may be read as necessary.

  The filtering unit 61 performs a filtering process for removing minor image differences unrelated to image noise and defects for each of the inspection target image and the reference image, and sends each image to the difference extraction unit 62. The difference extraction unit 62 corresponds to an example of the “image extraction unit” of the present invention, and extracts the regions having different image contents by obtaining the difference between the inspection target image and the reference image after the filtering process, The difference image is sent to the binarization processing unit 63. Then, the binarization processing unit 63 binarizes the difference image with an appropriate threshold value, generates binary image data BFI, and sends it to the labeling unit 64. The labeling unit 64 generates a plurality of runs by converting the binary image data BFI into run lengths and assigns labels to the plurality of runs. Further, the isolated removal processing unit 65 removes remaining isolated points from the labeled binary image. Thereby, the required defect position image is created and the image processing unit 6 is output.

  FIG. 3 is a block diagram showing a schematic configuration of a labeling unit corresponding to an embodiment of a labeling apparatus according to the present invention. The labeling unit 64 includes a GPU (Graphics Processing Unit) 642 having a plurality of processor cores 641. In this GPU 642, each processor core 641 functions as the individual update processing unit 643 and the individual search processing unit 644, and executes the individual update processing and individual search processing for each run as one thread. In this embodiment, the individual update process for each run corresponds to an example of the “update thread” of the present invention, and the individual search process for each run corresponds to an example of the “search thread” of the present invention. The individual update process and the individual search process will be described in detail later.

  The labeling unit 64 includes a labeling storage unit 645 for storing binary image data BFI, data for specifying the run r [m] (hereinafter referred to as “run data”), and run connection data id [m]. ing. These data will be described in detail later with specific examples.

  Further, the labeling unit 64 is provided with an arithmetic processing unit 646 configured by a CPU (Central Processing Unit), a memory, and the like. In the arithmetic processing unit 646, the CPU performs arithmetic processing according to a predetermined program, thereby executing the run generation unit 647 that generates the run r [m] by converting the binary image data BFI into run length, and is executed by each processor core 641 in the GPU 642. A parallel processing control unit 648 that executes the thread to be executed in parallel with exclusive control, and a data initial setting unit 649 that initializes the label number included in the run r [m] and the run connection data id [m]. Yes. Thus, the arithmetic processing unit 646 functions as a “control unit” of the present invention.

  Next, a labeling operation by the labeling unit 64 configured as described above will be described with reference to FIGS. FIG. 4 is a flowchart showing a labeling operation by the labeling unit. When the binary image data BFI created by the binarization processing unit 63 is given to the labeling unit 64, the arithmetic processing unit 646 of the labeling unit 64 controls each part of the labeling unit 64 to execute a labeling process. First, the binary image data BFI is temporarily registered in the labeling storage unit 645. In addition, a run length process is performed on the binary image data BFI by the run generation unit 647 to generate a plurality of runs r [m] (step S1), which are stored in the labeling storage unit 645. Many known techniques have been proposed for the run-length process, and the general run-length process is used as it is in this embodiment. Therefore, the description of the run length process is omitted.

  FIG. 5 is a diagram showing an example of binary image data given to the labeling unit. FIG. 6 is a diagram showing the run data of the run obtained by executing the run length process on the binary image data of FIG. 5 and the initial value of the run connection data indicating the connection relationship between the runs. The numerical values (0, 1, 2,...) In FIG. 5 and FIG. 6A indicate the coordinate positions in the row direction and the column direction of the pixels arranged in a matrix, that is, the row index and the column index. Show. In FIG. 5, hatched pixels indicate pixels having pixel data “1”, while pixels not hatched are pixels having pixel data “0”. It is shown that. When run length conversion processing for each row is applied to the binary image data BFI having such pixel data, the run shown in FIG. 6A is obtained. Specifically, since the pixel “1” does not exist in the 0th and 4th rows, no run is created, and in the 1st to 3rd rows, 2, 2 and 1 respectively. Runs are created. In this specification, the run number of each run created by the run length process is represented by “m”, and the values of 0, 1, 2,... Are taken in the run creation order. [M].

  Further, the run data of the runs r [0] to r [4] created by the run-length processing of the binary image data BFI of FIG. 5 are stored in the run data storage area of the labeling storage unit 645 in the order of run numbers. The state is illustrated in FIG. In FIG. 7B, “label number” indicates a label assigned to the run, “start pixel position” indicates a column index indicating the start pixel position of the run in the binary image data BFI, and “end” “Pixel position” indicates a column index indicating the end pixel position of the run in the binary image data BFI, and “Row index” indicates a row index of the run in the binary image data BFI. Among these, the “label number” is a value indicating a connection relationship between the run and a run other than the run, and the update process (step S3) and the search process (step S4) described later are executed. In the binary image data BFI shown in FIG. 5, the same values are assigned to the runs connected to each other. In the present embodiment, in order to efficiently execute the search process (step S4), a run connection table indicating a connection relationship between runs is provided. In this run connection table, run connection data id [m] is provided as data indicating the connection status of the run r [m], and the connection status between the run r [m] and other runs is indicated by the label number. For example, it is stored in the run connection data storage area of the labeling storage unit 645 in a table format as shown in FIG.

  Returning to FIG. 4, the description of the labeling process will be continued. In this embodiment, when a run is created from the binary image data BFI, a different label number is set for each run as a label indicating the connection relationship between the run and a run other than the run (step S2). In the present embodiment, the label number of the run r [m] and the label number of the run connection data id [m] are matched with the run number m. That is, the label numbers of the runs r [0], r [1],... Are initialized to “0”, “1”,... And the labels of the run connection data id [0], id [1],. The numbers are also initially set to “0”, “1”,.

  In the next step S3, update processing (step S3) is executed. In this update process, an individual update process is executed for each run. This “individual update process” is to determine whether or not there is a connected run that has one run as the processing target run and is connected vertically and diagonally in the lower row of the processing target run. In the present embodiment, individual update processing is performed for each of a plurality of (for example, n) runs. Run. Here, it is possible to perform these n individual update processes serially, for example, in the order of run numbers, but in this case, a great deal of time is spent on the update process.

  Therefore, in this embodiment, the individual update process for each run is set as one thread, and the threads are executed in parallel by the processor cores 641 in the GPU 642. That is, as shown in FIG. 3, the 0th processor core 641 executes the individual update process with the run r [0] as the processing target run, and the first processor core 641 processes the run r [1] in parallel with this. The individual update process is executed as the target run. The other runs r [2], r [3],..., R [n] are also executed in parallel with the individual update process. However, since each individual update process involves an update of the label number in the run connection data as will be described later, the parallel processing control unit 648 performs exclusive control between the processor cores 641 in this embodiment.

  FIG. 7 is a flowchart showing an individual update process executed by each processor core. FIG. 8 is a schematic diagram showing an example of an individual update process executed by each processor core. Here, first, the basic operation of the individual update process for the run r [p] of the run number p (p = 0, 1, 2,... Or n) will be described with reference to FIG. 7, and then the specific example of FIG. An example of updating the label number will be described with reference to FIG.

  The individual update process for the run r [p] is executed by the p-th processor core 641 in the following procedure. In step S11, the run r [p] is set as the processing target run, and the run connected to the lower row of the processing target run r [p] is extracted as the first connected run. If the first connected run is not extracted (in the case of “NO” in step S12), the individual update process in the thread is terminated.

  On the other hand, when the first connected run is extracted (in the case of “YES” in step S12), the following steps S13 to S21 are executed to update the label number. Here, the description will be continued assuming that the run r [q] located in the lower row of the run r [p] is extracted as the first connected run.

  In the next step S13, the run connection data id [p] and id [q] respectively corresponding to the runs r [p] and r [q] are read from the labeling storage unit 645. Here, when the run connection data id [p] and id [q] have the same value, that is, the same label number is set and the connection relationship has already been set (in the case of “YES” in step S14). Ends the individual update process.

  On the other hand, if “NO” is determined in step S14, that is, if the run connection data id [p] and id [q] are different (different label numbers are set), update of the run connection data is performed. Is done. As described above, the fact that the run connection data id [p] and id [q] are different from each other means that no connection relationship is established between the run [p] and the run [q]. However, since the connection relationship is actually established, one label number is updated to the other label number according to the size relationship between the run connection data id [p] and id [q] (steps S15 to S19). .

  When the run connection data id [p] and id [q] are compared in step S15, and it is found from the comparison result that the run connection data id [p] has a larger label number than the run connection data id [q]. , The run connection data id [p] is rewritten with the run connection data id [q], and the run connection data id [p] is updated to a smaller label number (step S16). In addition, the label number of the run r [p] on the side to be changed in this way is registered as a value (old_dum), and the run connection data id [p] before the change is registered as a value (old_id). The run connection data id [p] after being registered is registered as a value (new_id) (step S17).

  Conversely, when it is found that the run connection data id [q] has a larger label number than the run connection data id [p], the run connection data id [q] is rewritten to the run connection data id [p]. The run connection data id [q] is updated to a smaller label number (step S18). Further, the label number of the run r [q] on the side to be changed in this way is registered as a value (old_dum), the run connection data id [q] before the change is registered as a value (old_id), and the change is made The run connection data id [q] after being registered is registered as a value (new_id) (step S19).

  When the value (old_dum) and the value (old_id) match (in the case of “YES” in step S20), the individual update process ends. On the other hand, if the two do not match (in the case of “NO” in step S10), the run whose label number has been updated (hereinafter referred to as “update run”) is a run other than the processing target run and the first linked run ( (Hereinafter referred to as “second connected run”). Therefore, when such a second linked run exists, step S21 is executed, and then the process returns to step S13. That is, in step S21, the run number p is rewritten to a value (old_id) and the run number q is rewritten to a value (new_id). As a result, the updated run is set as a new run r [p] and the second linked run is set as a new run r [q], and the process returns to step S13, and the label number is set between the updated run and the second linked run. (Step S14). If the label numbers are different, the larger label number is updated to the smaller label number (steps S15 to S19).

  As described above, in the individual update process, the label number is updated not only for the directly connected run directly connected to the processing target run but also for the indirectly connected run indirectly connected to the processing target run through the directly connected run. ing.

  Next, the update process (step S3) will be described with reference to an example (FIG. 8) in which the update process is executed for the runs r [0] to r [4] shown in FIGS. The contents will be described in more detail. The vertical axis in FIG. 8 represents the passage of time, and the run connection data id [0] to id [4] are set to initial values, that is, 0 to 4 at the start time. Of the run connection data id [0] to id [4], those with a pear pattern indicate the label number after the update change. Further, items (“connected run: existence”, “rewrite prohibition”, etc.) arranged below “0th thread” to “4th thread” indicate the contents of processing executed by each processor core 641.

Of the n processor cores 641, five of the processor cores 641 execute the individual update processes of the runs r [0] to r [4] in parallel. Of these, the individual update processes of the runs r [3] and r [4] are executed as the third thread and the fourth thread, respectively, but both do not descend and the first linked run does not exist. (Steps S3-3 and S3-4) are terminated, and the individual update process is completed. On the other hand, in the individual update processing of the runs r [0] to r [2], it is confirmed that there are linked runs, and read, contrast, rewrite, and the like of the run connection data corresponding to them are performed. Here, for example, in the second thread (individual update process of run r [2]), run connection data id [2] between run r [2], which is the process target run, and linked run r [4] connected thereto. , Id [4], that is, the label number is read out. here,
id [2] = 2,
id [4] = 4,
It has become. As a result of comparison, since the run connection data id [4] has a larger value than the run connection data id [2], the label number of the run connection data id [4] is changed to the label number of the run connection data id [2]. Need to be rewritten. Thus, when the rewriting preparation in the second thread is first, the parallel connection control unit 648 temporarily disables the data rewriting in the individual update processing of the 0th thread and the 1st thread, and the run connection data id [ 4] is executed, and the label number of the run connection data id [4] is updated from “4” to “2”. Further, the value (old_dum) that is the label number of the updated run r [4] and the value (old_id) that is the run connection data id [4] before the change are both “4”, which is the same value. Thus, the second thread (step S3-2) is terminated.

Next, as shown in the figure, when the first thread is ready for rewriting, the parallel connection control unit 648 temporarily prohibits data rewriting in the individual update process of the 0th thread, and the run connection data id [2] and id [3] are rewritten. More specifically, in the first thread (individual update process of run r [1]), run r [1] that is the process target run and runs r [2] and r [3] are connected in the lower row. These are recognized as the first consolidated run. Then, the run connection data id [1] of the processing target run r [1] and the run connection data id [2] and id [3] of the linked runs r [2] and r [3] are read. They are,
id [1] = 1,
id [2] = 2,
id [3] = 3,
It has become. As a result of comparison, since the run connection data id [2] and id [3] are both larger values than the run connection data id [1], the labels of the run connection data id [2] and id [3] Both numbers are rewritten to the label number of the run connection data id [1]. For the updated run r [2], the value (old_dum) that is the label number and the value (old_id) that is the run connection data id [2] before the change are both “2”. For the updated run r [3], the value (old_dum) that is the label number and the value (old_id) that is the run connection data id [3] before the change are both “3”. As described above, since the value (old_dum) and the value (old_id) are the same for both the runs r [2] and r [3], the first thread (step S3-1) is terminated.

  In the last remaining 0th thread (individual update process of run r [0]), run r [0], which is the process target run, and run r [2] are connected in the lower row, and this is used as a linked run. Similar to the first thread and the second thread, the label number of the run connection data id [2] is rewritten to the label number of the run connection data id [0]. However, the updated run connection data id [2] has already been rewritten once and becomes “1” from the initial value (initial label number = 2 of run r [2]). The value (old_dum) that is the label number and the value (old_id) that is the run connection data id [2] before the change are “2” and “1”, respectively, and are different. Therefore, the read connection data id [0] and id [1] are read, compared, and rewritten as executed in step S21 in FIG. 7, and the run connection data id is shown in the lower part of FIG. The label number of [1] is rewritten to the label number of run connection data id [0]. Thus, the value (old_dum) that is the label number of the updated run r [1] and the value (old_id) that is the run connection data id [1] before the change are both “1” and the same value. Thus, the 0th thread (step S3-0) is terminated. As described above, the run r [1] is not directly connected to the processing target run r [0], but is indirectly connected via the run r [2]. Updates are made.

  When all the threads are finished in this way, the update process (step S3) is completed, and the update process is executed on the runs r [0] to r [4] shown in FIGS. 6A and 6B. The connection data id [0] to id [4] is updated from (0, 1, 2, 3, 4) to (0, 0, 0, 1, 2). As can be seen from the above description, since parallel processing is performed in one thread and one run, the label numbers of the run connection data id [0] to id [4] after the update processing depend on which run is processed first. Can take different values.

  Next, returning to FIG. 4, the description of the labeling process will be continued. When the update process (step S3) is completed, a search process (step S4) is executed. Similarly to the update process, the individual search process for each run is set as one thread, and the thread is executed in parallel by each processor core 641 in the GPU 642 while the rewrite timing is controlled by the parallel processing control unit 648. In other words, as shown in FIG. 3, the 0th processor core 641 executes the individual search process with the run r [0] as the processing target run, and the first processor core 641 processes the run r [1] in parallel therewith. An individual search process is executed as the target run. The other runs r [2], r [3],..., R [n] are also executed in parallel with the individual search process. The “individual search process” refers to one run as a process target run and a connected run connected to the process target run in a data connection table (run connection data id [0], id [1],..., Id [N]), the label numbers of the processing target run and the linked run are rewritten to be the same. Hereinafter, the individual search process will be described in detail with reference to FIGS. 9 and 10.

  FIG. 9 is a flowchart showing an individual search process executed by each processor core. FIG. 10 is a schematic diagram showing an example of an individual search process executed by each processor core. Here, as in the case of the individual update process, first, the basic operation of the individual search process for the run r [p] will be described with reference to FIG. 9, and then the label number will be changed with reference to the specific example of FIG. An example of rewriting will be described.

  The individual search process for the run r [p] is executed by the p-th processor core 641 according to the following procedure. In step S31, the run r [p] is a process target run, and the run connection data id [p] corresponding to the process target run r [p] is read from the run connection table and acquired. Then, it is determined whether or not the run connection data id [p] is equal to the initial value of the label number of the processing target run r [p] (step S32). Here, if it is determined that they are the same, the individual search process is completed as it is. On the other hand, if it is determined that they are different (in the case of “NO” in step S32), steps S33 to S38 are executed to rewrite the label number.

  After registering the run connection data id [p] as a value (new_dum) in step S33, the run connection data id [new_dum] is read from the run connection table (step S34). Then, it is determined whether or not the run connection data id [new_dum] is equal to the value (new_dum) (step S35). While it is determined that they are not equal, the value (new_dum) is changed to the run connection data id [new_dum]. (Step S36), the process returns to step S34, and reading of new run connection data id [new_dum] is repeated.

  Then, when the run connection data id [new_dum] matches the value (new_dum) (when it is determined “YES” in step S35), the label number of the run r [p] of the run number p is set to the value ( new_dum) (step S37). Similarly, the run connection data id [p] is also rewritten to a value (new_dum) (step S38), and the individual search process is terminated. This rewriting is executed in a state in which the rewriting timing is controlled by the parallel processing control unit 648 as in the case of the individual update processing.

  Next, as shown in FIG. 8, the contents of the search process (step S <b> 4) with reference to an example (FIG. 10) when the search process is executed after the run connection table is updated from the initial state by the update process. Will be described in more detail. Here, an individual search process using the run r [4] as a process target run will be described with reference to FIG. The vertical axis in FIG. 10 represents the passage of time, and the initial values of the label numbers of the runs r [0] to r [4] are set to 0 to 4, respectively. Among the label numbers, those with a satin pattern indicate the label numbers to be searched. In addition, items arranged below “fourth thread” (such as “acquisition of id [4] → id [4] = 2”) indicate processing contents executed by the fourth processor core 641.

  The fourth processor core 641 sets the run r [4] as a process target run, reads the run connection data id [4] corresponding to the process target run r [4] from the run connection table, and sets this value (= 2). When compared with the initial value (= 4) of the label number of run r [4], it is confirmed that the two do not match. Then, the read target is moved to the run connection data id [new_dum] of the label number (new_dum) indicated by the run connection data id [4], that is, the run connection data id [2], and the run connection data id [[ 2] is read, and this value (= 0) is compared with the initial value (= 2) of the label number of run r [2]. When it is confirmed that they do not match, the read target is moved again to the run connection data id [new_dum] of the label number (new_dum) indicated by the run connection data id [2], that is, the run connection data id [0]. Then, the run connection data id [0] is read from the run connection table, and this value (= 0) is compared with the initial value (= 0) of the label number of the run r [0]. Then, it is confirmed that they match each other, the label number of the processing target run r [4] is rewritten with the value, and this thread is terminated. Note that the same individual search process is executed for the other threads, and as a result, the runs connected to each other are searched and given the same label. For example, the same label (= 0) is assigned to the runs r [0] to r [4] (see FIG. 6) obtained by performing the run length processing on the binary image data shown in FIG.

  As described above, in this embodiment, after the label numbers having different values are initialized for each run r [m], the two-pass process of the update process (step S3) and the search process (step S4) is executed. doing. Of these, in the update process (step S3), for each run r [m], is there a linked run that executes the individual update process (steps S3-0, S3-1,...) And connects to the process target run? The label number is updated according to whether or not. Further, in the search process (step S4), for each run r [m], the individual search process (steps S4-0, S4-1,...) Is executed and the linked run connected to the process target run is updated. Based on the number, the labels of the processing target run and the linked run are rewritten to be the same. Thus, the labeling process of the binary image data BFI can be performed with the same number of label numbers as the number m of runs, and the labeling process can be speeded up.

  In addition, since both the update process (step S3) and the search process (step S4) execute the individual update process and the individual search process for each run in parallel in one run and one thread, the total time required for the labeling process Can be greatly shortened.

  Furthermore, in the individual update process (steps S3-0, S3-1,...), When the label number of the first linked run directly connected to the processing target run is updated, not only the first linked run, When there is a second linked run other than the process target run connected to the first linked run, the second linked run is also updated. Therefore, the time required for the individual update process for the second linked run executed after the individual update process for the processing target run can be reduced. As a result, the total processing time can be shortened.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. For example, in the above embodiment, the individual update processing is executed in parallel by one run and one thread, but the division mode of the plurality of runs r [m] is not limited to this. That is, the division mode of the plurality of runs r [m] may be appropriately changed according to the number of the processor cores 641, and the plurality of runs r [m] are divided into a plurality of sets according to the number of the processor cores 641. An individual update process may be executed as an update thread for one or a plurality of runs into which each processor core is divided. These points are exactly the same in the individual search process.

  In the above embodiment, the GPU 642 having a plurality of processor cores 641 is used to execute the individual update process and the individual update process. However, instead of the GPU, a plurality of CPUs are provided, and each CPU has one thread. It may be configured to execute. Further, an arithmetic processing unit 646 and a labeling storage unit 645 may be provided in the GPU 642.

  In the above-described embodiment, the parallel processing control unit 648 positioned above the 0th to nth processor cores 641 performs access management of the labeling storage unit 645, and one thread is used in both the update processing and the search processing. According to the data rewriting by, data rewriting by other threads is temporarily prohibited. That is, for example, when rewriting run connection data id [m] (however, 0 ≦ m ≦ n) in one thread, other threads are temporarily prohibited from rewriting run connection data id [m]. In addition to avoiding label update conflict, rewriting of other run connection data is temporarily prohibited. Such access management is not necessarily optimal, and there is room for improvement. For example, in order to further improve the processing speed, it is desirable to execute access management of the labeling storage unit 645 by the parallel processing control unit 648 as follows. That is, when the run connection data id [m] is rewritten by one thread, the other thread temporarily prohibits rewriting the run connection data id [m] to avoid label update contention. It is preferable to permit data rewriting by liberalizing access to run connection data other than the run connection data id [m].

  Further, in the above embodiment, the binary image data BFI is run length for each row to obtain the run r [m], and the run that is connected to the processing target run in the vertical and diagonal directions is determined as a connected run. However, neither the run-length mode nor the connected run determination mode is limited to this. For example, the run-length mode is run for each column, and the run created by the run-length process is applied to the present invention. Such a labeling method can be applied. In the above embodiment, as described above, the linked run is determined by the 8-neighbor search for the descending run. However, the linked run may be determined by another search method, for example, the 4-neighbor search. In the run length conversion process of the above embodiment, the run is generated for the entire image at one time. However, the run may be generated in each divided range by dividing the image into appropriate size regions. However, in this case, it is necessary to perform a run connection determination process at the boundary between the divided regions.

  In the above-described embodiment, the serial number (0, 1, 2,...) Is used as the “label” of the present invention, but a different number or a value other than the number is used as the “label” of the present invention. May be used. Moreover, although the label is updated by rewriting to a smaller number, the manner of label update is not limited to this.

  Furthermore, in the above embodiment, the binary image data BFI is run-length inside the apparatus, and then the labeling process is performed on the run created by the run-length process. For example, the present invention may be applied to an apparatus or method for labeling a run created by an external apparatus.

  The present invention can be suitably applied to a labeling technique for assigning labels to a plurality of runs created by converting the binary image data into run lengths.

DESCRIPTION OF SYMBOLS 1 ... Inspection system 5 ... Image acquisition part 6 ... Image processing part 21 ... Imaging part 62 ... Difference extraction part (image extraction part)
63: Binarization processing unit 64: Labeling unit 641 ... Processor core 642 ... GPU
643 ... Individual update processing unit 644 ... Individual search processing unit 645 ... Labeling storage unit 646 ... Calculation processing unit (control unit)
647 ... run generation unit 648 ... parallel processing control unit 649 ... data initial setting unit

Claims (11)

A labeling method for performing a labeling process on a plurality of runs created by converting the binary image data into run lengths,
An initial label setting step for setting different labels for each run;
For each run, it is determined whether or not there is a first connected run connected to the target run and the target run. When the first connected run exists, the target run and An update process for performing an individual update process for updating one label of the first linked runs to the other label;
After the updating step, a search step of searching for a run connected to each other among the plurality of runs based on the plurality of labels and providing the same label,
In the update step, the plurality of runs are divided into a plurality of sets, and an individual update process executed in each set is used as an update thread, and the label is rewritten by one update thread among the plurality of update threads under exclusive control. A labeling method wherein the plurality of update threads are executed in parallel while prohibiting rewriting by another update thread of a label rewritten by the one update thread. The labeling method according to claim 1,
In the update step, a labeling method for permitting rewriting by the other update thread of a label other than the label rewritten by the one update thread. A labeling method according to claim 1 or 2,
In the individual update process, the run whose label has been updated among the process target run and the first connected run is set as an update run, and a second other than the process target run and the first connected run with respect to the update run. A labeling method for determining whether or not a connected run is connected, and further updating one label of the updated run and the second connected run to the other label when the second connected run exists. A labeling method according to any one of claims 1 to 3,
The plurality of runs are obtained by converting the binary image data into run lengths for each row,
In the individual update process, a labeling method for determining a run connected to the downstream of the processing target run as the first linked run. A labeling method according to any one of claims 1 to 4,
In the search step, for each of the runs, the run is set as a process target run and a connected run connected to the process target run is found based on the plurality of labels, and the label of the process target run and the connected run is the same. A labeling method that executes individual search processing to be rewritten. The labeling method according to claim 5,
In the search step, the plurality of runs are divided into a plurality of sets, and individual search processing executed in each set is set as a search thread, and rewriting of a label by one search thread among the plurality of search threads under exclusive control. A labeling method that executes the plurality of search threads in parallel while prohibiting rewriting by another search thread of a label that is rewritten by the one search thread. The labeling method according to claim 6,
In the searching step, a labeling method for permitting rewriting of a label other than the label rewritten by the one searching thread by the other searching thread. A labeling device that performs a labeling process on a plurality of runs created by converting binary image data into run lengths,
A storage unit for storing the label of each run;
A processor unit having a plurality of processor cores for performing arithmetic processing;
After initially setting different values for the plurality of labels, for each run, it is determined whether or not there is a first linked run that is the target run and connected to the target run. When the first linked run exists, an individual update process for updating one label of the processing target run and the first linked run to the other label is performed, and the plurality of labels are further updated based on the plurality of labels. A controller that searches for orchids connected to each other and assigns the same label,
The plurality of runs are divided into a plurality of sets according to the number of the processor cores,
Each processor core executes the individual update process as an update thread for one or more divided runs,
The control unit permits rewriting of a label by one update thread among the plurality of update threads under exclusive control and prohibits rewriting by another update thread of a label rewritten by the one update thread. A labeling device that performs an individual update process for each run by executing a plurality of update threads in parallel. A labeling device according to claim 8,
The control unit, for each run, sets the run as a processing target run and finds a connected run connected to the processing target run based on the plurality of labels, so that the label of the processing target run and the connection run is the same A labeling device that executes an individual search process to be rewritten. The labeling device according to claim 9,
Each processor core executes the individual search process as a search thread for one or a plurality of divided runs,
The control unit permits rewriting of a label by one search thread among the plurality of search threads under exclusive control and prohibits rewriting by another search thread of a label rewritten by the one search thread. A labeling device that performs individual search processing for each run by executing the search threads in parallel. An image acquisition unit for acquiring an image to be inspected;
An image extraction unit that inspects the inspection target image and extracts a defect image including a defective portion;
A binarization processing unit that binarizes the defect image to generate binary image data;
A run generation unit that generates a plurality of runs by converting the binary image data into a run length;
A labeling device having the same configuration as the labeling device according to any one of claims 8 to 10, further comprising labeling means for giving the same label to runs connected to each other among the plurality of runs. Defect inspection equipment.

Images