JP2015130074A - image processing system and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system and an image processing method for easily accelerating the detection of abnormality and the execution of processing based on the abnormality.SOLUTION: An image processing system of an embodiment includes: a first device having a first calculation processing unit that creates, from image data, division data having a first pixel and a second pixel divided according to a luminance value or a luminance value distribution; and a second device having a second calculation processing unit that determines a specific area from the division data created by the first calculation processing unit and a processing execution control unit that executes predetermined processing on the basis of information indicating the specific area.

Description

  Embodiments described herein relate generally to an image processing system and an image processing method.

  2. Description of the Related Art Conventionally, an image processing system that detects defects, abnormalities, and the like by image processing from captured image data is known.

JP 2007-327789 A

  In this type of image processing system, it may not be preferable that a delay occurs in detecting an abnormality, displaying the detected abnormality, executing a process based on the detected abnormality, and the like. In other words, it would be meaningful to obtain an image processing system and an image processing method that can facilitate the detection of an abnormality and the execution of processing based on the detected abnormality.

  The image processing system of the embodiment includes a first arithmetic processing unit that generates segment data in which the first pixel and the second pixel are segmented from the image data according to the luminance value or the luminance value distribution. One apparatus, a second arithmetic processing unit that determines a specific area from the classification data generated by the first arithmetic processing unit, and a process execution control unit that executes a predetermined process based on information indicating the specific area; And a second device.

  According to the embodiment, as an example, it is possible to obtain an image processing system and an image processing method that can easily speed up the detection of an abnormality and the execution of a process based on the detected abnormality.

FIG. 1 is a schematic diagram illustrating a state of imaging by an imaging unit of an example of an image processing system according to the embodiment. FIG. 2 is a block diagram of an example of an image processing system according to the embodiment. FIG. 3 is a flowchart of an example of a processing procedure in the image processing system according to the embodiment. FIG. 4 is a schematic diagram illustrating an example of part of image data before being captured and processed by the image processing system according to the embodiment. FIG. 5 is a schematic diagram illustrating an example of filtering by the image processing system according to the embodiment. FIG. 6 is a schematic diagram illustrating an example of an image showing a specific region obtained by the image processing system according to the embodiment. FIG. 7 is a schematic diagram illustrating an example of image data output by the image processing system according to the embodiment.

  Hereinafter, exemplary embodiments of the present invention are disclosed. The configuration and control of the embodiment shown below, and the operation and result (effect) brought about by the configuration and control are merely examples. The present invention can be realized by means other than the configuration and control disclosed in the following embodiments, and various effects (including derivative effects) obtained by the basic configuration and control can be obtained. is there.

  In the present embodiment, as illustrated in FIG. 1, the image processing system 1 performs image processing on the two-dimensional image data of the inspection target surface 101 of the inspection target 100 captured by the imaging unit 2, thereby performing an inspection target. The surface 101 is inspected. The inspection object 100 is a sheet (paper, film, film, plate, etc.) as an example, but may be anything as long as image data can be obtained by imaging by the imaging unit 2. In the example of FIG. 1, the conveyance belt 102 is conveyed by a roller 103 (part of the conveyance mechanism), and a plurality of sheet-like inspection objects 100 are arranged side by side along the longitudinal direction on the conveyance belt 102. ing. The imaging unit 2 sequentially images different positions of the moving conveyor belt 102 at predetermined (very short) time intervals. The image acquired by the imaging unit 2 includes the inspection target surface 101 of the inspection target 100 and the conveyor belt 102. The inspection target surface 101 is an example of an inspection target area, and the conveyor belt 102 is an example of a non-inspection target area.

  The imaging unit 2 is, for example, an area sensor (a solid-state imaging device such as a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS)) having two-dimensionally arranged photoelectric conversion elements (photoelectric conversion units). Image sensor). Alternatively, the image processing system 1 inspects two-dimensional image data configured by combining a plurality of one-dimensional image data and wider two-dimensional image data configured by combining a plurality of two-dimensional image data. Inspection of the target surface 101 (inspection object 100) can be performed. That is, the imaging unit 2 may be a line sensor (line camera, image sensor) that acquires one-dimensional image data, or may be an area sensor that acquires two-dimensional image data. The two-dimensional image data includes luminance value data (hereinafter also referred to as pixel data) of a plurality of pixels arranged two-dimensionally.

  In the present embodiment, as an example, as illustrated in FIG. 2, the image processing system 1 includes a first device 3 and a second device 4. The first device 3 is, for example, an integrated circuit in which a plurality of logic circuits are integrated. For example, a programmable logic device (PLD) such as a field-programmable gate array (FPGA) or a CPLD (complex programmable) is used. logic device), ASIC (application specific integrated circuit), and the like. The first device 3 is capable of high-speed arithmetic processing in parallel (relative to the second device 4). When the first device 3 includes a programmable logic device, the logic circuit can be rewritten after the first device 3 is manufactured, which is convenient and is suitable for the inspection object 100, imaging environment, device specifications, and the like. By changing the logic circuit, the accuracy and speed of operation (inspection) can be improved more easily.

  In addition, the second device 4 is, for example, a device (control unit 41, for example, a CPU (central processing unit), a DSP (digital signal) or the like that executes computation (sequential processing) according to a computer program (software program). processor) etc.). The second device 4 includes a storage unit 42 (for example, RAM (random access memory), ROM (read only memory), flash memory, HDD (hard disk drive), SSD (solid state drive), etc.), so-called. It can be configured as a computer. The computer program is installed (stored) in a part of the storage unit 42 (nonvolatile storage unit). The result calculated by the second device 4 is displayed on the display unit 5. The display unit 5 is, for example, an LCD (liquid crystal display) or an OELD (organic electro-luminescent display).

  As shown in FIG. 2, the first device 3 includes a preprocessing unit 31, a storage unit 32, a background identification unit 33, a distribution unit 34, a sorting processing unit 35, a combining unit 36, a compression unit 37, and the like. The second device 4 includes a control unit 41, a storage unit 42, and the like. The control unit 41 includes a quality determination unit 43, a synthesis unit 44, a display control unit 45, and the like.

  Hereinafter, configuration examples and operation examples of the first device 3 and the second device 4 will be described. FIG. 3 shows an example of a processing procedure performed by the first device 3 and the second device 4. First, imaging is performed by the imaging unit 2, and two-dimensional pixel data (an image Im captured by the imaging unit 2, hereinafter simply referred to as an image Im, see FIG. 4) is obtained (S1).

  Next, the preprocessing unit 31 performs preprocessing on the image Im as necessary (S2). In S2, in addition to general preprocessing (filtering, smoothing, etc.) of image processing, interpolation processing in a known Bayer array pattern can be executed. The Bayer array pattern is an element (not shown) corresponding to each pixel of the imaging unit 2 that acquires pixel data (luminance value) of one color of R (red), G (green), or B (blue). It is an array pattern. When the elements of the imaging unit 2 are arranged in a Bayer arrangement pattern, the preprocessing unit 31 can interpolate pixel data of other colors using pixel data of other pixels (surrounding pixels). it can. The storage unit 32 can store pixel data of each color of each pixel (preprocessed pixel data).

  The pixel data preprocessed by the preprocessing unit 31 is stored in the storage unit 32 (S3). The storage unit 32 is, for example, a nonvolatile rewritable storage unit 32 (for example, an auxiliary storage device).

  Next, the background identification unit 33 uses the pixel data of the inspection target surface 101 stored in the storage unit 32 (data that has been subjected to arithmetic processing in the preprocessing unit 31 and preprocessed image Im) as an area corresponding to the background ( A non-detection target area, in the example of FIG. 1, the conveyor belt 102) is identified (S4). In S4, the background image ImB in the image Im can be identified by the following three methods (1) to (3), for example.

(1) Identification by Position in Image Im In the example of FIG. 4, the transport direction (movement direction) is the downward direction. In this case, the background identification unit 33 has a known substantially constant width at the left end (or right end). The region can be distinguished from the background image ImB. When the timing of the image ImS on the inspection target surface 101 (inspection target 100, inspection target region) (that is, the position of the inspection target 100 on the transport belt 102) is known, the width direction of the image Im (see FIG. The region extending with a constant width in the left-right direction (4) can be determined as the background image ImB.

(2) Identification by the magnitude of the luminance value of each pixel In the example of FIG. 4, a pixel whose luminance value is lower than the threshold value or a pixel whose luminance value falls within the threshold value range is a pixel included in the background image ImB ( ).

(3) Identification by the total value of luminance values of each pixel and pixels adjacent to or around the pixel In the example of FIG. 4, the background image ImB is compared with the image ImS of the inspection target surface 101. The value becomes lower. Therefore, by comparing the total value with the corresponding threshold value, for example, a pixel whose total value is lower than the threshold value can be determined as the background pixel ImB. In S4, it is only necessary to identify at least a part of the background image ImB. The background identification unit 33 is an example of a third arithmetic processing unit.

  The sorting process of S6 in FIG. 3 is executed sequentially by one of a plurality (two in the example of FIG. 2) of the sorting processing units 35 provided in parallel as shown in FIG. . The distribution unit 34 determines the assignment of the sorting processing unit 35 (the sorting processing unit 35 that executes the sorting process of S6 next) (S5).

  The division processing unit 35 to which the execution of the process is assigned by the distribution unit 34 generates the division data in which the first pixel and the second pixel are divided from the image Im (executes the division process, S6). In S6, the classification processing unit 35 can execute a so-called binarization process, for example, depending on the magnitude of the pixel data threshold value of each pixel. In this case, for example, the classification processing unit 35 sets a pixel whose luminance value is lower than the threshold as the first pixel (defective estimated pixel), and a pixel whose luminance value is equal to or higher than the threshold as the second pixel (good estimated pixel), Classification data is generated by classifying the former (defective estimated pixel, first pixel) as “1” and the latter (good estimated pixel, second pixel) as “0”. The first pixel is an example of a necessary pixel that is used in arithmetic processing in the second device 4 later, and the second pixel is an unnecessary pixel that is not used in arithmetic processing in the second device 4 later. It is an example. In the example of FIG. 4, for example, the classification processing unit 35 determines the classification data of the pixels in the images Im1, Im2, and Im3 in the image Im as “1” (defective estimated pixel, necessary pixel), and other areas. The division data of the pixel in the image ImS (white region in FIG. 4) is determined as “0” (good estimated pixel, unnecessary pixel). In this example, the luminance value of the image data is, for example, 256 gradations from “0” to “255”, while the value of the division data is two gradations, “0” or “1”. Therefore, according to the present embodiment, as an example, the first device 3 to the second device 4 is compared with the case where the luminance value (pixel data) of each pixel is sent from the first device 3 to the second device 4. The size of data (partition data) sent to the device 4 is reduced, and the data transfer speed from the first device 3 to the second device 4 and the processing speed of the second device 4 are further increased. Cheap. Therefore, in the image processing system 1, arithmetic processing with less delay is easily realized. The classification processing unit 35 is an example of a first arithmetic processing unit that generates classification data according to the luminance value.

  Further, in S6, the classification processing unit 35 can set the classification data of the pixels identified as the background image ImB in S4 to be the same as the above-described good estimated pixels (“0” in the above-described example). This is because the pixel data in the background image ImB is not subject to defect determination in the pass / fail determination unit 43 of the second device 4 together with the good estimated pixels in the inspection target surface 101. The pixel data in the background image ImB may be assigned a value different from “0” or “1” assigned to the pixels in the inspection target surface 101 (for example, “2”). However, in this case, the size of the divided data transferred from the first device 3 to the second device 4 is 2 bits, which is larger than the case of 1-bit divided data of “0” or “1”. Prone.

  Further, in S6, the classification processing unit 35 can execute noise removal processing or hole filling processing (filtering) as shown in FIG. In FIG. 5 (the same applies to FIG. 6), a rectangle surrounded by vertical and horizontal broken lines is the pixel P. The division processing unit 35 determines the value of the pixel P1 by comparing the total value of the division data (or luminance value) of each pixel P in the region F adjacent to the central pixel P1 with the corresponding threshold value (change). To do). For example, as illustrated in FIG. 5, the luminance value of the central pixel P1 is low (value of the division data: “1”), and the luminance values of all the pixels P in the region F adjacent to the pixel P1 are high. In the case of (value of the division data: “0”), the total value of the division data (or luminance value) of the pixel P in the region F is “0” (or a value having a low luminance value). Therefore, in this example, by comparing the total value with the corresponding threshold value, when the total value is lower than the corresponding threshold value, the value of the segment data of the pixel P1 can be determined (changed) to “0”. On the contrary, even when the luminance values of the pixels P in the region F are all low, if the total value is higher than the corresponding threshold value by comparing the total value with the threshold value set corresponding to this case, the pixel The value of the division data of P1 can be determined as “1”. Note that the region F may or may not include the central pixel P1. The processing described here can be executed by the preprocessing unit 31 in S2. The division processing unit 35 is an example of a first calculation processing unit that generates division data according to the luminance value distribution.

  The synthesizing unit 36 combines the segment data processed by the plurality of segment processing units 35 in the original order (S7). The division data combined by the combining unit 36 in S7 is sent to the second device 4.

  In S2, S4, and S6, for example, processing that is easy to perform parallel operations at a relatively high speed by an integrated circuit in which a plurality of logic circuits are integrated, such as the same arithmetic processing for all the pixels, is executed. Since the size of the data sent from the first device 3 to the second device 4 and processed by the second device 4 is reduced by the processing in the first device 3, the first device 3 The data transfer speed from the first apparatus 4 to the second apparatus 4 and the data processing speed of the second apparatus 4 tend to be higher.

  The pass / fail judgment unit 43 performs image processing based on the classification data and makes a pass / fail judgment (S8). In S8, the pass / fail determination unit 43 can execute pass / fail determination, for example, by pattern matching with reference data. In addition, the pass / fail determination unit 43 performs the same group (group) by grouping, labeling, calculating the size of the region, comparing the size with a threshold value, and the like, for example, for the defective estimated pixels (pixels whose classification data is “1”) ) And the size of the pixel group (image) determined to be equal to or greater than a predetermined threshold value, the pixel group (image) can be determined as a specific region (defective (abnormal) location). The pass / fail judgment unit 43 is an example of a second arithmetic processing unit.

  In addition, the pass / fail determination unit 43 can generate information for displaying that the pixel group is determined to be the specific region. For example, as illustrated in FIG. 6, the quality determination unit 43 calculates the centroid C and the shape S (image ImM) with the centroid C as a reference position for the image Im3 determined as the specific region. Can do. In the example of FIG. 6, the shape S is calculated as a size (ellipse or circle) that surrounds the image Im3 with the center of gravity C as the center. The image Imm of the shape S is an example of an image showing a specific area.

  The synthesizing unit 44 synthesizes the data of the image Im obtained from the storage unit 32 of the first device 3 through the compression unit 37 and the data of the image ImM indicating the specific area obtained from the pass / fail judgment unit 43 (S9). . In step S9, the data of the image Im obtained from the first device 3 is data that has been reduced in size by the compression processing in the compression unit 37 or the like. The compression unit 37 can execute known image compression processing, thinning processing, and the like. The synthesizing unit 44 is an example of a fourth arithmetic processing unit.

  As shown in FIG. 7, the display control unit 45 controls the display unit 5 so that the image synthesized by the synthesis unit 44 is displayed (S10). The display control unit 45 is an example of a process execution control unit. The process execution control unit may be an alarm control unit that controls the display unit 5 and an audio output unit (not shown) so that an alarm is issued, a stop control unit that stops an inspection line and a production line, and the like. .

  As described above, in the present embodiment, as an example, the first device 3 having the classification processing unit 35 (first arithmetic processing unit), the pass / fail judgment unit 43 (second arithmetic processing unit), and the display control. And a second device 4 having a unit 45 (processing execution control unit). Therefore, according to the present embodiment, as an example, the data size can be reduced by the processing in the sorting processing unit 35, and as an example, the data transfer rate from the first device 3 to the second device 4 or The data processing speed in the second device 4 tends to increase.

  In the present embodiment, as an example, the background identification unit 33 (third arithmetic processing unit) can remove the background (non-inspection target region). Therefore, as an example, the processing execution time is likely to be shorter than when the classification processing unit 35 executes processing on image data including a background.

  In the present embodiment, as an example, the synthesis unit 44 (fourth arithmetic processing unit) is identified by the image data ImM obtained from the first device 3 and the pass / fail judgment unit 43 (second arithmetic processing unit). The image data Im indicating the specified specific area is synthesized. Therefore, as an example, an image in which the pass / fail determination result is easier to understand is easily displayed on the display unit 5.

  Furthermore, in this embodiment, as an example, the first device 3 includes a programmable logic device, and the second device 4 includes a control unit 41 (device) that operates according to a computer program. The calculation processing in S8 by the quality determination unit 43 of the second device 4 is not suitable for parallel calculation because variations and case classifications increase. On the other hand, in S2, S4, and S6 by each part of the first device 3, parallel calculation by a logic gate is possible for a plurality of pixels. That is, in the present embodiment, the first apparatus 3 performs parallel operations, and the second apparatus 4 performs operations that are not suitable for parallel operations, thereby further speeding up image processing according to the present embodiment. be able to. Thereby, as an example, it is possible to shorten the cycle of detecting an abnormality or executing a process based on the detected abnormality, which contributes to an improvement in efficiency and an improvement in product throughput.

  In the present embodiment, as an example, the first device 3 can include a programmable logic device. Therefore, according to the present embodiment, as an example, the specification of the first apparatus 3 is changed according to the inspection object, the inspection object region, the inspection environment, the required accuracy or cycle, and the inconvenience is further increased. Arithmetic processing (inspection) is easily performed in a small state.

  As mentioned above, although embodiment of this invention was illustrated, the said embodiment is an example to the last. The embodiment can be implemented in various other forms, and various omissions, replacements, combinations, and changes can be made without departing from the scope of the invention. In addition, the configuration and shape of the embodiment can be partially replaced with other configurations and shapes. In addition, specifications (structure, type, direction, angle, shape, size, length, width, thickness, height, number, arrangement, position, material, etc.) of each configuration, shape, etc. are changed as appropriate. Can be implemented.

  DESCRIPTION OF SYMBOLS 1 ... Image processing system, 3 ... 1st apparatus (programmable logic device), 4 ... 2nd apparatus, 33 ... Background identification part (3rd arithmetic processing part), 35 ... Classification processing part (1st arithmetic processing) Part), 41 ... control part (device), 43 ... pass / fail judgment part (second arithmetic processing part), 44 ... composition part (fourth arithmetic processing part), 45 ... display control part (process execution control part), Im2, Im3 ... (specific area) image, ImB ... (background, non-inspection area) image, ImS ... (inspection area) image.

Claims (5)

A first apparatus having a first arithmetic processing unit that generates segmented data in which a first pixel and a second pixel are segmented according to a luminance value or luminance value distribution from image data;
A second arithmetic processing unit that determines a specific area from the classification data generated by the first arithmetic processing unit; and a process execution control unit that executes a predetermined process based on information indicating the specific area; A second device having,
An image processing system comprising:   The image processing system according to claim 1, wherein the first apparatus further includes a third arithmetic processing unit that specifies an inspection target region and a non-inspection target region.   The second device further includes a fourth arithmetic processing unit that synthesizes the image data obtained from the first device and image data indicating a specific area specified by the second arithmetic processing unit. The image processing system according to claim 1 or 2.   The image processing system according to claim 1, wherein the first apparatus includes a programmable logic device, and the second apparatus includes a device that operates according to a computer program. An image processing method by an image processing system having a first device and a second device,
The first device generates segmented data in which the first pixel and the second pixel are segmented according to a luminance value or luminance value distribution,
The second device determines a specific area from the classification data, and executes a predetermined process based on information indicating the specific area;
Image processing method.

Images