JP4635760B2 - Image processing apparatus and method - Google Patents

Description

  The present invention relates to an image processing apparatus and method for extracting an edge from an input image.

An image processing apparatus (method) for extracting an edge from an input image is known and disclosed in [Patent Document 1] below. In what is described in [Patent Document 1] below, an image is binarized with a plurality of threshold values, and a boundary pixel determination operator is applied to the binarized image, and then the boundary image is integrated.
JP-A-5-225336

  Since the edge part of the boundary image is synthesized by the theoretical sum, all the edge parts are integrated, so it is not possible to extract only the edges that are continuous in the specific part (originally only the edges of the specific part) Therefore, an object of the present invention is to provide an image processing apparatus and method capable of extracting only a specific edge portion in an input image.

The image processing apparatus according to claim 1, wherein a reference edge extracting unit that extracts a reference edge from an input image, a reference edge extracting unit that extracts a reference edge before division, and a plurality of reference edges before dividing are divided into a plurality of pieces. Dividing means for generating a reference edge after division and the degree of overlap with the base edge for each of the reference edges after division are calculated, and among the plurality of reference edges after division, the reference after division having a degree of overlap of a predetermined standard or more and a edge configuration means for configuring the edge in the input image using the edge, the target object in the input image by comparing the both sides of the small area of the reference edge that determines whether a transparent object, the input It is characterized by detecting the edge of a transparent object in an image .

The image processing method according to claim 5 is an image processing apparatus that extracts an edge from an input image, extracts a reference edge and a reference edge before division from the input image, and divides the reference edge before division. A plurality of divided reference edges, and the degree of overlap with the reference edge is calculated for each of the divided reference edges, and among the plurality of divided reference edges, the divided reference having a degree of overlap of a predetermined standard or more Edges in the input image are configured using edges, and the edge of the transparent object in the input image is determined by comparing the areas on both sides of the reference edge to determine whether the target object in the input image is a transparent object. It is characterized by detecting .

The invention described in claims 2 and 6 is the image processing apparatus according to claim 1 or the image processing method according to claim 5 , wherein the reference edge before division is a division filter, a Sobel filter, or It is characterized by being extracted using a Canny filter.

According to the third and seventh aspects of the present invention, in the image processing apparatus according to the first aspect or the image processing method according to the fifth aspect , the reference edge is divided at a point where the change in the gradient of the edge is discontinuous. It is characterized by being performed.

The inventions according to claims 4 and 8 are the image processing apparatus according to claim 1 or the image processing method according to claim 5 , wherein the overlapping degree is a pixel overlap of the reference edge and the reference edge after the division. It is characterized by a ratio.

  According to the image processing apparatus or method of the present invention, it is possible to accurately extract only a specific edge portion (for example, an edge portion of a transparent object) in an input image. When the extraction of the edge portion of the transparent object is attempted, in order to adopt only the edge where the feature appears as the reference edge, the reference edge may be extracted after being divided into discontinuities. In such a case, only a desired edge and a continuous edge can be extracted by extracting a reference edge as an edge for a portion where a reference edge by a general edge extracting means and a reference edge overlap. It becomes possible.

  The image processing apparatus of the present invention includes an image acquisition unit, an image processing unit that processes the acquired image by applying various filters, and an image output unit that outputs the finally processed image. The image acquisition unit is a CCD camera or the like. The image processing unit not only applies various filters, but also performs arithmetic processing related to a standard edge and a reference edge (details will be described later) obtained by image processing, and includes an ECU including a CPU, GPU, ROM / RAM, etc. It is. The image output unit may be a display, a printer, or an output unit that outputs an image signal to another ECU that performs the next process.

  An example of image processing control by the above-described image processing apparatus will be described. A flowchart for this control example is shown in FIG. Here, the specific edge to be extracted is the edge of the transparent object. If it is an issue to detect and recognize a transparent object using an imaging system such as a CCD camera, polarization is used or a light source position is specified in order to derive a feature amount such as a pattern or a 3D shape. It is difficult if it is not in a special environment such as, and the most powerful clue at present is the edge.

  However, in the normal edge detection (Sobel, Canny) method, not only the edge of the transparent part but also the edge of every object is extracted uniformly, and it is difficult to extract only the edge of the transparent object from among them. It was. However, the extraction of transparent objects by detecting the “light” that increases the brightness specific to the transparent part is also under consideration, but this [light] phenomenon is not specific to transparent objects, It is not a definitive technique because it exists on the entire curved surface.

The detection control of this embodiment described below solves such a problem. As shown in FIG. 1, first, an image is acquired by an image acquisition unit such as a CCD camera (step 100: FIG. 2). This image is subjected to an edge extraction filter such as Sobel to extract a reference edge I _edge (step 105: FIG. 3). The filter for extracting the reference edge I _edge may be a filter such as a differential filter or a Canny filter. These filters are general filters for edge extraction, and can extract edges relatively clearly than a reference filter described later.

In parallel with the extraction of the reference edge I _edge , a multi-resolution analysis (K _i [here, a wavelet filter is applied to the image in FIG. 1) using a basis filter (here, a wavelet filter is applied). , I = 1 to N: N = 4] to extract the reference edge (step 110: FIGS. 4 to 7). In the present embodiment, edge detection of a transparent object (glass cup) is described as an example, and a base having a shape similar to a transparent edge portion is preferable. Here, FIGS. 4 to 7 show different frequency bands in the wavelet filter, and the frequencies increase in order from FIGS. 4 to 7.

  The filter used for extracting the reference edge may be a Gabor filter or the like in addition to the wavelet filter. These filters can extract desired frequency bands and directional edges. Alternatively, the reference edge may be extracted by a graph cut filter or a mean shift method. These filters can extract regions / outlines such as desired color, edge strength, etc. A characteristic common to these reference edge extraction filters is that it can be extracted by aiming at an edge having a characteristic to be extracted (for example, aiming at an edge of a transparent object and an edge of a specific color).

After step 105, the reference edge I _edge is thinned (extension processing may be performed if necessary) to obtain a junction point (a point where the gradient is discontinuous or a point where the gradient changes suddenly). Edge segmentation is performed (step 115). Here, since the line is complicated in the reference edge diagram of FIG. 2, an example of segmentation is shown by taking FIG. 8 as an example of FIG. 4 binarized with a certain threshold value. FIG. 9 is obtained by segmenting FIG. 8 and enclosing each segment with a line. A segment surrounded by one line is one segment. Such segmentation is performed on the reference edge of FIG. Thereby, the reference edge I _edge is divided and segmented into a plurality of segments.

On the other hand, for the image showing each reference edge (N = 1 to 4), after step 110, the following processing is performed for each. The reference edge and the reference edge I _edge are overlapped, and attention is paid to minute regions existing on both sides of the reference edge. Here, the edge of the transparent object is detected, but if it is the reference edge I _edge of the transparent object, the possibility that the backgrounds match on both sides should be very high (or match). However, since the transparent object is transmitted through the inner and outer regions of the reference edge I _edge of the transparent portion, blur exists in the inner region that transmits the transparent object. Therefore, the difference between the minute regions on both sides is measured (step 125), and when the threshold value is equal to or less than the reference value (substantially matches or matches), it is determined that the reference edge I _edge is the _edge of the transparent object. .

This process will be described based on the diagram of FIG. 10 and the flowchart of FIG. As shown in FIG. 10, a pair of minute regions (rectangular regions having a side of about 10 to 20 pixels) are set on both sides of the reference edge, and one of these is set to an outer region dS _out-i that does not transmit a transparent object. After performing a smoothing (blurring) process such as a Gaussian filter, a difference Diff _i = S _in−i −S _out−i from the inner region dS _in−i that passes through the transparent object is calculated (step 1110). Steps 1100 and S1105 correspond to steps 100 and 105 in the flowchart of FIG. 1 described above, respectively.

Note that S _in-i and S _out-i are values obtained by converting the states of the respective regions dS _in-i and dS _out-i into scalar quantities in order to obtain the difference Diff _i . In addition, when performing the smoothing (blurring) process, in fact, it is not known which side of the edge is the side that has passed through the transparent object or not, so the smoothing process is performed on one of the areas. Look at the results and decide which is inside and which is outside. Thereafter, the above-described difference Diff _i is obtained.

The method for obtaining the difference Diff _i may be based on various methods. For example, the color of all the pixels in the region (the density or the like in the case of gray scale) may be digitized, and the average value or total sum of all the pixels may be used as the S _in-i and S _out-i values. Alternatively, the hue, saturation, brightness, and luminance value may be digitized and used. Alternatively, a difference may be calculated for each pixel, and an average value or a sum of the differences may be obtained to obtain an overall difference Diff _i .

Then, after step 1110, in FIG. 11, only edges whose difference Diff _i is less than the predetermined threshold Th are extracted as extracted edges I _last that are edges of the transparent object (step 1115). The extracted edge I _last is extracted as a part of the outline of a transparent object (for example, a cup) that is partially missing. Therefore, after step 1115, the contour of the transparent object is determined using a contour extraction method such as Snake (step 1120). However, in the flowchart of FIG. 1, since a plurality of reference edges (FIGS. 4 to 7) are used, another step is performed before steps 1115 and 1120. However, the difference Diff _i in step 120 of FIG. 1 is calculated as described above.

In step 120 of the flowchart of FIG. 1, for each basic edge image, a layer K _{i in} which the ratio satisfying Diff _i <Th (= Ratio) exceeds a predetermined threshold Th-ratio is selected for all minute regions (step 125). ). FIGS. 12 to 15 are diagrams in which reference edge portions satisfying Diff _i <Th are extracted by overlapping the reference edge of FIG. 2 with respect to the reference edge of FIGS. 4 to 7. Here, K _{1 is the} only hierarchy that satisfies Ratio> Th-ratio. However, if there are a plurality of layers, those obtained by superimposing (integrating) all of them are extracted as integrated edge I _comb (step 130).

After step 130, compares the synthesized edge _{I comb} and "segmented" reference edge _{I edge,} overlapping the integrated edge _{I comb} region (multiplicity: Duplicate ratio is equal to or higher than a predetermined reference) of the reference edge _{I edge} A segment is extracted as an extraction edge I _last (step 135). This figure is shown in FIG. Here, by superimposing the segmented reference edge I _edge , the edge that has not been extracted as the integrated edge I _comb can be recovered, so that detection with higher accuracy can be performed. For example, as can be seen by comparing FIG. 12 (integrated edge I _comb ) and FIG. 16 (extracted edge I _last ), the original shape of the second cup and the like from the left is accurately extracted.

The extracted edge I _last is extracted as a part of the outline of a transparent object (for example, a cup) that is missing. Therefore, the outline of the transparent object is determined using an outline extraction method such as Snake as in Step 1120 of FIG. 11 described above (Step 140). FIG. 17 shows a contour of a transparent object determined using a contour extraction method such as Snake. In this way, the edge of a specific object that is difficult to detect (here, the edge of a transparent object).

In the above-described embodiment, the case where the edge of the transparent object is detected has been described as an example. However, the present invention is not necessarily limited to the detection of the transparent object. A flowchart in the case of generalization without specifying an object is shown in FIG. First, an image is acquired by an image acquisition unit such as a CCD camera (step 1800). The image is subjected to two different edge extraction filters to extract the reference edge I _edge1 (step 1805) and the reference edge I _edge2 (step 1810).

The two different edge extraction filters have different characteristics, the filter that extracts the reference edge I _{edge1 employs} a filter that can also extract weak edges (the edge can be strongly extracted), and the filter that extracts the reference edge I _edge2 has an outline. A filter that can extract the edge of the target object to be extracted more strongly (only the edge of the target object is stronger) is used in combination. Here, it is assumed that there is only one image from which the reference edge I _edge2 is extracted.

Next, with respect to the reference edge I _edge1 , as in step 115 of FIG. 1 described above, a junction point (a point where the gradient is discontinuous or a point where the gradient changes suddenly) is obtained, and edge segmentation is performed (step 1815). . Further, by superimposing the reference edge _{I edge2} and segmented reference edge _{I edge1,} for each segment, a segment of the reference edge _{I edge1} overlapping portion exceeds a predetermined threshold value Th-ratio as extracting an edge _{I last} (Step 1820: It can be said that steps 120 to 135 in FIG. 1 are simplified). Then, similarly to step 140 in FIG. 1, the contour of the transparent object is determined using a contour extraction method such as Snake (step 1825).

  In addition, this invention is not limited to embodiment mentioned above. For example, in the above-described embodiment, a monochrome image has been described as an example, but the present invention can also be applied to a color image.

It is a flowchart of one Embodiment of the image processing apparatus (method) of this invention. It is an acquired image. This is an image obtained by applying a Sobel filter to an acquired image and extracting a reference edge. This is an image obtained by applying a wavelet filter (first frequency) to an acquired image and extracting a reference edge. This is an image obtained by applying a wavelet filter (second frequency) to an acquired image and extracting a reference edge. This is an image obtained by applying a wavelet filter (third frequency) to an acquired image and extracting a reference edge. This is an image obtained by applying a wavelet filter (fourth frequency) to an acquired image and extracting a reference edge. It is the image which binarized the image of FIG. It is the image which segmented the image of FIG. It is explanatory drawing explaining the edge detection of a transparent object. It is a flowchart explaining edge detection of a transparent object. FIG. 5 is an image showing an edge satisfying a difference Diff <predetermined threshold Th based on the image of FIG. 4. FIG. 6 is an image showing an edge satisfying a difference Diff <predetermined threshold Th based on the image of FIG. 5. It is an image which shows the edge which satisfy | fills a difference Diff <predetermined threshold value Th based on the image of FIG. FIG. 8 is an image showing edges satisfying the difference Diff <predetermined threshold Th based on the image of FIG. 7. The segmented reference edge image and the integrated edge satisfying a predetermined ratio Th-ratio are overlapped to extract overlapping segments. It is the image which extracted the outline from the image of FIG. It is a flowchart of other embodiment of the image processing apparatus (method) of this invention.

Claims (8)

In an image processing apparatus that extracts edges from an input image,
A reference edge extracting means for extracting a reference edge from an input image; a reference edge extracting means for extracting a reference edge before division; a dividing means for dividing a reference edge before division to generate a plurality of divided reference edges; The degree of overlap with the base edge is calculated for each of the divided reference edges, and the edges in the input image are configured using the divided reference edges with a degree of overlap greater than or equal to a predetermined standard among the plurality of divided reference edges Edge forming means to
An image in which an edge of a transparent object in the input image is detected by determining whether or not a target object in the input image is a transparent object by comparing minute regions on both sides of the reference edge. Processing equipment.
An image processing apparatus.   The image processing apparatus according to claim 1, wherein the reference edge before division is extracted using a differential filter, a Sobel filter, or a Canny filter.   The image processing apparatus according to claim 1, wherein the reference edge is divided at a point where a change in edge gradient is discontinuous.   The image processing apparatus according to claim 1, wherein the degree of overlap is an overlap ratio of pixels of a reference edge and a base edge after division. In an image processing apparatus that extracts edges from an input image,
The reference edge and the reference edge before division are extracted from the input image, the reference edge before division is divided to generate a plurality of divided reference edges, and the degree of overlap with the reference edge for each of the divided reference edges And the edge in the input image is configured using the reference edge after the division having a degree of overlap equal to or greater than a predetermined standard among the plurality of reference edges after the division.
Image processing characterized in that an edge of a transparent object in the input image is detected by determining whether or not a target object in the input image is a transparent object by comparing regions on both sides of the reference edge Method.   The image processing method according to claim 5, wherein the reference edge before division is extracted using a differential filter, a Sobel filter, or a Canny filter.   6. The image processing method according to claim 5, wherein the reference edge is divided at points where the change in the edge gradient is discontinuous.   The image processing method according to claim 5, wherein the degree of overlap is an overlap ratio of pixels of a reference edge and a base edge after division.