JP2013037658A - Edge detection device and program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an edge detection device capable of accurately detecting each of frame images constituting a video image even when a photographic condition such as illumination is changed, and the program thereof.SOLUTION: An edge detection device 1 includes edge detection means 20 for using a first threshold and a second threshold to detect edge points of each frame image constituting a video image, edge tracking means 40 for tracking edge points of the preceding frame image having similar edge points, edge slope strength and edge direction from edge points of the current frame image, model generation and updating means 50 for distributing edge points detected by the edge detection means 20 to generate a Gaussian mixed model, and also adding edge points whose tracking is successful by the edge tracking means 40 to update the Gaussian mixed model, and threshold calculation means 60 for calculating a first threshold and a second threshold from the average value and standard deviation of the generated or updated Gaussian mixed model.

Description

  The present invention relates to an edge detection device for detecting an edge from an input video and a program thereof.

Conventionally, the Canny operator is mentioned as a typical method in the edge detection process. The Canny operator performs (1) noise removal by a Gaussian filter, (2) differentiation processing by a Sobel filter, (3) maximum position detection of a gradient, and (4) a threshold value having hysteresis characteristics. Four processes of edge detection and edge interpolation are performed (see Non-Patent Document 1). In particular, in the threshold processing (4), an edge start point is first obtained by the first threshold value h _th , joined to the start point of the edge, and those having a value _{equal to} or greater than the second threshold value l _th are determined as edges. Only the necessary edges removed are detected.

J. Canny, “A Computational Approach to Edge Detection”, IEEE Trans. Pattern Analysis and Machine Intelligence, .8, 6, .679-698, 1986.

  Here, the edge detection processing of video used in various image processing including computer vision depends on shooting conditions such as lighting and the subject in order to accurately detect the edge from each of the frame images constituting the video. Tuning needs to be done. In addition, even when the adjusted setting condition is not changed, it is necessary to keep the shooting condition and the subject condition constant so that the edge can always be accurately detected. However, it is difficult to manually adjust the tuning in accordance with the photographing conditions because it requires trial and error in the case of manual operation, is a complicated operation, and it is difficult to keep the edge detection state constant.

For example, in the image edge detection process using the Canny operator proposed in Non-Patent Document 1, the two thresholds h _th and l _th are fixed even when the shooting conditions such as illumination change. Threshold processing was performed in the state. Therefore, conventionally, when a shooting condition such as illumination changes, a shadow that is not originally an edge is detected as an edge, so that the edge state that can be detected changes depending on the shooting condition.

  The present invention has been made in view of the above points, and an edge detection device capable of accurately detecting an edge from each of frame images constituting a video even when a shooting condition such as illumination changes. And providing a program thereof.

  In order to solve the above-described problem, an edge detection apparatus according to claim 1 is an edge detection apparatus that detects edge points constituting an edge from an input video, and includes edge detection means, edge tracking means, and model generation. -It was set as the structure provided with an update means and a threshold value calculation means.

  Thus, the edge detection device uses the edge detection unit to detect, for each frame image constituting the video, a pixel having an edge gradient strength exceeding the first threshold and a pixel adjacent to the pixel, the pixel being equal to or lower than the first threshold and A pixel having an edge gradient strength equal to or higher than a second threshold lower than the first threshold is detected as the edge point. Further, the edge detection device tracks the edge point of the previous frame image whose edge gradient strength and edge direction are similar to the edge point from the edge point of the current frame image by the edge tracking means. The edge detection device generates a Gaussian mixture model by distributing the edge points detected by the edge detection means by the model generation / update means, and the edge points of the current frame image successfully tracked by the edge tracking means. Is added to the Gaussian mixture model to update the Gaussian mixture model sequentially for each frame image. The Gaussian mixture model may be generated or updated for each frame image, or may be generated or updated for each predetermined block obtained by dividing the frame image into blocks of a predetermined size.

  In addition, the edge detection device has a weight greater than or equal to a predetermined value from each Gaussian model constituting the Gaussian mixture model generated and updated by the model generation / update unit by the threshold calculation unit, and has the largest weight. The first calculated value, which is a value obtained by subtracting a predetermined number times the standard deviation of the Gaussian model from the average value of the Gaussian model, is a value obtained by dividing the average value by the predetermined value. When the second calculated value is exceeded, the first calculated value is set as the first threshold value and the second calculated value is set as the second threshold value. When the first calculated value is equal to or less than the second calculated value, the second calculated value is set. Is the first threshold and the first calculated value is the second threshold. As described above, in the edge detection process using the Canny operator, the edge detection apparatus converts the first threshold value and the second threshold value that have been fixed in the past into a Gaussian mixture model in which the edge detection results before the previous frame image are learned. Can be determined automatically based on.

  Further, the edge detection device according to claim 2 is the edge detection device according to claim 1, wherein the threshold value calculation means adds a value obtained by multiplying the average value by a predetermined number of times to the average value so that the first detection value is higher than the first threshold value. 3 threshold values are calculated, and the edge detection means includes, for each frame image constituting the video, a pixel having an edge gradient strength equal to or lower than the third threshold value and exceeding the first threshold value, and a pixel adjacent to the pixel, A pixel having an edge gradient strength equal to or lower than the threshold and equal to or higher than the second threshold is detected as an edge point. As described above, the edge detection apparatus can automatically determine the third threshold value having a value larger than the first threshold value based on the Gaussian mixture model in which the edge detection result before the previous frame image is learned.

  The edge detection device according to claim 3 is the edge detection device according to claim 1 or 2, wherein the edge detection unit calculates the first threshold value and the second threshold value when the threshold value calculation unit calculates the first threshold value and the second threshold value. When the edge point is detected using the calculated first threshold value and the second threshold value, and the first threshold value and the second threshold value are not calculated by the threshold value calculation means, the predetermined fixed value is set to the first threshold value and the second threshold value. It was set as the structure which detects as an edge point. As described above, the edge detection device can detect an edge point from a frame image using a predetermined fixed threshold even when the threshold is not calculated by the threshold calculation means.

  An edge detection apparatus according to claim 4 is the edge detection apparatus according to any one of claims 1 to 3, wherein the model generation / update means cannot detect an edge point by the edge detection means. Or if there is no edge point successfully tracked by the edge tracking means, the Gaussian mixture model is discarded, and if the Gaussian mixture model is discarded, the edge detection means sets a predetermined fixed value as the first threshold and The edge point is detected using the second threshold value. As described above, when the edge detection device fails to detect the edge point from the input frame image, or when it cannot track the edge point of the previous frame image similar to the detected edge point, it learns so far. The generated Gaussian mixture model is discarded, and the Gaussian mixture model is generated again by the next processing.

  According to a fifth aspect of the present invention, in the edge detection device according to any one of the first to fourth aspects, the edge tracking means is configured so that the edge tracking means detects a predetermined edge point in the edge point of the current frame image. The edge point of the previous frame image having similar edge gradient strength and edge direction to the edge point is tracked. As described above, the edge detection apparatus can track only a specific edge point to be tracked, instead of tracking all edge points detected from the current frame image.

  In order to solve the above-mentioned problem, an edge detection program according to a sixth aspect of the present invention provides a computer, an edge detection unit, an edge tracking unit, a model generation / update unit, in order to detect an edge point constituting an edge from an input video. The threshold value calculating means is configured to function.

  Thus, the edge detection program uses the edge detection unit to detect, for each frame image constituting the video, a pixel having an edge gradient strength exceeding the first threshold and a pixel adjacent to the pixel, the pixel being equal to or lower than the first threshold and A pixel having an edge gradient strength equal to or higher than a second threshold lower than the first threshold is detected as the edge point. Further, the edge detection program tracks the edge point of the previous frame image whose edge gradient strength and edge direction are similar to the edge point from the edge point of the current frame image by the edge tracking means. Further, the edge detection program generates a Gaussian mixture model by distributing the edge points detected by the edge detection means by the model generation / update means, and the edge points of the current frame image successfully tracked by the edge tracking means. Is added to the Gaussian mixture model to update the Gaussian mixture model sequentially for each frame image. The Gaussian mixture model may be generated and updated for each frame image, or may be generated and updated for each predetermined area obtained by dividing the frame image into predetermined areas.

  The edge detection program has a weight greater than or equal to a predetermined value from the individual Gaussian models constituting the Gaussian mixture model generated and updated by the model generation / update unit by the threshold calculation unit, and the largest weight. The first calculated value, which is a value obtained by subtracting a predetermined number times the standard deviation of the Gaussian model from the average value of the Gaussian model, is a value obtained by dividing the average value by the predetermined value. When the second calculated value is exceeded, the first calculated value is set as the first threshold value and the second calculated value is set as the second threshold value. When the first calculated value is equal to or less than the second calculated value, the second calculated value is set. Is the first threshold and the first calculated value is the second threshold. As described above, in the edge detection process using the Canny operator, the edge detection program converts the first threshold value and the second threshold value that have been fixed in the past into a Gaussian mixture model in which the edge detection results before the previous frame image are learned. Can be determined automatically based on.

  According to the inventions according to claim 1 and claim 6, using the first threshold value and the second threshold value determined based on the Gaussian mixture model in which the edge detection result before the previous frame image is learned, Since edge detection is performed in consideration of the edge detection result, edge points can be accurately detected from each of the frame images constituting the video even if the shooting conditions such as illumination are changed.

  According to the second aspect of the invention, by using the third threshold value as the upper limit value in edge detection, a portion having a high edge gradient strength exceeding the first threshold value (for example, a shadow) is detected as an edge point. Can be suppressed, and the accuracy of edge detection can be further improved.

  According to the third aspect of the present invention, even when the threshold value used in edge detection cannot be calculated, for example, when an edge point is not detected from the previous frame image, the frame image constituting the video Edge points can be detected sequentially.

  According to the fourth aspect of the present invention, since the edge point can be detected from the input frame image and the Gaussian mixture model is generated or updated only when the edge point can be tracked, the edge point before the previous frame image is obtained. The threshold value can be calculated based on the Gaussian mixture model accurately reflected.

  According to the invention of claim 5, since only specific edge points are tracked, the overall processing speed can be improved.

It is a block diagram which shows the whole structure of the edge detection apparatus which concerns on this invention. It is a schematic diagram showing an example of a Gaussian mixture model used in the edge detection device according to the present invention, (a) is a diagram showing a Gaussian mixture model composed of Gaussian models having a weight equal to or greater than a predetermined value; (B) is a figure which shows the Gaussian mixture model comprised by the Gaussian model which has a weight less than a predetermined value. It is the schematic which shows an example of the Gaussian mixture model used with the edge detection apparatus which concerns on this invention, (a) is a figure which shows a Gaussian mixture model in case a 1st calculated value is larger than a 2nd calculated value, (b) ) Is a diagram showing a Gaussian mixture model when the first calculated value is smaller than the second calculated value. It is a flowchart which shows the whole operation | movement of the edge detection apparatus which concerns on this invention. It is the schematic which shows the Example of the edge detection apparatus which concerns on this invention, (a) is a figure at the time of applying edge detection by applying the conventional edge detection apparatus to model-based camera tracking, (b), It is a figure at the time of applying edge detection apparatus concerning the present invention to model base camera tracking, and performing edge detection. It is the schematic which shows the Example of the edge detection apparatus which concerns on this invention, (a) is the enlarged view of the number 1-3 of Fig.5 (a), (b) is the number 1-of FIG.5 (b). 3 is an enlarged view of FIG.

  An edge detection apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the following description, the same configuration is given the same name and symbol, and detailed description is omitted.

  The edge detection device 1 detects edge points that constitute an edge from an input video. The edge detection apparatus 1 is used, for example, when detecting an edge in model-based camera tracking. Here, as shown in FIG. 1, the edge detection apparatus 1 includes a video input means 10, an edge detection means 20, a detection result output means 30, an edge tracking means 40, a model generation / update means 50, and a threshold calculation. Means 60. Hereinafter, each element of the edge detection apparatus 1 will be described in detail.

  As shown in FIG. 1, the video input means 10 is for inputting video from outside. The video input means 10 is a video interface, for example, and outputs the input video to the edge detection means 20 for each frame image constituting the video.

  The edge detection means 20 detects (extracts) edge points for each frame image constituting the video. Here, the basic configuration of the edge detection means 20 is the same as that of the above-mentioned Canny operator. That is, the edge detection means 20 smooths the image by removing noise from the input frame image using a Gaussian filter. Next, the edge detection means 20 performs a differentiation process on the smoothed image using a Sobel filter, and calculates the edge gradient strength and edge direction of each pixel. Next, the edge detection unit 20 identifies a pixel having a maximum edge gradient strength, and sets the pixel as a candidate for an edge point. Then, the edge detection means 20 narrows down the edge point candidates by threshold processing and detects the edge points.

In the threshold processing described above, the edge detection unit 20 detects an edge point using two threshold values, ie, a first threshold value h _th and a second threshold value l _th lower than the first threshold value h _th . The first threshold value h _th is a threshold value for detecting a dark edge point in the frame image, that is, an edge point having a high edge gradient strength. The second threshold value l _th is a threshold value for detecting a thin edge point in the frame image, that is, an edge point having a low edge gradient strength.

The edge detection means 20 first detects a pixel having an edge gradient strength exceeding the first threshold value h _th as an edge point. The edge detection means 20 is a pixel having an edge gradient strength equal to or lower than the first threshold value h _{th and} equal to or higher than the second threshold value l _th, and is adjacent to the pixel detected as an edge point by the first threshold value h _th (connection). Detect a pixel as an edge point. The details of the first threshold value h _th and the second threshold value l _th used in the edge detection means 20 will be described later.

Here, the edge detection means 20 uses the first threshold value h _th and the second threshold value l _th calculated by the threshold value calculation means 60 described later in the threshold value processing described above, or a predetermined fixed first threshold value h _th and The edge point is detected using either the second threshold value l _th (hereinafter referred to as the initial threshold value). For example, when the first threshold value h _th and the second threshold value l _th are calculated by the threshold value calculation means 60 described later, the edge detection means 20 performs an edge detection process using the calculated threshold values.

On the other hand, the edge detection unit 20 performs the edge detection process using the initial threshold when, for example, the first threshold h _th and the second threshold l _th are not calculated by the threshold calculation unit 60 described later. As a case where the first threshold value h _th and the second threshold value l _th are not calculated by the threshold value calculation means 60 in this way, for example, the frame image input from the video input means 10 is the first (first) video image. If the frame and Gaussian mixture model has not been generated yet, the second and subsequent frame images have been input, but the edge point could not be detected in the previous edge detection process and the Gaussian mixture model was discarded For example, the second and subsequent frame images are input, but there are no edge points that have been successfully tracked by the edge tracking unit 40 and the Gaussian mixture model is discarded. As described above, the edge detection apparatus 1 detects an edge point from a frame image using a predetermined fixed threshold (initial threshold) even when the threshold is not calculated by the threshold calculation means 60 described later. Can do. Therefore, as described above, the edge detection apparatus 1 can sequentially detect edge points from the frame images constituting the video even when the threshold used for edge detection cannot be calculated. Details of generation and destruction of the Gaussian mixture model will be described later.

Here, when the third threshold value g _th is calculated in addition to the first threshold value h _th and the second threshold value l _th by the threshold value calculation unit 60 described later, the edge detection unit 20 sets the third threshold value g _th . It is preferable to perform threshold processing using the upper limit value of the edge gradient strength. The third threshold value g _th is a threshold value for suppressing, for example, a portion that is not actually an edge point, such as a shadow, from being detected as an edge point by the first threshold value h _th described above.

In this case, the edge detection unit 20 first detects a pixel having an edge gradient strength equal to or less than the third threshold value g _th and exceeding the first threshold value h _th as an edge point. The edge detection means 20 is a pixel having an edge gradient strength equal to or lower than the first threshold value h _{th and} equal to or higher than the second threshold value l _th, and is adjacent to (connected to) a pixel detected as an edge point by the first threshold value h _th . ) Detect pixels as edge points. As described above, the edge detection apparatus 1 can perform edge detection under more limited conditions by using the third threshold value as the upper limit value in edge detection, thereby further improving the accuracy of edge detection. Can be made. The details of the third threshold value g _th used in the edge detection means 20 will be described later.

  As shown in FIG. 1, frame images constituting a video are sequentially input from the video input unit 10 to the edge detection unit 20. Then, the edge detection means 20 detects edge points from the frame image by the above-described method, and the detection result is output as a detection result output means 30, edge tracking means 40, model generation / update means as shown in FIG. And 50.

  The detection result output means 30 outputs the result of edge detection by the edge detection means 20. As shown in FIG. 1, the edge detection means 20 outputs the detection result input from the edge detection means 20 to, for example, an external personal computer or storage means (not shown).

  The edge tracking means 40 tracks the edge point of the previous frame image (hereinafter referred to as the previous frame image) corresponding to the edge point of the current frame image (hereinafter referred to as the current frame image). Specifically, the edge tracking means 40 compares the edge point of the current frame image with the edge point of the previous frame image, and the edge point of the previous frame image whose edge gradient strength and edge direction are similar to those of the current frame image. To track. Further, the edge tracking means 40 searches for a predetermined range (predetermined length) in the normal direction of the edge constituted by the edge point with reference to the edge point of the current frame image, for example, so that the previous frame image Track edge points. The range in which the edge tracking unit 40 searches in the normal direction of the edge is not particularly limited, but is preferably 30 pixels, for example. Then, the edge tracking means 40 outputs the edge point of the current frame similar to the edge point of the previous frame image, that is, the edge point of the current frame image that has been successfully tracked, as the edge tracking result.

  The edge tracking unit 40 performs edge tracking when the frame image input from the video input unit 10 is the second and subsequent frames and an edge point is detected in the previous edge detection process. On the other hand, the edge tracking means 40 is used when the frame image input from the video input means 10 is the first (first) frame of the video, or when the frame image input from the video input means 10 is the second and subsequent frames. If an edge point has not been detected in the previous edge detection process in the frame, no edge point to be tracked (hereinafter referred to as a tracking target edge) does not exist, and therefore edge tracking is not performed.

  Here, the edge tracking means 40 may track the edge points of the previous frame image having similar edge gradient strength and edge direction to all edge points detected from the current frame image. Tracking may be performed only from a specific edge point of the current frame image. In this case, the edge tracking means 40 is input with a tracking target edge indicating a specific edge point among edge points detected from the current frame image, for example, as indicated by a broken line arrow in FIG. The tracking target edge can be input, for example, when the user directly designates a specific edge point among the edge points detected from the current frame image. Then, the edge tracking unit 40 tracks the edge point of the previous frame image whose edge gradient strength and edge direction are similar to the edge point indicated by the tracking target edge. In this way, the edge detection apparatus 1 can track only a specific edge point to be tracked instead of tracking all the edge points detected from the current frame image, thereby improving the overall processing speed. Can be made.

  As shown in FIG. 1, the edge point of the current frame image is input from the edge detection unit 20 to the edge tracking unit 40. Then, the edge tracking unit 40 tracks the edge point by the above-described method, and outputs the edge point of the current frame image that has been successfully tracked to the model generation / update unit 50.

  The model generation / update unit 50 generates and updates a Gaussian mixture model. Here, the Gaussian mixture model is obtained by modeling the feature of the edge point in the input video, that is, the edge gradient strength. More specifically, the Gaussian mixture model is a distribution of edge gradient strengths of edge points detected from frame images constituting an image.

Here, the edge gradient strength of the edge point detected from the frame image by the edge detection means 20 varies greatly depending on the photographing conditions such as illumination. Accordingly, when the distribution of the edge intensity of the frame image is illustrated, for example, as shown in FIG. 2A, not a simple Gaussian model but a plurality of Gaussian models (here, Gaussian models G ₁ , G ₂ , G ₃ ) becomes a mixed Gaussian mixture model. Note that the Gaussian mixture model shown in FIGS. 2A and 2B can be considered as a result of applying a Sobel filter to a frame image (or a predetermined block thereof as described later) constituting a video.

  In FIGS. 2A and 2B, | g | on the horizontal axis is the edge gradient strength of the edge point, and w on the vertical axis is the weight. The weight w is a parameter that increases as the number of edge points having the same edge gradient strength increases. For example, in FIG. 2A, since the weight w at the point where the gradient strength | g | is 100 is the largest, as a result of the edge detection process before the previous frame image, the edge point where the gradient strength | g | Indicates that most are detected.

  The model generation / update unit 50 generates a Gaussian mixture model by distributing the edge points detected by the edge detection unit 20. More specifically, when an edge point of the current frame image is detected by the edge detection unit 20, the model generation / update unit 50 classifies and counts for each edge gradient strength of the edge point. A Gaussian mixture model is generated by modeling a stack of edge points having the same edge gradient strength.

  Then, when the edge detection unit 20 detects the edge point of the next frame image after the current frame image and the edge tracking unit 40 tracks the edge point, the model generation / update unit 50 performs tracking. Update the Gaussian mixture model by adding the edge points of the successful current frame image to the already generated Gaussian mixture model. Therefore, the Gaussian mixture model is updated each time an edge point is input, and the shape of the mountain shown in FIGS. 2A and 2B changes each time. The model generation / update unit 50 includes a storage unit (not shown) that stores a Gaussian mixture model inside or outside the model generation / update unit 50.

  Here, when the model generation / update unit 50 cannot detect an edge point from the current frame image by the edge detection unit 20 or when there is no edge point successfully tracked by the edge tracking unit 40 Discards a Gaussian mixture model that has already been generated or updated. As described above, the edge detection device 1 generates or updates the Gaussian mixture model by the model generation / update unit 50 only when the edge point can be detected from the input frame image and the edge point can be tracked. The threshold can be calculated based on a Gaussian mixture model that accurately reflects the edge points before the frame image.

  The model generation / update unit 50 may generate a Gaussian mixture model for each frame image in which an edge point is detected, and sequentially update the Gaussian mixture model for each frame image. X It may be divided into blocks having a predetermined height, and a Gaussian mixture model may be generated and updated for each divided block. The number of blocks to be divided is not particularly limited. For example, a frame image is divided into 10 blocks in the vertical direction and 10 blocks in the horizontal direction (for example, when the image size is 640 × 480 pixels, one block is divided into 64 × 48 pixels). It can be performed. The edge detection apparatus 1 can improve the processing speed of edge detection by generating and updating the Gaussian mixture model for each block of the frame image in this way.

  As shown in FIG. 1, the model generation / update unit 50 receives an edge point of the current frame image from the edge detection unit 20 or an edge point that has been successfully tracked from the edge tracking unit 40. Then, the model generation / update unit 50 generates or updates a Gaussian mixture model by the above method, and outputs the generated or updated Gaussian mixture model to the threshold value calculation unit 60.

The threshold value calculation unit 60 calculates the first threshold value h _th and the second threshold value l _th used in the threshold processing in the edge detection unit 20. Specifically, the threshold value calculation means 60 calculates the first threshold value h _th and the second threshold value l _{th according} to the following procedure.

First, when a Gaussian mixture model as shown in FIG. 2A, for example, is generated or updated by the model generation / update unit 50, the threshold value calculation unit 60 includes Gaussian models G ₁ and G constituting the Gaussian mixture model. from the _{G 2,} G _3, selects the Gaussian model _G 1, _{G 3} having _{w min} or more weights w determined in advance. Here, the predetermined w _min is set in a range of 0 to 1, and is set to 0.7 here, for example.

Next, the threshold value calculating means 60, as shown in FIG. 2 (a), if the Gaussian model there are multiple with w _min or more weight w predetermined Gaussian model G ₁ having a higher weight w Select. These processes correspond to selecting the most reliable Gaussian model with the most learned edge points among the Gaussian models constituting the Gaussian mixture model.

Then, the threshold calculating means 60 calculates the first calculated value and the second calculated value by using the average value of the Gaussian model G ₁ selected and the standard deviation. As shown in the following formula (1), the first calculated value can be calculated by subtracting a value obtained by multiplying the Gaussian model standard deviation σ by k from the average value μ of the Gaussian model. The second calculated value can be calculated by dividing the average value μ of the Gaussian model by 2, as shown in the following formula (2).

First calculated value = μ−kσ (1)
Second calculated value = μ / 2 Formula (2)

Then, when the first calculated value exceeds the second calculated value, the threshold value calculation means 60 sets the first calculated value as the first threshold value h _th and sets the second calculated value as the second threshold value l _th . On the other hand, when the first calculated value is less than or equal to the second calculated value, the threshold value calculating means 60 sets the second calculated value as the first threshold value h _th and sets the first calculated value as the second threshold value l _th . These processes can be expressed as, for example, the following equation (3).

If (μ-kσ)> μ / 2
h _th = μ-kσ, l _th = μ / 2
else,
h _th = μ / 2, l _th = μ-kσ (3)

  Here, k in the formulas (1) to (3) is preferably in the range of 1.0 to 2.5, for example, and more preferably 1.5. If k in Expression (3) is less than 1.0, the edge detection condition may be too strict to detect an edge point. If k in Expression (3) exceeds 2.5, the edge detection condition may be Therefore, a pixel that is not an edge point may be detected as an edge point.

On the other hand, when the Gaussian mixture model as shown in FIG. 2B, for example, is generated or updated by the model generation / update unit 50, the threshold value calculation unit 60, the Gaussian model G ₄ , constituting the Gaussian mixture model, since the Gaussian model having a predetermined _{w min} or more weights w in G _{5, G 6} is absent, without selecting the Gaussian model, nor calculated first threshold value _{h th} and the second threshold _{l th.} Therefore, in this case, the edge detection means 20 detects an edge point from the frame image using a predetermined initial threshold value.

Here, it is preferable that the threshold value calculation unit 60 calculates the third threshold value g _th in addition to the first threshold value h _th and the second threshold value l _th . The third threshold value g _th has a value larger than the first threshold value h _th and functions as an upper limit threshold value at the time of edge detection. As described above, the edge detection device 1 uses the third threshold value g _th as the upper limit value in edge detection, so that a portion having a high edge gradient strength exceeding the first threshold value h _th (for example, a shadow or the like) is an edge point. Can be suppressed and the accuracy of edge detection can be further improved.

As shown in FIG. 3A, the threshold value calculation means 60 has a weight w _{equal to} or greater than a predetermined w _min from the Gaussian mixture model generated or updated by the model generation / update means 50, and After selecting the Gaussian model G ₁ having the highest large weight w, as shown in the following equation (4), the average value μ of the Gaussian model G ₁ is added to the standard deviation σ of the Gaussian model multiplied by k. The third threshold value g _th is calculated. In the following formula (4), k is preferably in the range of, for example, 1.0 to 2.5, and more preferably 1.5, as in the above formulas (1) and (3). .

g _th = μ + kσ (4)

The first threshold value h _th , the second threshold value l _th, and the third threshold value g _th calculated by the threshold value calculation means 60 are, for example, a case where the first calculated value exceeds the second calculated value, and k is 1. When set to 5, it can be illustrated as shown in FIG. On the other hand, the first threshold value h _th , the second threshold value l _th, and the third threshold value g _th calculated by the threshold value calculation means 60 are cases where the first calculated value is less than or equal to the second calculated value, and k is When set to 1.5, for example, it can be illustrated as shown in FIG. Thus, when the first calculated value is equal to or smaller than the second calculated value, for example, as shown in FIG. 3B, the Gaussian model has a wide mountain and “μ−kσ” is “μ / 2”. It means the case where it becomes below.

As shown in FIG. 1, a Gaussian mixture model is input from the model generation / update unit 50 to the threshold calculation unit 60. Then, the threshold value calculation unit 60 calculates the first threshold value h _th , the second threshold value l _th, and the third threshold value g _th by the above-described method, and outputs them to the edge detection unit 20.

In the edge detection apparatus 1 having the above-described configuration, in the edge detection process using the Canny operator, the first threshold value h _th and the second threshold value l _th that have been fixed in the past are used as the edge detection results before the previous frame image. Can be automatically determined based on the Gaussian mixture model learned. Therefore, according to the edge detection apparatus 1, using the first threshold value h _th and the second threshold value l _th determined based on the Gaussian mixture model in which the edge detection result before the previous frame image is learned, Since edge detection is performed in consideration of the edge detection result, edge points can be accurately detected from each of the frame images constituting the video even if the shooting conditions such as illumination are changed.

  Moreover, the edge detection apparatus 1 can construct a highly accurate and robust system by incorporating it into an algorithm relating to automatic processing using the edge of a video. For example, even when the edge detection device 1 is used to detect an edge of a product in a production line, the edge detection device 1 can accurately detect the edge without strictly restricting the lighting environment or the like.

  Hereinafter, the operation of the edge detection apparatus 1 will be briefly described with reference to FIG. In the following description, for convenience of explanation, the description will be made on the assumption that the edge detection apparatus 1 can always detect an edge point from a frame image.

First, when a video is input to the video input unit 10 from, for example, an external video storage unit (not shown) (Yes in step S1), the edge detection device 1 uses the edge detection unit 20 to form a current frame image that constitutes the video. An edge point is detected from (step S2). At that time, the edge detection means 20 detects an edge point using a predetermined initial threshold value as the first threshold value h _th and the second threshold value l _th .

  Next, the edge detection apparatus 1 determines whether there is a tracking target edge to be tracked by the edge point detected from the current frame image by the edge tracking unit 40 (step S3). When there is no tracking target edge (No in step S3), the edge detection device 1 generates a Gaussian mixture model by the model generation / update unit 50 (step S4). The case where there is no tracking target edge includes the case where the current frame image is the first (first) frame of the video as described above.

Next, the edge detection apparatus 1 _adds a weight w _{equal to} or greater than a predetermined w _min to the individual Gaussian models constituting the Gaussian mixture model generated by the model generation / update unit 50 by the threshold calculation unit 60. It is determined whether or not there is a Gaussian model possessed (step S5). Then, when there is a Gaussian model having a weight w greater than or equal to the predetermined w _min (Yes in step S5), the edge detection device 1 uses threshold values (first threshold value h _th , second threshold value l _th, and third threshold value). The threshold value g _th ) is calculated (step S6), and the process returns to step S1.

  On the other hand, if there is an edge to be tracked in step S3 (Yes in step S3), the edge detection apparatus 1 tracks the edge point of the previous frame image from the edge point of the current frame image by the edge tracking unit 40, and An edge point of the current frame that is similar in edge gradient strength and edge direction to the edge point of the frame image is detected (step S7). Next, the edge detection apparatus 1 determines whether there is an edge point that has been successfully tracked by the model generation / update unit 50 (step S8). Then, when there is an edge point that has been successfully tracked (Yes in step S8), the edge detection device 1 updates the already generated Gaussian mixture model by the model generation / update unit 50 (step S9), and then proceeds to step S5. move on.

  On the other hand, if there is no edge point that has been successfully tracked (No in step S8), the edge detection device 1 discards the already generated Gaussian mixture model by the model generation / update unit 50 (step S10), and the process proceeds to step S1. Return.

  The edge detection apparatus 1 that performs the operation as described above generates or sequentially updates a Gaussian mixture model in which the edge detection results before the previous frame image are learned, and based on the Gaussian mixture model, the edge detection results before the previous frame Therefore, the edge point can be accurately detected from each of the frame images constituting the video even if the photographing condition such as illumination is changed.

[Edge detection program]
Here, the edge detection apparatus 1 described above can be realized by operating a general computer with a program that functions as each of the above-described means. This program can be distributed via a communication line, or can be written on a recording medium such as a CD-ROM for distribution.

  Examples for confirming the effects of the present invention will be described below. In this embodiment, the edge detection device according to the present invention (hereinafter referred to as the present method) and the edge detection device according to the prior art (hereinafter referred to as the conventional method) are respectively applied to the edge detection of model-based camera tracking. A comparison was made. Here, model-based camera tracking is to estimate a camera posture by tracking edge points having a three-dimensional shape in real space with a camera-captured video and comparing them with a three-dimensional model prepared in advance. That is, in model-based tracking, an edge point in a camera-captured video is detected, and the detected posture is matched with an edge point of a three-dimensional model prepared in advance to estimate the camera posture at the time of shooting.

Here, in the present embodiment, in order to reproduce the conventional method, in the edge detection process of model-based camera tracking, edge detection is performed using a fixed first threshold value h _th (= 200) and second threshold value l _th (= 100). Went. Further, in order to reproduce this method, in the edge detection process of model-based camera tracking, edge detection is performed using the first threshold value h _th and the second threshold value l _th calculated from a Gaussian mixture model generated in advance.

Next, by matching the edge points obtained by the conventional method and the present method with the corresponding edge points of the three-dimensional model prepared in advance, the camera posture information (three translation parameters t _{1 to} t ₃ , Three rotation parameters r _{1 to} r ₃ ) were calculated. Then, the camera posture information (three translation parameters t _{1 to} t ₃ and three rotation parameters r _{1 to} r ₃ ) as a reference, which has been estimated in advance using the C language library “ARTToolKit”, and the calculation The attitude error of the camera was obtained by calculating the difference from the attitude information of the camera. At the same time, the temporal change (standard deviation) of the camera posture error was also calculated. The results are shown in Table 1. “Error” in Table 1 indicates the attitude error of the camera described above, and “jitter” indicates a temporal change in the attitude error of the camera described above.

  As shown in Table 1, in the conventional method, the average error of the translation parameter and the rotation parameter is 4.265 mm and 0.474 degrees, respectively, and the average jitter of the translation parameter and the rotation parameter is 1.403 mm / frame, 0.183 ° / frame. On the other hand, in this method, the average error of the translation parameter and the rotation parameter is 3.919 mm and 0.409 degrees, respectively, and the average jitter of the translation parameter and the rotation parameter is 1.289 mm / frame and 0.166, respectively. ° / frame.

  In this way, in model-based camera tracking, if edge detection is performed using this method, edge points can be accurately detected from camera-captured images, so errors and jitter may be reduced compared to conventional methods. Recognize. Further, in the conventional method, when matching an edge point detected from a camera-captured video image and an edge point of a three-dimensional model prepared in advance, 1.8998 corresponding (similar) edge point candidates on average. However, in this method, the average number was 1.793. Therefore, it can be seen that when edge detection is performed using this method, the possibility of tracking an incorrect edge point is reduced as compared with the conventional method.

  Next, FIGS. 5 and 6 show actual images obtained by performing edge detection by the present method and the conventional method. Here, the upper right diagram in FIG. 5A shows an original image in which edge detection is performed by the present method and the conventional method. FIGS. 6A and 6B show the enlarged numbers 1 to 3 in FIGS. 5A and 5B, respectively. In addition, the empty circle and the filled circle drawn along the edges in FIGS. 5B and 6B are the edge points of the three-dimensional model prepared in advance and the edge points detected by the present method, respectively. The short line segment passing through the two circles indicates the tracking range of the edge point detected by this method.

Referring to the edge detection result according to the conventional method, it can be seen that the shadow portion is erroneously detected as an edge, as indicated by reference numerals e ₁ and e ₂ in FIG. Further, as indicated by reference numeral e ₃ of FIG. 6 (a), it is found that it is detected as an edge incorrectly moieties not an edge.

On the other hand, referring to the edge detection result obtained by this method, as shown in FIG. 6B, it can be seen that a shadow portion or a non-edge portion is not detected as an edge. Comparing the edges B ₁ and B ₂ at the edge of the box in FIGS. 6A and 6B, many other edges are detected around the edge B _{1 in} FIG. 6A. another edge to the edge B ₂ around in FIG 6 (b) is not detected. Therefore, in model-based camera tracking, for example, when a three-dimensional model of this box is prepared in advance and tracking is performed, it is compared with the conventional method in which other useless edges are detected in the vicinity (see FIG. 6A). Thus, it can be seen that the present technique (see FIG. 6B) in which no other useless edges are detected in the periphery is superior in tracking accuracy.

  The edge detection device according to the present invention has been specifically described above with reference to the embodiments for carrying out the invention. However, the gist of the present invention is not limited to these descriptions, and is based on the descriptions in the claims. Must be interpreted widely. Needless to say, various changes and modifications based on these descriptions are also included in the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Edge detection apparatus 10 Image | video input means 20 Edge detection means 30 Detection result output means 40 Edge tracking means 50 Model production | generation / update means 60 Threshold calculation means

Claims (6)

An edge detection device for detecting edge points constituting an edge from input video,
For each frame image constituting the video, a pixel having an edge gradient strength exceeding a first threshold and a pixel adjacent to the pixel, the pixel being equal to or lower than the first threshold and lower than the second threshold lower than the first threshold Edge detection means for detecting pixels having edge gradient strength as the edge points;
Edge tracking means for tracking the edge point of the previous frame image whose edge gradient strength and edge direction are similar to the edge point from the edge point of the current frame image;
The Gaussian mixture model is generated by distributing the edge points detected by the edge detection unit, and the edge point of the current frame image that has been successfully tracked by the edge tracking unit is added to the Gaussian mixture model. Model generation / update means for sequentially updating the mixed model for each frame image;
A Gaussian model having a weight greater than or equal to a predetermined value and having the largest weight is selected from the individual Gaussian models constituting the Gaussian mixture model generated and updated by the model generation / update means, and the Gaussian is selected. When the first calculated value that is a value obtained by subtracting the standard deviation of the Gaussian model from the average value of the model by a predetermined number exceeds the second calculated value that is a value obtained by dividing the average value by the predetermined value, When the first calculated value is the first threshold value and the second calculated value is the second threshold value, and the first calculated value is less than or equal to the second calculated value, the second calculated value is the second threshold value. Threshold calculation means for setting the first calculated value as the second threshold and the first calculated value;
An edge detection apparatus comprising: The threshold value calculation means calculates a third threshold value higher than the first threshold value by adding a value obtained by multiplying the average value by a predetermined number of times to the average value,
The edge detection means includes, for each frame image constituting the video, a pixel having an edge gradient strength that is equal to or less than the third threshold and exceeds the first threshold, and a pixel adjacent to the pixel, the first threshold The edge detection apparatus according to claim 1, wherein a pixel having an edge gradient strength equal to or greater than the second threshold is detected as the edge point.   When the first threshold value and the second threshold value are calculated by the threshold value calculation means, the edge detection means detects the edge point using the calculated first threshold value and the second threshold value, When the threshold value calculation unit does not calculate the first threshold value and the second threshold value, the edge point is detected using a predetermined fixed value as the first threshold value and the second threshold value. The edge detection apparatus according to claim 1 or 2. The model generation / update unit discards the Gaussian mixture model when the edge point cannot be detected by the edge detection unit or when there is no edge point successfully tracked by the edge tracking unit,
The edge detection unit detects the edge point by using a predetermined fixed value as the first threshold and the second threshold when the Gaussian mixture model is discarded. The edge detection apparatus according to claim 3.   The edge tracking means is configured to track an edge point of a previous frame image having a similar edge gradient strength and edge direction from an edge point designated in advance in an edge point of a current frame image. The edge detection apparatus as described in any one of Claims 1-4. In order to detect the edge points that make up the edge from the input video,
For each frame image constituting the video, a pixel having an edge gradient strength exceeding a first threshold and a pixel adjacent to the pixel, the pixel being equal to or lower than the first threshold and lower than the second threshold lower than the first threshold Edge detection means for detecting pixels having edge gradient strength as the edge points;
Edge tracking means for tracking the edge point of the previous frame image whose edge gradient strength and edge direction are similar from the edge point of the current frame image,
The Gaussian mixture model is generated by distributing the edge points detected by the edge detection unit, and the edge point of the current frame image that has been successfully tracked by the edge tracking unit is added to the Gaussian mixture model. Model generation / update means for sequentially updating the mixed model for each frame image,
A Gaussian model having a weight greater than or equal to a predetermined value and having the largest weight is selected from the individual Gaussian models constituting the Gaussian mixture model generated and updated by the model generation / update means, and the Gaussian is selected. When the first calculated value that is a value obtained by subtracting the standard deviation of the Gaussian model from the average value of the model by a predetermined number exceeds the second calculated value that is a value obtained by dividing the average value by the predetermined value, When the first calculated value is the first threshold value and the second calculated value is the second threshold value, and the first calculated value is less than or equal to the second calculated value, the second calculated value is the second threshold value. Threshold calculation means for setting the first calculated value as the second threshold and the first calculated value;
Edge detection program characterized by functioning as

Images