JPH01184588A - System for corresponding to closed curve by dp matching - Google Patents

Abstract

PURPOSE:To apply DP matching also on a closed curve by utilizing the fact that on open curve can be formed by cutting the closed curve representing a mobile body, etc., in a moving image at a certain point, and deciding the similarly of a feature point extracted on the open curve. CONSTITUTION:The contour of the mobile body is extracted from the area of a targeted mobile body in the moving image by a first means 2. The feature point is extracted by a second means 3 based on the amount of change of the density level of an image element on an extracted contour. The similarity between the local irregular state of an extracted feature point and the state change of the density level is decided by a third means 4. By those first-third means 2-4, the cut point of the closed curve representing the mobile body in the moving image is extracted, and by applying the DP matching 5 on the cut point assuming as a point whose correspondence is established, the correspondence of the closed curve representing the mobile body can be performed.

Description

[Detailed Description of the Invention] [Summary] For example, optical flow is extracted by associating closed curves indicating a moving object, etc. in a moving image, and the translation, expansion, contraction, etc. of the object in the moving image are extracted. The purpose of this paper is to apply OP matching, which could only be applied to closed curves, to closed curves, for example, to accurately extract the optical flow of a moving object in a moving image. Time t from the target moving object area in
Extract the contour (closed curve) of the moving object at tjtz, extract feature points from the density level of pixels on the contour using Laplacian (second-order differential), and calculate the local unevenness of the plurality of extracted feature points. Based on the similarity between the state and the change state of the density level, the most similar painting is extracted as a cutting point, and this cutting point is used as the point where the pairing is determined, and by performing DP matching processing, the two The optical flow of the moving object in the moving image is extracted by selecting the corresponding column of pixels that are most similar among the two closed curves.

[Industrial application field]

The present invention extracts optical flow by, for example, associating closed curves indicating moving objects etc. in a moving image,
The present invention relates to a method for automatically recognizing translation, rotation, expansion, contraction, etc. of a moving object in a moving image.

With the recent spread of television cameras and television receivers, it has become possible to obtain economical input/output devices for moving images. There is a growing need for automatic recognition of the movements of specific objects inside.

For example, in order to recognize the translation, expansion, contraction, etc. of a specific object in a visual system of a robot, it is required to accurately determine the optical flow of the specific object in the moving image.

[Prior Art and Problems to be Solved by the Invention] FIG. 4 is a diagram illustrating a conventional method for associating closed curves.

In the field of computer vision, the problems associated with the correspondence between two images are the correspondence of epipolar lines in binocular stereoscopic vision, and the process of extracting the optical flow of a specific object (object movement) in video processing. This is a process of grasping translation (9 rotations, expansion, contraction, etc.).

Among these, in binocular stereopsis, the two end points of the left and right epipolar lines correspond to each other on the left and right, and matching is performed using the well-known "rDP matching".

(For example, “Stereo image interval correspondence method using dynamic programming”, Transactions of the Institute of Electronics and Communication Engineers, 1985. VOL, 4
．． J68-D, 1lh4, pages 554-561)
However, in moving image processing, in order to associate each object in the moving image, it is necessary to associate closed curves, that is, the outlines of objects.

This figure shows the conventional mapping method for closed curves, in which the contour extraction unit 2 subtracts the image for object 1 at time t2 from the image for object 1 at time 1° to create a background image. is deleted, only the image of the moving object is extracted, and the two extracted images are subjected to binarization processing to extract the outline of the moving object.

Next, in the feature point extraction unit 3, the extracted contour (
The Laplacian, that is, the second derivative of the density change on the closed curve, is determined, and the feature points of the closed curve are extracted.The next correspondence is made by determining the optical flow for the extracted feature points in the processing unit 6. , since the motion mode of the moving object, such as translation and one rotation, was determined, there was a problem in that it did not correspond to all pixels of the closed curve, resulting in a lack of accuracy.

In view of the above-mentioned conventional drawbacks, the present invention extracts an optical flow by associating closed curves indicating moving objects etc. in a moving image, and extracts one rotation of translation of the object in the moving image.
In the method of automatically recognizing expansion, contraction, etc., originally F
DP matching" has the lowest cost (8 correspondence errors between two points is small) to transition from one fixed state (that is, where the matching is fixed) to another fixed state. It is a method to search for a path (correspondence) ° that is the most similar to two closed curves. It is an object of this invention to provide a method for accurately associating each pixel of a closed curve.

[Means for solving problems]

FIG. 1 is a diagram illustrating the principle of a method for associating closed curves by DP matching according to the present invention.

The above problem can be solved by a method for matching closed curves using DP matching configured as follows.

For example, a method extracts an optical flow by associating closed curves indicating a moving object, etc. in a moving image, and extracts the outline of the moving object from the region of the moving object in the moving image. a first means 2; a second means 3 for extracting a feature point based on the amount of change (secondary differential) in the density level of the pixel on the extracted contour; A third means 4 for determining the similarity between the uneven state and the change state at the '/a degree level is provided, and the cutting point of the closed curve is extracted from the first means 2 to the third means 4, By performing DP matching 5 using this cutting point as a point for which correspondence has been determined, the system is configured to perform correspondence between closed curves representing moving objects in the moving image.

[Effect]

That is, according to the present invention, for example, the optical flow is extracted by associating closed curves indicating a moving object, etc. in a moving image, and automatic processing such as translation, rotation, expansion, contraction, etc. of the object in the moving image is performed. To perform recognition, we use the fact that the closed curve becomes a closed curve when cut at a certain point, and the minute intervals of feature points extracted by Laplacian (second-order differential) etc. on the closed curve are the most similar. Select a point, use that point as a point for which the correspondence for performing DP matching processing has been established, and perform DP matching processing in the same way as for the closed curve.
Find the correspondence between the most similar pixel columns of the two closed curves,
Since it is designed to determine the optical flow of a moving object, DP, which could previously only be applied to closed curves,
Matching can now be applied to closed curves, and has the effect of, for example, allowing accurate optical flow extraction for moving objects in video processing.

〔Example〕

Embodiments of the present invention will be described in detail below with reference to the drawings.

The above-mentioned FIG. 1 is a diagram showing the principle of the method of matching closed curves by DP matching of the present invention, and FIG. 2 is a diagram showing an embodiment of the present invention, and (a) shows an example of the configuration. (b) shows a method for detecting irregularities, and FIG. 3 is a processing diagram for associating closed curves according to the present invention. This is a necessary measure. Note that the same reference numerals indicate the same objects throughout the figures.

Hereinafter, with reference to FIG. 1 and FIGS. 2 and 3,
A method for associating closed curves using DP matching according to the present invention will be explained.

First, in the object contour extraction unit 2, for example, two different images at time t+ and h in a moving image are extracted at time 1. After subtracting the image at time t2 from the image of , the background image is deleted and the extracted moving object (each is referred to as the object at time t1 + the object at time F) 1 is extracted, and each image is The image is binarized and scanned using an 8-connected or 4-connected window (not shown) to extract pixels whose density values have changed, that is, contours. (Figure 3)
(See a) and (b)) Next, the contour pixel string data extracted from each image is temporarily stored in the object contour pixel string section 31 of the feature point extraction section 3, and then the stored data is used. Then, along the contour, a feature point extraction unit 32 using a -dimensional Laplacian extracts feature points (for example, pixels whose density values have changed) using a one-dimensional Laplacian (-dimensional quadratic differential). ((c in Figure 3)
), the pixels indicated by black dots) Next, the geometrical unevenness near the extracted feature point is examined in the unevenness determination unit 43 near the feature point.

The geometric unevenness can be detected using the method shown in FIG. 2(b), for example, by scanning with a 3×3 window.
Perform expansion filter treatment ■ or contraction filter treatment ■,
By looking at the difference from the original image, concavities or convexities can be detected.

Based on the data from the contour pixel array section 31 of the object and the feature point data section 42, the pixel where the unevenness of the feature point and the local shading change around the feature point are most compatible is determined.
The cutting point is selected by the cutting point determining unit 41 and is set as a point at which the correspondence between the outlines of the two objects has been determined. (Refer to the pixels indicated by x in FIG. 3(d)) Finally, the contour pixel string of the moving object is cut at the cutting point, and the DP matching section 5 performs a known DP matching process to combine the two contours. The motion of the moving object is recognized by finding the most similar pixel row (that is, the least correspondence error) for the pixel rows that make up the pixel row, and finding the optical flow between the corresponding pixels. do.

Note that in the above embodiment, when recognizing the movement of a moving object in a moving image, the outlines of two images taken at different times are extracted and the outlines are associated with pixels. However, in view of the gist of the present invention, the present invention is not limited to mapping to moving objects in moving images;
It goes without saying that this method can also be applied to the correspondence between two images of the same object.

As described above, the present invention extracts an optical flow by associating closed curves indicating a moving object, etc. in a moving image, and automatically performs translation, rotation, expansion, contraction, etc. of the object in the moving image. To perform recognition, we focus on the fact that a closed curve becomes a closed curve by cutting it at a certain point, find the outline of a moving object, and find out if the tops of multiple feature points on the outline are also similar. The feature is that the feature point is detected, and DP matching is performed using that point as a point with a confirmed correspondence, and the correspondence is made for each pixel to obtain an accurate optical flow of the moving object.

〔Effect of the invention〕

As explained above in detail, the method of associating closed curves by DP matching of the present invention is, for example, by associating closed curves indicating moving objects in a moving image and extracting optical flows. One rotation of the translation of the object in the image,
In order to automatically recognize expansion, contraction, etc., we utilize the fact that the closed curve becomes a closed curve if you cut it at a certain point, and we use a micro section of feature points extracted by Laplacian (second order differentiation) etc. on the closed curve. Select the point that is most similar, and use that point as the point for which the correspondence for performing DP matching processing has been established.From then on, DP matching processing is performed in the same way as for closed curves, and the most similar point of the two closed curves is selected. Since this method calculates the optical flow of a moving object by determining the correspondence between pixel rows that are This has the effect of making it possible to accurately extract optical flow for a moving object in ° processing.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing the principle of matching closed curves using DP matching according to the present invention. FIG. 2 is a diagram showing an embodiment of the present invention. FIG. 3 is a processing diagram for mapping closed curves according to the present invention. FIG. 4 is a diagram illustrating a conventional method for associating closed curves. It is. In drawings, l is an image or an object. 2 is an object contour extraction section or a contour extraction section. 3 is a feature point extraction unit. 31 is a contour pixel row part of the object. 32 is a feature point extraction unit using one-dimensional Ubrasian. 4 is a cutting point extraction part. 41 is a cutting point determining section, and 42 is a feature point data section. 43 is a determination unit for determining unevenness near the feature point. 5 is the DP matching section. 6 is a processing section with correspondence. ■ is expansion filter processing, ■ is contraction filter processing. are shown respectively. Small\light color DP pine marten 7-f; 1
Dimensions: 17 ka J~ Power formula 1,000 T shea 1,000 1st mouth (

Claims (1)

[Claims] A method for extracting an optical flow by associating at least two closed curves representing a moving object, etc. in two images, and extracting an optical flow from a target object region in the images. (1) A first means (2) for extracting the contour; and a second means (3) for extracting feature points based on the amount of change (secondary differential amount) in the density level of pixels on the extracted contour. and a third means (4) for determining the similarity between the local unevenness state of the extracted feature point and the change state of the density level, and the first means (2) to the third means Means (4) and above 2
By extracting the cutting points of the two closed curves and performing the DP matching (5) process using these cutting points as points for which the correspondence has been established, the correspondence between the two closed curves representing moving objects, etc. in the two images can be determined. A closed curve matching method using DP matching.