JP2000276596A - Automatic tracking device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an automatic tracking device capable tracking with high accuracy without limiting the shape of a tracking object. SOLUTION: The template update judging circuit 19 of this automatic tracking device inputs a tracking coordinate signal 17 and a template coordinated signal 23 of the preceding frame and performs update decision of a template on the basis of the change of positional relation between frames. There are switching of the template to another one, changing of template sizes, updating of the template and not updating the template as update decision results of the template. A template update circuit 22 updated the template in accordance with an update judging result signal 20 from the circuit 19.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic tracking device which is used for tracking an object which moves relatively and which is mounted on, for example, a flying object and used when the flying object lands on the ground surface or the like. .

[0002]

2. Description of the Related Art In order for an unmanned spacecraft to land on the surface of an unknown celestial body such as an asteroid, it is necessary to detect the relative speed between the surface of the celestial body and the spacecraft and control the landing operation so as to cancel the speed. In order to know the relative speed between the celestial body surface and the spacecraft, an automatic tracking device that automatically finds a characteristic region of the earth surface such as a crater and tracks the region with high accuracy is required.

A conventional tracking device is disclosed in, for example,
Japanese Patent No. 334682 discloses an apparatus for correcting the position of a template image with respect to a tracking target object to eliminate a cumulative tracking error. FIG. 19 is a block diagram showing an example of such an automatic tracking device.

In the automatic tracking device shown in FIG. 19, an object to be tracked is picked up by an image pickup device 68, and an image (analog video signal) 69 is sent to an AD converter 70. And
The digital video signal 71 is stored in the frame memory 72 for each screen. The template setting circuit 74 outputs the digital image signal 73 output from the frame memory 72.
To set the template image. Correlation operation circuit 76
Performs a correlation operation between the template image signal 75 and the digital image data of the next frame, and outputs the obtained correlation image signal 77 to the minimum value search circuit 78.

[0005] The minimum value search circuit 78 outputs a correlation image signal 77.
Search for the minimum value of the signal value at. Then, the coordinate position indicating the minimum value is output as the tracking coordinate signal 79. The straight line detection circuit 82 detects the contour of the tracking target object based on the tracking coordinate signal 79, and outputs a four-sided straight line group parameter signal 83 representing the contour. The vertex detection circuit 84 detects the vertex coordinates of the tracking target from the four-side straight line group parameter signal 83 representing the contour, and sends a vertex coordinate signal 85 to the template update circuit 80. The template updating circuit 80 corrects the position of the template based on the vertex coordinates.

By performing the above processing periodically, even if the set position of the template image is deviated from the object to be tracked, the position of the template image is corrected. Can be performed.

[0007]

However, in the conventional automatic tracking apparatus described above, the position of the template is corrected by moving the center of the template to the vertices of the polygon by the straight line detection circuit 82 and the vertex detection circuit 84. ing. Then, the tracking target is limited to a polygon, and cannot be applied to a circle or another complicated shape. Therefore, when the unmanned spacecraft lands on an unknown celestial body, it becomes inoperable if there is no polygonal ground feature point on the celestial body surface.

Further, the template setting circuit 74 is not provided with a function for automatically detecting the vertices of a polygon, so that the template in the first frame cannot be automatically set. Therefore, in the case of an unmanned spacecraft, it is necessary to set a template by remote manual operation. Then, there is a possibility that the tracking target has disappeared off the screen due to a delay in the communication time.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and provides an automatic tracking device capable of performing accurate tracking without limiting the shape of a tracking target. With the goal.

[0010]

SUMMARY OF THE INVENTION The present invention relates to an automatic tracking device for tracking an object using a captured image, and particularly to an object for tracking an object having an unknown shape. Automatically set and track multiple tracking targets without using the geometrical features of the objects, and determined the motion of the image based on the inter-frame variation of the positional relationship of the multiple tracking targets. The object is accurately tracked by updating the template.

The automatic tracking apparatus according to the first aspect of the present invention provides a template generating means for setting a plurality of templates in a captured image, and a template based on a change in a positional relationship between frames of tracking coordinates set in each template. Template update determining means for determining whether or not to update, template update means for outputting each template image according to the determination result of the template update determining means,
The configuration includes a correlation calculating unit that calculates a correlation between each template image and a captured image of a next frame, and a maximum value searching unit that outputs a coordinate having a maximum correlation value in each template as a tracking coordinate signal.

[0012] The template generating means includes a feature extracting circuit for outputting, as a template candidate, an area including a portion in which the change in shading exceeds a predetermined value in the captured image; And a new template generation circuit using the object as a template.

The template update judging means inputs a template coordinate signal indicating the coordinates of each template from the template updating means, inputs a maximum correlation value and a tracking coordinate signal from the maximum value searching means, and outputs the template coordinate signal and the maximum correlation value. It may be configured to determine whether the template is updated, deformed, or maintained using the tracking coordinate signal.

The template update judging means calculates the positional relationship of each tracking target obtained from the tracking coordinate signal, calculates the positional relationship of each tracking target obtained from the template coordinate signal, and determines the current frame and the current frame based on each calculation result. Calculate the amount of change in the positional relationship of each tracking target with the previous frame and update the template from the amount of change in the positional relationship,
It may be configured to determine whether to enlarge or reduce or maintain.

The template updating means inputs a new template coordinate signal indicating the coordinates of each template from the template generating means, and a tracking area coordinate signal indicating an area including the tracking target from the template updating determining means.
An update determination result signal and an enlargement ratio signal indicating a template enlargement / reduction ratio are input. If the update determination result signal indicates an update, a new template image is generated according to the new template coordinate signal, and the update determination result is output. If the signal indicates enlargement or reduction, the template image is enlarged or reduced according to the enlargement ratio, and if the update determination result signal indicates that the signal is not updated, the template image according to the tracking area coordinate signal is used. It may be configured to output.

The automatic tracking device may be configured so that the correlation calculating means performs a normalized correlation process.

Further, the automatic tracking device may include a pre-processing unit for performing a contrast enhancement process on an image output from the imaging unit.

The automatic tracking device may be provided with a result output circuit for outputting the tracking coordinate signal from the maximum value searching means as a result, or the template coordinate signal output from the template updating means and the template updating judging means. And a result output circuit for outputting the result of the update determination result signal output from the CPU.

[0019]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a first embodiment of the automatic tracking device according to the present invention. In FIG. 1, an object to be tracked is imaged by an imaging device 1 and an image is
-Sent to the D converter 3. Then, the digital video signal 4 is stored in the frame memory 5 for each screen.

The pre-processing circuit 7 reads the digital image signal 6 from the frame memory 5 and adjusts the state of the image.
The feature extraction circuit 9 receives the pre-processed image signal from the pre-processing circuit 7 and automatically changes the shading in the image in order to automatically set the tracking target and to supplement the template when tracking is lost. The intense portion is set as a candidate for a feature point to be tracked, and the coordinates of the candidate on the screen are output as a label data signal 10 to the new template generating circuit 11.

The new template generating circuit 11 receives the label data signal 10 and outputs the feature points of a size satisfying the condition in descending order. The correlation calculation circuit 13 performs a correlation calculation between the template image signal 24 output from the template update circuit 22 and the pre-processed image signal 8 to perform tracking, and converts the correlation image signal 14 into a maximum value search circuit 15.
Send to The maximum value search circuit 15 searches for the maximum value of the correlation value, and sends the maximum correlation value signal 16 to the result output circuit 25.

Further, in order to increase the tracking accuracy, the template update determining circuit 19 receives the tracking coordinate signal 17 and the template coordinate signal 23 of the previous frame, and determines whether to update the template based on the change in the positional relationship between the frames. Do. As a result of the template update determination, there are such things as switching the template to another one, changing the template size, updating the template, and not updating the template as it is. The template update circuit 22 updates the template according to the update determination result signal 20 from the template update determination circuit 19.

More specifically, the image pickup apparatus 1 picks up an external image including a tracking target object, and outputs an image (analog video signal) 2 to the AD converter 3. The A / D converter 3 converts the analog video signal 2 into a digital video signal 4
And outputs it to the frame memory 5. The frame memory 5 stores the digital video signal 4 in units of one screen,
A pre-processing circuit 7 for each screen as a digital image signal 6
Output to The pre-processing circuit 7 receives the digital image signal 6 and performs contrast enhancement and noise removal processing to adjust the state of the image. Then, the pre-processed image signal 8 as a processing result is converted into a feature extraction circuit 9 and a correlation operation circuit 13.
And outputs it to the template update circuit 22.

The feature extraction circuit 9 receives the pre-processed image signal 8, automatically searches for a plurality of sharply changing portions in the image, that is, edge portions, and uses the coordinates on the screen as a label data signal 10 as a new template. Output to the generation circuit 11. The new template generation circuit 11 receives the label data signal 10 and selects only one having an appropriate size. Further, the coordinates are rearranged in the size order, and the coordinates are output to the template updating circuit 22 as a new template coordinate signal 12.

The correlation operation circuit 13 performs a correlation operation between the pre-processed image signal 8 from the pre-processing circuit 7 and the template image signal 24 from the template update circuit 22, and searches the obtained correlation image signal 14 for the maximum value. Output to the circuit 15.
The maximum value search circuit 15 finds the maximum value in the correlation image signal 14 and outputs the maximum value to the template update determination circuit 19 as the maximum correlation value signal 16. The center coordinates at the maximum are output to the template update determination circuit 19 and the result output circuit 25 as the tracking coordinate signal 17, and the circumscribed coordinates at the maximum are output to the template update circuit 22 as the tracking area coordinate signal 18. Output.

The template update determination circuit 19 receives the maximum correlation value signal 16 and the tracking coordinate signal 17 from the maximum value search circuit 15 and the template coordinate signal 23 from the template update circuit 22, and updates the template. Judge whether to do. Then, an update determination result signal 20 is output to the template update circuit 22.

The function of the template update determination circuit 19 is as follows.
First, when the input maximum correlation value is small, it is determined that the tracking reliability is low, and a determination to switch the tracking target to another part, that is, a determination to switch the template is made. Second, the positional relationship of each tracking target obtained from the tracking coordinates is calculated, the positional relationship of each tracking target obtained from the template coordinates is calculated, and the change amount of the positional relationship between the current frame and the previous frame is calculated. Calculate and determine the enlargement, rotation, parallel movement, etc. of the image from the amount of change in the positional relationship, and decide whether to enlarge the template, if so, at what magnification, and whether to update the template. Is to decide. The result of these determinations is output as an update determination result signal 20.

The template update circuit 22 includes a pre-processed image signal 8 from the pre-processing circuit 7, a new template coordinate signal 12 from the new template generation circuit 11, a tracking area coordinate signal 18 from the maximum value search circuit 15, and a template update decision. An update determination result signal 20 and an enlargement ratio signal 21 from the circuit 19 are input. Then, a template update process is performed according to the update determination result, and a template coordinate signal 23 and a template image signal 24 are output.

When the result of the update determination is template switching, the template update circuit 22 sets the new template coordinates from the new template generation circuit 11 as template coordinates, and sets an image obtained by cutting out the portion from the preprocessed image as the template image. . If the result of the update determination is template enlargement, template coordinates are obtained by enlarging the tracking coordinates, and an image obtained by cutting out the portion from the preprocessed image is used as the template image. If the update determination result is a template update, the tracking coordinates become the template coordinates, and an image obtained by cutting out that part from the preprocessed image is used as the template image. If the update determination result has not been updated, the tracking coordinates become template coordinates, and the template image holds the previous image.

The result output circuit 25 includes a maximum value search circuit 15
, The tracking coordinate signal 17 is input, and the tracking result is displayed or output to the automatic navigation and guidance control device.

Next, the configuration of the preprocessing circuit 7 will be described. FIG. 2 is a block diagram illustrating a configuration example of the preprocessing circuit 7 according to the present embodiment. 2, the preprocessing circuit 7 includes a histogram generation circuit 26, a contrast enhancement circuit 28, and a smoothing circuit 30.

The histogram generation circuit 26 generates a histogram of the digital image signal 6 supplied from the frame memory 5 and outputs a histogram data signal 27 to a contrast enhancement circuit 28. Contrast enhancement circuit 2
8 receives the digital image signal 6 supplied from the frame memory 5 and the histogram data signal 27 from the histogram generation circuit 26, and sets the minimum pixel value of the histogram to 0 and the maximum pixel value to the maximum gradation of the image. The contrast conversion processing of the image to be converted is performed, and the converted image is output to the smoothing circuit 30 as the emphasized image signal 29. The smoothing circuit 30 receives the enhanced image signal 29, performs smoothing by averaging surrounding pixels for noise removal, and outputs a pre-processed image signal 8.

Next, a detailed configuration of the feature extraction circuit 9 will be described. FIG. 3 is a block diagram showing a configuration example of the feature extraction circuit 9 in this embodiment. 3, the feature extracting circuit 9 includes an edge component extracting circuit 31, a histogram generating circuit 33, a threshold setting circuit 35, a binarizing circuit 37, a labeling circuit 39, and a measuring circuit 41.

The edge component extraction circuit 31 includes a pre-processing circuit 7
, And performs an edge component extraction process on the image, and outputs an edge image signal 32. The histogram generation circuit 33 generates a histogram of the edge image signal 32 from the edge component extraction circuit 31 and converts the histogram data signal 34 into a threshold value setting circuit 3
5 is output. The threshold value setting circuit 35 receives the histogram data signal 34 from the histogram generation circuit 33, calculates an average value of the histogram, and sets a threshold value for binarization using the result. Then, the binarization threshold signal 36 is output to the binarization circuit 37.

The binarizing circuit 37 includes an edge component extracting circuit 3
1 and the threshold setting circuit 35
, And an edge image which is multi-valued data is set to “0”,
A binary image signal 38 divided into “1” is output to a labeling circuit 39. The labeling circuit 39 is a binarizing circuit 3
The labeling process is performed by inputting the binary image signal 38 from 7, and the label image signal 40 is output to the measurement circuit 41.
The measurement circuit 41 receives the label image signal 40 from the labeling circuit 39, calculates the circumscribed coordinates of each label, and outputs the label number and the circumscribed coordinates as the label data signal 10.

Next, a detailed configuration of the new template generation circuit 11 will be described. FIG. 4 is a block diagram illustrating a configuration example of the new template generation circuit 11 according to the present embodiment. In FIG. 4, the new template generation circuit 11 includes an area calculation circuit 42, a selector 44, and a sort circuit 46.

The area calculation circuit 42 receives the label data signal 10 supplied from the feature extraction circuit 9, calculates the circumscribed area of each label, and outputs the label number and its circumscribed area to the selector 44 as an area signal 43. . Selector 4
4 sets an upper limit value for the circumscribed area in order to reduce the calculation load, and outputs only the area signal 43 input from the area calculation circuit 42 that is smaller than the upper limit to the sort circuit 46 as the selected area signal 45. .

The sorting circuit 46 receives the label data signal 10 supplied from the feature extracting circuit 9 and the selected area signal 45 from the selector 44, sorts the selected area signal 45 in descending order of area, and sorts the label data signal. The label numbers and their circumscribed coordinates after rearrangement with reference to 10 are output as new template coordinate signals 12.

Next, a detailed configuration of the template update determination circuit 19 will be described. FIG. 5 is a block diagram showing a configuration example of the template update determination circuit 19 according to this embodiment. 5, the template update determination circuit 19 includes a correlation value comparator 47, a first relative vector calculation circuit 49, a second relative vector calculation circuit 52,
Vector length ratio calculation circuit 55, vector angle difference calculation circuit 56, enlargement ratio comparator 58, rotation angle comparator 6
0 and an update determination result generation circuit 62.

The correlation value comparator 47 receives the maximum correlation value signal 16 supplied from the maximum value search circuit 15 and outputs a tracking failure signal 48 if the maximum correlation value is small. The first relative vector operation circuit 49 includes a maximum value search circuit 15
And the tracking failure signal 48 from the correlation value comparator 47, and two points are extracted from the tracking coordinates that are not tracking failure, and the two points are defined as (starting point,
Create a vector as the end point) and calculate its length and slope. This calculation is performed for all combinations that take out two points from tracking coordinates that are not tracking failures.

Then, the first relative vector operation circuit 49
Outputs the result of the vector length as the relative vector length signal 50 of the current frame and the result of the vector inclination as the relative vector angle signal 51 of the current frame. The second relative vector calculation circuit 52 receives the template coordinate signal 23 supplied from the template update circuit 22 and the tracking failure signal 48 from the correlation value comparator 47, and performs the first relative vector calculation circuit on the template coordinates. 4
A vector similar to 9 is created, and its length is output as a relative vector length signal 53 of the previous frame, and its inclination is output as a relative vector angle signal 54 of the previous frame.

The vector length ratio calculation circuit 55 includes a current frame relative vector length signal 50 from the first relative vector calculation circuit 49 and a previous frame relative vector length signal 53 from the second relative vector calculation circuit 52. Is calculated, the ratio of the vector length of the current frame and the vector length of the previous frame having the same combination is calculated for each combination, and the average value is output as the magnification signal 21.

The vector angle difference calculation circuit 56 is provided with a relative vector angle signal 51 of the current frame from the first relative vector calculation circuit 49 and a relative vector angle signal 54 of the previous frame from the second relative vector calculation circuit 52. Is calculated, and the difference between the vector angles of the current frame and the previous frame having the same combination is calculated for each combination, and the average value is output to the rotation angle comparator 60 as the rotation angle signal 57.

The enlargement ratio comparator 58 receives the enlargement ratio signal 21 from the vector length ratio calculation circuit 55 and outputs an enlargement request signal 59 to enlarge or reduce the template when the enlargement ratio is large or small to some extent. The result is output to the update determination result generation circuit 62. The rotation angle comparator 60 receives the rotation angle signal 57 from the vector angle difference calculation circuit 56 and outputs a rotation signal 61 to the update determination result generation circuit 62 when the rotation angle is somewhat large.

The update determination result generation circuit 62 receives the tracking failure signal 48 from the correlation value comparator 47, the enlargement request signal 59 from the enlargement ratio comparator 58, and the rotation signal 61 from the rotation angle comparator 60, and receives the tracking status or It determines the state of enlargement, reduction, and rotation of the image, and outputs an update determination result signal 20.

The update determination result is defined as follows. When the tracking failure signal 48 is input from the correlation value comparator 47, it is determined that the template needs to be switched to another tracking target, and the update determination result is set to mode 1.

When the tracking failure signal 48 is not inputted from the correlation value comparator 47 and the enlargement request signal 59 is inputted from the enlargement ratio comparator 58, the image is enlarged or reduced and the template is enlarged or reduced. Is determined to be necessary, and the update determination result is set to mode 2.

When the tracking failure signal 48 is not inputted from the correlation value comparator 47, the enlargement request signal 59 is not inputted from the enlargement ratio comparator 58, and the rotation signal 61 is inputted from the rotation angle comparator 60 Although the image is not enlarged or reduced, it is determined that the image is rotated and the template needs to be updated at the same magnification, and the update determination result is set to mode 3.

When the tracking failure signal 48 is not input from the correlation value comparator 47, the enlargement request signal 59 is not input from the enlargement ratio comparator 58, and the rotation signal 61 is not input from the rotation comparator 60, Determines that there is no need to update the template without performing enlargement / reduction and rotation, and sets the update determination result to mode 4. The update determination result generation circuit 62 outputs the update determination result defined as described above as the update determination result signal 20.

Next, a detailed configuration of the template updating circuit 22 will be described. FIG. 6 is a block diagram illustrating a configuration example of the template update circuit 22 according to the present embodiment. In FIG. 6, the template update circuit 22
It comprises a size conversion circuit 63, a multiplexer 65 and a template image generation circuit 66.

The size conversion circuit 63 is a maximum value search circuit 1
5 and the enlargement ratio 21 supplied from the template update determination circuit 19, the tracking region is enlarged at the same ratio as the enlargement ratio, and the enlarged coordinates are converted into the size conversion coordinate signal. Multiplexer 65 as 64
Output to The multiplexer 65 includes a new template coordinate signal 12 supplied from the new template generation circuit 11, a tracking area coordinate signal 18 supplied from the maximum value search circuit 15, an update determination result signal 20 supplied from the template update determination circuit 19, and A size conversion coordinate signal 64 from the size conversion circuit 63 is input, and a new template coordinate signal 1
2. Tracking area coordinate signal 18 and size conversion coordinate signal 6
4 and the template coordinate signal 23 is selected.
Output as

When the content of the update determination result signal 20 is mode 1, it is necessary to switch the template to another tracking target, so the new template coordinate signal 12 is selected as the template coordinate signal 23. If the update determination result is mode 2, the image is enlarged or reduced,
Since the template needs to be enlarged or reduced, the size conversion coordinates 64 are selected as the template coordinate signal 23. In modes 3 and 4, the tracking area coordinate signal 18 is selected as the template coordinate signal 23.

The template image generation circuit 66 includes a preprocessed image signal 8 supplied from the preprocessing circuit 7 and an update determination result signal 2 supplied from the template update determination circuit 19.
0, and the template coordinate signal 23 from the multiplexer 65, and generates or holds a template image according to the update determination result.
Is output. If the update determination result is in modes 1 to 3,
An image in the area of the template coordinates is cut out from the preprocessed image and output as a template image signal 24. If the update determination result is mode 4, the template image of the previous frame is held as it is.

Next, the operation will be described. Imaging device 1
Captures an asteroid surface scene including a crater or other ground feature region that is a tracking target, the imaging apparatus 1 outputs the image data to the A / D converter 3 as an analog video signal 2. The analog video signal 2 is supplied to an A / D converter 3
Is converted into a digital video signal 4 and stored in the frame memory 5 in units of one screen. Frame memory 5
The digital image signal 6 read from the
Sent to FIG. 7 shows an example of a digital image.

In the preprocessing circuit 7 shown in FIG. 2, the histogram generation circuit 26 generates a histogram of the digital image. Next, the contrast emphasizing circuit 28 performs grayscale value conversion of the image. FIG. 8 shows an example of a histogram conversion process for performing a grayscale value conversion. That is, the minimum gray value A of the input histogram is set to 0, and the maximum gray value B of the input histogram is set to the maximum gray value C of the image (for example, 255 in the case of 8-bit gradation). And
The data between the minimum value and the maximum value of the input histogram is linearly enlarged.

The contrast emphasizing circuit 28 performs image gray level conversion based on the histogram thus converted. Assuming that the gray value before conversion, that is, the gray value of the input digital image is x, and the gray value after conversion, that is, the gray value of the output enhanced image, is y, the following conversion is performed.

[0057]

(Equation 1)

FIG. 9 shows an example of an enhanced image output when the digital image shown in FIG. 7 is input to the contrast enhancement circuit 28.

Then, the smoothing circuit 30 smoothes the emphasized image. Assume that the emphasized image data input to the smoothing circuit 30 is f (x, y). x is the horizontal coordinate of the image, y is the vertical coordinate of the image, and f (x, y) is the gray value of the emphasized image at that coordinate. Assuming that the preprocessed image data output from the smoothing circuit 30 is g (x, y), the smoothing circuit 30
Performs the following calculation.

[0060]

(Equation 2)

The enhanced image shown in FIG.
FIG. 10 shows an example of a pre-processed image output when the image is input to.

In the feature extraction circuit 9 shown in FIG.
The edge component extraction circuit 31 extracts an edge component of the pre-processed image. For example, the four masks shown in FIGS. 11A to 11D are separately scanned on the preprocessed image in synchronization with the raster scan, and the product-sum operation of each mask and the preprocessed image is performed. The one with the largest value is the output value of the mask center pixel. Then, an output image obtained by performing the above processing on all pixels of the preprocessed image is defined as an edge image.

In FIGS. 11A to 11D, the mask data at the (i, j) coordinate of the k-th mask from the left is represented by m
(K, i, j). Here, (i, j) has the center coordinate as the origin, and −2 ≦ i ≦ 2 and −2 ≦ j ≦ 2. Assuming that the input pre-processed image data is g (x, y) and the output edge image data is e (x, y), the edge component extraction circuit 31 performs the following operation.

[0064]

(Equation 3)

FIG. 12 shows an example of an edge image output when the preprocessed image shown in FIG. 10 is input to the edge component extraction circuit 31. Next, the histogram generation circuit 33 generates a histogram of the edge image. Further, the threshold value setting circuit 35 sets a threshold value for binarizing the edge image. For example, the average value of the grayscale values of the edge image is calculated from the histogram data, and twice the average value is set as the threshold value. Then, the binarization circuit 37 binarizes the input edge image with a binarization threshold.

An example of a binarized image output when the edge image shown in FIG. 12 is input to the binarizing circuit 37 is shown in FIG.
Shown in

Next, the labeling circuit 39 performs labeling on the binary image. That is, the area of “1” of the binary image is extracted, and the same label number is assigned to pixels belonging to the same continuous area in the extracted area. The above processing 2
The result performed on all pixels of the value image is defined as a label image.
That is, the label image is obtained by replacing the data “1” in the white area of the binary image with the label number.

Finally, the measuring circuit 41 measures the circumscribed coordinates of each label, that is, the minimum and maximum values of the x coordinate and the minimum and maximum values of the y coordinate.

In the new template generating circuit 11 shown in FIG. 4, the area calculating circuit 42 calculates the circumscribed area of each label. Xmi is the minimum value of the x coordinate of the i-th label
If n (i), the maximum value of the x coordinate is Xmax (i), the minimum value of the y coordinate is Ymin (i), and the maximum value of the y coordinate is Ymax (i), the circumscribed area is S (i), and the area calculation is performed. The circuit 42 performs the following operation.

[0070]

(Equation 4)

The selector 44 selects only the circumscribed area smaller than the set upper limit value. The upper limit to set is
For example, 3000 or the like. The sorting circuit 46 sorts the selected labels in descending order of the circumscribed area and outputs the sorted labels together with the circumscribed coordinates.

The correlation value calculating circuit 13 calculates a correlation value between the template image signal 24 from the template updating circuit 22 and the pre-processed image signal 8 in the next frame. In the n-th template image, horizontal Xn pixels,
A vertical Yn pixel and a gray value at coordinates (i, j) are gn
(I, j).

However, the coordinates (i, j) have the origin at the center of the template image. The gray value at the coordinates (x, y) of the pre-processed image of the next frame for which a correlation value is to be obtained is represented by f
(X, y). Assuming that the correlation value between the coordinates (x, y) and the n-th template image is rn (x, y),
The correlation value operation circuit 13 performs the operation operation of the following equation. here,
The correlation value rn (x, y) is the standard deviation of the gray value in the template image and the standard value of the gray value in the partial image within the range of Xn / 2 pixels around (x, y) in the pre-processed image. By dividing by the product of the deviation, the fluctuation due to the brightness of the image is normalized.

[0074]

(Equation 5)

However,

(Equation 6) And Correlation value calculation circuit 13
Calculates the correlation value rn (x, y) for the entire image and outputs the number of correlation image signals 14 corresponding to the number of templates to the maximum value search circuit 15.

The maximum value search circuit 15 searches the correlation image signal 14 of each template for the maximum value and the maximum coordinates. Assuming that the correlation image for the n-th template is rn (x, y) and the maximum value is Rn, the maximum value search circuit 15 performs the following calculation.

[0077]

(Equation 7)

Then, the maximum value search circuit 15 outputs Rn as the maximum correlation value signal 16. Further, (x, y) at the time of Rn is output as the tracking coordinate signal 17. Further, the tracking coordinates in the n-th template are represented by (x
n, yn), the template size is Xn pixels × Yn pixels, the maximum x-coordinate of the tracking area is Xmax (n), the minimum is Xmin (n), and the maximum y-coordinate is Ymax (n),
Assuming that the minimum value is Ymin (n), the maximum value search circuit 15
Calculates the following equation and outputs the calculated coordinates as the tracking area coordinate signal 18.

[0079]

(Equation 8)

In the template update judging circuit 19 shown in FIG. 5, the correlation value comparator 47 judges the magnitude of the input maximum correlation value signal 16. If the maximum correlation value is smaller than the set threshold value, a tracking failure signal 48 is output. The set correlation value threshold is, for example, 0.9.

Next, the first relative vector operation circuit 49
Calculates the relative positional relationship between tracking coordinates as a vector. Hereinafter, such a vector is called a relative vector. For example, if the input tracking coordinates are four, the combination of the relative vector (start point, end point) is (1, 2)
(1,3) (1,4) (2,3) (2,4) (3,4)
It is six. At this time, the tracking coordinates at which the tracking failure signal 48 has been output by the correlation value comparator 47 are excluded from the combination.

The first relative vector calculation circuit 49 calculates the length and the angle with respect to the relative vectors combined as described above. Let the coordinates of the starting point of the relative vector be (X0, Y
0), the end point coordinates are (X1, Y1), the length of the relative vector is l, and the angle is θ, the first relative vector calculation circuit 49 performs the following calculation.

[0083]

(Equation 9)

The first relative vector calculation circuit 49 calculates the relative vector length signal 50 of the current frame by
, And outputs the θ of all the relative vectors as the relative vector angle signal 51 of the current frame.

Similarly, the second relative vector operation circuit 52
Calculates the center coordinates of the template from the template coordinates, and calculates the relative positional relationship between the templates, that is, the relative positional relationship between the tracking coordinates of the previous frame, as a vector. The combination of the relative vector (start point and end point) and the calculation method are described in the first relative vector calculation circuit 5.
Same as those for 0.

FIG. 14 is a schematic diagram showing a change in the position of the tracking coordinates and a change in the relative vector when the probe moves parallel to the ground surface. If the probe only moves in parallel, the tracking coordinates only move up, down, left, and right, so that both the length and the angle of the relative vector do not change as shown in FIG.

FIG. 15 is a schematic diagram showing a change in the position of the tracking coordinates and a change in the relative vector when the probe relatively approaches the ground. When the spacecraft approaches, the relative distance between the tracking coordinates increases, so that the relative vector changes in length as shown in FIG.

FIG. 16 is a schematic diagram showing a change in the position of the tracking coordinates and a change in the relative vector when the probe rotates relative to the ground. When the spacecraft rotates,
As shown in FIG. 6, the angle of the relative vector changes.

In the case illustrated in FIG. 14, the enlargement ratio is 1 in the vector length ratio operation circuit 55 and the rotation angle is 0 in the vector angle difference operation circuit 56. Request signal 59
Also, neither the rotation signal 61 from the rotation angle comparator 60 is output. Therefore, in the update determination result generation circuit 62, the update determination result is Mode 4, and it is determined that there is no need to update the template.

In the case illustrated in FIG. 15, the enlargement ratio is larger than 1 in the vector length ratio calculation circuit 55, and the rotation angle is 0 in the vector angle difference calculation circuit 56. Accordingly, although the enlargement request signal 59 is output from the enlargement ratio comparator 58, the rotation angle comparator 6
From 0, the rotation signal 61 is not output. Therefore, the update determination result is the mode 2 in the update determination result generation circuit 62, and it is determined that the template needs to be enlarged and updated.

In the case illustrated in FIG. 16, the enlargement ratio is 1 in the vector length ratio calculation circuit 55, and the rotation angle is a numerical value larger than 0 in the vector angle difference calculation circuit 56. Accordingly, the enlargement request signal 59 is not output from the enlargement ratio comparator 58, and the rotation angle comparator 60
Outputs a rotation signal 61. Therefore, the update determination result is the mode 3 in the update determination result generation circuit 62,
It is determined that it is necessary to update the template at the same size.

The template updating circuit 22 shown in FIG.
In, the size conversion circuit 63 enlarges the tracking area at the same ratio as the enlargement ratio. Here, enlargement is performed so that the center point does not change. The enlargement ratio signal 21 input from the template update determination circuit 19 is represented by a, and the minimum x coordinate value of the tracking area coordinate signal 18 output from the maximum value search circuit 15 is represented by x
0, maximum value is x1, y coordinate minimum value is y0, maximum value is y1
In the size conversion coordinate signal 64 output after enlargement, the minimum value of the x coordinate is X0, the maximum value is X1, the minimum value of the y coordinate is Y0,
When the maximum value is Y1, the size conversion circuit 63 performs the calculation of the following equation.

[0093]

(Equation 10)

In the case illustrated in FIG. 14, the update determination result signal 20 is in the mode 4. In that case, the multiplexer 65 selects the tracking area coordinate signal 18 as the template coordinate signal 23. Therefore, the template image generation circuit 66 holds and outputs the template image of the previous frame as it is as the template image signal 24.

In the case illustrated in FIG. 15, the update determination result signal 20 is in the mode 2. In that case, the multiplexer 65 selects the size converted coordinate signal 64 as the template coordinate signal 23. Therefore, the template image generating circuit 66 cuts out the template coordinate signal 23 from the pre-processed image signal 8 and outputs the template image signal 24.

In the case illustrated in FIG. 16, the update judgment result signal 20 is in the mode 3. In that case, the multiplexer 65 selects the tracking area coordinate signal 18 as the template coordinate signal 23. Therefore, the template image generating circuit 66 cuts out the template coordinate signal 23 from the pre-processed image signal 8 and outputs the template image signal 24.

As described above, in this embodiment, the feature extraction circuit 9 does not extract a target object by shape recognition but extracts a portion where the density of an image changes sharply as a tracking target. The tracking can be performed without limiting the shape of. In addition, the feature extraction circuit 9 detects an object by spatial differentiation, and sets a portion having a strong gradation change in the image as a template. Therefore, unlike the conventional case, the template is automatically extracted in the first frame. Can be set to

Further, the template update judging circuit 19
Determines the movement of the image, determines whether or not to update the template, and determines how to perform the update process. Therefore, the processing efficiency of updating the template is good.

In this embodiment, since the normalized correlation is used in the correlation calculation circuit 13, accurate tracking is possible even when the brightness of the entire input image fluctuates. Further, since the contrast of the image can be kept constant by the pre-processing circuit 7, it is possible to cope with a change in the contrast of the input image.

Next, another embodiment of the present invention will be described in detail with reference to the drawings. In the second embodiment shown in FIG. 17, the digital video signal 4 output from the A / D converter 3 is input to the preprocessing circuit 7. Then, the preprocessing circuit 7 outputs the preprocessed video signal 67 to the frame memory 5. The frame memory 5 outputs the pre-processed image signal 8 to the feature extraction circuit 9, the correlation operation circuit 13, and the template update circuit 22. That is, A-
The digital video signal 4 output from the D converter 3 is
After the pre-processing by the pre-processing circuit 7 is performed, the pre-processing is stored in the frame memory 5.

In this embodiment, the feature extraction circuit 9, the correlation calculation circuit 13 and the template update circuit 22 directly access the frame memory 5 without repeating the preprocessing when the preprocessing image signal 8 is input. Since this is good, there is a new effect that the processing efficiency can be improved.

In the third embodiment shown in FIG. 18, the maximum value search circuit 15 outputs the tracking coordinate signal 17 only to the template update judging circuit 19, and does not output it to the result output circuit 25. Instead, the template update circuit 22 outputs the template coordinates 23 to the result output circuit 25. Further, the template update determination circuit 19 outputs an update determination result signal 20 to the result output circuit 25.

According to this embodiment, when tracking is missed, it is possible to output which tracking target has missed and where a new template has been set.

[0104]

As described above, according to the present invention, the automatic tracking device determines whether or not to update a template based on a change in the positional relationship between the frames of the tracking coordinates set in each template. Since the configuration includes the template update determination means, there is an effect that accurate tracking can be performed without limiting the shape of the tracking target.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration example of a preprocessing circuit.

FIG. 3 is a block diagram illustrating a configuration example of a feature extraction circuit.

FIG. 4 is a block diagram illustrating a configuration example of a new template generation circuit.

FIG. 5 is a block diagram illustrating a configuration example of a template update determination circuit.

FIG. 6 is a block diagram illustrating a configuration example of a template update circuit.

FIG. 7 is an explanatory diagram illustrating an example of a digital image.

FIG. 8 is an explanatory diagram illustrating an example of a histogram conversion process in the contrast enhancement circuit.

9 is an explanatory diagram illustrating an example of an enhanced image output from a contrast enhancement circuit 28. FIG.

FIG. 10 is an explanatory diagram illustrating an example of a pre-processed image output from a smoothing circuit.

FIG. 11 is an explanatory diagram illustrating an example of a mask used when an edge component extraction circuit extracts an edge component.

FIG. 12 is an explanatory diagram illustrating an example of an edge image output from an edge component extraction circuit.

FIG. 13 is an explanatory diagram illustrating an example of a binarized image output from a binarization circuit.

FIG. 14 is a schematic diagram showing a change in the position of a tracking coordinate and a change in a relative vector when the probe moves parallel to the ground surface.

FIG. 15 is a schematic diagram illustrating a change in the position of a tracking coordinate and a change in a relative vector when the spacecraft relatively approaches the ground surface.

FIG. 16 is a schematic diagram showing a change in the position of a tracking coordinate and a change in a relative vector when the probe rotates relative to the ground.

FIG. 17 is a block diagram showing another embodiment of the present invention.

FIG. 18 is a block diagram showing still another embodiment of the present invention.

FIG. 19 is a block diagram showing a configuration of a conventional automatic tracking device.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 imaging device 3 A / D converter 5 frame memory 7 preprocessing circuit 9 feature extraction circuit 11 new template generation circuit 13 correlation operation circuit 15 maximum value search circuit 19 template update determination circuit 22 template update circuit 25 result output circuit

Continued on the front page F-term (reference) 5C054 AA01 CA04 CC03 EA01 EA05 EB05 ED12 EJ04 FC03 FC13 FC14 FC16 HA26 HA31 5L096 AA06 BA05 DA02 EA02 EA06 EA43 FA06 FA34 FA35 FA59 FA67 GA34 GA51 HA04 HA05 JA09 KA04 BZ28 KK04 KA03

Claims (9)

[Claims] 1. An automatic tracking device, comprising: an imaging unit that outputs a captured image including a tracking target object one frame at a time, and tracking an object based on a template set in the captured image. Template generating means to be set; template update determining means for determining whether or not to update a template based on a change in the positional relationship between frames of tracking coordinates set in each template; and determination results of the template update determining means Template updating means for outputting each template image in accordance with the following, correlation calculating means for calculating the correlation between each template image and the captured image of the next frame, and coordinates representing the maximum correlation value in each template as tracking coordinate signals. An automatic tracking device comprising: a maximum value searching means for outputting. 2. A feature extraction circuit for outputting, as a template candidate, an area including a portion in which a change in shading in a captured image exceeds a predetermined value, and a predetermined size of each area output by the feature extraction circuit. 2. The automatic tracking apparatus according to claim 1, further comprising: a new template generating circuit using the following as a template. 3. The template update judging means inputs a template coordinate signal indicating the coordinates of each template from the template updating means, and a maximum correlation value and a tracking coordinate signal from a maximum value searching means. The automatic tracking apparatus according to claim 1, wherein it is determined whether the template is updated, deformed, or maintained using the maximum correlation value and the tracking coordinate signal. 4. A template update judging means calculates a positional relationship of each tracking target obtained from the tracking coordinate signal, calculates a positional relationship of each tracking target obtained from the template coordinate signal, and calculates a current relationship based on each calculation result. 4. The method according to claim 3, wherein the amount of change in the positional relationship of each tracking target between the frame and the previous frame is calculated, and it is determined whether the template is updated, enlarged or reduced, or maintained based on the amount of change in the positional relationship. Automatic tracking device. 5. The template updating means inputs a new template coordinate signal indicating the coordinates of each template from the template generating means, a tracking area coordinate signal indicating an area including a tracking target, and an update determination result signal from the template updating determining means. And an enlargement ratio signal indicating a template enlargement / reduction ratio. If the update determination result signal indicates update, a new template image is generated according to the new template coordinate signal, and the update determination result signal If it indicates enlargement or reduction, the template image is enlarged or reduced according to the enlargement ratio signal, and if the update determination result signal indicates not to be updated, the template image is adjusted according to the tracking area coordinate signal. 5. The automatic tracking device according to claim 4, wherein the automatic tracking device outputs a template image. 6. The automatic tracking device according to claim 1, wherein the correlation calculating means performs a normalized correlation process. 7. The automatic tracking apparatus according to claim 1, further comprising a preprocessing unit that performs a contrast enhancement process on an image output by the imaging unit. 8. The automatic tracking device according to claim 1, further comprising a result output circuit for outputting a tracking coordinate signal from the maximum value search means as a result. 9. The automatic tracking according to claim 5, further comprising a result output circuit for outputting a template coordinate signal output from the template update unit and an update determination result signal output from the template update determination unit. apparatus.