JPH10150593A - Device and method for extracting target - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve followup performance by reducing possibility due to miss or erroneously recognize a target by performing matching concerning all the areas of image, while using an evaluation value considering that the possibility of target existence is improved getting closer to a predictive position in template matching. SOLUTION: A template setting part 14 sets a template in a prescribed size with a pixel, which is discriminated as one part of an infiltrator from the difference of luminance with a background image as a reference, as a center. An evaluation value calculating part 16 finds the degree of similarity with the total sum of luminance difference as scaling for each partial area in the same size as the template over all the surface of frame. Concerning the coordinate of predictive position on the template from a predictive position calculating part 15 and the coordinate of partial area defined as an object, a position evaluation value calculating part 15 sends the sum of absolute values of separate distances in X-axis and Y-axis directions to the evaluation value calculating part 16 as a position evaluation value. The result of adding the position evaluation value to the degree of similarity is sent to a target-extracting part 17 as the evaluation value of matching by the evaluation value calculating part 16.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a target extracting device and a target extracting method for extracting a target from an image obtained by an image pickup device such as a TV camera and an infrared camera.
For example, the present invention relates to a target extraction device and a target extraction method used for extracting a monitoring target in a monitoring device.

[0002]

2. Description of the Related Art For example, in a system for monitoring a suspicious person using an image pickup device such as a TV camera or an infrared camera, an image obtained every moment by an image pickup device is used for the purpose of detecting a movement of a target. Extract goals from
A tracking device has recently been required.

Conventionally, as a method for extracting such a target, a template matching method has been used in which the target is represented as a template and the similarity between the template and a predetermined search area in an image is determined. FIG. 7 is an explanatory diagram of this template matching method, in which the center of a template t (m, n) representing a target to be extracted is set to a point () in a search area S in an image f (x, y). i,
j), the similarity (matching degree) between the template t (m, n) and the part of the image overlapping the template t (m, n) is measured, and the value is regarded as a certainty that the target exists at the point (i, j). The basic idea is that this operation is performed on all points in the search area S, and the position where the similarity is maximized is determined as the target position. For the measurement of the similarity, for example, a value D calculated by the following equation (1) is used. That is,

[0004]

D (i, j) = Σ | f (i + m, j + n) −t
(M, n) |. Note that Σ means taking the sum of the differences for all points of the template t (m, n), and f, t
Represents luminance. Here, the search area S
Is a region where a target may be present, and is intended to prevent a region other than the target from being erroneously recognized as a target by limiting the target region of template matching only to the search region S.

In a conventional target extracting apparatus for extracting a target, the position of the target does not change significantly between frames separated by a very short time, so that the position of the target extracted in the immediately preceding frame is centered. It is set as a region of a predetermined size.

[0006]

However, in the above-mentioned conventional method, if the target exists outside the search area S, the target cannot be detected. On the other hand, if the search area S is set large, the search area cannot be detected. There is a problem that the effect of the provision is reduced. The present invention has been made in view of such a problem, and the inventions described in claims 1 to 12 can accurately extract a target without providing a search area and a target extraction method. The purpose is to provide.

[0007]

According to one aspect of the present invention, there is provided an imaging apparatus for imaging a target and outputting a video signal, and sequentially outputting a video signal output from the imaging apparatus. A / D converted and image f (x, y) for each frame
Image generation means for generating the image f (x, y)
And a target extracting means for extracting the target from the target and outputting a target position signal, wherein the target extracting means has a template t (m, n) representing the target and the same size as the template. Each part p (x,
y), an evaluation value for evaluating the probability that the part p (x, y) is a target is represented by a template t (m, n).
And the similarity between the part p (x, y) and the part p
From the position evaluation value c determined by the degree of separation between the representative point of (x, y) and the target predicted position (x0, y0) in the current frame predicted based on the target position extracted before the previous frame. It is characterized in that a target is extracted from the image f (x, y) based on the obtained and obtained evaluation value of each part p (x, y).

[0008] According to a second aspect of the present invention, in the first aspect, the evaluation value is calculated based on a template t.
The sum D of the luminance differences between (m, n) and the corresponding pixels of the part p (x, y), and the representative point of the part p (x, y) and the position of the target extracted before the previous frame Target position (x0, y) in the current frame predicted based on
0) and a position evaluation value c which is a non-decreasing function of the degree of separation.

According to a third aspect of the present invention, in the second aspect, the position evaluation value c includes
From the representative point (i, j) of (x, y) and the predicted position of the target (x0, y0),

[0010]

C = k (| i−x0 | + | j−y0 |) k: It is characterized by being obtained by a positive constant. According to a fourth aspect of the present invention, there is provided an imaging apparatus for imaging a target and outputting a video signal, and sequentially performing A / D conversion of a video signal output from the imaging apparatus to generate an image f (x, y) for each frame. ), And a target extraction unit that extracts the target from the image f (x, y) and outputs a target position signal, wherein the target extraction unit represents a target. For a template t (m, n) and each part p (x, y) of the image having the same size as the template, an evaluation value for evaluating the probability that the part p (x, y) is a target To the template t (m, n) and its part p
The similarity of (x, y) and the degree of separation between the representative point of the part p (x, y) and the target position (x1, y1) extracted in the image f (x, y) in the previous frame Each part p (x, y) obtained from the determined position evaluation value c
And extracting a target from the image f (x, y) based on the evaluation value.

According to a fifth aspect of the present invention, in the fourth aspect, the evaluation value is calculated based on a template t.
The sum D of the difference in luminance between (m, n) and the corresponding pixel of the part p (x, y), and the representative point of the part p (x, y) and the image f (x, y) in the previous frame ) Is obtained by adding the position evaluation value c, which is a non-decreasing function of the degree of separation from the target position (x1, y1) extracted in step (1).

According to a sixth aspect of the present invention, in the fifth aspect, the position evaluation value c is calculated by using
The representative point (i, j) of (x, y) and the image f in the previous frame
The target position (x1, y) extracted at (x, y)
1) From

[0013]

C = k (| i−x1 | + | j−y1 |) k: a positive constant. According to a seventh aspect of the present invention, a video signal output from an imaging device that captures an image of a target is sequentially subjected to A / D conversion, and an image f (x, y) is obtained for each frame.
And extracting a target from the image f (x, y) in the target extraction method for extracting the target from the image f (x, y)
For each part p (x, y) of the image having the same size as (m, n) and the template, the part p (x, y)
The evaluation value for evaluating the likelihood that y) is the target is represented by the similarity between the template t (m, n) and its part p (x, y), and the representative point of the part p (x, y) and the previous frame. Each position p obtained from the position evaluation value c determined by the degree of separation from the target predicted position (x0, y0) in the current frame predicted based on the previously extracted target position.
It is characterized in that it is performed based on the evaluation value of (x, y).

According to an eighth aspect of the present invention, in the seventh aspect of the present invention, the evaluation value is a template t.
The sum D of the luminance differences between (m, n) and the corresponding pixels of the part p (x, y), and the representative point of the part p (x, y) and the position of the target extracted before the previous frame Target position (x0, y) in the current frame predicted based on
0) and a position evaluation value c which is a non-decreasing function of the degree of separation.

According to a ninth aspect of the present invention, in the ninth aspect of the present invention, the position evaluation value c is calculated based on a part p thereof.
From the representative point (i, j) of (x, y) and the predicted position of the target (x0, y0),

[0016]

C = k (| i−x0 | + | j−y0 |) k: a positive constant. According to a tenth aspect of the present invention, a video signal output from an imaging device for imaging a target is sequentially subjected to A / D conversion and an image f (x,
y), and extracting the target from the image f (x, y).
The extraction of the target from y) is based on a template t (m, n) representing the target and each part p (x, y) of the image having the same size as the template.
The evaluation value for evaluating the likelihood that y) is the target is represented by the similarity between the template t (m, n) and its part p (x, y), and the representative point of the part p (x, y) and the previous frame. The target position (x extracted in the image f (x, y) at
1, y1), and a position evaluation value c determined by the degree of separation, and is performed based on the obtained evaluation values of the respective parts p (x, y).

The invention according to claim 11 is the first invention.
0, the evaluation value is a template t
The sum D of the difference in luminance between (m, n) and the corresponding pixel of the part p (x, y), and the representative point of the part p (x, y) and the image f (x, y) in the previous frame ) Is obtained by adding the position evaluation value c, which is a non-decreasing function of the degree of separation from the target position (x1, y1) extracted in step (1).

The invention according to claim 12 is the first invention.
1, the position evaluation value c is calculated based on the representative point (i, j) of the portion p (x, y) and the target position (x1) extracted in the image f (x, y) in the previous frame. , Y
1) From

[0019]

C = k (| i-x1 | + | j-y1 |) k: It is characterized by being obtained by a positive constant. In the present invention, the degree of separation is an index indicating the degree of separation between two points, and an arbitrary function can be selected. Claim 3, as an example of the degree of separation
In addition to the one represented by c described in 6, 9, or 12, the distance between two points, {(i−x0) ² + (j−y0) ² }
Or {(i-x1) ² + (j-y1) ² }.

The prediction of the target can be calculated by various methods based on the position of the target before the previous frame. For example, it can be calculated by using the α-β tracker method or the like in which an α-β filter is applied. it can.

[0021]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a target extracting apparatus according to the present invention.
1 is an imaging device, 12 is a timing signal generator, 13 is A
A / D conversion unit, 20 is a target extraction unit, and 10 is a display unit.

The imaging device 11 includes a timing signal generator 1
2 receives a vertical synchronization signal and a horizontal synchronization signal from the camera 2, and captures an image of a monitoring area and outputs the image as an electric video signal. The timing signal generator 12 generates a vertical synchronizing signal, a horizontal synchronizing signal, and a pixel clock signal. In addition to outputting a synchronizing signal to the imaging device 11 as described above,
These signals are output to the D conversion unit 13, the template setting unit 14, the position evaluation value calculation unit 15, the evaluation value calculation unit 16, and the target frame weight unit 19 that constitute the target extraction unit 20.

The A / D converter 13 (image generating means) sequentially A / D converts the video signal from the image pickup device 11 to make each pixel a gray value of, for example, 8 bits and 256 values, and outputs an image f for each frame. The (x, y) raster scan data is output. Based on this data, a template setting unit 14, a position evaluation value calculation unit 15, an evaluation value calculation unit 16, a target extraction unit 17, a predicted position calculation unit 18, and a target frame weight unit 19, which will be described in detail below, are configured. Target extraction means 2
At 0, the target is extracted.

The display unit 10 displays a video signal from the image pickup device 11, and is constituted by, for example, a CRT display device. The above-described imaging device 11, timing signal generator 12, A / D converter 13, and display 1
0 is the same as used in conventional target extraction devices and works similarly.

Next, the target extracting means 20 will be described in detail. Prior to the automatic extraction of the target, the template setting unit 14 selects a part of the image f (x, y) as a template t (m, n) representing the target, and every time a template setting signal is input, the template f (x, y) is selected. The data is sent to the evaluation value calculation unit 16. For example, when the present target extraction device is used in a suspicious person monitoring system, the template setting unit 14
Detects an intruder and targets it, and outputs the intruder part in the image as a template representing the target.

In order to realize this function, the template setting unit 14 includes, for example, a background image b (x, y) representing a monitoring area when no intruder exists and an A / D conversion unit 13
And a frame memory for storing raster scan data of the image f (x, y) from the image memory. The image f (x, y) captured in the frame memory and the background image b (x, y) are compared for each pixel. Then, when the difference in luminance is equal to or larger than a predetermined value, it is determined that the pixel of the image f (x, y) is a part of the intruder. Then, an intruder is detected as a set of such pixels, and an area of a predetermined size centered on the center is sent to the evaluation value calculation unit 16 as a template t (m, n). FIG. 4A shows an image f composed of X × Y pixels.
With respect to (x, y), the shaded portion represents a portion where the difference in luminance is equal to or greater than a predetermined value, and the center of this portion (xi, yi)
And a template t (m, n) composed of M × N pixels whose centers match.

The function of constantly taking in the image f (x, y) is as follows.
It is realized by a counter, a logic circuit, and a memory based on the vertical synchronization signal, the horizontal synchronization signal, and the pixel clock signal from the timing signal generator 12. Also, an intruder is detected, and a template is calculated based on the detected intruder.
Can be transmitted by, for example, software processing executed by the CPU and the memory.

The position evaluation value calculation unit 15 calculates an evaluation value calculation unit 16 for each part p (x, y) of the image f (x, y) having the same size as the template t (m, n). , Which calculates a position evaluation value c used when calculating an evaluation value for evaluating the probability that the part p (x, y) is a target, and the position evaluation value c is a representative of the respective parts. This is a non-decreasing function of the degree of separation between the point and the target predicted position in the current frame predicted by the predicted position calculation unit 18 described later.

As the position evaluation value c, for example, the following equation (2) can be used. Here, (i, j) is the center position of the portion p (x, y), (x0, y0) is the predicted position of the image f (x, y), and k is a positive constant.

[0030]

C = k (| i−x0 | + | j−y0 |) (2) FIG. 4B shows the center (i, j) of the part p (x, y) and the predicted position of the target ( x0, y0). That is, c
Can be defined as k times the degree of separation, which is the sum of | i−x0 | representing the degree of horizontal separation and | j−y0 | representing the degree of vertical separation. However, as the degree of separation, in addition to this example, the center (i, j) of the portion p (x, y)
And the distance between the predicted position and the predicted position (x0, y0). In any case, an arbitrary function representing the degree of separation between (i, j) and (x0, y0) can be obtained. c is
A non-decreasing function of the degree of separation may be used.

As will be described later, since the evaluation value calculation unit 16 inputs the raster scan data of the image f (x, y) as it is and obtains the evaluation value of each part p (x, y), It is preferable that the position evaluation value c for p (x, y) is sent to the evaluation value calculation unit 16 at the timing when the evaluation value of that part is obtained by the evaluation value calculation unit 16. The unit 15 may have, for example, the configuration shown in FIG.

In FIG. 2, 21 and 24 are up / do.
wn counters, 22 and 25 are 0 detection circuits, 23 and 2
6 is a flip-flop (F / F), 27 is an adder, 28
Is a shift register. up / down counter 21
Is used to calculate | i−x0 |, and when a horizontal synchronization signal is input to the LOAD terminal LD, x input from the DATA terminal
By initially counting down the pixel clock signal input to the CK terminal with 0 as an initial value, x
Output 0-i. Note that the origin of the coordinate system is the first data in the raster scan, that is, the upper left of the image.
Then, whether or not the output value x0-i has become 0 is detected by a 0 detection circuit 22 composed of a comparator or the like.
When the 0 detection circuit 22 detects that it has become 0, the output of the flip-flop 23 is inverted and the up / down switching U /
By switching D and counting up,
Output i-x0. Thus, | i−x0 | can be output.

Similarly, the up / down counter 24
| J−y0 | is calculated. When a vertical synchronizing signal is input to the LOAD terminal LD, the horizontal synchronizing signal input to the CK terminal is initially counted down with y0 input from the DATA terminal as an initial value. By doing so, y0-j is output. Then, whether or not the output value becomes 0 is detected by a 0 detection circuit 25 composed of a comparator or the like. When the 0 detection circuit 25 detects that it has become 0,
The output of the flip-flop 26 is inverted, the up / down switching U / D is switched, and the count-up is performed, thereby outputting j-y0. Thus, | j−y0 | can be output.

Up / down counter 21 and up / d
The output of the own counter 24 is added by the adder 27. Then, the output of the adder 27 is sent to the shift register 28 and shifted, so that the multiplication in the case where k in the equation (2) is a power of 2 can be substituted by the shift operation, and the position evaluation is performed. The value c is output. In the examples of FIGS. 2 and 4B, the center (i,
j) and the target predicted position (x0, y0) were used to calculate the position evaluation value c. For example, the lower right point (i0, j0) of the part p (x, y) and the corresponding point It is equivalent to calculate the position evaluation value c using the position coordinates of the lower right point (x0-M / 2, y0-N / 2) of the target predicted position.

Next, the evaluation value calculation unit 16 evaluates the probability that each part of the image f (x, y) having the same size as the template t (m, n) is the target. The sum of the differences D (similarity) between the points of the template t (m, n) and the corresponding points of the part p (x, y), and ii) the position. The position evaluation value c for the part p (x, y) obtained by the evaluation value calculation unit 15 is obtained by adding both.

The evaluation value calculation section 16 can be configured, for example, as shown in FIG. In FIG. 3, reference numeral 30 denotes a template matching LSI (for example, IP90C08 manufactured by Sumitomo Metal Industries, Ltd.), and 31 denotes a 1H line delay for outputting image data for one horizontal axis by FIFO. The template matching LSI 30 includes a template register 30a, a shift register array 30b, a difference absolute value circuit 30c, a summation circuit 30d, a position calculation circuit 30e, an addition circuit 30f, and a register 30g.
It has.

The template register 30a stores the template t (m,
n), and the template setting unit 14 determines that the template t (m, n) is a template matching LS
The stored content can be rewritten each time it is sent to I30. Also, N-1 1H line delays 31 obtained by subtracting 1 from the number N of pixels in the vertical direction of the template t (m, n) are connected in series, and the output from each 1H line delay 31 and the 1H line delay 31 are provided. Are input to the template matching LSI 30 at the same time. The N pieces of data are input to the shift register array 30b, and the image f (x,
Image data of the M × N part p (x, y) of y) is temporarily stored.

The absolute difference circuit 30c calculates the absolute value of the difference between the corresponding data in the template register 30a and the shift register array 30b, that is, the absolute value of the difference in luminance between the corresponding pixels. The summation circuit 30d calculates the sum D of the absolute values of the differences.
The sum D is

[0039]

D (i, j) = Σ | p (i + m, j + n) −
t (m, n) | Further, the position calculation circuit 30e calculates the part p (x, y) currently stored in the shift register array 30b.
A vertical synchronizing signal, a horizontal synchronizing signal, and a pixel clock signal from the timing signal generator 12 are input. Based on these, the latest input raster scan data Position (i0,
j0), and then the center position (i, j) of the portion p (x, y) stored in the shift register array 30b can be obtained (i = i0−M / 2,
j = j0-N / 2).

The D from the summing circuit 30d and the position evaluation value c obtained by the position evaluation value calculator 15 are input to the addition circuit 30f, and D and c are added. The register 30g compares the output of the adder circuit 30f with the already stored data. If the output of the adder circuit 30f is smaller, the data is rewritten to the smaller one and at the same time, the position calculation circuit 30e The obtained position (i, j) is also stored, and the position is output when the processing for one frame is completed.

In this way, the evaluation value D + c for each part p (x, y) can be obtained in real time, and the position (i, j) of the part having the smallest evaluation value D + c, that is, the largest matching is output. . In addition, IP90C08
Is used, the IP90C08 has an input terminal EXin for cascade connection, and is configured to output the sum of the values input from D and EXin as the final matching result. By inputting the position evaluation value c from the position evaluation value calculator 15 to EXin, D and c can be added.

The target extracting section 17 regards the center position (x1, y1) of the portion p (x, y) having the smallest evaluation value D + c output from the evaluation value calculating section 16 as the target position and outputs the target position signal. Output to the predicted position calculation unit 18 and the target frame weight unit 19. The target extraction unit 17 can be configured by software processing executed by a CPU and a memory, for example. Next, the predicted position calculating section 18 predicts the target position in the next frame based on the target position signal extracted by the target extracting section 17.

The calculation of the target predicted position can be performed, for example, by applying the α-β filter expressed by the equation (3) in each of the horizontal direction x and the vertical direction y. In the equation (3), Xm is the position of the target extracted by the target extraction unit 17, Xs is the target smooth position, Xp is the target predicted position, Vs is the target speed, α and β are filter constants,
T represents a time interval between frames, and n represents a frame number.

[0044]

Xs (n) = Xp (n) + α [Xm (n) −Xp (n)] Vs (n) = Vs (n−1) + β [Xm (n) −Xp (n)] / T Xp (n + 1) = Xs (n) + Vs (n) · T (3) This α-β filter is a kind of low-pass filter,
When the value of α or β is reduced, the smoothing characteristics of the filter become remarkable, and when the value of α or β is increased, the tracking characteristics of the filter become remarkable (for the frequency characteristics of the α-β filter, see, for example, DEMayiatis “Comparison of
α-β and Kalman filter in track while scan radar
s "). FIG. 5 shows how to obtain the target predicted position Xp (n + 1) using the α-β filter.

The above-mentioned predicted position calculating section 18 can be constituted by software processing executed by the CPU and the memory. The target frame weight unit 19 superimposes a target frame indicating the position of the target extracted by the target extraction unit 17 on the video signal from the imaging device 11. For example, as shown in FIG. The switching signal generation circuit 42 can be used. The video switch 41 switches between a video signal from the imaging device 11 and a “white” level signal representing a target frame by a switching signal from a switching signal generation circuit 42. The switching signal generation circuit 42
Based on the target position signal, the vertical synchronization signal, the horizontal synchronization signal, and the pixel clock signal, so as to be displayed as a target frame of a predetermined size around the target position extracted in
At a timing corresponding to the target frame, a switching signal for switching the video switch 41 from the video signal to the “white” level signal is output.

In this way, by displaying the extracted target in the target frame centered on the target position on the display unit 10, it is possible to track the target of, for example, an intruder and monitor the action. Further, as another embodiment, the predicted position calculating unit 18 in the target extracting unit 20 is omitted, and the target position signal from the target extracting unit 17 is directly sent to the position evaluation value calculating unit 15 instead. And the position evaluation value c calculated by the position evaluation value calculation unit 15 in the image f (x, y) at the center (i, j) of each part p (x, y) of the image f (x, y) and the previous frame. The extracted target position (x1,
y1) may be a non-decreasing function of the degree of separation.

In this case, the following expression (4) can be used as the position evaluation value c.

[0048]

C = k (| i−x1 | + | j−y1 |) (4) If the target has a relatively slow movement or a small movement, the embodiment sufficiently extracts the target. can do. As described above, according to each embodiment, by introducing the position evaluation value c, the image f (x, y)
Since the matching with the template is performed not only in a part of the region but also in the entire image f (x, y), the possibility that the target cannot be extracted and the target is lost can be reduced, and the target It can also prevent misperception.

By adjusting the value of k in the position evaluation value calculator 15, the weight of the degree of separation as the evaluation value can be changed. Therefore, for example, when the target is an irregular movement, the value of c is reduced, and conversely,
In the case where the target is that the movement is predictive, an appropriate evaluation can be performed according to the situation by increasing the value of c.

Further, according to the present embodiment, an existing commercially available LSI can be used for simple configuration, matching can be performed in real time, and target extraction can be performed. In the present embodiment, the position evaluation value calculation unit 15 and the evaluation value calculation unit 16 use ICs and LSIs. However, the entire target extraction unit may be configured by software executed by a CPU and a memory. is there.

[0051]

As described above, according to the first to third and seventh to ninth aspects of the present invention, the target is extracted from the image f (x, y) by extracting the target. For a template t (m, n) to represent and each part p (x, y) of the image having the same size as the template,
An evaluation value for evaluating the likelihood that the part p (x, y) is the target is calculated using the template t (m, n) and the part p (x, y).
The target predicted position (x0, y0) in the current frame predicted based on the similarity of (x, y) and the representative point of the part p (x, y) and the target position extracted before the previous frame.
From the position evaluation value c determined by the degree of separation from
By performing on the basis of the obtained evaluation value of each part p (x, y), it is taken into consideration that the closer to the predicted position, the higher the possibility that the target exists, so that misrecognition of the target is prevented. And highly reliable target tracking can be performed. Further, since it is not necessary to use a conventional search area, the problem of setting a conventional search area is solved.

Similarly, according to the inventions of claims 4 to 6 and 10 to 12, the image f
The extraction of the target from (x, y) is performed for a template t (m, n) representing the target and each part p (x, y) of the image having the same size as the template.
An evaluation value for evaluating the likelihood that (x, y) is the target is defined as a template t (m, n) and its part p (x, y).
And a position evaluation value c determined by the degree of separation between the representative point of the part p (x, y) and the target position (x1, y1) extracted in the image f (x, y) in the previous frame. , And based on the obtained evaluation value of each part p (x, y), by taking into account that the closer to the target position of the previous frame, the higher the possibility that the target exists will be taken into account. Therefore, erroneous recognition of the target can be prevented, and highly reliable target tracking can be performed. Further, since it is not necessary to use a conventional search area, the problem of setting a conventional search area is solved.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of a target extracting apparatus according to the present invention.

FIG. 2 is a block diagram showing a specific configuration example of a position evaluation value calculation unit 15 in FIG. 1;

FIG. 3 is a block diagram showing a specific configuration example of an evaluation value calculation unit 16 in FIG. 1;

FIG. 4A is an explanatory diagram of an image showing a setting of a template, and FIG. 4B is an explanatory diagram showing a degree of separation between a certain part p (x, y) of the image and a target predicted position of the image. is there.

FIG. 5 is an explanatory diagram of a principle of calculating a predicted position of a target by applying an α-β filter performed by a predicted position calculation unit 18 in FIG. 1;

FIG. 6 is a block diagram showing a specific configuration example of a target frame weight unit 19 in FIG. 1;

FIG. 7 is an explanatory diagram showing a conventional template matching method.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Imaging device 13 A / D conversion part (image generation means) 20 Target extraction means

Claims (12)

[Claims] An image pickup apparatus for picking up an image of a target and outputting a video signal, and sequentially outputting a video signal output from the image pickup apparatus to A / A
A target comprising: an image generating means for performing D conversion and generating an image f (x, y) for each frame; and a target extracting means for extracting the target from the image f (x, y) and outputting a target position signal. In the extraction device, the target extraction means includes a template t (m,
n) and each part p (x, y) of the image having the same size as the template, an evaluation value for evaluating the probability that the part p (x, y) is a target is represented by a template t ( m, n) and the similarity between the part p (x, y) and the target in the current frame predicted based on the representative point of the part p (x, y) and the position of the target extracted before the previous frame. From the image f (x, y) based on the evaluation value of each part p (x, y) obtained from the position evaluation value c determined by the degree of separation from the predicted position (x0, y0). A target extraction device for extracting a target. 2. The evaluation value is a template t (m,
n) and the sum D of the luminance differences between the corresponding pixels of the part p (x, y) and the representative point of the part p (x, y) and the target position extracted before the previous frame. 2. The target extraction apparatus according to claim 1, wherein the target extraction position is obtained by adding a predicted value (x0, y0) of the target in the current frame and a position evaluation value c, which is a non-decreasing function of the degree of separation. . 3. The position evaluation value c is calculated based on a portion p (x,
y) and the target predicted position (x0,
3. The target extraction device according to claim 2, wherein c = k (| i−x0 | + | j−y0 |) where k is a positive constant. 4. An image pickup apparatus for picking up an image of a target and outputting a video signal, and sequentially outputting a video signal output from the image pickup apparatus to A / A
A target comprising: an image generating means for performing D conversion and generating an image f (x, y) for each frame; and a target extracting means for extracting the target from the image f (x, y) and outputting a target position signal. In the extraction device, the target extraction means includes a template t (m,
n) and each part p (x, y) of the image having the same size as the template, an evaluation value for evaluating the probability that the part p (x, y) is a target is represented by a template t ( m, n) and the similarity between the part p (x, y) and the representative point of the part p (x, y) and the image f in the previous frame
The target position (x1, y) extracted at (x, y)
1) and a position evaluation value c determined by the degree of separation, and a target is extracted from the image f (x, y) based on the obtained evaluation value of each part p (x, y). Target extraction device. 5. The evaluation value is a template t (m,
n) and the sum D of the luminance differences between the corresponding pixels of the portion p (x, y) and the representative point of the portion p (x, y) and the image f (x, y) in the previous frame. 5. The target extracting apparatus according to claim 4, wherein the target extracting apparatus obtains the target position by adding the position evaluation value c, which is a non-decreasing function of the degree of separation from the target position (x1, y1). 6. The position evaluation value c is represented by a part p (x,
y) and the image f (x,
From the target position (x1, y1) extracted in y), c = k (| i-x1 | + | j-y1 |) where k is a positive constant. Item 6. The target extraction device according to Item 5. 7. A video signal output from an imaging device for capturing an image of a target is sequentially A / D converted and an image f
In the target extraction method of generating the target as (x, y) and extracting the target from the image f (x, y), extracting the target from the image f (x, y) is performed by using a template t ( m, n) and each part p (x, y) of the image having the same size as the template,
An evaluation value for evaluating the likelihood that the part p (x, y) is the target is calculated using the template t (m, n) and the part p (x, y).
The target predicted position (x0, y0) in the current frame predicted based on the similarity of (x, y) and the representative point of the part p (x, y) and the target position extracted before the previous frame.
From the position evaluation value c determined by the degree of separation from
A target extraction method characterized in that the target extraction method is performed based on the obtained evaluation value of each part p (x, y). 8. The evaluation value of the template t (m,
n) and the sum D of the luminance differences between the corresponding pixels of the part p (x, y) and the representative point of the part p (x, y) and the target position extracted before the previous frame. 8. The target extraction method according to claim 7, wherein the target extraction position is obtained by adding a predicted value (x0, y0) of the target in the current frame and a position evaluation value c that is a non-decreasing function of the degree of separation. . 9. The position evaluation value c is calculated based on the portion p (x,
y) and the target predicted position (x0,
9. The target extraction method according to claim 8, wherein c = k (| i-x0 | + | j-y0 |) k: a positive constant is obtained from y0). 10. A video signal output from an imaging device for imaging a target is sequentially subjected to A / D conversion, and an image f
In the target extraction method for generating (x, y) and extracting the target from the image f (x, y), extracting the target from the image f (x, y) includes a template t ( m, n) and each part p (x, y) of the image having the same size as the template,
An evaluation value for evaluating the likelihood that the part p (x, y) is the target is calculated using the template t (m, n) and the part p (x, y).
The similarity of (x, y) and the degree of separation between the representative point of the part p (x, y) and the target position (x1, y1) extracted in the image f (x, y) in the previous frame Each part p (x, y) obtained from the determined position evaluation value c
A target extraction method characterized in that the target extraction method is performed based on the evaluation value of the target. 11. The evaluation value is a template t (m,
n) and the sum D of the luminance differences between the corresponding pixels of the portion p (x, y) and the representative point of the portion p (x, y) and the image f (x, y) in the previous frame. 11. The target extraction method according to claim 10, wherein the target extraction method is obtained by adding the calculated target position (x1, y1) and a position evaluation value c, which is a non-decreasing function of the degree of separation. 12. The position evaluation value c is calculated based on the part p
The representative point (i, j) of (x, y) and the image f in the previous frame
The target position (x1, y) extracted at (x, y)
12. The target extraction method according to claim 11, wherein c = k (| i-x1 | + | j-y1 |) k: a positive constant is obtained from 1).