JPH0345898A - Image identifying and tracing apparatus - Google Patents

Abstract

PURPOSE:To satisfactorily continue a target tracing even with an image having many noises, under various disturbing environments by employing a shape extraction filter for extracting a specific shape such as circular, elliptical, parallel shape, from a dense and pale image in a target candidate extractor. CONSTITUTION:A target candidate extractor 31 is formed of an edge information extractor 32 and a shape extraction filter 33. An image signal obtained from an image sensor 1 is converted into a dense and pale image signal by an image input circuit 21, and the extractor 32 extracts an edge intensity, dense/ pale change direction at each image according to the dense and pale image signal. The filter 33 extracts a preset specific shape such as circular, elliptical, parallel shape, by using the edge intensity and dense and pale change direction data quantized in eight directions. Feature amounts such as a position of center of gravity, an area, etc., are calculated at each region of a target candidate obtained by the filter 33 via a feature extractor 41, and stored in a feature data memory 62. A feature amount of each candidate of each image stored in the memory 62 is employed in a corresponding relation evaluator 63, which evaluates the corresponding relation of the candidate and determined a target region.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an image identification/tracking device, and particularly to a target identification device and an improvement for improving tracking ability.

[Prior Art] FIG. 5 is a diagram showing the configuration of a conventional image identification/tracking device disclosed in, for example, Japanese Patent Application No. 63-251139. In the figure, (1) is an image sensor, and <2) is an image sensor. Sensor (1)
Converting the image signal obtained by A to a grayscale image signal
/D converter, (3) is a binarization circuit that binarizes the grayscale image signal with a certain threshold, (4> is a binarization circuit 3)
An area measuring circuit that measures the area of the target, etc. that has been converted into a value, <5) a radar device that measures the distance to the target, and (6)
) inputs the density image signal from the A/D converter (2), the area data from the area measuring circuit (4), and the distance measurement data from the radar device (5), and determines whether it is a target or not. (7> is a target extraction circuit that creates a target binarized image signal having level 1 only in the target area from the output of the binarization circuit (3) and the output of the judgment circuit 6); 8〉 is eye +
Measure the center of gravity position of the target area from the IA binary image signal,
This is a center of gravity position measurement circuit that outputs target position data as a tracking signal.

Next, the operation will be explained. Image data captured by the image sensor 1> is converted into a grayscale image signal by an A/D converter (2). This signal is processed by the binarization circuit (3
), the target area is binarized, and the area measurement circuit (4)
Then, the area of each target region is measured. The area data, the gray scale image signal, and the distance data to the target obtained by the radar device (5) are input to the determination circuit (6>).

Now, let us say that the area of the target is S, the brightness level is A, and the distance data is R. When you want to detect a target and the target is far away, S is sufficiently small and A has a value greater than or equal to the bottom value. 1 judgment circuit 6〉 is A/S.
The determination circuit (9> The area with the largest curvature is determined to be the target area.

The target data determined by the determination circuit <6> is detected by the target detection circuit (7>) as a target 2 which has level 1 only in the target area.
is converted into an image signal, and the center of gravity position of the target area is calculated by the center of gravity position measuring circuit (8) and output as a tracking signal.

[Three problems to be solved by the invention] Since the conventional image tracking device is configured as described above, it is possible to determine the target area using only the distance measurement data, the area data after binarization, and the luminance information. Therefore, when the radar equipment is interfered with and cannot obtain correct distance measurement data, erroneous detection of small, high-intensity IR flares, etc. may occur, and the extraction of information from image data is difficult. Since the method only uses brightness information and does not make judgments based on shape information, there was a problem in that it was insufficient to separate out background noise such as horizon lines that appear to be highly bright in infrared images, clouds, and pseudo heat sources.

The second invention was made to solve the above-mentioned problems, and uses shape information to separate background noise such as IR flares, horizon lines, clouds, and pseudo heat sources, and to eliminate noisy images and The purpose of the present invention is to obtain an image identification/tracking device that can effectively identify and track a target even under various disturbing environments.

[Means for Solving the Problems] An image identification/tracking device according to the present invention includes an edge information extraction circuit that uses the output of an image input circuit as input to extract edge intensity and direction of density change, and this edge information extraction circuit. A shape extraction filter that extracts specific shapes such as circles, ellipses, and parallel shapes from the output of the target candidate extraction circuit that outputs target candidates, and a target candidate in multiple consecutive images obtained from the feature extraction circuit. Based on the feature data storage unit that stores the feature amounts related to the image, the feature data of the area and center of gravity position information of the target candidates between multiple consecutive images, and the movement information of the amount of movement between the target candidates that have already been matched, It consists of a correspondence evaluation section that evaluates the correspondence of target candidates, and uses a determination circuit that determines the target area.

[Function] The shape extraction filter of the present invention extracts a specific shape such as a circle, ellipse, parallel shape, etc. from a grayscale image at high speed, and the determination circuit extracts a specific shape such as a circle, an ellipse, a parallel shape, etc.
Since the target area is determined as the bottom of the tank, determination using shape information and movement information becomes possible, and good identification and tracking becomes possible.

[Example] Hereinafter, an example of the present invention will be described with reference to the drawings. 1st
In the figure, (1> is an image sensor, (21> is an image input circuit that multi-values the image signal obtained from the image sensor (1) for each pixel, and (32) is an edge strength and density Edge information extraction circuit that extracts the direction of change, etc. (33
〉 is a shape extraction filter that extracts a set specific shape such as a circle, ellipse, parallel shape, etc. from edge strength and density change direction data, 〈31) is an edge information extraction circuit (32〉) and a shape extraction filter (33) (41) is a feature extraction circuit that extracts feature quantities such as area and centroid position for each region of the target candidate, and (62) is a feature obtained by the feature extraction circuit (41). (63) is a correspondence evaluation unit that evaluates the correspondence from the feature data of the target candidate between the plurality of consecutive images; <61> is the feature data Storage part (
62) and a correspondence evaluation section 63).

Next, the operation will be explained. The image signal returned from the image sensor (1>) is converted into a grayscale image signal by the single image input circuit (21>).The edge information extraction circuit (32>) uses the grayscale image signal to determine the edge strength and direction of grayscale change in each image. This is performed, for example, by a spatial product-sum calculation of 3 rows and 3 columns.Processing countermeasure image data X + #, n l +
When the weighting coefficients W ((, j) are arranged as shown in FIG. 2(a), the spatial product-sum calculation results Y +s, n+ are expressed by a linear equation.

At this time, when the two types of load coefficients shown in Fig. 2 <b> are used as load coefficients to determine the amount of change S8 in the horizontal direction and the amount S8 of change in the vertical direction, the direction of edge strength and density change is determined respectively. SX I +l S, l tan-'Sy/Sx.

The shape extraction filter (33) uses edge strength and direction data of shading changes quantized in 8 directions to extract circles, ellipses,
Extract a specific shape set in advance, such as a parallel shape. The extraction principle of the shape extraction filter (33) will be explained with reference to FIG. The figure is an explanatory diagram when extracting a circle, and as shown in FIG. 3(a), the direction of the change in density at the edge portions that make up the circle all points toward the center. Therefore, Figure 3(b)
As shown in , corresponding to the direction of the density change at the edge,
Spokes of length S apart (line elements)
to occur. As shown in FIG. 3(C), when spokes are generated from each direction, the spokes intersect near the center of the circle. The shape extraction filter is a method of determining the presence or absence of a circle by looking at this intersection situation, and S corresponds to the size of the circle to be extracted and L corresponds to its allowable amount. Furthermore, when extracting parallel shapes, the center line of the parallel lines is extracted by utilizing the property that the directions of edges from opposite parallel lines are opposite. As described above, shape extraction filters can perform stable extraction that is resistant to noise by limiting the shape and size to be extracted, and simultaneously using features with shape information added to the intensity and direction information of edges. conduct. In particular, in infrared images showing heat distribution, targets often have a circular, elliptical, or parallel shape, and this is an effective method for extracting a target area. Furthermore, since shape information is used, minute high-intensity ]R flare,
This means that the influence of clouds, horizontal 11A, etc. can be removed.

The present inventors have already developed an ``image shape recognition method'' (Japanese Patent Application No. 62-330960) as a method for extracting a specific shape, and a ``spoke register generation circuit'' as a high-speed execution method for a shape extraction filter.
(Japanese Patent Application No. 63-255102), and this invention relates to their application.

For the target candidates returned by the shape extraction filter (33), feature quantities such as the center of gravity position and area are calculated for each region of the target candidate in the feature extraction circuit <Step 4>.The feature quantities calculated for each image is further stored in the calculated feature data storage unit <62) for each of the plurality of consecutive images. The correspondence evaluation unit (63) uses the feature amount for each target candidate in each image stored in the feature data storage unit <62> to evaluate the correspondence of the target candidates and determines the target area.

Now, the target candidate Q obtained from the image at a certain time t,
(t), its center of gravity is x, (t>, y:
(t>, area is expressed as S+(t). Here, i=
(..., Nt, Nt is the total number of target candidates drawn from the image at time t.

For the sake of explanation, let us consider the case where the correspondence between four consecutive frames of time tt+t++++t++2+tl or 3 is evaluated. Target candidate Q- at time t,,tj
(tk>, Qb(tz>[n=1.-, Ntm,
b=1. −Ntt, tb≠tt, t, tz=t,・
・Select 2 frames from 4 frames for t,, 3 images between all 6 combinations of images (1, and t, .

t, and ti**, J and ij+l+fj+I and t3+2.
tl*+ and t, +2. il+2 and t, -3), the correspondence is evaluated for round-robin combinations of target candidates. The presence or absence of a correspondence relationship is determined by the distance n o between target candidates shown in the following formula
, b, is determined based on the area difference M, b.

Dab2: [xb(tz)-xs(tk)]'+[yb
(tz)-y, (t,)]2...(2)M-b"
Sb (尤)-Ss (ih)...
...(3) Threshold value T at which both Dab and Mab exist
+, when smaller than T2, Qa (tk) and Qb (
Suppose that there is a correspondence relationship between tz ). Furthermore, if two or more target candidates correspond to one target candidate in the same frame, the target candidate that minimizes the product of Dab, mouth, and b is selected.

Next, the connection between the four frames in correspondence will be evaluated. That is, 1. When viewed in 4 frames of ~t,,3,
Between all 2 frames (1, and il”l+ ij+1 and i
4 +2+ i j + 2 and ij+3), or whether there is a correspondence in only 3 out of 4 frames ((1,,1, or I
+il+2)* (jj+t)+1° to door 3)・
(tj・ti・2・fi+3)+<f''・1・t
i, 2, t'', and 3>), or whether there is a correspondence only between two frames in four frames.

Naturally, a target area is determined to be a target area if a correspondence relationship can be established in many frames.

The flow of processing will be explained using FIG. Each time fi, -
The feature data of the target candidate extracted from the image 1i+"3 is stored in the feature data storage section <62>.

FIG. 4 shows each time ti, -ti. The numbers of target candidates extracted from images No. 3 are 5, 6, 3, and 4, respectively. Next, the correspondence between the two frames is evaluated using equation 2.3. As a result, as shown in the figure, Q3(t,) and Q2(t4.+), Q5(J) and Q
e(t;-+).

Q2(t, 1) and Qtomi(t7*3>-Qs(ti, 2
) and Qa(tl.3), we further examine the correspondence between 4 frames and find that the correspondence exists in 3 frames: Q3<t,), Q2(t;.+), Q+
<t7 or 3> is determined to be the target area and output. That is, by looking at the correspondence between successive frames, it becomes possible to perform good tracking even if there are frames in which it is difficult to detect ζ due to the influence of lJ, clouds, etc. Also, the next frame 1.・
Regarding the target candidates in 4, it is sufficient to simply look at the correspondence with the already detected target areas, or similarly to the above, the target candidates tJ and 1 to t, 1 may be evaluated.

Furthermore, the target area has already been determined, and at the stage of tracking it, the amount of movement V in the X direction and Y direction per frame,
, V, and use the following equation instead of equation 2 to utilize motion information.

Dab2”[Xb(tt)−(L(ttt)+vx(t
t-ttt)l]2+[Xb(tz>-fL(tb)
+V, (tt-ttt)l]2... (4) In the above embodiment, the correspondence evaluation unit evaluates the correspondence between two frames and the correspondence between four frames. Although the case of evaluation has been described, evaluation may be performed using any frame.

In addition, target candidates are extracted by approximating the shape of the target to a circle, ellipse, parallel shape, etc., so when the target area expands due to the approach of the target, etc., it can be approximated to a circle, ellipse, parallel shape, etc.! ? There is a problem that the target candidates cannot be extracted easily.

For this purpose, the determination circuit (61) detects that the area of the target area exceeds a certain value, and the image input circuit (21) detects that the area of the target area exceeds a certain value.
), the image is reduced in size in response to the increase in area (control indicated by the wavy arrow in FIG. 1), and tracking is continued. kOh, image reduction is done by thinning out pixels or by 2
Equation ζJ can be easily realized by a method of integrating neighboring pixels, such as taking the average value of two pixels as one pixel.

[Effects of the Invention] As described above, according to the present invention, the target candidate extraction circuit has the following functions:
Using a shape extraction filter that extracts specific shapes such as circles, ellipses, parallel shapes, etc. from grayscale images, a determination circuit identifies target candidates based on the area, center of gravity position, and movement information of target candidates between multiple consecutive images. Since it is configured to evaluate the correspondence quantity 1 system, it is possible to identify the target well even in a noisy image, and it has the effect of being able to continue tracking the target well even under various interference environments. be.

[Brief explanation of drawings]

Fig. 1 is a block diagram showing the configuration of an image identification/tracking device according to an embodiment of the present invention, Fig. 2 is an explanatory diagram of spatial product-sum calculation for performing edge information extraction processing, and Fig. 3 is an extraction diagram of a shape extraction filter. FIG. 4 is an explanatory diagram of the principle, FIG. 4 is an explanatory diagram showing the processing contents of the determination circuit, and FIG. 5 is a block diagram showing the configuration of a conventional image tracking device. In the figure, (1) is an image sensor, <2> is an A/D converter, (21> is an image input circuit, <3) is a binarization circuit, and (3) is an image input circuit.
1> is a target candidate extraction circuit (32> is an edge information extraction circuit, (33) is a shape extraction filter, <4> is an area measurement circuit, (41> is a feature extraction circuit, (5> is a radar device, <6
〉, <61> is a determination circuit, <62) is a feature data storage unit, (63) is a correspondence evaluation unit, (7> is a target extraction circuit,
8> is a center of gravity position measuring circuit. Note that the same symbols in the figures indicate the same or equivalent parts.

Claims (1)

[Claims] An image input circuit that multivalues the image signal obtained from the image sensor for each pixel, and 2
a target candidate extraction circuit that extracts target candidates by converting them into values, etc.;
An image identification/tracking device comprising a feature extraction circuit that extracts feature quantities such as area and centroid position for each region of the target candidate, and a determination circuit that determines a target region by evaluating the feature quantities, An edge information extraction circuit that takes the output of the image input circuit as input and extracts the edge strength and direction of density change, and a shape extraction filter that extracts specific shapes such as circles, ellipses, and parallel shapes from the output of this edge information extraction circuit. a target candidate extraction circuit that outputs target candidates; a feature data storage unit that stores feature amounts related to the target candidates in the plurality of consecutive images obtained from the feature extraction circuit; and a correspondence evaluation unit that evaluates the correspondence of target candidates based on the feature data of the area and center of gravity position information of the target candidates and the movement information of the amount of movement between target candidates that have already corresponded. An image identification/tracking device characterized by comprising: a determination circuit for determining.