JP3461143B2 - Color image target position detection device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a color target position detecting device for performing target tracking with a visible image taken by a television camera.

[0002]

2. Description of the Related Art Conventionally, in a target tracking device of this type, automatic tracking in the image pickup direction is realized by detecting the movement of a target object from an image taken by a television camera. As a method of detecting this motion, a method of detecting the position of the block having the maximum correlation with respect to the reference image block defined by the peripheral image of the target object or the image of the characteristic portion from the captured image is generally used. Has been.

In the case of this method, when the size of the target object is changed or deformed, there is a drawback that tracking becomes difficult, and in addition, the circuit scale for obtaining the correlation becomes extremely large. Further, there has been a problem that when the circuit scale is reduced by the CPU processing, the processing load increases and the real time performance is lost.

Therefore, some methods have been proposed which use the color information of the target based on the color image signal of the target. As one of them, for example, R of the color image signal,
Consider G and B as components of a vector on Cartesian coordinates, and use the unit vector of the color to be tracked as a reference vector, the dot product with the R, G, and B vectors of the image signal.

[0005]

[Equation 5] There is a method of obtaining the difference in the vector direction as cos θ (where θ is the angle formed by the vector of the corresponding pixel of the captured image and the reference vector) by obtaining

Further, Japanese Patent Laid-Open No. 4-184283 discloses R, G, B of color image signals as a method for improving the ability of target identification.

[0007]

[Equation 6] Or

[0008]

[Equation 7] According to the method of converting to r, g, b colorimetric chromaticity values and comparing, a determination range is set for two sets of reference values from the target r or b and g, and the correspondence of the captured image is set. The comparison circuit determines r or b and g of the pixel to be processed, and if both values are within the determination range, the pixel is binarized to obtain the target area, and then the brightness value (R + G + B) is obtained. The method of detecting the position at the maximum brightness point in the target area of 1) is proposed.

[0009]

In the above-mentioned conventional example, although the information on the lightness and darkness of the image is excluded and the influence of the lightness and darkness is suppressed, since the evaluation is made by cos θ, the characteristics of the cosine function when the colors are close to each other. In addition, the values are almost the same, which makes it extremely difficult to identify the target point.

Further, in order to obtain the value of cos θ, it is necessary to divide by the absolute values of the R, G and B vectors of the image signal. Therefore, in digital high speed processing, the reciprocal is generated by tabulation by ROM or the like. The method of multiplying this is used,
This method also has a drawback that the reciprocal decreases and the ratio of the quantization error increases as the values of R, G, and B increase, and this increases by the multiplier, which increases the target detection error.

Further, according to the method of the above-mentioned Japanese Patent Laid-Open No. 4-184283, the target point can be identified more easily and the detection capability is improved compared to the former method, but the same reason as the former method is used. Therefore, in the case of digital calculation, since direct division is performed to obtain r, g, and b, (R
In the image region where the value of + G + B) is large and the image is bright, the quantization noise increases.

Therefore, the drawback of causing an error in the target determination due to this error has not been solved, and since the maximum luminance point is the target detection position at the final stage, it is within the target area determined by the chromaticity determination. However, there is a drawback that the detection position jumps due to the change in the luminance distribution of the.

On the other hand, the target is obtained by using two stimulus value signals of RY and BY having information relating to the hue and saturation with respect to the Y stimulus value, without using the above division operation. A method of determining the position of an object and issuing a tracking signal is disclosed in Japanese Patent Laid-Open No. Hei 8 (1998)
-51564.

However, in this method, in order to determine the target without being influenced by the brightness of the target, the relationship between the two stimulus values of RY and BY of the target and the Y stimulus value (luminance) is calculated. It is necessary to set the modeled color prediction function and the allowable error range for judging the target considering this functional model, and whether the model setting and the allowable error range setting are good or bad affects the characteristics of detection or misidentification. Has the drawback.

An object of the present invention is to provide a color target position detecting means capable of eliminating the drawbacks of the above methods.

[0016]

A color target position detecting apparatus of the present invention captures a target color image and outputs three color signals of red (R), green (G) and blue (B). By performing a predetermined calculation on the image signals R, G, B from the sensor, the tristimulus value signals X, Y corresponding to the color signal input in real time for each pixel output from the sensor,
After conversion into Z, signals x and y at chromaticity coordinate points that do not include information on the magnitude of the stimulus value corresponding to the lightness and darkness of the image are further calculated from these three stimulus value signals X, Y, and Z, and this color is calculated. It is characterized in that the chromaticity difference signal ΔC is obtained from the signals x and y of the degree coordinate point, and the target is detected using this chromaticity difference signal ΔC.

Since the chromaticity coordinate point is directly related to the unique spectral reflectance characteristic of the target, it is possible to make an accurate determination without being caught by the brightness of the target and to detect the target position with few errors. . Furthermore, since the values of x and y are values of "1" or less, it becomes easy to scale the processing values required in the circuit design in the subsequent processing.

Further, according to the present invention, it is also possible to combine the chromaticity signals x and y and the stimulus value signal Y and perform the target detection based on the signals x and y of the chromaticity coordinate point and the stimulus value signal Y.
This is because the stimulus value signal Y corresponds to the brightness of the color light, and thus the Y stimulus value is used as it is for target detection and the chromaticity values, x, and y are combined to enable more accurate detection determination.

Further, according to the present invention, the signals x and y at the chromaticity coordinate points are used.
A method of performing logarithmic compression and then subtracting can be used for the calculation for obtaining. This is because the color signals R, G, B are converted into the stimulus value signals X, Y, Z, so long as all of R, G, B are "0", the stimulus that is the numerator of x, y. Since the values X and Y never become "0", it can be adopted due to the effect obtained by the stimulus value conversion into X and Y that stable logarithmic compression is possible in the area where the image is captured. Therefore, the dynamic range of the stimulus values that can be handled to obtain x and y can be extremely widened, and the division circuit can be omitted.

Further, according to the present invention, the positional deviation of the picked-up image from the reference of the coordinates in the chromaticity space with reference to the target chromaticity value to be detected is converted into one signal value corresponding to the distance information. Then, the target point is detected by detecting the point where this value is the minimum. As a result, the scale of the processing circuit can be reduced.

The features of the present invention will be described below with reference to FIG. In the present invention, the color image sensor 101
The three image signals R, G, and B of red, green, and blue from the image are converted by the stimulus value conversion unit 1 in real time for each pixel according to the color signal input.

[0022]

[Equation 8] Is converted into tristimulus value signals of X, Y, and Z.

R, G, B from the color image sensor 101
As shown in FIG. 13, the signals are time-series signals R (α, β), G (α, β), B on the entire surface of the frame of the image pickup screen 501 pixel by pixel according to the signal output order 502.
Since it is output as (α, β), the tristimulus value signals of X, Y, and Z are time-series signals X similar to the signals of R, G, and B.
(Α, β), Y (α, β), Z (α, β). Where α represents the pixel position of the horizontal pixel row on the screen, and β
Represents the position of the vertical pixel row on the screen and corresponds to the coordinates on the screen.

In the present invention, from the values of the tristimulus value signals of X, Y and Z,

[0025]

[Equation 9] Is converted into a signal of x, y chromaticity coordinate points that does not include information on the magnitude of the stimulus value corresponding to the lightness and darkness of the image, and time-sequential chromaticity signals x (α, β), y (α, β ) Is output.

The chromaticity coordinate signal x (α,
β), y (α, β), and a designated point data holding unit 13 to which the stimulus value Y (α, β) is input, and the coordinates α, β
X of the pixel that coincides with the designated initial capture point Gto represented by
(Α, β), y (α, β) and the stimulus value Y (α, β) are selected and held, and this is updated for each image frame.

This held data is the specified point chromaticity signal xt.
o, yto, and the designated point stimulus value Yto are output to the reference chromaticity updating unit 4 and the reference luminance updating unit 10 as outputs of the designated point data holding unit 13.

In the reference chromaticity updating section 4, the designated point chromaticity signal xt is set only when the setting completion timing signal To is input.
The values o and yto are updated as they are to the values of the reference chromaticity signals xo and yo and are output. On the other hand, the reference brightness update unit 10 updates the designated point stimulation value Yto to the value of the reference brightness signal Yo and outputs it only when the setting completion timing signal To is input. That is, the reference chromaticity signals xo and yo and the reference luminance signal Yo are initial chromaticity and luminance values of the target point captured at the input of the setting completion timing signal To.

After the setting completion timing signal To is received, the reference chromaticity updating unit 4 outputs the minimum minimum point chromaticity value which is the output of the minimum point data holding unit 6 corresponding to the chromaticity of the obtained tracking target point. xpm and ypm are input for each frame, the reference chromaticity of the next frame is predicted or filtered, and new reference chromaticity signals xo and yo are output.

Further, in the reference luminance updating section 10, similarly,
The minimum minimum point luminance value Ypm of the minimum point data holding unit 6 for which the minimum minimum point chromaticity values xpm and ypm corresponding to the chromaticity of the tracked target point are input, and prediction or filtering processing is performed, It is used as a new reference luminance signal Yo.

Therefore, using the values of the reference chromaticity signals xo and yo and the reference luminance signal Yo as reference values, the range of the chromaticity coordinate difference distance of the target and the distribution limit of the stimulus value Y are set. Since the chromaticity difference signal binarization unit 7 and the luminance signal binarization unit 8 capable of binarizing an inner region are included, the target existing region can be output as a binarized image. it can.

According to the present invention, the reference chromaticity signals xo and yo.
And the chromaticity signals x (α, β), y from the chromaticity coordinate calculation unit 2
The chromaticity difference calculation unit 3 calculates a distance from (α, β) and outputs the result as a chromaticity difference signal ΔC (α, β) (ΔC map). The chromaticity difference signal ΔC ( a minimum point detection unit 5 that determines α, β) and detects a point where this signal has a minimum.
have.

Further, in the present invention, the minimum point data holding unit 6 is provided, and the stimulus value Y (α, β), the chromaticity signal x (α, β), y.
(Α, β) and the chromaticity difference signal ΔC (α, β) are input, and the chromaticity difference signal ΔC (α, β) is input from the minimum point detection unit 5.
Local minimum point detection timing signal p
Although m is input, the minimum point data holding unit 6 takes in as a chromaticity difference signal ΔC (α, β) of the minimum point each time pm occurs, and compares this value with the value obtained in the past and the current value. Then, the chromaticity difference signal ΔC (α, β) of the smaller minimum point is held.

At this time, the held chromaticity difference signal ΔC (α,
The stimulus value Y (α, β) and the chromaticity signals x (α, β), y (α, β) input in correspondence with β) are also held at the same time. As a result, Y (α, β), x (α, β), y (α, β) corresponding to the minimum local minimum value of the chromaticity difference signal ΔC (α, β) at the time of completion of inputting the color signal of one frame. ),
And ΔC (α, β) can be obtained, which are respectively the minimum minimum point luminance value Ypm and the minimum minimum point chromaticity value xpm, y.
pm and the minimum minimum point chromaticity difference ΔCpm.

In the present invention, the chromaticity difference signal binarization unit 7 and the luminance signal binarization unit 8 are provided with the chromaticity difference signal ΔC (α, β) and the stimulus value Y.
It has a ΔC map frame memory 71 shown in FIG. 4 and a luminance map memory 81 shown in FIG. 5 for temporarily storing (α, β) as an image map for one frame. Therefore, the map information stored from here is stored. By reading, the same chromaticity difference signal ΔC (α,
β) and the image map of the stimulus value Y (α, β) by ΔCpm
And Ypm as reference values, the range of the chromaticity coordinate difference distance that the target has and the distribution limit of the stimulus value Y are set, and two regions are binarized by the threshold value that binarizes the region where both are within the limits. The image outputs BΔC (α, β) and BY (α, β) can be obtained.

The logical product B (α,
Since it has the binarized image centroid point calculation unit 11 that calculates the final surface centroid according to β), the binarized area centroid point of the target existing region can be output as the target detection position signal G.

As described above, in the present invention, since the chromaticity coordinate point directly related to the unique spectral reflectance characteristic of the target is used as the target of evaluation for target detection, the spectral condition of the irradiation light with respect to the target is determined. If the change is small, it means that an accurate feature amount that is not caught by the brightness of the target is determined. Therefore, the target position can be detected with less error.

Further, in the chromaticity coordinate calculation unit 2 for obtaining chromaticity coordinates, the stimulus values X (α, β) and Y (α,
A method of obtaining chromaticity coordinates in a state where β) and Z (α, β) are logarithmically compressed by the following calculation is adopted.

[0039]

[Equation 10] This is because the stimulus values X and Y will never be "0" unless R, G, and B are all "0", so stable logarithmic compression is possible in the area where the image is captured. It became possible to be adopted because.

Therefore, when logarithmic compression is used to perform digital operation on the image signals R, G and B in an extremely wide range, the number of bits of data is greatly increased in consideration of the data range of the stimulus value. It also has the excellent feature that it is possible to prevent an increase in the error in the chromaticity calculation.

Further, as for the method of obtaining the chromaticity difference, as shown in FIG. 8, the reference chromaticity signals xo and yo obtained by the reference chromaticity updating unit 4 are used as chromaticity coordinates 2 at the target designated point in the chromaticity coordinates.
03, the chromaticity signals x (α, β), y (α, β) of each pixel output from the chromaticity coordinate calculation unit 2 in time series are used as the input image chromaticity coordinates 202, and between these two points. Direct distance r of coordinates
Is considered to be the chromaticity difference, the chromaticity difference calculation unit 3 of the present invention

[0042]

[Equation 11] The calculation is adopted as the chromaticity difference ΔC. Note that ABS represents an absolute value.

According to this method, with respect to the direct distance r between the target chromaticity coordinate point and the reference chromaticity coordinate point,

[0044]

[Equation 12] If the direct distance between the target chromaticity coordinate point and the reference chromaticity coordinate point is not “0”, it will be larger than the distance r in the above equation between the coordinate points in most cases, and In addition to the characteristic that it has a sharp target discriminating ability, it also has the characteristic that it can be realized by a simple addition / subtraction circuit.

In addition, since the present invention is not intended for display, the output of the image pickup device is directly output as an RGB signal without performing masking processing or the like in consideration of color reproducibility so as to approximate an ideal characteristic having a negative spectral characteristic. It is supposed to be used as. In this case, since the output of the image pickup device of the color image sensor does not have a negative sensitivity region, as shown in FIG. 9, the chromaticity value of the coordinates located outside the chromaticity range 201 of the RGB signals of the color image sensor It never happens.

The range 201 of the chromaticity by the RGB signal of this color image sensor is a straight line 301 where x + y = 1,
It is in the range surrounded by the coordinate axes of x and y and the limit line 302 that satisfies ΔC ≦ 1.0 with respect to the point O, and therefore Δ
The magnitude of C is also always in the range of "0" to "1", which has an advantage that it is easy to scale the data of the circuit that processes ΔC as in the case of r.

[0047]

FIG. 1 is a block diagram showing an embodiment of a color image target position detecting device of the present invention. 2 to 5 show examples of the chromaticity coordinate calculation unit 2, the chromaticity difference calculation unit 3, the chromaticity difference signal binarization unit 7, and the luminance signal binarization unit 8 of FIG. 1, respectively. Shows. Further, FIG. 6 is a block diagram showing an example of connection using the color image target position detection device of FIG.

First, the outline of the target tracking operation using the color image target position detecting device of the present invention will be described with reference to FIG.

Color image target position detecting apparatus 10 of the present invention
Reference numeral 0 denotes a color image sensor 101 that captures an image of the imaging target 106 and outputs three image signals R, G and B of red, green and blue.
An image display device 103 that displays a picked-up image by image signals R, G, and B, an initial capture designation input device 102 that designates a target to be tracked by an operator from the display screen at an initial stage, and a target detection from a color target position detection device 100. The tracking signal output device 105 that outputs the target tracking drive signal DT for tracking the target by the position signal G and the gimbal device 104 that changes the visual axis thereof by the target tracking drive signal DT are used in combination.

In the tracking operation, first, the gimbal device 104 is directly operated to align the visual axis of the color image sensor 101 with the approximate target direction, and the image display device 10 is operated.
The tracking target 108 is imaged on the display screen 107 of No. 3.

The operator displays the target to be tracked on the display screen 107.
Then, the initial capture designation input device 102 is used to set a tracking target point and to set a window for prohibiting capture to a similar target to be excluded. On the display screen 107 of the image display device 103, the tracking target point and the window set by the initial capture designation input device 102 are displayed in an overlapping manner with the images captured as the initial capture designated point marker 107b and the initial capture window marker 107a, respectively. Therefore, the tracking target point and the window can be set while monitoring this.

This set value is used for the initial capture designation input device 10.
Initial acquisition designated point Gto represented by coordinates α and β on the screen in 2
And an initial capture window WGto are generated and sequentially output to the color image target position detection device 100 of the present invention.

When the operator completes the setting, he or she performs a switch operation or the like, which causes the initial capture designation input device 102.
Then, the setting completion timing signal To is generated and output to the color image target position detection device 100 of the present invention.

The color image target position detection device 100 uses the initial capture designation point Gto from the initial capture designation input device 102.
And the initial acquisition window WGto at the time of inputting the setting completion timing signal To, and this point is set as the initial tracking target point. After that, the color image target position detection device 100
The target position is detected based on the feature amount of the tracking target point obtained from the color image, and the target detection position signal G is sequentially output.

The tracking signal output device 105 outputs a target tracking drive signal DT for controlling the gimbal device 104 so that the target detection position signal G becomes the coordinate point at the center of the screen. As a result, after the setting completion timing signal To is generated, the visual axis of the color image sensor 101 continues the operation of tracking the movement of the target for which capture is designated.

Next, the configuration of the color image target position detecting device 100 of the present invention will be described with reference to FIGS.

The color image target position detecting device of the present invention is
The color image signal (red) R, the color image signal (green) G, the color image signal (blue) B, which are the inputs from the color image sensor 101, and the initial, which is the input from the initial capture designation input device 102 shown in FIG. The target detection position G is output by inputting six signals including the designated capture point Gto, the initial capture window WGto, and the setting completion timing signal To.

Color image signals (red) R, (green) G,
The three (blue) B are connected to the three input terminals of the stimulus value conversion unit 1, and the three output terminals of the stimulus value conversion unit 1 for the X, Y, and Z stimulus values are connected to the chromaticity coordinate calculation unit 2. It is connected. The outputs of the chromaticity coordinates x and y of the chromaticity coordinate calculation unit 2 are connected to the chromaticity difference calculation unit 3, and the output chromaticity difference signal ΔC is input to the minimum point detection unit 5.

At the input terminal of the minimum point data holding section 6, the minimum point detection timing signal pm from the minimum point detection section 5, the chromaticity difference signal ΔC from the chromaticity difference calculation section 3, and the chromaticity coordinate calculation section 2 are input. The chromaticity signals x and y and the luminance signal Y (Y stimulus value) from the stimulus value conversion unit 1 are input.

In the minimum point detecting section 5, the evaluation area generating section 1 for limiting the detection area of the minimum point of the chromaticity difference signal ΔC.
The detection area signal WG which is the input from FIG.
As shown in FIG. 2, the chromaticity difference signal ΔC input in time series is sequentially determined in the range of the pm generation range designation area 404, and the point ΔCi at which the signal of the chromaticity difference signal ΔC satisfies the minimum value condition, When j is detected and the pm generation point 403 is scanned, the minimum point detection timing signal pm is output.

The minimum point data holding unit 6 takes in the chromaticity difference signal ΔC of the minimum point every time pm occurs, and this value is compared with the value fetched in the past and the present value, and the minimum point of the smaller one is compared. The chromaticity difference signal ΔC is held. At this time, the luminance signal Y and the chromaticity signals x and y input corresponding to the held chromaticity difference signal ΔC are simultaneously held.

As a result, Y, x, y, and ΔC corresponding to the minimum minimum point value of the chromaticity difference signal ΔC can be obtained at the time when the color signal input for one frame is completed, and the minimum point data holding unit 6 is obtained. These are output from the output terminal of as the minimum minimum point luminance value Ypm, the minimum minimum point chromaticity value xpm, ypm, and the minimum minimum point chromaticity difference ΔCpm.

Further, the chromaticity signals x and y obtained from the chromaticity coordinate calculation unit 2 and the luminance signal Y obtained from the stimulus value conversion unit 1 are input to the designated point data holding unit 13 and the main color image is obtained. The chromaticity x, y and the stimulus value Y of the pixel at the screen coordinate point that coincides with the initial capture designated point Gto, which is the input of the target position detection device 100, are held and updated for each image frame.

The chromaticity x and y and the luminance signal Y are designated point chromaticity signals xto and yto and designated point stimulus value Yto, respectively, from the designated point data holding section 13 to the reference chromaticity updating section 4 and the reference luminance updating. It is output to the unit 10.

At the input end of the reference chromaticity updating section 4, designated point chromaticity signals xto and yto from the designated point data holding section 13 and the minimum minimum point chromaticity value xp from the minimum point data holding section 6 are input.
m and ypm are input.

In the reference chromaticity updating unit 4, the designated point chromaticity signal xt is set only when the setting completion timing signal To is input.
The values of the reference chromaticity signals xo and yo are updated as they are with o and yto, and in other cases, the minimum minimum point chromaticity values xpm and ypm are updated.
Is used to predict or filter the reference chromaticity of the next image frame, update to new reference chromaticities xo and yo, and output. The output of the reference chromaticity update unit 4 is connected to the input end of the chromaticity difference calculation unit 3.

On the other hand, at the input end of the reference brightness updating section 10,
The designated point stimulus value Yto from the designated point data holding unit 13 and the minimum minimal point luminance value Yp from the minimal point data holding unit 6
m is input. In the reference luminance updating unit 10, the designated point stimulus value Y is obtained only when the setting completion timing signal To is input.
At to, the value of the reference luminance signal Yo is updated and output as it is. In other cases, the minimum minimum point luminance value Ypm is used.
The reference chromaticity of the next image frame is predicted or filtered, updated to a new reference luminance signal Yo, and output.

The output Yo of the reference brightness updating unit 10 is connected to one of the two input terminals of the brightness signal binarizing unit 8. The luminance signal Y to be binarized from the stimulus value conversion unit 1 is input to the other input end of the luminance signal binarization unit 8, and the luminance image 2 is output from the output end of the luminance signal binarization unit 8. The binarized signal BY is output and connected to one of the three input terminals of the AND circuit 9.

The minimum minimum point chromaticity difference ΔCpm from the minimum point data holding section 6 is input to one of the two input terminals of the chromaticity difference signal binarization section 7, and the chromaticity difference is input to the other input terminal. Calculation unit 3
The chromaticity difference signal ΔC to be binarized from is input. The ΔC map binarization signal BΔC is output to the output end of the chromaticity difference signal binarization unit 7 and is connected to another one of the three input ends of the AND circuit 9. A binarized window area signal w that limits the binarized output image area, which is the output of the evaluation area generation unit 12, is connected to the third input terminal of the AND circuit 9.

In the AND circuit 9, as shown in FIG. 10, the binarized image signal B obtained by the logical product of the luminance image binarized signal BY, the ΔC map binarized signal BΔC and the binarized window area signal w. To output the binarized image centroid point calculation unit 1
You have entered 1.

In the binarized image centroid calculation unit 11, FIG.
As shown in, the binarized area S having the area centroid (αG, βG) of the binarized area of B = 1 as the target detection position G and the value of the number of pixels of B = 1 at the same time Is calculated and output from the binarized image centroid calculation unit 11. In addition,
The target detection position G is output from the color image target position detection device.

The evaluation area generating unit 12 for generating the detection area signal WG for limiting the detection area of the minimum point and the binarized window area signal w for limiting the binarized output image area which coincides with this area. At the input end of, the target detection position G and the binarized area S from the binarized image centroid calculation unit 11, the initial capture window WGto and the setting completion timing signal To input to the color image target position detection apparatus are input. Is entered.

At the time of receiving the setting completion timing signal To, the evaluation area generating unit 12 sets the initial capture window WGto corresponding to the initial capture window marker 107a in FIG. 6 as the initial value of the detection area signal WG, and Generate a corresponding binarized window area signal w to ensure initial capture.

After the time point when the setting completion timing signal To is received, the evaluation area generation unit 12 sets the target detection position G which is the input from the binarized image centroid calculation unit 11 and the number of pixels where B = 1. From the binarized area S, the target detection position G is set to the center position, and the ratio of the initial capture window WGto in the vertical and horizontal directions is made constant, while the ratio of the binarized area S to the area area of WGto becomes constant. Then, the detection area signal WG and the binarized window area signal w generation area are updated.

Therefore, the minimum point detecting section 5 can detect the minimum minimum point value of the chromaticity difference signal ΔC only in the area near the tracking target, and the binarized image signal B is converted into the binarized image centroid. It is possible to input to the point calculation unit 11.

In the construction of this apparatus, the chromaticity coordinate calculation unit 2 is shown in FIG.
, One 3-input adder circuit 21, three logarithmic compression circuits 22, 23, 24, and two 2-input subtractor circuits 2
5, 26 and two exponential function generating circuits 27, 28.

Signal lines for X, Y, and Z stimulus values are connected to the 3-input addition circuit 21, and the output terminal thereof is a logarithmic compression circuit 2.
Connected to 2. An X stimulus value signal line is connected to the logarithmic compression circuit 23, and a Y stimulus value signal line is connected to the logarithmic compression circuit 24.

The output terminal of the logarithmic compression circuit 22 is connected to the subtraction value input terminals of the two-input subtraction circuits 25 and 26, and the output terminal of the logarithmic compression circuit 23 and the output terminal of the logarithmic compression circuit 24 are exponential function generating circuits. 27 and the exponential function generating circuit 28 are connected to the subtracted value input terminals. Exponential function generator 27
The signal of the chromaticity coordinate x is output to the output terminal of, and the signal of the chromaticity coordinate y is output to the output terminal of the exponential function generating circuit 28.

In the configuration of this apparatus, the chromaticity difference calculating section 3 is composed of two 2-input subtraction circuits 31, 32, two absolute value conversion circuits 33, 34 and a 2-input addition circuit 35 as shown in FIG. ing. The signal line of the chromaticity coordinate x and the reference chromaticity coordinate xo is connected to the input end of the 2-input subtraction circuit 31, and the signal line of the chromaticity coordinate y and the reference chromaticity coordinate yo is connected to the input end of the 2-input subtraction circuit 32. Are connected.

The output terminals of the 2-input subtraction circuits 31 and 32 are connected to the absolute value conversion circuits 33 and 34, respectively, and their output terminals are connected to the input terminals of the 2-input addition circuit 35. Two
The chromaticity difference signal ΔC is output from the output end of the input addition circuit 35.

In the configuration of the present apparatus, the chromaticity difference signal binarization unit 7 has the ΔC map frame memory 71, Δ as shown in FIG.
It is composed of a Cmax calculation processing circuit 72 and a binarization circuit 73. The signal line of the chromaticity difference signal ΔC from the chromaticity difference calculation unit 3 shown in FIG. 1 is connected to the input end of the ΔC map frame memory 71, and its output end is connected to the input end of the binarization circuit 73. There is.

A signal output of the minimum minimum point chromaticity difference ΔCpm from the minimum point data holding unit 6 shown in FIG. 1 is connected to the input terminal of the ΔCmax calculation processing circuit 72, and its output terminal is 2
It is connected to the binarization threshold value setting input terminal of the binarization circuit 73. This minimum minimum point chromaticity difference ΔCpm is the chromaticity closest to the chromaticity of the target designated point, and can be treated as the latest target chromaticity at the time of image capturing.

Therefore, a value larger than the target judgment limit value by the target model is set from this value, and this value is used as a threshold value to binarize ΔC below this value, whereby the target feature existing area is binarized. Can be realized.

In the ΔCmax calculation processing circuit 72, in order to binarize the chromaticity difference signal ΔC, the target judgment limit value is added to the value of the minimum minimum point chromaticity difference ΔCpm held in the minimum point data holding unit 6. Binarization threshold value ΔCma obtained by addition
Output x. From the output end of the binarization circuit 73, the chromaticity difference signal ΔC read from the ΔC map frame memory 71.
Is set to "1", and the ΔC map binarized signal BΔC smaller than the binarized threshold value ΔCmax is output as a binarized image.

In the configuration of the present apparatus, the luminance signal binarization unit 8 has the luminance map frame memory 81 and Ym as shown in FIG.
ax, Ymin calculation processing circuit 82 and binarization circuit 83
It consists of A luminance signal Y from the stimulus value conversion unit 1 shown in FIG.
Signal line is connected to the output terminal of the binarization circuit 8
3 is connected to the input end.

The line of the reference luminance Yo from the reference luminance updating unit 10 shown in FIG. 1 is connected to the input terminal of the Ymax and Ymin calculation processing circuit 82, and the output terminal thereof is the binarization circuit 8.
3 is connected to the binarized threshold value setting input terminal. A luminance binarization signal BY is output from the output terminal of the binarization circuit 73.

The minimum minimum point luminance value Ypm is also ΔCpm
Since it is considered to be the target feature quantity like
A reference luminance value Yo is calculated based on pm, and the luminance binarization limit values Ymax and Ym are obtained with the target characteristics taken into consideration.
By obtaining in, binarizing the value of Y within this range and taking the logical product with the ΔC map binary signal BΔC, the binarized image of the target area can be obtained with high accuracy.

In order to binarize the brightness signal Y, the reference minimum brightness value Ypm shown in FIG. Reference luminance Yo obtained by the filtering process
However, in the Ymax and Ymin calculation processing circuit 82,
A luminance binarization limit value Ymax obtained by adding the target determination limit upper limit width considering the characteristics of the target to this Yo and a luminance binarization limit value Ymin obtained by subtracting the target determination limit lower limit width from the Yo are obtained.

In the binarization circuit 73, these Ymax and Ym
The luminance image binarization signal BY is set to "1" for the value of Y read from the luminance map frame memory 81 in the range between in and the binarized image is output.

The operation of the color image target position detecting apparatus of the present invention will be described below with reference to FIGS.

First, the three received color image signals R, G,
B is input to the stimulus value conversion unit 1 at the same rate as the output of the sensor pixel by pixel,

[0092]

[Equation 13] The calculation results of are sequentially output.

This output is output to the chromaticity coordinate calculation unit 2 as X, Y, Z stimulus values. Of these three stimulus values, the stimulus value Y corresponds to the stimulus value of the primary color representing the brightness of the image. These X, Y, and Z signals are summed by the 3-input addition circuit 21 of the chromaticity coordinate calculation unit 2 shown in FIG. 2 and input to the logarithmic compression circuit 22.

The X and Y signals are the logarithmic compression circuit 2 respectively.
3 and the logarithmic compression circuit 24, and the 2-input subtraction circuit 2
5 and 26 are output. The two-input subtraction circuits 25 and 26 subtract the output value of the logarithmic compression circuit 22 from the input value,

[0095]

[Equation 14] To get These x'and y'signals are the exponential function generating circuit 2
Input to 7, 28, where

[0096]

[Equation 15] After obtaining the value of, the output signal is output from the chromaticity coordinate calculation unit 2. The values of the chromaticity signals x and y are

[0097]

[Equation 16] Of the color image signals R, G, B
Shows the chromaticity coordinate points for the.

On the other hand, when the chromaticity signals x and y from the chromaticity coordinate calculation unit 2 and the stimulus value Y from the stimulus value conversion unit 1 are both input to the designated point data holding unit 13 and a target is designated here. X and y of the pixel that coincides with the initial capture designated point Gto given to the above and the stimulus value Y are selected and held, and this is updated for each image frame, and the designated point chromaticity signals xto, yto and
The designated point stimulus value Yto is output from the designated point data holding unit 13.

Of these, the specified point chromaticity signals xto and yto are input to the reference chromaticity updating section 4. In the reference chromaticity update unit 4, when the setting completion timing signal To indicating that the initial capture designated point Gto has been set as the initial capture point is received, the designated point chromaticity signals xto and yto are used as they are as the reference chromaticity update unit. 4 and the initial reference chromaticity signals xo and yo.

The designated point stimulus value Yto is input to the reference luminance updating unit 10 and is set as the initial reference luminance signal Yo of the reference luminance updating unit 10 when the setting completion timing signal To is received. Further, the evaluation area generator 12 generates a detection area signal WG having an initial value of the initial capture window WGto given at the time of receiving the setting completion timing signal To.

Next, the values of the chromaticity signals x and y from the chromaticity coordinate calculation unit 2 are subtracted by the two-input subtraction circuits 31 and 32 of the chromaticity difference calculation unit 3 shown in FIG. Value circuit 3
After obtaining the absolute values in 3 and 34, by adding them in the 2-input adding circuit 35,

[0102]

[Equation 17] The chromaticity difference signal ΔC for each pixel according to the equation is obtained at the same rate as the image signal from the sensor.

This chromaticity difference signal ΔC is input to the minimum point detecting section 5, and every time when the minimum point of ΔC is detected, the minimum point detection timing signal pm indicating the time of occurrence is input to the minimum point data holding section 6. Is output.

In the minimum point data holding section 6, the chromaticity difference signal ΔC from the chromaticity difference calculating section 3, the chromaticity coordinate signals x and y from the chromaticity coordinate calculating section 2, and the stimulus value converting section 1 are sent. The luminance signal Y (Y stimulus value) is input, but the minimum point detection signal p
When m comes, it is determined whether or not ΔC smaller than the previously stored chromaticity difference signal ΔC is input, and if a small value is input, all the input signals are updated and held.

Therefore, when the input of one frame on the entire screen is completed in this apparatus, the minimum points ΔC, x, y, and Y having the minimum value of the chromaticity difference signal ΔC are stored in the minimum point data holding unit 6. Are held, and each of these is ΔC
Output as pm, xpm, ypm, Ypm.

The ΔCpm signal from the minimum point data holding unit 6 is a binarization threshold value ΔCmax obtained by adding the target judgment limit value modeled from the target characteristic by the chromaticity difference signal binarization unit 7 shown in FIG. Calculated by the calculation processing circuit 72, ΔCmax is 2
The value is set in the digitizing circuit 73.

Similarly, in the luminance signal binarization unit 8 shown in FIG. 5, the target determination limit upper limit width considering the characteristics of the target is added based on the reference luminance signal Yo from the reference luminance updating unit 10. The luminance binarization limit value Ymin obtained by subtracting the luminance binarization limit value Ymax and the target determination limit lower limit width is obtained as a binarization range by the Ymax and Ymin calculation processing circuit 82 and set in the binarization circuit 73.

When the setting of ΔCmax, Ymax, and Ymin is completed, ΔC map data for one frame is obtained from the ΔC map frame memory 71 shown in FIG. 4, and Y map data is obtained from the luminance map frame memory 81 shown in FIG.
, And two binary image signals BΔC and BY for ΔC and Y are obtained by the binarizing circuits 73 and 83, respectively.

In the evaluation area generator 12, if the detection area signal WG has an initial capture window WGto given at the time of receiving the setting completion timing signal To as an initial value, and the area is on the corresponding image frame, The binarized area signal w of "1" is generated and output, and the binarized area signal w and the binarized image signal BΔ
The AND circuit 9 ANDs C and BY to obtain the final target binarized image B.

The obtained binarized image B is subjected to labeling processing in the binarized image centroid calculation unit 11 to distinguish and separate into a plurality of independent binarized areas, and then the movement of the past detected position. One of the binarized areas is selected according to the prediction by, and the center of gravity of the surface is calculated, and the target detection position G is output from this apparatus.

At the time of shifting to the processing of the next frame,
Before inputting the next image frame, the reference chromaticity updating unit 4 inputs the latest target chromaticities xpm and ypm held in the minimum point data holding unit 6, and the currently held reference chromaticity value xo. , Yo as a reference, the reference chromaticity of the next frame is predicted or filtered to obtain a new reference chromaticity value x.
o and yo.

Similarly, the reference luminance updating unit 10 uses the target luminance Ypm held in the local minimum point data holding unit 6 to obtain a new reference luminance Y in the prediction or filtering process.
Update to o.

In the evaluation area generation unit 12, the target detection position G is set as the center position from the target detection position G and the binarized area S from the binarized image centroid calculation unit 11, and the vertical and horizontal directions of the initial capture window WGto are set. WGto while keeping the ratio of
The detection area signal WG is updated so that the ratio of the binarized area S to the area area of is constant, and the binarized area signal w corresponding to the WG updated in the next frame can be generated. .

The above is the color image signal R for each frame,
By repeating from the input of G and B, the target of the initial capture designated point when the setting completion timing signal To is first received is detected, and the position is output for each frame. When the setting completion timing signal To is input next, the initial capture designated point Gto given at that time is input.
The goal of is a new goal.

In the above embodiment, the minimum point data holding unit 6
The data held by is stored in a data group corresponding to the minimum point having the minimum ΔC and is output as information for tracking the target, but as another embodiment, the minimum point data holding unit 6 can also hold the position coordinates of the minimum points on the screen, and can also be configured to hold a data group for a plurality of minimum points in the form of a list.

In this configuration, if the chromaticity to be searched is set as the target designated point chromaticity, the coordinates of the target candidate point to be searched on the screen are ranked in the order of a plurality of small ΔC, and the data group is obtained. Becomes a so-called target candidate search device capable of organizing and outputting these.

[0117]

According to the present invention, the three image signals R, G and B of red, green and blue are supplied in real time for each pixel output from the sensor in response to a color signal input, and a tristimulus value signal X, Since the signals are converted into Y and Z, and the signals x and y of the chromaticity coordinate points directly related to the spectral characteristics of the target reflected light are calculated from these tristimulus value signals X, Y, and Z, it is possible to calculate the target brightness and darkness. Accurate target detection that is not captured is possible.

Furthermore, since the values of x and y are always values of "1" or less, the scaling of the processing values required for the circuit design in the subsequent processing becomes easy. Further, since the Y stimulus value obtained by the stimulus value conversion unit corresponds to the brightness of the colored light, if the Y stimulus value is used as it is for the target detection and the chromaticity values, x, and y are combined, it is more accurate. It is possible to make various detection determinations.

Further, the present invention is a chromaticity coordinate calculation unit, wherein x,
Since a method of performing logarithmic compression and then subtracting can be used for the operation for obtaining y, the dynamic range of the stimulus value that can be handled for obtaining x and y can be made extremely wide, and the division circuit can be eliminated. Has an effect.

Further, the chromaticity difference calculating section 3 uses the target chromaticity value to be detected as a reference and the positional deviation of the captured image from the reference of the coordinates in the chromaticity space as one signal value called distance information. Since the conversion is performed, the target point can be set as the target point detection point only by detecting the point having the minimum value, and the scale of the processing circuit can be reduced. Furthermore, since the chromaticity difference is calculated when obtaining the distance information about the positional deviation from the coordinate reference in the chromaticity space, the distance information can be easily obtained without the square calculation and the square root. , Always ΔC
Since there is a relation of ≧ r, it can be obtained as a value of a difference in chromaticity larger than the value of r.

Further, according to the present invention, since the reference chromaticity updating unit 4 is provided, the target distance changes to a distant position with time,
Even if the imaging chromaticity changes due to the influence of the atmospheric scattered light and the spectral characteristics of the transmittance, the reference chromaticity can be tracked, and the effect of the imaging distance can be suppressed.

Further, since the final target position is calculated by the binarized image barycenter calculation unit 11 after the labeling process, it is output as the surface barycenter after the binarized area is selected by the target motion prediction determination. The same-color object intersection determination can be performed, and the detection positions can be averaged within the area to reduce the fluctuation of the detection point position or the jump width.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is a block diagram showing an example of a chromaticity coordinate conversion unit 2 in FIG.

3 is a block diagram showing an example of a chromaticity difference calculation unit 3 in FIG.

FIG. 4 is a block diagram showing an example of a chromaticity difference signal binarization unit 7 in FIG.

5 is a block diagram showing an example of a luminance signal binarization unit 8 in FIG.

FIG. 6 is a block diagram showing an example of use connection of FIG.

FIG. 7 is a diagram for explaining the operation of each block in FIG.

FIG. 8 is a diagram for explaining the operation of the present invention.

FIG. 9 is a diagram for explaining the operation of the present invention.

FIG. 10 is a diagram for explaining the operation of the present invention.

FIG. 11 is a diagram for explaining the operation of the present invention.

FIG. 12 is a diagram for explaining the operation of the present invention.

FIG. 13 is a diagram for explaining the operation of the present invention.

[Explanation of symbols]

1 Stimulus value conversion unit 2 Chromaticity coordinate calculation unit 3 Chromaticity difference calculation unit 4 Reference chromaticity update unit 5 Minimum point detection unit 6 Minimum point data holding unit 7 Chromaticity difference signal binarization unit 8 Luminance signal binarization unit 9 AND circuit 10 Reference luminance update unit 11 Binary image centroid calculation unit 12 Evaluation area generation unit 13 Specified point data storage unit 21 3 Input addition circuit 22, 23, 24 Logarithmic compression circuit 25, 26 Subtraction circuit 27, 28 Exponent Function circuit 31, 32 Subtraction circuit 33, 34 Absolute value conversion circuit 35 2-input addition circuit 71 ΔC map frame memory 72 ΔCmax calculation processing circuit 73 Binarization circuit 81 Luminance map frame memory 82 Ymax / Ymin calculation processing circuit 83 Binarization Circuit 100 Color image target detection device 101 Color image sensor 102 Initial capture designation input device 103 Image / designated position display device 104 Gimbal device 105 Tail signal output device 106 Imaging target 107 Display screen 108 Tracking target 201 Chromaticity range by RGB signal of color image sensor 202 Chromaticity coordinate 203 of input image Target designated point chromaticity coordinate (reference chromaticity coordinate) 301 x + y = 1 A straight line 302 a limit line 401 that satisfies ΔC ≦ 1.0 based on 0 points ΔC map 402 minimum point 403 pm generation point 404 pm generation range designation area WG 501 imaging screen 502 signal output order

─────────────────────────────────────────────────── ─── Continuation of front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06T ⁷ /00-7/60 G06T 1/00 H04N 5/222-5/257 H04N 9/04-9 / 11 H04N 7/18

Claims (3)

(57) [Claims] 1. An image signal R, G, B from a color image sensor that captures a target color image and outputs three color signals of red (R), green (G), and blue (B) 1] And a stimulus value conversion unit for converting each pixel output from the sensor into a tristimulus value signal of X, Y, Z in real time corresponding to a color signal input, and the X, Y, Z 3 stimulus value signals of [Equation 2] And a chromaticity coordinate calculation unit for converting into chromaticity coordinate signals x and y by performing the following calculation, and reference chromaticity coordinate signals xo and yo and the chromaticity coordinate signals x and y.
From, The chromaticity difference calculation unit that calculates the chromaticity difference ΔC by performing the above calculation, and the point where the chromaticity difference ΔC becomes the minimum in the detection area set by the detection area signal from the evaluation area generation unit is detected. And a chromaticity coordinate corresponding to the chromaticity difference ΔC of the minimum point and the chromaticity difference ΔC of the minimum point each time the minimum point detection timing signal pm is generated. The signals x and y are held, and the minimum one of the chromaticity differences ΔC of the minimum points and the chromaticity coordinate signals x and y corresponding to the minimum ΔC at the time point when the color signal input for one frame is completed are the minimum minimum points. Chromaticity difference ΔCpm and minimum minimal point chromaticity value xpm, y
A minimum point data holding unit that outputs as pm, and a chromaticity difference signal that inputs the chromaticity difference ΔC and the minimum minimum point chromaticity difference ΔCpm and converts the chromaticity difference ΔC into a binarized chromaticity difference BΔC. AND which outputs a logical product signal B of the binarization unit and the binarization chromaticity difference BΔC and the binarization window area signal w that limits the binarization output image area output from the evaluation area generation unit A circuit, and a binarized image centroid point calculation unit for inputting the logical product signal B and computing a binarized area S having a value of the number of pixels where B = 1 and the centroid point G of the binarized image. A designated point data holding unit that selects and holds the chromaticity coordinate signals x and y of the pixels that match the initially captured designated point and outputs the designated point chromaticity signals xto and yto; and the designated point chromaticity signals xto and yto. And the minimum minimum point chromaticity values xpm and ypm are input, and the reference chromaticity signal is input. o,
and a reference chromaticity updating unit that outputs yo. 2. The minimum point data holding unit further holds a stimulus value signal Y corresponding to the chromaticity difference ΔC of the minimum point every time the minimum point detection timing signal pm is generated, and stores the stimulus value signal Y for one frame. The pointing point data holding unit further comprises means for outputting a stimulus value signal Y corresponding to the minimum ΔC in the chromaticity difference ΔC of the minimum points at the time of completion of signal input, as the minimum minimum point stimulus value signal Ypm. , A means for selecting and holding a stimulus value signal Y of a pixel that coincides with the initial capture designated point and outputting it as a designated point stimulus value signal Yto, further, the minimum minimum point stimulus value signal Ypm and the designated point. A reference brightness update unit that inputs a stimulus value signal Yto and outputs a reference brightness signal Yo; and inputs the stimulus value signal Y and the reference brightness signal Yo,
And a luminance signal binarization unit that converts the stimulation value signal Y into a binarized luminance signal BY and outputs the binarized luminance signal BY to the AND circuit, and the binarized chromaticity difference BΔC and the luminance signal BY are output from the AND circuit. 3. The color image target position detecting device according to claim 1, wherein a logical product signal B of the binary window area signal w and the binary signal is output. 3. The color coordinate conversion unit converts the chromaticity coordinate signals x and y into 3. The color image target position detecting device according to claim 1 or 2, further comprising a calculating unit that obtains