JP2939608B2 - Target extraction method, target extraction device, and target tracking device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a target extracting method, a target extracting device, and a target tracking device. More specifically, the present invention relates to a target extraction method and a target extraction device for extracting a target from an image so that the target can be clearly separated from the background. In addition, the present invention relates to a target tracking device that tracks a moving target so that the moving target is always captured at a specific position in an image.

[0002]

2. Description of the Related Art Japanese Unexamined Patent Publication No. Hei 4-373079 discloses a method of setting a background window and a target window in an image.
A one-dimensional histogram of pixel values is obtained for all the pixels in the background window, and an evaluation function for outputting an evaluation value from the pixel values so that the background value and the target value are clearly separated from each other is represented by the one-dimensional histogram. "Image signal separation method and apparatus" that obtains based on a histogram, converts a pixel value of each pixel included in a target window in the image into an evaluation value using the evaluation function, and extracts a target from the image is disclosed. Have been.

Japanese Patent Application Laid-Open No. Hei 5-197804 discloses that an R, G, B color image signal is obtained by taking an image with a color image pickup device, and from the R, G, B color signals, Ir = 255 · R / (R + G + B) Ig = 255 · G / (R + G + B) The color vector signals Ir and Ig are obtained, the background window and the target window are set in the image, and the color vector signals Ir and Ig are set for all the pixels in the background window.
Is obtained from the color vector signals Ir and Ig based on the two-dimensional histogram, and an evaluation function for outputting an evaluation value such that a background value and a target value are clearly separated values is obtained based on the two-dimensional histogram. An "image signal separation method and apparatus" that converts color vector signals Ir and Ig of each pixel included in a target window in the image into an evaluation value using the evaluation function and extracts a target from the image is disclosed. I have.

[0004] As another related conventional technique, there is a technique described in Japanese Patent Application No. 2-239056.

[0005]

SUMMARY OF THE INVENTION The above-mentioned Japanese Patent Laid-Open No. 4-373 is disclosed.
In many cases, a target can be effectively extracted from an image by the "image signal separation method and apparatus" disclosed in Japanese Patent Application Laid-Open No. 079.
However, since the evaluation value for separating the background and the target is obtained from the one-dimensional histogram of only the pixel values, there is a problem that if there is a background having the same pixel value, the background and the target cannot be separated. .

On the other hand, Japanese Unexamined Patent Publication No.
In the "image signal separation method and apparatus" disclosed in Japanese Patent Application Laid-Open No. H11-157, an evaluation value for separating a background and a target is determined by a color vector signal Ir, based on two signals of an R color image signal and a G color image signal. It is obtained from a two-dimensional histogram of Ig. Since the probability that two pixel values of the R color image signal and the G color image signal match between the background and the target is smaller than the probability that one pixel value matches, the method disclosed in Japanese Patent Laid-Open No. 4-37307 is used.
The probability that the background and the target can be separated is higher than that of the "image signal separation method and apparatus" disclosed in Japanese Patent Application Laid-Open No. 9-90. However, the color vector signals Ir and Ig are all (R + G + B)
Since this is a relative index indicating the ratio of a part (R, G) to the background, if there is a background that matches this ratio, there is a problem that the background cannot be separated from the target.

Accordingly, a first object of the present invention is to provide a target extraction method and a target extraction device which can increase the probability that the background can be separated from the target as compared with the above-mentioned prior art. A second object of the present invention is to provide a target tracking device that tracks a target by using the target extracting method and the target extracting device.

[0008]

According to a first aspect of the present invention, the same field of view is imaged in two different wavelength bands a and b, and a pixel value D of a corresponding pixel of each obtained image is obtained.
The magnitude I and the phase Φ of the pixel value vector (Da, Db) having the components a and Db are obtained, and the two-dimensional histogram of the magnitude I and the phase Φ is obtained for all the pixels included in the background area in the image. Calculating an evaluation function that outputs an evaluation value such that the background value and the target value are clearly separated from the magnitude I and the phase Φ based on the two-dimensional histogram,
A target extraction method for converting the magnitude I and the phase Φ of a pixel value vector of each pixel included in a target area in the image into an evaluation value using the evaluation function is provided. In the target extraction method having the above configuration, instead of converting the magnitude I and the phase Φ of the pixel value vector into an evaluation value, the components Da and D of the pixel value vector are used.
You may make it convert b into an evaluation value directly.

In a second aspect, the present invention captures the same field of view in n (≧ 3) different wavelength bands a, b,..., And obtains pixel values Da, D of corresponding pixels of each obtained image.
pixel value vector (Da, Db,...) having b,.
, And the direction ratios Pa, Pb,... Are obtained. An n-dimensional histogram of the size I and the direction ratios Pa, Pb,. An evaluation function that outputs an evaluation value that clearly separates the target value from the magnitude I and the directional ratios Pa, Pb,... Is obtained based on the n-dimensional histogram, and a target area in the image is obtained. Are converted into an evaluation value using the evaluation function, the magnitude I of the pixel value vector of each pixel and the direction ratio Pa, Pb,. In the target extraction method having the above structure, instead of converting the magnitude I of the pixel value vector and the direction ratio Pa, Pb,... Into an evaluation value, the components Da, Db,. It may be.

In a third aspect, the present invention provides an image pickup means (1a, 1b) for picking up the same visual field in two different wavelength bands a, b, and a pixel value Da of a corresponding pixel of each obtained image. , Db as pixel components (Da,
Calculation means (5, 5) for calculating the magnitude I and the phase Φ of Db)
6) and calculating a two-dimensional histogram of the magnitude I and the phase Φ for all the pixels included in the background region in the image, and evaluating the evaluation value so that the background value and the target value are clearly separated from each other. An evaluation function generator (7) for obtaining an evaluation function output from the magnitude I and the phase Φ based on the two-dimensional histogram; and a magnitude I of a pixel value vector of each pixel included in the target area in the image. And an evaluation value output means (8) for converting the phase Φ into an evaluation value using the evaluation function.

[0011] In a fourth aspect, the present invention provides an image processing apparatus comprising: n (≧ 3)
, And a pixel value vector (D) having the pixel values Da, Db,... Of the corresponding pixels of each obtained image as components.
a, Db,...) and the direction ratio Pa, Pb,..., and the size I and the direction ratio Pa, Pb,. An evaluation function for obtaining an n-dimensional histogram and outputting an evaluation value such that the background value and the target value are clearly separated from the magnitude I and the direction ratios Pa, Pb,. An evaluation function generating means for determining the magnitude I of the pixel value vector of each pixel included in the target area in the image and the direction ratio Pa, Pb,... Into an evaluation value using the evaluation function And a target extracting device provided with an output unit.

According to a fifth aspect, the present invention calculates the position of the center of gravity of a target based on the target extracting apparatus (100) having the above-described configuration, and the coordinates and evaluation values of each pixel included in the target area in the image. A target tracking device comprising: a center of gravity calculating means (20); and a tracking means (22, 23) for driving an imaging means so that the position of the center of gravity matches a specific position in the target area. provide.

[0013]

In the target extraction method according to the first aspect and the target extraction apparatus according to the third aspect, an evaluation value for separating a background and a target is determined by using pixels obtained in two different wavelength bands a and b. It is obtained from a two-dimensional histogram of the magnitude I and the phase Φ of the pixel value vector (Da, Db) having the components Da and Db as components. Since the probability that two pixel values of the pixel values Da and Db match in the background and the target is smaller than the probability that one pixel value matches, the “Image signal separation method and image signal separation method” disclosed in JP-A-4-373079 is disclosed. The probability that the background and the target can be separated is higher than that of the device. The magnitude I of the pixel value vector (Da, Db) is not a relative index but an absolute index having a physical unit. On the other hand, the phase Φ is a relative index indicating the ratio of the other (Da) to the one (Db). As described above, since the absolute index and the relative index are combined, the probability that both the indexes match the background and the target is smaller than the case where only the relative index is used.
The probability that the background and the target can be separated is higher than that of the “image signal separation method and apparatus” disclosed in Japanese Patent No. 97804.

In the target extraction method according to the second aspect and the target extraction apparatus according to the fourth aspect, the evaluation values for separating the background and the target are set to n (≧ 3) or more different wavelength bands a and b. , The magnitude I of the pixel value vector (Da, Db,...) Having the pixel values Da, Db,.
And the direction ratios Pa, Pb,. Since the probability that three or more pixel values of the pixel values Da, Db,... Match in the background and the target is smaller than the probability that one pixel value matches, the “image signal” disclosed in The probability that the background and the target can be separated is higher than that of the “separation method and device”. The magnitude I of the pixel value vector (Da, Db,...) Is not a relative index,
Absolute index with physical units. On the other hand, the direction ratios Pa, P
are part (Da, Db,...) of the whole (I)
Is a relative index indicating the ratio of. As described above, since the absolute index and the relative index are combined, the probability that these indexes match the background and the target is smaller than the case where only the relative index is used. The probability that the background and the target can be separated is higher than that of the “method and apparatus for separating image signals”.

In the target tracking device according to the fifth aspect,
Since the target extracting device according to the second aspect or the target extracting device according to the fourth aspect is used, it is possible to reliably track the target.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below in more detail with reference to the embodiments shown in the drawings. It should be noted that the present invention is not limited by this.

FIG. 1 is a block diagram of a target extracting apparatus 100 according to a first embodiment of the present invention. The camera 1 has a wavelength band a (for example, 3 μm
1 to 5 μm) and a second imaging tube 1b mainly having a wavelength band b (for example, 8 μm to 12 μm). Both imaging tubes 1a, 1b
Have the same field of view. In addition, both imaging tubes 1a, 1
b outputs analog video signals Ca and Cb in synchronization with the synchronization signal SY from the synchronization control circuit 2. The pre-processing circuit 3a subjects the analog video signal Ca to A / D conversion and noise elimination processing to obtain a pixel value Da.
(Digital value) is output. The preprocessing circuit 3b performs A / D conversion and noise removal processing on the analog video signal Cb, and outputs a pixel value Db (digital value).

The first arithmetic circuit 5 calculates the magnitude I of the pixel value vector (Da, Db) by the following equation. I = {Da · Da + Db · Db} The second arithmetic circuit 5 calculates the phase Φ of the pixel value vector (Da, Db) by the following equation. Φ = Arctan {Da / Db} The evaluation function generator 7 sets the magnitude I and the phase Φ to an evaluation value e such that the background value and the target value are values that are clearly separated from each other.
Generate an evaluation function for converting to. The evaluation function generator 7 will be described later in detail with reference to FIG. The evaluation value output unit 8 obtains and outputs the evaluation value E of each pixel in the target window WT (the rectangular area shown in FIG. 4) using the evaluation function. The evaluation value output unit 8 will be described later in detail with reference to FIG.

The background window setting circuit 10 calculates the coordinate values X, Y of the pixels input from the synchronization control circuit 2 (as shown in FIG. 4, the horizontal coordinate value of the screen is X, and the vertical coordinate value is Y). ) Based on the current pixel value vector (D
a, Db) to output a background window gate signal Bw for determining whether or not the pixel belongs to a pixel in the background window WB (the frame region shown in FIG. 4). The target window setting circuit 11 generates a target window gate signal for determining whether or not the current pixel value vector (Da, Db) belongs to a pixel in the target window WT based on the coordinate values X, Y of the pixel. Tw is output.

FIG. 2 is a detailed block diagram of the evaluation function generator 7. The magnitude I and the phase Φ of the pixel value vector (Da, Db) are input to the L channel address of the dual port RAM 71. Histogram creation control circuit 72
Reads the data stored at the address defined by the magnitude I and the phase Φ only when the current pixel value vector (Da, Db) is a pixel in the background window, and To be held. The adder circuit 74 adds “+1” to the data held in the latch circuit 73. The histogram creation control circuit 72 includes:
Only when the current pixel value vector (Da, Db) is that of a pixel in the background window WB, the “+1” data is stored at an address defined by the magnitude I and the phase Φ. The contents of the dual port RAM 71 are cleared for each frame in synchronization with the vertical synchronization signal VD as described later. Therefore, the contents of the dual port RAM 71 immediately before being cleared are the two-dimensional histogram H of the magnitude I and the phase Φ for all the pixels included in the background window WB in the latest frame (shown in FIG. 5A). Yamagata solid).

When the vertical synchronization signal VD is input from the synchronization control circuit 2, the evaluation value setting control circuit 75 reads the contents of the dual port RAM 71, that is, the two-dimensional histogram H shown in FIG. Next, the evaluation value setting control circuit 75 obtains a threshold value by multiplying the total number of pixels of the two-dimensional histogram H by an appropriate coefficient k (where 0 <k ≦ 1), and calculates the two-dimensional histogram H with the threshold value. The clipped and remaining portion below the threshold value is normalized such that the threshold value matches the maximum value of the evaluation value, and an evaluation value normalized histogram solid is created. In FIG. 5B, the threshold value is smaller than the peak value of the two-dimensional histogram H, and the maximum value of the evaluation value is 2
An example of the evaluation value normalized histogram solid H ′ when 55 is set is shown. Next, the evaluation value setting control circuit 75 inverts the curved surface formed by the IΦ plane and the surface of the evaluation value normalized histogram solid H ′ with respect to the maximum value of the evaluation value, performs min processing, and performs the evaluation value function ( Surface). For example, in the evaluation value normalized histogram space shown in FIG.
By performing the operation of h ′ (I, Φ) = 255−E (I, Φ), a curved surface h ′ shown in FIG. 5C is obtained. Thereafter, h (I, Φ) = min ｛h ′ (I−ΔI, Φ−ΔΦ),..., H ′ (I + Δ
I, Φ + ΔΦ)｝ (min ｛｝ is a function that takes a minimum value. ΔI, ΔΦ are appropriate section parameters.) A min process is performed to obtain an evaluation value function h shown in FIG. . Next, the evaluation value setting control circuit 75 outputs the evaluation value function h to the evaluation value output unit 8.
Transfer to 1 to 3, the evaluation value function h transferred from the evaluation function generation unit 7 to the evaluation value output unit 8 is represented by e, i, and φ in FIGS. Evaluation value function h
After the transfer, the evaluation value setting control circuit 75 clears the contents of the dual port RAM 71 as described above to prepare for the creation of a histogram for the next frame.

FIG. 3 is a detailed block diagram of the evaluation value output unit 8. On the R channel side of the dual port RAM 81,
Addresses i and φ and data e are sent from the evaluation function generator 7 and the evaluation function h is rewritten for each frame. The L-channel address of the dual port RAM 81 includes the magnitude I of the pixel value vector (Da, Db).
And the phase Φ are input. As a result, the pixel value vector (D
a, Db) and the evaluation value e ′ corresponding to the phase Φ are output as L channel data. Output control circuit 82
When the pixel inputting the magnitude I and the phase Φ to the L channel address is a pixel outside the target window WT, the latch circuit 83 is cleared, and the magnitude I and the phase Φ are input to the L channel address. When the present pixel is a pixel within the target window WT, the evaluation value e ′ is held in the latch circuit 83. As a result, as shown in FIG.
On the image based on the evaluation value E output from the target 3, only the target T in the target window WT is clearly extracted. That is, as shown in FIG. 4, the target T and the background are confusing on the original image captured by the camera 1, but only the target T can be clearly separated and extracted from the target.

In the first embodiment, instead of performing the operation of I = {Da · Da + Db · Db} in the first operation circuit 5, the operation of I = Da · Da + Db · Db or I = Da + Db is performed. (These cases are also included in the scope of the present invention). In the first embodiment, instead of performing the operation of Φ = Arctan {Da / Db}, the second operation circuit 6 may perform the operation of Φ = Da / Db or Φ = Da / I. (These cases are also included in the scope of the present invention). Further, the evaluation value output unit 8 of the first embodiment described above.
In the above, the magnitude I and the phase Φ of the pixel value vector were converted into the evaluation value E using a look-up table, but the evaluation function is provided as a procedure (or function) on a program, and the conversion is performed by a series of processing. (This case is also included in the scope of the present invention). In the first embodiment, instead of converting the pixel value vector magnitude I and the phase Φ into the evaluation value E, the evaluation value output unit 8 directly converts the pixel value vector components Da and Db into the evaluation value E. It may be converted (this case is also included in the scope of the present invention). Second Embodiment FIG. 7 is a configuration diagram of a target extraction device 200 according to a second embodiment of the present invention. The target extraction device 200 has basically the same configuration as the target extraction device 100 of the first embodiment, but differs in the following points. The camera 1 has three image pickup tubes each having main sensitivities in different wavelength bands a, b, and c. The arithmetic circuit 5 calculates the magnitude I of the pixel value vector (Da, Db, Dc) by I = {Da.Da + Db.Db + Dc.Dc}.
Is calculated. The operation circuits 6a and 6b calculate the directional ratios Pa and Pb, respectively, according to Pa = Da / IPb = Db / I.

The evaluation function generator 7 evaluates the magnitude I and the directional ratios Pa and Pb into an evaluation value E for converting the background value and the target value to values clearly separated from each other. h
Generate The operation of the target extraction device 200 is basically the same as the operation of the target extraction device 100 according to the first embodiment, and a description thereof will be omitted.

In the second embodiment, instead of performing the operation of I = {Da.Da + Db.Db + Dc.Dc} in the first arithmetic circuit 5, I = Da.Da + Db.Db + Dc.Dc or I = Da + Db + Dc (These cases are also included in the scope of the present invention). Further, in the second embodiment, instead of performing the operation of Pa = Da / IPb = Db / I in the second arithmetic circuits 6a, 6b, 6c, Pa = Arccos ｛Da / I｝ Pb = Arccos ｛Db / The calculation of I｝ may be performed (these cases are also included in the scope of the present invention). Further, in the second embodiment, the evaluation value output unit 8 uses the pixel value vector magnitude I and the directional ratios Pa and Pb.
Is converted to an evaluation value E instead of the component D of the pixel value vector.
a, Db, and Dc may be directly converted to the evaluation value E (this case is also included in the scope of the present invention).

Third Embodiment FIG. 8 is a block diagram of a target tracking device 800 according to a third embodiment of the present invention. The target tracking device 800 includes the target extracting device 100 of the first embodiment (or the target extracting device 200 of the second embodiment), the coordinate values X and Y and the evaluation value E of each pixel in the target window TW. , A latch circuit 21 for holding the center of gravity gx, gy calculated for each frame, a center of gravity gx, gy and the center of the target window TW. The target extracting device 100 is set so that the position matches the position.
(Or 200) tracking drive 2 for driving camera 1
3 is provided.

That is, as shown in FIG. 9A, the target T moves and the center of gravity position gx, gy and the target window T
When a deviation occurs from the center position Wo of W, the camera 1 is driven so that the deviation becomes “0”. As a result,
As shown in FIG. 9B, the moving target T is
Will automatically track.

[0028]

According to the target extracting method and the target extracting apparatus of the present invention, a target can be extracted from an image and clearly separated from a background. Further, according to the target tracking device of the present invention, a moving target can be always caught at a specific position in an image. Therefore, it is particularly useful for a foreign substance inspection device, an intruder monitoring device, a moving object monitoring device, a visual sensor of a palletizing robot, and the like.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a target extraction device according to a first embodiment of the present invention.

FIG. 2 is a detailed configuration diagram of an evaluation function generation unit in the target extraction device of FIG. 1;

FIG. 3 is a detailed configuration diagram of an evaluation value output unit in the target extraction device of FIG. 1;

FIG. 4 is an exemplary view of an image obtained by imaging with a camera.

FIG. 5 is an explanatory diagram of a method for generating an evaluation function.

FIG. 6 is a view showing an example of a target image extracted by the target extracting device of FIG. 1;

FIG. 7 is a configuration diagram of a target extraction device according to a second embodiment of the present invention.

FIG. 8 is a configuration diagram of a target tracking device according to a third embodiment of the present invention.

9 is an exemplary view of a screen for explaining an operation of the target tracking device of FIG. 8;

[Explanation of symbols]

 100, 200 Target tracking device 1 Camera 1a, 1b Image pickup tube 5, 6, 6a, 6b Arithmetic circuit 7 Evaluation function generation unit 8 Evaluation value output unit 20 Center of gravity calculation circuit 22 Tracking control circuit 23 Tracking drive device WT Target window WB Background window T target H two-dimensional histogram H 'evaluation value normalized histogram solid h evaluation function gx, gy center of gravity position Wo center of target window

Claims (7)

(57) [Claims] 1. A pixel field vector (Da, Db) having the same field of view in two different wavelength bands a, b, and pixel values Da, Db of a corresponding pixel of each obtained image as components.
b), the magnitude I and the phase Φ are obtained, and a two-dimensional histogram of the magnitude I and the phase Φ is obtained for all the pixels included in the background area in the image, and the background value and the target value are clearly separated. An evaluation function that outputs an evaluation value based on the magnitude I and the phase Φ is calculated based on the two-dimensional histogram, and the magnitude I of the pixel value vector of each pixel included in the target area in the image is obtained. And a phase φ are converted into the evaluation value using the evaluation function. 2. The target extraction method according to claim 1, wherein instead of converting the magnitude I and the phase Φ of the pixel value vector into an evaluation value, the components Da and Db of the pixel value vector are directly converted into an evaluation value. A target extraction method, characterized in that: 3. n (≧ 3) different wavelength bands a, b,
, The same field of view is respectively imaged, and the magnitude I of the pixel value vector (Da, Db,...) Having the pixel values Da, Db,.
.. are obtained, and an n-dimensional histogram of the size I and the directional ratios Pa, Pb,... is obtained for all the pixels included in the background area in the image. The evaluation value which becomes a value is determined by the magnitude I and the direction ratio Pa,
The evaluation function output from Pb,... Is obtained based on the n-dimensional histogram, and the magnitude I of the pixel value vector of each pixel included in the target area in the image and the direction ratio Pa, Pb,
... is converted into the evaluation value using the evaluation function. 4. The target extraction method according to claim 3, wherein the magnitude I of the pixel value vector and the directional ratios Pa, Pb,.
Is converted to an evaluation value, a component D of a pixel value vector
a, Db,... are directly converted into evaluation values. 5. An image pickup means (1a, 1b) for picking up the same visual field in two different wavelength bands a, b, and a pixel value having a pixel value Da, Db of a corresponding pixel of each obtained image as a component. Calculating means (5, 6) for calculating the magnitude I and the phase Φ of the vector (Da, Db), and obtaining a two-dimensional histogram of the magnitude I and the phase Φ for all the pixels included in the background area in the image In addition, an evaluation function generating means (7) for obtaining, based on the two-dimensional histogram, an evaluation function for outputting an evaluation value from the magnitude I and the phase Φ so that the background value and the target value are clearly separated values. ) And evaluation value output means (8) for converting the magnitude I and the phase Φ of the pixel value vector of each pixel included in the target area in the image into an evaluation value using the evaluation function. Characteristic target extraction device. 6. n (≧ 3) different wavelength bands a, b,
, And the magnitude I and the direction ratio of the pixel value vector (Da, Db,...) Having the pixel values Da, Db,... Of the corresponding pixels of each obtained image as components. Calculating means for calculating Pa, Pb,..., Calculating an n-dimensional histogram of the size I and the directional ratio Pa, Pb,. Evaluation function generating means for obtaining, based on the n-dimensional histogram, an evaluation function for outputting an evaluation value from the magnitude I and the directional ratios Pa, Pb,... The magnitude I of the pixel value vector of each pixel included in the target area and the direction ratio Pa, P
a target value extracting device for converting b,... into an evaluation value using the evaluation function. 7. A center of gravity for calculating a position of a center of gravity of a target based on the target extracting apparatus (100) according to claim 5 and a coordinate and an evaluation value of each pixel included in a target area in an image. A target tracking device comprising: calculating means (20); and tracking means (22, 23) for driving an imaging means so that the position of the center of gravity coincides with a specific position in the target area.