JP2866528B2 - Image signal separation method and apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal separating method and apparatus, and more particularly to a novel improvement for separating and extracting a target signal based on two digital image signals obtained from two kinds of image pickup apparatuses. .

[0002]

2. Description of the Related Art There have been various image signal separation methods of this type which have been used in the past. Among them, a typical one is described as follows. The digital video signal Gd is converted by a / D converter (not shown), and the digital video signal Gd is input to the window comparator circuit 1, the MAX circuit 2, the MIN circuit 3, and the average value calculation circuit 4, and Each of the circuits 2, 3, and 4 has a first gate signal TG
ATE has been entered.

The average value calculating circuit 4 includes a second gate signal BGATE, a clock signal DCLK, and a video signal V.
D is input via a delay circuit 5, and the video signal VD is input to latch circuits 6, 7, and 8 connected to the circuits 2, 3, and 4, respectively.

The output G from each of the latch circuits 6, 7, 8
HI, GLO and GAV are the first and second subtraction circuits 9 and 10
The outputs 9a and 10a of the subtraction circuits 9 and 10 and the output GAV are input to a first addition circuit 11 and a third subtraction circuit 12, and are input to the window comparator circuit 1. Twelve outputs 11a and 12a are input to the window comparator circuit 1.

That is, the digital video signal Gd is converted into one field or one field by an average value calculation circuit 4 for calculating the average brightness in the background window, a MAX circuit 2 for obtaining the maximum brightness in the target window, and a MIN circuit 3 for obtaining the minimum value. The output GHI, GLO, GAV for each frame
And then the window comparator circuit 1
The comparator setting value of is set.

The aforementioned outputs GAV, GLO, GHI are:
10 and 11, wherein the comparator set value is given by the following equation. CBL = GLO, CBH = G
AV-K2 × (GAV-GLO), CWL = GAV + K
By setting 2 × (GHI−GAV) and CWH = GHI, the digital video signal Gd is converted and binarized by the window comparator circuit 1 in FIG. 9 to obtain an output Gd as a target signal.

That is, the processing of Gd = fb (Gd) is performed, whereby only the target signal (target image) is extracted. The window comparator circuit 1
Is used to select whether to extract a target signal darker or brighter than the background signal.

[0008]

Since the conventional image signal separating method is configured as described above, there are the following problems. That is, when the background signal is simple as shown in FIG. 12, the target signal can be easily extracted by the conventional method as shown in FIG. However, as shown in FIGS. 14 and 15, when the histogram of the background signal is bimodal and the target signal exists between the background signals, the background signal is more It was to be extracted and misrecognized. Therefore, when the prior art is applied, extraction is impossible unless the distribution of the background signal is simple and the distribution of the target signal is outside the background signal. Also, since the imaging device is only a visible camera, if there is no difference between the brightness distribution of the background and the target,
It was not possible to separate and extract objects from the background.

The present invention has been made to solve the above problems, and in particular, separates and extracts a target signal based on two digital video signals obtained from two types of image pickup devices, visible and infrared. It is an object of the present invention to provide a method and apparatus for separating image signals.

[0010]

An image signal separating method according to the present invention comprises first and second image pickup devices (21, 21) of different types.
22), a first image signal (21a) composed of a visible image signal and a second image signal (22a) composed of an infrared image signal are processed to obtain two types of digital image signals (21aA, 22aA) and each digital image signal
From the (21aA, 22aA), a two-dimensional histogram only in the background window (40) is extracted using the histogram processing circuit (23), and the two-dimensional histogram is looked up by the background removal function data obtained by the processing including the minimum processing. The digital image signals (21a, 22a) latched by the up-table circuit (24) are exchanged to remove the background signal in the target window (90), to separate and extract only the target signal (24a), and to extract the target signal. This is a method of determining the position of the center of gravity of the signal (24a). Further, a first image pickup device (21) and a second image pickup device (22) of different types, a first image signal (21a) composed of visible image signals obtained from the respective image pickup devices (21, 22), and an infrared ray An image preprocessing circuit (20) for inputting a second image signal (22a) composed of an image signal; a first digital image signal (21aA) and a second digital image obtained from the image preprocessing circuit (20) A histogram processing circuit (23) for receiving a signal (22aA) and extracting a two-dimensional histogram in a background window (40); and a digital image signal (21aA, 22aA) and a histogram processing circuit (23). Background removal signal (23
a) and a look-up table circuit (24) having a function of discriminating a difference in contrast at the same temperature and a difference in temperature at the same contrast having a two-port ram (32), Table circuit (24)
Is the look-up table circuit (2) based on the background removal function data obtained by processing the two-dimensional histogram.
4) convert each digital image signal (21a, 22a) latched by the flip-flop (70, 71) to the digital image signal
(21aA, 22aA) is configured to remove the background signal component and separate and extract only the target signal (24a), and the look-up table circuit (24) is connected to a center of gravity detection circuit (80). Configuration.

[0011]

In the image signal separation method and apparatus according to the present invention, the first image signal and the second image signal from the two kinds of imaging devices, visible and infrared, which image the same visual field, are converted into the first image signal by the image preprocessing circuit. , Converted to a second digital image signal,
Each of the image signals is input to a histogram processing circuit together with a timing signal and the like generated at the same time. The histogram processing circuit creates a two-dimensional histogram in a background window. At the same time, a background removal signal is created and input to a look-up table circuit. You. Each of the digital image signals is input to the look-up table at the same time. In the look-up table, a target signal from which the background signal is removed from each of the digital image signals is separated and extracted.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the image signal separating method and apparatus according to the present invention will be described below in detail with reference to the drawings.
1 to 8 show an image signal separation method and apparatus according to the present invention. FIG. 1 is a block diagram, FIG. 2 is a detailed block diagram of a main part of FIG. 1, and FIG. FIG. 4 is an explanatory diagram showing a creation method, FIG. 4 is an explanatory diagram showing extraction of a target signal, FIG. 5 is a block diagram for detecting a center of gravity position of the target signal,
6 is an explanatory diagram showing a first image signal, FIG. 7 is an explanatory diagram showing a second image signal, and FIG. 8 is an explanatory diagram showing an image of a target signal.

First, in FIG. 1, reference numeral 20 denotes first and second image pickup devices 21 and 22 each comprising a visible CCD camera and an infrared CCD camera of different types.
Is an external pre-processing circuit for receiving a first image signal (visible image signal) 21a and a second image signal (infrared image signal) 22a from the CPU and outputting an external synchronization signal 20a to each of the imaging devices 21 and 22. The image preprocessing circuit 20 converts each of the image signals 21a and 22a into a first digital image signal 2
1aA and the second digital image signal 22aA are input to the histogram processing circuit 23. The imaging devices 21 and 22 are configured to view the same field of view by the external synchronization signal 20a.

The BGATE signal 20b, the horizontal synchronizing signal 20c, the vertical synchronizing signal 20d, the X-direction counter value signal QX, the Y-direction counter value signal QY, the TGATE signal 20e, which are various timing signals output from the image preprocessing circuit 20, The clock signal 20f is input to the histogram processing circuit 23.

The background removal signal 23a from the histogram processing circuit 23 and the digital image signals 21aA,
22aA is a look-up table (abbreviated LUT) circuit 2
4 and at the same time, the clock signal 20f is input to the look-up table circuit 24. The look-up table circuit 24 has a function of determining a difference in contrast between the image signals 21aA and 22aA at the same temperature and a difference between the temperatures at the same contrast.

The above-mentioned histogram processing circuit 23 and look-up table circuit 24 are configured as shown in FIG. 2, and the histogram processing circuit 23 is connected to an arithmetic unit 30 composed of a CPU system and this arithmetic unit 30. It comprises a first storage section 31 composed of an SRAM.

The arithmetic unit 3 of the histogram processing circuit 23
0 is connected to a second storage unit 32 composed of a two-port RAM, and the lookup table circuit 24 is configured to output a target signal 24a.

Next, a description will be given of a case where image signals are separated using the image signal separating apparatus having the above-described configuration.
First, the same field of view is imaged by each of the imaging devices 21 and 22.
The obtained first and second image signals 21a and 22a are input to the image preprocessing circuit 20.

The image signals 21a and 22a are A / D-converted by an A / D converter (not shown), and are converted into a first digital image signal 21aA and a second digital image signal 22a.
aA is input to the histogram processing circuit 23 and the look-up table circuit 24. At the same time, the BGATE signal 20b, which is each timing signal, and the horizontal synchronization signal 20
c, vertical synchronization signal 20d, X-direction counter value signal QX,
The Y direction counter value QY, the TGATE signal 20e, and the clock signal 20f are input to the histogram processing circuit 23, and the clock signal 20f is input to the look-up table circuit 24.

The clock signal 20f is composed of a dot clock signal, and its cycle indicates one pixel. Each of the aforementioned signals 21aA, 22aA, 20b, 20
The c, 20d, QX, QY, 20e, and 20f create a two-dimensional histogram only in the background window 40 shown in FIGS. 3 and 8 in the histogram processing circuit 23, and obtain the background removal signal 23a obtained as a result. The background signal is transferred to the lookup table circuit 24 and input.

The look-up table circuit 24 has a function of removing the background signal in real time.

Next, each digital image signal 21aA,
22aA is latched by the D-type flip-flops 35 and 36 of the histogram processing circuit 23 in synchronization with the clock signal 20f, and the latched digital image signal 2
1a and 22a are multiplexed by the multiplexers 50 and 51 together with the operation output 30a from the operation unit 30, and are connected to the address buses A0 to AF of the first storage unit 31 having the clear terminal 31a.

The data outputs D00 to D00 of the first storage unit 31
D0F passes through a +1 circuit 54 and a multiplexer 55,
Connected to data inputs D10-D1F. The multiplexer 55 outputs +1 when the BGATE signal 20b is active, that is, when the BGATE signal 20b is within the background window 40.
The result is connected. In other cases, the output is connected as it is.

Within the above range, the MW / signal 31b
Is connected to the clock signal 20f,
Only when the E signal 20b is active, a two-dimensional histogram can be obtained. This two-dimensional histogram can be accessed by the arithmetic unit 30, which creates a background removal function 60 (corresponding to FT shown in FIG. 4).

Next, the background removal function data obtained by processing the histogram from the arithmetic unit 30 in the histogram processing circuit 23 is transferred to the second storage unit 32, and the D-type flip-flops 70, The first and second digital image signals 21a and 22a latched at 71 are converted, and the target signal 24a from which the background signal has been removed is output.

Next, a method of creating the background removal function 60 will be described. First, First Step: Histogram Creation FIG. 3A shows a two-dimensional histogram in the background window 40. NB = NB (IBW, IIR) where IBW is the first digital image signal 21aAIIR and the second digital image signal 22aA is the second stage: αCUT processing As shown in FIG. 3B, αCUT processing is performed. Where B
S is the total number of pixels (area) in the background window, and α is a coefficient having a range from 0 to 1.0. Third stage: Normalization is performed as shown in FIG. For example, the following processing is performed to normalize the maximum value to 255. NB2 = 255 · NB1 / (α · BS) The meaning of this numerical value indicates the degree to which IBW and IIR are background signals. Fourth stage: As shown in FIG. 3D, an inversion process is performed. NB3 = 255-NB2 The meaning of this numerical value indicates the degree to which IBW and IIR are not background signals. Fifth step: As shown in FIG. 3E, MIN processing is performed by the following equation (1) to create a background removal function FT.

(Equation 1) Here, min (minimum) (•) takes the minimum value.

FIG. 4 shows an example of the operation. FIG. 4A shows a two-dimensional histogram of the background signal 24A in the background window 40. FIG. FIG. 4B shows the background removal function 60 created by the histogram. FIG. 4C shows a histogram in the target window. This histogram is a mixture of the background signal 24A and the target signal 24a. FIG. 4D shows a graph in which FIG. 4B and FIG. 4C are overlapped so that the effect of the present invention can be easily understood. As can be seen, the background signal 24A is removed and only the target signal 24a is extracted. That is, since the look-up table 24 has a function of distinguishing the difference in contrast at the same temperature and the difference in temperature at the same contrast as described above, even if there is no brightness distribution between the background and the object, The background can be easily removed using the difference or the brightness distribution difference.

Next, FIG. 5 shows another embodiment of FIG. 1, in which the above-mentioned digital image signals 21a, 22A, B
GATE signal 20b, horizontal synchronization signal 20c, vertical synchronization signal 20d, X direction counter value QX, Y direction counter value Q
Y, TGATE signal 20e and clock signal 20f are
The center of gravity of the target (target) 100 in the target window 90 in the X and Y directions can be calculated by being input to the center of gravity detection circuit 80 connected to the lookup table circuit 24.

[0029]

Since the image signal separating method and apparatus according to the present invention are configured as described above, the following effects can be obtained. That is, a two-dimensional histogram in the background window is obtained from two types of digital image signals, visible and infrared, and the histogram is processed to remove the background signal component in the target window. Even when there is no difference, an object can be obtained very easily from the background using the temperature difference or the brightness distribution difference.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an image signal separation method and apparatus according to the present invention.

FIG. 2 is a detailed block diagram of FIG.

FIG. 3 is an explanatory diagram showing a method of creating a background removal function.

FIG. 4 is an explanatory diagram showing extraction of a target signal.

FIG. 5 is a block diagram for detecting the position of the center of gravity of the target signal.

FIG. 6 is an explanatory diagram showing a first image signal.

FIG. 7 is an explanatory diagram showing a second image signal.

FIG. 8 is an explanatory diagram showing an image of a target signal.

FIG. 9 is a block diagram according to a conventional method.

FIG. 10 is a conventional image configuration diagram in which the average brightness GAV in a background window is obtained.

FIG. 11 shows a conventional maximum brightness GHI in a target window,
FIG. 5 is an image configuration diagram in which a minimum brightness GLO is calculated and a window comparator is set.

FIG. 12 is an image configuration diagram showing a conventional histogram in a background window.

FIG. 13 is an image configuration diagram showing a conventional background window average value calculation, a histogram in a target window, and a window comparator setting value setting.

FIG. 14 is an image configuration diagram showing a conventional histogram in a background window.

FIG. 15 is an image configuration diagram showing conventional background window average value calculation, target window histogram, and window comparator set value setting.

[Explanation of symbols]

 Reference Signs List 20 image preprocessing circuit 21, 22 imaging device 21a, 22a image signal 21aA, 22aA digital image signal 23 histogram processing circuit 23a background removal signal 24 look-up table circuit 24a target signal 40 background window 90 target window

Claims (2)

(57) [Claims] 1. First and second imaging devices of different types from each other
The same field of view is imaged by (21, 22), and the first image signal (21a) composed of the obtained visible image signal and the second image signal (22a) composed of the infrared image signal are processed to obtain two types of digital image signals. (21aA, 22aA), and a two-dimensional histogram only in the background window (40) is extracted from each of the digital image signals (21aA, 22aA) using the histogram processing circuit (23), and the two-dimensional histogram is subjected to minimum processing. The digital image signals (21a, 22a) latched by the look-up table circuit (24) are exchanged with the background removal function data obtained by the processing including the processing, and the background signal component in the target window (90) is removed. And separating and extracting only the target signal (24a) and taking the position of the center of gravity of the target signal (24a). 2. A first image pickup device (21) and a second image pickup device (22) of different types, and a first image signal (21a) composed of visible image signals obtained from each of the image pickup devices (21, 22). ) And a second image signal (22a) composed of an infrared image signal, and a first digital image signal (21aA) and a second digital image signal (21aA) obtained from the image preprocessing circuit (20). 2 Input digital image signal (22aA) and background window (40)
A histogram processing circuit (23) for extracting a two-dimensional histogram from the digital image signals (21aA, 22aA)
And a background removal signal from the histogram processing circuit (23).
(23a) and a look-up table circuit (24) having a two-port ram (32) and having a function of discriminating a difference in contrast at the same temperature and a difference in temperature at the same contrast. Uptable circuit (2
4) uses the look-up table circuit based on the background removal function data obtained by processing the two-dimensional histogram.
The digital image signals (21a, 22a) latched by the flip-flops (70, 71) of (24) are converted to remove the background signal portion of the digital image signals (21aA, 22aA), and only the target signal (24a) is removed. An image signal separating apparatus characterized in that the look-up table circuit (24) is connected to a center-of-gravity detecting circuit (80).