JPS62208783A - Automatic tracking video camera - Google Patents

Abstract

PURPOSE:To automatically track the human being of the commonest object by providing an infrared ray photoelectric conversion part for photoelectric conversion the infrared ray, detecting the position of the object from a photoelectrically converted infrared ray video signal and controlling to direct to the direction of the position of the object. CONSTITUTION:A light signal incident through a zoom lens 1 is separated into visible ray and the infrared ray by a dichroic mirror 2 and respectively photoelectrically converted by a visible ray photodetecting image pickup element 3 and an infrared ray photodetecting image pickup element 4. The infrared ray video signal obtained from the infrared ray photodetecting image pickup element 4 obtains the center of gravity of the object and the center position in an infrared ray video signal processing circuit 6 by an operation. Then, the quantity of displacement in the direction of a camera to set the center of gravity of the object to the center of a picture is calculated in a control circuit 7 and the direction of the camera is moved horizontally or vertically by a controller 8 to automatically track the human being.

Description

[Detailed Description of the Invention] [Field of Industrial Application] The present invention relates to an automatic tracking video camera, and in particular, to a video camera that performs automatic tracking using infrared light. The popularity of video cameras, both for home use and home use, is remarkable.

In the field of home video cameras, advances in LSI technology have made it possible to mass produce high-performance solid-state image sensors, which has led to smaller and cheaper cameras.
The user base is also expanding from specific users to general users.

As described above, as video cameras become more popular among the general public, it becomes important that anyone can easily operate them. For this reason, automation of essential functions has progressed, and functions such as automatic white balance adjustment and automatic focus have already become commonplace, and there is a desire to further improve operability.

Therefore, this is one of the functions that should be automated next.

There is an automatic tracking function that automates the shooting itself, which was traditionally left to the video camera photographer, and automatically causes the camera to follow the subject.

This automatic tracking function is expected to be used not only in home use but also in surveillance cameras for monitoring purposes, and is more effective than cameras with a fixed camera direction.

Incidentally, as a conventionally proposed video camera equipped with an automatic tracking function, for example, Japanese Patent Application Laid-Open No. 59-208983
There are some that are described in the No.

[Problem that the invention seeks to solve]

The automatic tracking camera described in Japanese Patent Application Laid-Open No. 59-20898:3 detects the movement of a subject from the signal, recognizes the moving object as a target for imaging, and detects the movement of the subject. Automatic tracking according to movement.

However, with such conventional technology, if an object other than the object in the video signal that the photographer is targeting moves, it is not possible to distinguish between the target object and other objects, making it difficult to properly automatically track the target object. was sometimes impossible.

Additionally, complex pattern recognition is required to accurately recognize a moving object as an object, which also has the drawback of increasing the circuit scale.

An object of the present invention is to overcome the drawbacks of the prior art described above.
To provide a video camera that does not require miscellaneous fL pattern recognition and can therefore automatically track a person, who is the most common subject, with a simple circuit configuration.

[Means for solving problems]

The above purpose is to photoelectrically convert infrared light in a video camera equipped with a visible light photoelectric conversion section that photoelectrically converts visible light and a signal processing section that synthesizes a video signal from the video signal obtained from the visible light photoelectric conversion section. An infrared light 1 conversion section is provided, a detection means for detecting a subject position from a photoelectrically converted infrared light video signal, and a camera is controlled to face in the direction of the subject position in accordance with a signal indicating the subject position from the detection means. This is achieved by providing a control means.

[Effect]

Infrared light video signals function as a sensor that recognizes people by utilizing the fact that the human body emits infrared rays. That is, when an image of a person is captured by the image capturing element, an electric signal corresponding to the infrared light is obtained from the infrared light photoelectric conversion section of the nuclear element. In the present invention, the position of the person is detected by the detection means based on this electrical signal. Depending on the position of the person as the subject detected in this manner, the control means controls the camera direction so that the camera always faces toward the subject. This makes it possible to always automatically track the direction of the camera in the direction of the person in response to the movement of a person. [Embodiment] The present invention will be described below with reference to the drawings.

The first section is a block diagram of an automatic tracking video camera according to a first embodiment of the present invention. In the figure, 1 is a zoom lens, and 2 is a dichroic mirror.

3 is an image sensor for receiving visible light, 4 is an image sensor for receiving infrared light, 5 is a visible light video signal processing circuit, 6 is an infrared light video signal processing circuit, 7 is a control circuit, 8 is a control device, and 9 is a control device. This is an image sensor drive circuit.

The optical signal incident through the zoom lens 1 is separated by the dichroic mirror 2 into oTtA light and infrared light I, which are respectively used for visible light reception. ! Photoelectric conversion is performed by the element 3 and the infrared light receiving image sensor 4.

The visible light receiving image sensor 3 is a single-tube or single-plate image sensor, and a color separation filter is provided on the light-receiving surface to produce a color image.
It is now possible to receive an elephant signal. on the other hand.

The infrared light-receiving image sensor 4 may be an image sensor that obtains a monochrome video signal, but is preferably one that has good sensitivity to infrared light. The color signal corresponding to the color separation filter obtained from the visible light receiving image sensor 3 is as follows.

Visible light video signal processing circuit 5 is supplied with signal processing similar to that of a normal video camera, and output as a video signal from output terminal 10. Infrared light obtained from infrared light receiving image sensor 4 The optical video signal is processed by an infrared optical video signal processing circuit 6.

First, signal processing such as amplification and clamping is performed.

Thereafter, the infrared light video signal processing circuit 6 calculates the center of gravity or center position of the subject based on the infrared light video signal. As mentioned earlier, the infrared light video signal can be regarded as a person recognition signal, so the center of gravity or center position of the object detected from the infrared light video signal can be used as is for automatic tracking of a single person. A detection signal of the center of gravity or center position of the subject, that is, a person, is input to the control circuit 7. The control circuit 7 calculates, for example, a displacement amount in the camera direction to align the center of gravity of the subject with the center of the screen based on the detection signal, and outputs a corresponding control signal.

Based on the control signal thus obtained, the control device 8 moves the camera direction left and right or up and down to automatically track the person.

FIG. 2 shows an example of the infrared light video signal processing circuit 6 in FIG. 1, where 61 is an amplifier, and 6° is an analog
Digital converter, 6. is a frame memory, and 64 is an arithmetic circuit.

The infrared light image signal obtained from the infrared light receiving image sensor 4 is sent to an amplifier 6. and analog-to-digital converter 6
．． Analog-to-digital conversion is performed by Analog-
The digitally converted signal.

Frame memory 6. is accumulated in Based on the digital values stored in the frame memory 6 in this way, the arithmetic circuit 6. Now, perform calculations to find the center of gravity of the subject. The center of gravity of the subject (XG + y□) is as follows (1),
Calculated using equation (2).

・・・・・・・・・＜1) ・・・・・・・・・(2) However, M and N are the number of horizontal and vertical sample points, x.

y is the coordinate o f (x+Y) is the value expressed by binarizing (1 or 0) the infrared light video signal at each coordinate.The coordinate is the sampling point on the image sensor, for example, the lower left corner. (The sampling point at the upper right corner of 111L is (M, N).

When the position of the center of gravity of the tracking target is detected in this manner, the control circuit 7 and the control device 8 automatically track the object by directing the camera toward the center of gravity of the subject.

The arithmetic circuit 64 and control circuit 7 will be explained in more detail using FIGS. 14 and 15.

FIG. 14 shows the arithmetic circuit 6 of FIG. This is a specific example of b
641 + '46 is the timing generator, 64! l
64? is the multiplier 'X unit, '43 + 646 is the integrator,
6416411 is a divider.

646 is an integrator.

Timing generator 64. The coordinate X is output from the timing generator 646, and the locus IIAy is output from the timing generator 646.
and y are multipliers 66. and y, respectively. and 63. The signal from the frame memory 6, [(x+Y), is multiplied by the +A divider 64. and integrated at 648. As a result, the numerator of the above formula (1) and (2j) is obtained. Also, the signal fcx*Y) is passed through the integrator 64. It is integrated by As a result, the denominators of equations (1) and (2) are obtained.

Exclude: J! The integrator 64 . output signal of integrator 64. By dividing the output signal of the integrator 6.8, that is, by normalizing (signal lid x coordinate) by the signal amount, the center of gravity The center of gravity y is obtained by dividing the output signal of the device 64.

FIG. 15 shows a specific example of the control circuit 7 shown in FIG.

7. is the reference coordinate generator %7. The reference coordinate generator 7t, which is a comparator, outputs coordinate data (Xo*Yo) specifying the position of the subject on the monitor screen. .

+ Yt3 is comparator 7. are compared. As a result, comparator 7. outputs a control signal according to the deviation between the given center of gravity and the coordinate data.

For example, X. >XQ, it is determined that the center of gravity of the subject is to the right of the monitor screen, and a control signal is output to move the video camera to the right. therefore.

For example, the coordinates of the center of the monitor screen are X. If +YO is set, the center of gravity of the subject is automatically tracked via the control device 8 so that it is always displayed at the center of the monitor screen.

FIG. 3 is a block diagram showing a second embodiment of the automatic tracking video camera of the present invention. In the figure, 5' is an oT visual aperture signal processing circuit, and 11 is a variable amplification circuit. The same parts or equivalent parts as in the embodiment shown in FIG. 1 are given the same reference numerals, and the explanation thereof will be omitted.

The feature of this embodiment is that when the illuminance is low and the signal-to-noise ratio (8/N ratio) of the video signal decreases, a part of the infrared light video signal obtained from the infrared light receiving image sensor 4 is amplified. The brightness signal is amplified by the variable intensity amplifier circuit 11, inputted to the image signal processing circuit 5', and added to the video signal obtained from the image sensor 3 for receiving the 9th congratulatory light to generate a luminance signal.

When a luminance signal is generated in this manner, a luminance signal with a high signal-to-noise ratio can be obtained even when the illuminance is low, and sensitivity (8/N ratio) can be significantly improved.

For example, in the video signal obtained from the image sensor 3 for receiving visible light, the noise voltage normalized by the signal voltage is N1m.Similarly, the noise voltage in the infrared light video signal obtained from the image sensor 4 for receiving infrared light is the signal. N is the value normalized by voltage.

Then, in the case of uncorrelated random noises, the SIN ratio of the luminance signal obtained by adding the visible light signal and the infrared light signal ts+i will be in the best condition.

FIG. 4 is a block diagram showing a third embodiment of the automatic tracking video camera of the present invention. In the same figure.

The same reference numerals as in FIG. 1 represent the same or equivalent parts. Further, 9' is a drive circuit, 12 is a separation circuit, and 13 is an image sensor.

In this embodiment, a visible light receiving section and an infrared light receiving section are provided in a single image capturing element 13, and the oJ visible light and infrared light are received by the same image capturing element 13. The circuit 12 connects a video signal obtained by the visible light receiver (hereinafter simply referred to as visible light video signal) and a video signal obtained by the infrared light receiver (hereinafter simply referred to as infrared light video (hereinafter referred to as N)). The visible light video signal is input to the visible light 1!S'! filter signal processing circuit 5, and the infrared video signal is input to the infrared video signal processing circuit 6. Automatic tracking is performed by the same operation as in the embodiment.

FIG. 5 is a diagram schematically showing the light-receiving surface of the image sensor 13 shown in FIG. 4, and 11 indicates one picture element. The other picture elements are picture elements for visible light reception, and these picture elements are similar to ordinary single-tube or single-chip sensor elements, with color filters of primary colors or complementary colors placed on each picture element in a regular pattern. It will be arranged.

A picture element for receiving infrared light can be easily realized by, for example, arranging color filters having spectral characteristics with high sensitivity to long wavelengths in the picture elements. Also.

As will be described later, in the f-image element 13, by deepening the P-type well layer below the photodiode,
Since the long wavelength sensitivity can be improved, the above-mentioned picture amateur can be realized.

FIG. 6 is a schematic diagram of the cross-sectional structure of the micrograph 13 taken along the line a-b in FIG. 5. In the figure, 14 is a P-type well, 15 is an N-type substrate, and 22 is a protective film (insulating film). The photographing element 13 shown in FIG. 5 has a structure in which only the P-type well layer at the bottom of the infrared light receiving element is deepened to improve long wavelength sensitivity, as in the previous MtJ. It has become. As a result, the infrared optical image II
The signal component can be obtained exactly. Details will be described later.

FIG. 7 is a schematic diagram showing another row of light-receiving surfaces of the image sensor 13 shown in FIG. 4. The difference between the image sensor shown in FIG. 7 and the above-described image sensor shown in FIG. 5 is that the infrared light-receiving picture element is arranged only at the periphery of the light-receiving surface. With such a picture element arrangement, it is possible to increase the degree of viewing of the visible light video signal at the center of the screen. However, compared to the one in FIG. 5, the automatic tracking function will be somewhat degraded.

FIG. 8 is a block diagram showing the configuration of the separation circuit 12 of FIG. 4. In the figure, h I J + 121 is a sample and hold circuit.

The video signal S from the sensor +* element 13 inputted to the separation circuit 124 is supplied to sample and hold circuits 121 and 122, respectively. In addition, the sample hold circuit 12. ．．
12□, each clock φ1 from the drive circuit 9'
．． φ, is supplied. The clock φ1 is a signal for sampling and holding only the visible light image signal component in the video signal S, and the clock φ2 is a signal for sampling and holding only the infrared light image signal component. As a result, the sample and hold circuit 121 r l 2! From the OT Shokoei Deep Sea No. 6 S separated from the video signal S
I and infrared light receiving section - Is! are output respectively.

Figure 9 shows the sample hold circuit 121°12 of Figure 6.
, a video signal S during operation, a visible original video signal 81 and an infrared light video signal S2 separated from the video signal S, and a sample hold clock (sample hold pulse) φ1. 3 is a waveform diagram (time chart) showing the relationship between φ and φ. The code 17 in the video signal S indicates the OT optical video signal component, and the shaded code 18 indicates the infrared optical video signal component. Since there is no pulse at the timing at which the outside light image fl 1 lit signal component 18 is output, the previous visible light image signal component 17 is held during this period. In this way, locations where there is no visible light receiving section (locations where there is an infrared light receiving section) are interpolated using video signal components obtained from the adjacent same light receiving section.

Infrared light reflection! The signal S is also obtained by holding the adjacent infrared light video signal component 18 by the same operation. It is an enlarged view of. In the same figure, 19 is an infrared light photoelectric conversion unit, and 19' is a cuttable light beam '14.
c conversion section, 20 is aluminum wire, 21 is gate, 40.41
indicates an N-swelled diffusion layer. In addition.

As described above, only the P-type well layer at the bottom of the infrared light receiving picture element is deepened to improve the long wavelength sensitivity (infrared sensitivity).

Since infrared light is long wavelength light, carriers (1
child). On the other hand, visible light generates carriers in a relatively shallow area. Therefore, visible light photoelectric conversion section 19'
Infrared light carriers generated deep in the substrate are collected in the N-type substrate 15t and flow to an external vL source that supplies a bias voltage for the substrate 15. As a result, in the sOr photoelectric conversion section', only carriers of visible light generated in the shallow portion of the substrate are absorbed in the N-type wave dispersion layer 40.

On the other hand, in the infrared photoelectric conversion section 19 , infrared light carriers generated in the deep part of the substrate and visible light carriers generated in the shallow part of the substrate are both collected in the N-swelled diffusion layer 41 . Therefore, when a voltage is applied to the gate 21, an infrared light video signal component (corresponding to 18i in FIG. 9) is obtained from the infrared light photoelectric conversion section 19 via the aluminum wire 20 or the like. Also, a visible light video signal component (corresponding to 17 in FIG. 9) is obtained from the visible light photoelectric conversion section 19' via the aluminum wire 20 or the like.

FIG. 12 is an enlarged view showing the cross-sectional structure of another example of the one-image sensing element 13 shown in FIG. 4. In the same figure, 23
24 is an N-type substrate, 24 is a P-type well, 25 and 26 are N-swelled diffusion layers, 1 is an infrared and visible light photoelectric conversion section, and B is a visible light photoelectric conversion section.

Since the infrared lights 27 and 28 are long wavelength lights, carriers 31 and 32 (electrons) are generated deep in the substrate.

Visible light 29.30 is relatively shallow and carrier 33°34
generate. The carriers 33, 34 are collected in the diffusion layer 25 and read out to the outside by any suitable known means to form a normal video signal. On the other hand, the carrier 31 is the diffusion layer 2
6, and by reading this out to the outside using known means, an infrared light video signal can be obtained. Note that the carrier 32 is collected on the N-type substrate 23 and flows to an external power source that supplies a bias voltage for the substrate 23.

Using FIG. 13, it is assumed that the carriers of infrared light are
The principle by which carriers of visible light are collected in the N-swelled diffusion layer 25 will be explained in more detail.

Figures (m) and (b) show the potentials between Eero and Harney in Fig. 12, respectively.

Since the lowest voltage is applied to the P-type well 24, as shown in FIGS. 13(a) and 13(b), the P-type well 24
The potential is highest in this part. Now, since the carriers 33 and 34 are generated closer to the surface (light receiving surface) than where the potential is maximum, they move towards the lower bottom, that is, the surface. On the other hand, carriers 31 and 32 are generated closer to the back surface than where the potential is maximum, so they move towards the lower potential, that is, the front surface.

Therefore, in the t14 & image element shown in FIG. 12, carriers of infrared light can be collected in the N-shaped wave scattering r@za, and carriers of visible light can be collected in the N-swelled diffusion layer 25. As a result, if an image sensor having the structure shown in FIG. 12 is used, an infrared light video signal and a visible light video signal can be separated in the image sensor. Therefore, it is possible not only to arrange the infrared photoelectric conversion sections sparsely as shown in FIGS. 5 and 7, but also to arrange them in every picture element.

FIG. 11 is a block diagram showing another embodiment of the automatic tracking video camera of the present invention. In FIG. 1, the same reference numerals as in FIG. 4 represent the same or equivalent parts.

This embodiment differs from the embodiment shown in FIG. 4 in that the control circuit 7# not only outputs a control signal for controlling the camera direction in accordance with the detection signal of the center of gravity position of the subject, but also outputs a control signal for controlling the camera direction. The point is that zoom control is also performed at the same time.

That is, when the subject deviates from the field of view, that is, when the detection signal of the center of gravity position of the subject cannot be obtained from the infrared light video signal processing circuit 6, the control circuit 7' causes the zoom lens 1 to widen the zoom accordingly. Outputs a signal to electrify the side (wide-angle end).

By doing this, even if the subject deviates from the field of view, it is possible to bring the subject back into the field of view, and the position of the center of gravity of the subject can be detected.

As a result, automatic tracking can be performed appropriately.

〔Effect of the invention〕

As is clear from the above description, the present invention does not require complicated pattern recognition as in the prior art, and therefore uses a relatively simple circuit configuration.

This has the effect of appropriately automatically tracking a person, who is the most common subject.

[Brief explanation of drawings]

FIG. 1 is a block diagram of a first embodiment of the present invention. FIG. 2 is a block diagram showing an example of the infrared light video signal processing circuit shown in FIG. 1, and FIGS. 3 and 4 are respectively @
5 and 7 are schematic diagrams of the light-receiving surface of the image sensor shown in FIG. 4, and FIG. 6 is along line a-b in FIG. 5. Cross-sectional view of the image sensor, No. 8
9 is a block diagram showing an example of the separation circuit of FIG. 4.
The figure is a signal waveform diagram for explaining the operation of the separation circuit, Figure 1O is an enlarged view of the part surrounded by the broken line 16 in Figure 6, and Figure 11 is a block diagram showing another embodiment of the present invention. , 12th
The picture is shot 1a#! A sectional view showing another example of the element, FIG. 13 is a diagram for explaining the movement of the carrier in FIG. 12, and FIG.
The figure is a block diagram showing a specific example of the arithmetic circuit shown in Fig. 2, and Fig. 15 is a block diagram showing an example of the control circuit shown in Fig. 1. Light receiving element % 4...Image sensor for infrared light receiving, 5.5'
... Visible light video signal processing circuit, 6... Infrared light video signal processing circuit, 7, 7'... Control circuit, 8... Control device. 12...Separation circuit, 13...Image sensor agent Michihito Hiraki, patent attorney Fig. 1 Fig. 3 Fig. 5 Fig. 6 Fig. 7 Fig. 8 Fig. 9 Fig. 10 Fig. 19 Fig. 11 Fig. 12 Fig. 13 fa) (b) Both 7 noyal Both 7 noyal front -111yu □ Back side

Claims (4)

[Claims] (1) In an automatic tracking video camera having a visible light photoelectric conversion section that photoelectrically converts visible light, and a visible light video signal processing circuit that generates a visible light video signal from the electrical signal photoelectrically converted by the visible light photoelectric conversion section. , an infrared light photoelectric conversion unit that photoelectrically converts infrared light; a detection unit that detects the position of the center of gravity of the subject from the electrical signal photoelectrically converted by the infrared light photoelectric conversion unit; and a center of gravity of the subject detected by the detection unit; An automatic tracking video camera characterized by comprising a control means for controlling the camera direction according to the position. (2) The electric signal photoelectrically converted by the infrared photoelectric conversion section and the electric signal photoelectrically converted by the visible light photoelectric conversion section are added to generate a luminance signal at low illuminance. An automatic tracking video camera according to claim 1. (3) In an automatic tracking video camera having a visible light photoelectric conversion section that photoelectrically converts visible light, and a visible light video signal processing circuit that generates a visible light video signal from the electrical signal photoelectrically converted by the visible light photoelectric conversion section. , an infrared light photoelectric conversion unit that photoelectrically converts infrared light; a detection unit that detects the position of the center of gravity of the subject from the electrical signal photoelectrically converted by the infrared light photoelectric conversion unit; and a center of gravity of the subject detected by the detection unit; a control means for controlling the camera direction according to the position; and when the detection means cannot detect the center of gravity position of the subject;
An automatic tracking video camera characterized by comprising means for generating and outputting a signal for widening the zoom of the camera. (4) The electric signal photoelectrically converted by the infrared photoelectric converter and the electric signal photoelectrically converted by the visible light photoelectric converter are added to generate a luminance signal at low illuminance. An automatic tracking video camera according to claim 3.