JP2002252805A - Imaging apparatus and method, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus that quickly and properly controls an imaging means, and acquires an appropriate image, even if an object whose image is photographed by an object instruction is changed. SOLUTION: This imaging apparatus has the imaging means 10 for picking up an image over a wide range of an angle of view to obtain a photographed image, an object direction indication means 20 for indicating the direction of the object, a control information acquiring means 30 for acquiring plural control information for each direction, a control information retaining means 50 for allowing the direction to correspond to the plural control information acquired by the control information acquiring means 30 for retaining, and a control information selection means 40 for selecting the plural control information, corresponding to the direction that is indicated by the object direction indication means 20 from the control information retaining means 50. When the direction is indicated by the object direction indicating means 20, the control information corresponding to the indicated direction is selected from information that has been retained already by the control information retaining means 50, and the imaging means 10 is controlled, based on the selected control information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an imaging apparatus, an imaging method, a program, and a recording medium, and particularly suitable for application to a conference content capture system, a video conference system monitoring camera, and the like.

[0002]

2. Description of the Related Art There have been conventionally known some video recording systems for recording conference contents and monitoring indoors with a wide viewing angle using a video camera or the like. These are roughly classified as follows. A method of recording a video camera in a direction of a target subject by a driving unit, for example, Japanese Patent Application Laid-Open No. 8-223551.
No.-A method of recording at a wide angle of view and electronically cutting out and displaying a target object portion, for example, Japanese Patent Application Laid-Open No. 2000-83242 The advantage of the former is that only the target object is imaged, Focus adjustment and the like can be performed. However, on the other hand, there is a disadvantage in that the image cannot be switched in real time if the object is changed during shooting in order to point the camera at the object using the driving means. In the latter advantage, since images are recorded with a wide angle of view, the image can be changed in real time even if the object is changed. However, there is a disadvantage in that an exposure amount and a focus adjustment suitable for a target object are not performed because a wide angle of view is imaged. To solve these problems, see JP-A-11-41
In Japanese Patent No. 496, dimming means is provided for each of a plurality of objects within a wide angle of view so as to obtain images suitable for the plurality of objects. However, this method requires a special light control means, and cannot cope with focusing other than light control.

[0003]

According to the above-mentioned prior art, when a plurality of subjects are captured in a wide angle of view and a target subject is extracted later, an image is optimally taken except for the exposure conditions of the target subject. No method has been proposed. Further, in the method of changing the shooting direction according to the subject, a mechanical driving unit is required to change the direction, and it takes time to change the direction, and a captured image during that time cannot be properly captured. SUMMARY An advantage of some aspects of the invention is to solve the above-described problem. Even when a subject to be imaged is changed by a subject instruction, the present invention can quickly and appropriately control an imaging unit to obtain an optimal image. It is an object to provide an apparatus, an imaging method, a program, and a recording medium.

[0004]

In order to solve the above-mentioned problems, the invention according to claim 1 of the present application provides an image pickup means for picking up an image in a wide range of view angle to obtain a picked-up image, and instructs a direction of a subject. Subject direction instructing means, control information obtaining means for obtaining a plurality of control information for each direction, control information holding means for holding the direction and a plurality of control information obtained by the control information obtaining means in association with each other, Control information selecting means for selecting, from the control information holding means, a plurality of pieces of control information corresponding to the direction designated by the subject direction designating means, and when the direction is designated by the subject direction designating means, the designated direction Is selected from the control information holding means that has already been stored in the control information holding means, and the imaging means is controlled based on the selected control information. According to a second aspect of the present invention, in the imaging device according to the first aspect, the control information acquisition unit creates a plurality of pieces of control information using a captured image obtained from the imaging unit. I do. The invention according to claim 3 is:
3. The imaging apparatus according to claim 1, further comprising a range image creation unit that creates a partial image of a captured image obtained from the imaging unit in a direction specified by the subject direction specification unit. And

[0005] The invention described in claim 4 is based on claim 1,
4. The imaging device according to 2 or 3, further comprising a range information holding unit that holds range information of a plurality of partial images of the captured image, wherein the control information acquisition unit is configured to store the range information held by the range information holding unit. And acquiring a plurality of pieces of control information based on the control information. The invention according to claim 5 is
The imaging apparatus according to any one of claims 2 to 4, further comprising a mode instructing unit for instructing a recording mode for recording a captured image and a non-recording mode for not recording, wherein the control information acquisition unit includes: When the non-recording mode is instructed by the mode instructing means, a plurality of pieces of control information are created by using a captured image obtained from the imaging means.
According to a sixth aspect of the present invention, in the imaging device according to any one of the first to fifth aspects, the pointing direction indicating means detects a sound emitted from the subject, and determines a direction of the sound. It is characterized by specifying a subject. Also,
According to a seventh aspect of the present invention, in the imaging apparatus according to any one of the first to fifth aspects, the control information acquired by the control information acquisition unit is an exposure amount in the imaging unit. And

According to an eighth aspect of the present invention, there is provided an image pickup apparatus according to any one of the first to fifth aspects, wherein
The imaging unit has an imaging optical system for imaging a subject image, and the control information acquired by the control information acquisition unit includes:
It is characterized in that it is an extension amount of the imaging optical system.
According to a ninth aspect of the present invention, in the imaging device according to any one of the first to fifth aspects, the imaging means has a variable-magnification imaging optical system for imaging a subject image,
The control information obtained by the control information obtaining means is a magnification of the photographing optical system. Further, according to a tenth aspect of the present invention, a plurality of pieces of control information relating to imaging obtained before the direction of the subject is designated are stored in the control information holding means in association with each direction, and the control information storing unit stores the plurality of pieces of control information. When a direction is designated, control information corresponding to the designated direction is selected from those already held in the control information holding means, and imaging is performed based on the selected control information.

[0007] The invention according to an eleventh aspect provides an image capturing means for capturing a wide field angle range to obtain a captured image, a subject direction designating means for designating a subject direction,
Control information obtaining means for obtaining a plurality of control information for each direction, control information holding means for holding the direction and a plurality of control information obtained by the control information obtaining means in association with each other,
Control information selecting means for selecting, from the control information holding means, a plurality of pieces of control information corresponding to the direction designated by the subject direction designating means, and when the direction is designated by the subject direction designating means, the designated direction Is selected from those already stored in the control information storage means, and the function is made to control the imaging means based on the selected control information. The invention according to claim 12 is a computer-readable recording medium that records a program for causing a computer to function as an imaging device, and an imaging unit that obtains a captured image by capturing a wide angle of view range, Subject direction instructing means for instructing the subject direction, control information acquiring means for acquiring a plurality of control information for each direction, and control for associating the direction with a plurality of control information acquired by the control information acquiring means Information holding means, and control information selecting means for selecting from the control information holding means a plurality of pieces of control information corresponding to the direction instructed by the subject direction instructing means, wherein a direction is designated by the subject direction instructing means. At this time, the control information corresponding to the indicated direction is selected from among those already held in the control information holding means, and based on the selected control information, Recording a program for causing the functions to control the imaging means.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The construction and operation of an embodiment of the present invention will be described below in detail with reference to the drawings. First Embodiment (1) Functional Configuration of First Embodiment FIG. 1 shows a functional configuration for describing an embodiment (first embodiment) of an imaging apparatus according to the present invention. It is a block diagram. The imaging apparatus according to the first embodiment includes an imaging unit 10, a subject direction instruction unit 20, a control information acquisition unit 30, a control information selection unit 40, and a control information holding unit 50. The imaging unit 10 captures a subject image and creates a captured image. In this method, an image signal is output from an image sensor that converts the formed light amount into an electric signal from a subject image that has passed through a photographing lens and an aperture. After removing noise etc. by the circuit and gain control, the A / D converter
The image signal is converted to an optimal sampling frequency (for example, NTS
This is performed by converting into a digital signal at an integer multiple of the subcarrier frequency of the C signal. The control information acquisition unit 30 acquires a plurality of pieces of control information using the imaging information obtained by the imaging unit 10 and records the acquired plurality of pieces of control information in the control information holding unit 50 such as a memory.
This control information includes aperture control information, shutter control information,
There are focus control information, scaling control information, and the like.

The control information holding means 50 has a data structure as shown in FIG. 2, and holds, for each of a plurality of divided image areas, an area and a plurality of pieces of control information for the area in association with each other. You. Or, the control information holding means 5
0 may be associated with the direction of each subject as shown in FIG. 3, and the subject and a plurality of pieces of control information corresponding to the subject may be stored in association with each other. In this image division, the captured image acquired by the imaging unit 10 is divided into regions simply arranged as shown in FIG. 4 or regions set for each subject as shown in FIG. Subject direction indicating means 20
Indicates the direction of the subject. For example, in the case of a conference scene, this instruction may be one in which the sound generated by the subject is captured and the direction of the subject is detected. Two microphones are prepared as shown in FIG. 6 to detect the direction of the subject, and when the sound source (person) makes a voice (speech),
This sound reaches two microphones, and at this time, there is a time difference between audio signals reaching the microphones. By detecting this time difference, the direction of the sound source can be specified.
The direction discriminating circuit for detecting the time difference can be formed by a circuit as shown in FIG.

The signals of the microphones 1 and 2 are arbitrarily delayed by delay circuits, and their outputs are compared by a comparator. The amount of delay at which the output of the comparator becomes 0 corresponds to Δt (see FIG. 8). That is, the time difference Δt can be detected by varying the delay amount of the delay circuit by the delay amount setting means and detecting the delay amount at which the output of the comparator becomes 0. For example, a signal as shown in FIG. 8 is sampled at a certain sampling interval St, and the sampled sound pressure V1 (n)
And V2 (n) are detected.

This is because the correlation value F is given by the following equation 1.
(K) is obtained for k = −m to m (n> m), and the phase difference Δt = k × St is obtained by detecting the smallest F (k). Since the obtained phase difference Δt corresponds to the time difference,
2 where B is the distance between the two microphones and v is the speed of sound.
Is obtained by Expression 2. Further, the subject direction designating means 20 is configured to display the captured image obtained by the imaging means 10 on a display device such as a touch panel, and to designate the pointing direction by directly touching the captured image. Can also. When the direction is instructed by the subject direction instructing means 20, the control information selecting means 40 selects corresponding control information from a plurality of pieces of control information held in the control information holding means 50, and selects the selected control information. The imaging unit 10 is controlled by the information.

(2) Operation of First Embodiment FIG. 9 is a flowchart for explaining the operation of the image pickup apparatus of the first embodiment. Here, for the sake of simplicity, the control information will be described by taking the exposure amount of the imaging unit as an example, but other control information can be similarly realized. First, control information (aperture control amount A and CCD
The shutter control amount T of the electronic shutter is initialized with a predetermined initial value (step S100). The imaging unit 10 is controlled by a control amount set as control information. That is, the diaphragm driving is performed, and the diaphragm is changed to the diaphragm corresponding to the control amount A.
The electronic shutter of the CD is changed to a shutter time corresponding to the shutter control amount T (step S200). The image is captured by the CCD, and the captured image is obtained (step S300). A plurality of pieces of control information are obtained (Step S400).

The brightness of each area S in an area in which the obtained captured images are simply arranged as shown in FIG. 4 or an area set for each subject as shown in FIG. The level L is detected and compared with a specified luminance level L0 that provides an appropriate exposure level, and an amount that provides an appropriate exposure level is calculated. Here, when k is a coefficient, A is an aperture value, and T is a shutter speed, an appropriate exposure amount (E) is
E = k · A · T. For example, when the shutter control amount (T (m)) is changed without changing the aperture for the m-th region of the divided region, assuming that the shutter control amount at the time of acquiring the captured image is T, It can be calculated from Equation 3. T (m) = T * L0 / L (Equation 3) When the aperture for the m-th area is set to A (m), the aperture control amount when the captured image is acquired is A Can be calculated from the following Expression 4. T (m) = (A / A (m)) * T * L0 / L (Equation 4) Such processing is performed for all the divided areas of the captured image from m = 1 to N Thereby, control information of a plurality of areas can be obtained. The control information (aperture control information A and shutter control amount T) for a plurality of areas obtained in this way is stored in the control information holding means 50 such as a memory (step S500).

Next, for example, a sound from a sound source is detected using a plurality of microphones, a phase difference is detected from the plurality of sounds, and a subject direction is calculated from the phase difference (step S600). Next, for example, when control information is acquired in the range of each divided area as shown in FIG. 4 or FIG. 5 and stored in the control information holding unit 50, the image position V corresponding to the subject direction in the control information holding unit 50 The area S (m) that contains'
And control information (A (m) and T
(M)) is taken out from the control information holding means 50 (step S700). As another method, when control information is obtained in a range divided into regions as shown in FIG. 4, control information for a region around the position of V ′ may be obtained by interpolation. . While the photographing is being continued, the process returns to step S200 and continues. By adopting such a configuration, a plurality of pieces of control information are obtained before the subject is instructed, and when the subject is actually instructed, it is only necessary to select the already calculated and held control information. As a result, high-speed processing can be performed, and an appropriate image can be obtained because the imaging unit 10 performs control appropriate for the instructed subject.

In the same manner as described above, the focus adjustment of the photographing lens is performed by extending the lens to a plurality of positions, detecting the frequency component for each image range, and detecting the amount of extension in which the high-frequency component is the largest. Can be. However, in this case, the captured image is blurred because the image is captured with a plurality of feeding amounts. Since it is not preferable to record such an image, a mode in which a captured image is not recorded and a mode in which a captured image is recorded are provided, and focus detection is performed when a non-recording mode is designated. For example, in the case of a system for recording the contents of a meeting, it is preferable to set the imaging apparatus in a non-recording mode prior to the meeting, acquire focus information of a plurality of subjects, and then set the recording mode for recording the meeting. Image can be obtained. Alternatively, the non-recording mode may be set when the speaker is not speaking. Also, regarding the magnification, control information may be set in advance so that a human outline is extracted from the image information and the size becomes appropriate. Alternatively, control information may be set such that a distance is calculated from the above-mentioned focus information, and the person is changed in magnification so as to have an approximately appropriate size. By instructing a scaling factor suitable for the subject in this way, a high-resolution image can be obtained.

<Second Embodiment> FIG. 10 is a block diagram showing a functional configuration for explaining another embodiment (a second embodiment) of the imaging apparatus according to the present invention. The imaging apparatus according to the second embodiment includes an imaging unit 10, a subject direction indicating unit 20, a control information acquiring unit 30, and a control information selecting unit 4.
0, a control information holding unit 50, and a range image creating unit 60. In the configuration of FIG. 10, components having the same functions as those of the first embodiment are denoted by the same reference numerals, and only different points will be described here. Based on the subject direction instructed by the subject direction instructing means 20, control information is selected by the control information selecting means 40, and the imaging means 10 is controlled by the selected control information to obtain a captured image. The means 60 extracts a partial image in a predetermined range in the direction of the subject from the obtained captured image, and sets this partial image as an output image. With the configuration as in the second embodiment, a partial image suitable for a target subject can be obtained, so that it is not necessary to record the entire image, and a small recording amount is required.

<Third Embodiment> FIG. 11 is a block diagram showing a functional configuration for explaining another embodiment (third embodiment) of an imaging apparatus according to the present invention. The imaging apparatus according to the third embodiment includes an imaging unit 10, a subject direction instructing unit 20, a control information acquiring unit 30, and a control information selecting unit 4.
0, a control information holding unit 50, a range image creating unit 60, and a range information holding unit 70. In the configuration of FIG. 11, components having the same functions as those of the first embodiment and the second embodiment are given the same reference numerals, and only different points will be described here. Range information holding means 70
Holds range information of a plurality of divided areas as shown in FIG. 4 or FIG. The imaging device according to the third embodiment having such a configuration operates as follows. Based on the range information of the plurality of areas held in the range information holding unit 70, a plurality of pieces of control information are obtained from the captured images in the range, and the plurality of pieces of control information are associated with the range information of the area. The information is recorded in the control information holding means 50. Next, the subject direction instructing means 20 designates the subject direction, and the control information selecting means 40 calculates which of the plurality of regions is hit from the subject direction, and stores a plurality of control information corresponding to the region in the control information. Taken out from the means 50, and
To obtain a captured image. The range image creation means 60
The range of a region corresponding to the subject direction specified by the subject direction specifying means 20 is calculated, and a range image of this range is created. With the configuration according to the third embodiment, a partial image suitable for a target subject can be obtained, so that it is not necessary to record the entire image, and a small recording amount is required.

The following effects can be achieved by configuring each embodiment as described above. By controlling the imaging means at high speed and appropriately in accordance with the target object, an appropriate target image can be obtained, and disturbance of the captured image can be reduced even if the subject is changed. -By obtaining a plurality of accurate control information from the captured image, the imaging means can be accurately controlled. Obtaining a partial image suitable for the intended subject eliminates the need to record the entire image, and requires only a small amount of recording. By holding an imaging range for obtaining control information, control information suitable for the subject can be obtained, so that an appropriate image can be obtained. Obtaining control information in the non-recording mode allows quick control of the imaging unit in the recording mode, and eliminates image disturbance even when the target subject is changed. The subject direction is changed according to the sound generated by the subject, so that the subject direction can be easily specified. Obtaining control information on focus and magnification in the non-recording mode enables quick control of the image pickup means in the recording mode, so that an image of a focused target object and a high-resolution image can be obtained.

<Implementation by Program> FIG. 12 is a block diagram illustrating an embodiment in which the image pickup apparatus of the present invention is configured using a microprocessor. First, a program for realizing the processing functions of the above-described imaging device is RO
M (Read Only Memory), and the program uses the CPU (Cen) based on the given information.
(Tral Processing Unit). The light from the subject forms a subject image through the taking lens, and the formed subject image is CC.
A subject image is captured by D (charge coupled device), and an imaging signal is generated. This imaging signal is converted from an analog signal into digital information by a CDS (correlated double sampling) / A / D converter, and the imaging image information described in the above-described imaging means 10 is created. The audio signals from the plurality of microphones specify the subject direction by executing a program having the function of the subject direction indicating means 20 described above,
Get audio information. When the direction of the subject is detected, a program having a function of the control information selecting means 40 is executed, and
The control information is selected from the control information holding unit 50, and the aperture driving unit controls the aperture of the lens and the electronic shutter of the CCD. Further, the intermediate data created during the processing and the contents of the control information holding means 50 and the range information holding means 70 are stored in a memory, and read out by the CPU as necessary.
Correction and writing are performed. The created image is L
The sound is output to a CD and the speaker is output. Further, a program for realizing the function of the imaging apparatus of the present invention is stored in a server computer provided with a storage device such as a magnetic disk, and the server computer and a user terminal (even a computer such as a camera and a video camera) (It may be an imaging device) via a network, and it can be downloaded to the user's terminal or distributed from the server computer to the terminal.

[0020]

As described above, according to the present invention,
Even if the subject to be imaged is changed by the subject instruction, it is possible to control the imaging means quickly and appropriately and acquire an optimal image.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a functional configuration for describing an embodiment of an imaging device of the present invention.

FIG. 2 is a diagram illustrating an example of a data structure of a control information holding unit.

FIG. 3 is a diagram showing another example of the data structure of the control information holding means.

FIG. 4 is a diagram illustrating an example of screen division of a captured image.

FIG. 5 is a diagram showing another example of screen division of a captured image.

FIG. 6 is a diagram for explaining a method of detecting a subject direction.

FIG. 7 is a circuit diagram illustrating a configuration example of a direction discriminating circuit that detects a time difference.

FIG. 8 is a graph showing sound pressure when a signal from a sound source is sampled by each microphone.

FIG. 9 is a flowchart for explaining the operation of the first embodiment.

FIG. 10 is a block diagram illustrating a functional configuration according to a second embodiment;

FIG. 11 is a block diagram illustrating a functional configuration according to a third embodiment;

FIG. 12 is a block diagram when the imaging apparatus of the present invention is configured using a microprocessor.

[Explanation of symbols]

 10 imaging means 20 subject designating means 30 control information obtaining means 40 control information selecting means 50 control information holding means 60 range image creating means 70 range information holding means

Claims (12)

[Claims] An image capturing unit that captures a wide field angle range to obtain a captured image; a subject direction instructing unit that instructs a direction of a subject; a control information acquiring unit that acquires a plurality of pieces of control information for each direction; Control information holding means for holding a direction in association with a plurality of pieces of control information obtained by the control information obtaining means; and a plurality of pieces of control information corresponding to a direction instructed by the subject direction instructing means. Control information selecting means for selecting from among the following: when a direction is designated by the subject direction designating means, control information corresponding to the designated direction is selected from those already held in the control information holding means, An imaging apparatus, wherein the imaging unit is controlled based on selected control information. 2. The imaging apparatus according to claim 1, wherein the control information acquisition unit creates a plurality of pieces of control information using a captured image obtained from the imaging unit. 3. A range image creating unit according to claim 1, wherein a partial image of a captured image obtained from said imaging unit is created in a direction designated by said subject direction designating unit. An imaging device comprising: 4. The image pickup apparatus according to claim 1, further comprising a range information holding unit for holding range information of a plurality of partial images of the picked-up image, wherein said control information acquiring unit includes said range information. An imaging apparatus for acquiring a plurality of pieces of control information based on range information held by a holding unit. 5. The imaging apparatus according to claim 2, further comprising: a mode instructing unit for instructing a recording mode for recording a captured image and a non-recording mode for not recording, wherein the control is performed. An image capturing apparatus, wherein the information acquiring unit creates a plurality of pieces of control information using a captured image obtained from the image capturing unit when the mode instruction unit designates a non-recording mode. 6. The imaging apparatus according to claim 1, wherein the pointing direction indicating means detects a sound emitted from the subject and specifies the subject based on a direction of the sound. An imaging device characterized by the above-mentioned. 7. The imaging apparatus according to claim 1, wherein the control information acquired by the control information acquisition unit is an exposure amount in the imaging unit. apparatus. 8. The image pickup apparatus according to claim 1, wherein the image pickup unit has an image pickup optical system for picking up an object image, and the control unit obtains the image. The control information to be performed is an extension amount of the imaging optical system, wherein 9. The imaging apparatus according to claim 1, wherein the imaging unit has a variable-magnification imaging optical system for imaging a subject image, and the control information acquisition unit. Wherein the control information acquired in step (1) is a variable power of the photographing optical system. 10. A plurality of pieces of control information relating to imaging obtained before a direction of a subject is designated is stored in control information holding means in association with each direction, and the direction of the subject is designated. An imaging method characterized in that control information corresponding to the indicated direction is selected from those already held in the control information holding means, and an image is picked up based on the selected control information. 11. An image capturing means for capturing a wide field angle range to obtain a captured image, a subject direction instructing means for instructing a subject direction, and a control information acquiring means for acquiring a plurality of pieces of control information for each direction. A control information holding unit that holds the direction and the plurality of pieces of control information obtained by the control information obtaining unit in association with each other; and a plurality of pieces of control information corresponding to the direction specified by the subject direction indicating unit. Control information selecting means for selecting from the holding means, and when a direction is designated by the subject direction designating means, selects control information corresponding to the designated direction from among those already held in the control information holding means. A program for functioning to control the imaging unit based on the selected control information. 12. A computer-readable recording medium in which a program for causing a computer to function as an imaging device is recorded, wherein the imaging means obtains a captured image by capturing a wide range of angle of view, and instructs a subject direction. Subject direction instructing means, control information obtaining means for obtaining a plurality of control information for each direction, control information holding means for holding the direction and a plurality of control information obtained by the control information obtaining means in association with each other, Control information selecting means for selecting, from the control information holding means, a plurality of pieces of control information corresponding to the direction designated by the subject direction designating means, and when the direction is designated by the subject direction designating means, the designated direction The control information corresponding to is selected from those already held in the control information holding means, and the imaging means is selected based on the selected control information. A computer-readable recording medium on which a program for causing the computer to function is controlled.