JPH11341336A - Device and method for controlling camera, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To control the direction of a camera by voice based on intuition. SOLUTION: When a voice is detected, a camera direction specification processor 40 takes out the present direction information from a camera and stores it in a camera direction storage device 42. A speaker direction specification processor 44 calculates the direction of a speaker by utilizing image recognition and/or voice directivity and stores it in a speaker direction storage device 46. A relative direction calculation processor 48 calculates relative direction between the camera and the speaker from camera direction information and speaker direction information and stores a relative direction information in a relative direction storage device 50. A voice recognition processor 52 performs recognition processing of inputted voice, identifies speaking content that shows the relative direction control of the camera and stores recognition result in a recognition result storage device 54. A relative representation analysis processor 56 calculates a camera direction control parameters from the relative direction information and the voice recognition result. A direction control parameter output processor 60 outputs a direction control parameter information to the camera.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera control apparatus and method, and a storage medium for storing a program for implementing the camera control method.

[0002]

2. Description of the Related Art Conventionally, a camera control device for instructing a photographing direction of a camera by voice has conventionally been designed to indicate a direction viewed from the camera.

[0003]

Therefore, in the prior art,
For example, when the speaker faces the camera, the direction of the camera may be controlled in a direction opposite to the intention of the speaker.

The present invention proposes a camera control apparatus and method which solves such inconveniences and enables more realistic camera control, and a storage medium for storing program software for implementing the method. The purpose is to do.

[0005]

SUMMARY OF THE INVENTION A camera control device according to the present invention is a camera control device for controlling a camera whose shooting direction can be controlled externally, and a camera direction specifying means for extracting current direction information of the camera. Speaker orientation identification means for obtaining the orientation of the speaker, relative orientation calculation means for calculating the relative orientation of the camera and the speaker, voice input means, and the input of the camera from the input voice of the voice input means. Voice recognition processing means for recognizing the utterance content related to the azimuth control, and the camera from the viewpoint of the speaker based on the relative orientation calculated by the relative azimuth calculation means and the recognition result of the voice recognition processing means. Relative expression analysis processing means for calculating an azimuth control parameter for performing azimuth control, and information indicating the azimuth control parameter calculated by the relative expression analysis processing means. And having a direction control parameter output processing means for outputting to the camera.

A camera control method according to the present invention is a camera control method for controlling a camera whose shooting direction can be controlled externally. The camera direction identification step for extracting current direction information of the camera and the direction of the speaker are determined. A desired speaker direction identification step, a relative direction calculation step for calculating a relative direction between the camera and the speaker, and a speech recognition for recognizing a speech content related to a direction control of the camera from an input voice of a voice input unit. From the processing step, the relative azimuth calculated in the relative azimuth calculation step and the recognition result of the voice recognition processing step, an azimuth control parameter for controlling the azimuth of the camera when viewed from the viewpoint of the speaker is calculated. The relative expression analysis processing step and information indicating the azimuth control parameter calculated in the relative expression analysis processing step And having a direction control parameter output processing step of outputting to.

[0007] In the storage medium according to the present invention, program software for implementing the above-described camera control method is stored in an externally readable manner.

[0008]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of hardware according to an embodiment of the present invention. 10 is a camera, 20
Is a camera control device for controlling the camera 10.

The camera 10 includes an imaging device 12 and an imaging device 1
An optical parameter control circuit 14 for controlling the optical parameters (focus, iris, shutter speed, gain, white balance, etc.) and an azimuth control circuit 16 for controlling the imaging azimuth (pan angle, tilt angle and zoom)
Is provided.

The camera control circuit 20 includes a CPU 22, an RA
M24, ROM 26, I / O 28 and secondary storage device 30
And these are connected to the bus 32. Further, the output of the microphone 34 is converted into a digital signal by the A / D converter 36 and connected to the bus 32. The I / O 28 connects to the optical system parameter control circuit 14 and the azimuth control circuit 16 of the camera 10 via a control signal line, and connects to a video output of the imaging device 12 via a video signal line. The control signal line may be RS232C or parallel IO. The video signal may be in the NTSC format or the YC separation format, but the video signal output format of the imaging device 12 may be any one. The video signal communication system is not limited to a specific standard. The ROM 26 stores a control program according to a control procedure as shown in the flowchart shown in FIG.

FIG. 2 is a schematic functional block diagram of the embodiment. The camera direction identification processing unit 40 extracts the current direction information of the camera from the direction control circuit 16. The camera azimuth holding unit 42 holds information on a relative azimuth from a predetermined azimuth of the camera 10 as a reference. The speaker direction identification processing unit 44 obtains the direction of the speaker using image recognition and / or the directivity of the microphone 34. For example, the speaker is caused to wear an identifier such as an emblem having a predetermined color and shape. Then, the speaker direction characteristic processing unit 44 determines whether or not an emblem worn by the speaker exists in the image captured from the camera 10. When it is determined that the emblem exists on the image, the relative orientation of the speaker with respect to the orientation of the camera 10 is obtained from the position of the emblem on the image and the current zoom magnification of the camera 10. If no speaker is present in the image, the current orientation of the camera 10 is maintained and the camera 10 is rotated until the emblem is in the center of the image. A relative direction is obtained from the direction at the center and the stored direction. The relative azimuth of the speaker with respect to the azimuth of the camera 10 thus obtained is stored in the relative azimuth holding unit 50. As described later, it may be determined whether or not the speaker is within the range of the angle of view of the camera 10. When the direction of the speaker is obtained by using the directivity of the microphone 34, a sound collecting microphone or a microphone array is used, which is a known technique.

The speaker orientation holding unit 46 holds speaker orientation information obtained by using the directivity of the microphone 34. The relative azimuth calculator 48 calculates a relative azimuth (angle) between the camera 10 and the speaker. The relative orientation holding unit 50 includes the camera 1
The relative orientation (angle) between 0 and the speaker is held. The voice recognition processing unit 52 detects voice and recognizes utterance contents related to the azimuth control of the camera 10. Recognition result holding unit 54
Holds the recognition result of the voice recognition processing unit 52. The relative expression analysis processing unit 56 determines an azimuth control parameter value for controlling the azimuth of the camera 10 based on the relative azimuth information and the recognition result. Specifically, the relative expression analysis processing unit 56
Next, a voice command and a direction control command of the camera 10 corresponding to the voice command are provided in a table (not shown). Then, the relative expression analysis processing unit 56 determines which of the voice commands in the above table the recognition result of the voice recognition processing unit 52 is, and performs azimuth control based on the azimuth control command of the camera 10 corresponding to the determined voice command. Determine the parameter value. For example, when the voice command is determined to be “here”, a direction control parameter value is determined so that the relative direction between the camera 10 and the speaker becomes 0. If it is determined that the direction to be controlled by the camera 10 is different depending on the positional relationship between the speaker and the camera 10, such as “right” or “left”, the command is stored in the relative orientation holding unit 50. The direction of the camera to be controlled is determined according to the relative direction of the speaker with respect to the direction of the camera 10, and the direction of the camera 10 is controlled at a predetermined angle. For example, the direction to be controlled by the camera may be different when the relative direction of the speaker with respect to the direction of the camera 10 is ± 90 degrees.

The azimuth control parameter holding unit 58 holds azimuth control parameter values. The azimuth control parameter output processing unit 60 outputs the azimuth control parameter value stored in the azimuth control parameter holding unit 58 or information indicating this to the I / O
Output to the azimuth control circuit 16 via O28.

A camera direction identification processing section 40, a speaker direction identification processing section 44, a relative direction calculation section 48, a voice recognition processing section 5
2. In the present embodiment, the relative expression analysis processing unit 56 and the azimuth control parameter output processing unit 60 are realized by software.
And / or stored in the secondary storage device 30. The camera direction holding unit 42, the speaker direction holding unit 46, the relative direction holding unit 50, the recognition result holding unit 54, and the direction control parameter holding unit 58 are the RAM 24 or the secondary storage device 30.

FIG. 3 shows an operation flowchart of the present embodiment. The operation of the present embodiment will be described with reference to FIG.

The voice of the speaker is captured by a microphone 34, converted into a digital signal by an A / D converter 36, and input to the CPU 22 via the bus 32. The voice recognition processing unit 52 checks whether or not the input sound is a voice, and waits until a voice is detected (S1).

When a voice is detected (S1), the camera azimuth identification processing unit 40 extracts the current azimuth information of the camera 10 from the azimuth control circuit 16 of the camera 10 and stores it in the camera azimuth holding unit 42 (S2). Speaker direction identification processing unit 44
However, using the image recognition and / or the directivity of the microphone 34, the direction of the speaker is obtained and stored in the speaker direction holding unit 46 (S3).

A relative direction calculation processing unit 48 reads out the camera direction information stored in the camera direction holding unit 42 and the speaker direction information stored in the speaker direction holding unit 46,
The relative orientation (angle) between the camera 10 and the speaker is calculated,
The calculation result, that is, the relative azimuth information is stored in the relative azimuth holding unit 50 (S4).

A voice recognition processing unit 52 performs a recognition process on the input voice, and performs "more right", "turn here" and the like.
Recognize the utterance contents indicating the direction control of the camera relatively,
It is stored in the recognition result holding unit 54 (S5).

A relative expression analysis processing unit 56 reads out the relative orientation information stored in the relative orientation holding unit 50 and the recognition result stored in the recognition result holding unit 54, and
Is calculated (S6). That is, the direction such as “right” or “this way” when viewed from the speaker's viewpoint is calculated, and the azimuth of the camera 10 is controlled to the azimuth corresponding to the direction. The calculation result is stored in the azimuth control parameter holding unit 58.

Next, the azimuth control parameter output processing unit 60
Transmits the azimuth control parameter information stored in the azimuth control parameter holding unit 58 to the camera 10 via the I / O 28.
(S7).

For example, the operation when the voice instruction is "turn here" will be described with reference to FIGS. 4 and 5 show plan views of the positional relationship between the camera 10 and the speaker. FIG. 4 shows a state before control, and FIG. 5 shows a state after control.
In FIG. 4, the photographing center line of the camera 10 is rotated counterclockwise by an angle -α (clockwise angle α) from the reference azimuth, and the speaker changes the angle of β degrees counterclockwise from the reference azimuth. Shall be located.

In response to the speaker's "turn here" command, the camera 10 is controlled by the above-described processing to rotate the center line of the photographing center line clockwise from the reference azimuth by an angle β. You. The state after rotation is shown in FIG.

An operation in response to a voice instruction of "more right" will be described. 6 and 7 are plan views showing the positional relationship between the camera 10 and the speaker. As shown in FIG. 6, when the angle difference α between the speaker and the photographing center line of the camera 10 is within a specific angle β (here, it is assumed to be half the viewing angle of the camera 10), that is, the talk. When the user is within the range of the angle of view j of the camera 10, the camera control device 20 rotates the camera 10 counterclockwise by a predetermined angle in response to the command “right”. For the command "left", the camera controller 2
0 rotates the camera 10 clockwise. Conversely, FIG.
When α> β, that is, when the speaker
If the angle is out of the angle of view range of 0, the camera control device 20 rotates the camera 10 clockwise by a predetermined angle in response to the command “right”, and rotates the camera 10 counterclockwise in response to the command “left”.

In the above embodiment, the “right” or “left” when viewed from the viewpoint of the speaker is considered in the control direction of the camera 10 in consideration of the relative positional relationship between the camera 10 and the speaker. Although the azimuth of the camera 10 is controlled, the speaker 10 does not always control the camera 10 with “right” or “left” when viewed from his / her own viewpoint. For example, it is conceivable that the speaker gives a command that “right” or “left” when viewed from the viewpoint of the camera 10 while being careful that the language understanding ability of the camera 10 is low. In this case, in the previous embodiment, “right”
Is rotated "left" in response to the command. In this,
Control different from the speaker's intention is performed.

In such a case, it is desirable to recognize the voice of the speaker saying "different, opposite" or "opposite", and thereafter switch the viewpoint to the camera side to perform camera control.
That is, in response to the command of “right”, the shooting direction is rotated to “right” as it is.

FIG. 8 is a schematic functional block diagram of the modified embodiment. The same components as those in FIG. 2 are denoted by the same reference numerals. A new viewpoint extraction processing unit 62 and a new viewpoint holding unit 6
4 have been introduced. The relative expression analysis processing unit 56a
Refers to the viewpoint information stored in the viewpoint holding unit 64,
It has been modified to calculate the azimuth to be taken by the camera 10.

FIG. 9 shows an operation flowchart of the modified embodiment shown in FIG. S11 to S15 correspond to S1 in FIG.
Same as S5. After the voice recognition (S15), the viewpoint extraction processing unit 62 extracts viewpoint information (camera or speaker) related to the relative expression from the recognition result stored in the recognition result holding unit 54, and stores the viewpoint information in the viewpoint holding unit 64. Yes (S1
6). The relative expression analysis processing unit 56a includes a relative orientation holding unit 5
0, relative orientation information, recognition result holding unit 54
Then, the azimuth control parameters of the camera 10 are calculated from the recognition result stored in the camera and the viewpoint information stored in the viewpoint holding unit 64 (S17). Thereafter, as in the case of FIG.
Is changed.

The viewpoint may be changed in real time according to the content of the speaker's remark, but the speaker or the user may set the viewpoint to the speaker or the camera in advance.

In the embodiment shown in FIG. 8, the viewpoint is changed in accordance with the content of a speaker's remark (eg, "different, opposite", "opposite", etc.) for a certain action. , But is not limited to this. For example, the camera 10 may be controlled in accordance with an expression that simultaneously specifies a viewpoint and a direction, such as “right when viewed from the camera side” or “left when viewed from here”.

In the embodiment shown in FIG. 8, the viewpoint is maintained, but the present invention is not limited to this.
For example, the operation for each instruction related to the relative direction may be separately held. In other words, the viewpoint may be sequentially updated using a speaker operation command for camera control and an instruction operation table in which the rotation operation of the camera at that time is set.

In each of the above embodiments, the data holding means is arranged on a memory (RAM), but may be arranged on any storage medium.

In each of the above embodiments, each unit is configured on the same computer, but the present invention is not limited to this. For example, each unit may be configured by being divided into computers and processing devices distributed on a network.

In the above embodiments, the program is stored in the control memory (ROM), but the present invention is not limited to this. For example, it can be stored in any storage medium.
All or a part of the same operation may be realized by hardware.

The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of one device. A storage medium storing software program codes for realizing the functions of the above-described embodiments is supplied to a system or an apparatus, and a computer (or CPU or MPU) of the system or the apparatus stores the program stored in the storage medium. It goes without saying that this can also be achieved by reading and executing the code.

In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

As a recording medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, C
DR, magnetic tape, nonvolatile memory card and RO
M or the like can be used.

When the computer executes the readout program code, not only the functions of the above-described embodiment are realized, but also the OS (Operating System) running on the computer based on the instruction of the program code. It is needless to say that the system may perform some or all of the actual processing, and the processing may realize the functions of the above-described embodiments.

Further, after the program code read from the recording medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, based on the instruction of the program code, It goes without saying that the CPU included in the function expansion board or the function expansion unit performs a part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

[0041]

As can be easily understood from the above description, according to the present invention, it is possible to control the camera from the viewpoint of the speaker in consideration of the relative orientation. as a result,
Camera control can be realized in response to intuitive instructions from the speaker. Furthermore, by incorporating viewpoint information when determining the relative orientation, more flexible camera control can be realized.

[Brief description of the drawings]

FIG. 1 is a schematic configuration block diagram of hardware according to an embodiment of the present invention.

FIG. 2 is a schematic functional block diagram of the present embodiment.

FIG. 3 is an operation flowchart of the embodiment shown in FIG. 2;

FIG. 4 is an explanatory diagram of camera control (before control) of the embodiment shown in FIG. 2;

FIG. 5 is an explanatory diagram of camera control (after control) of the embodiment shown in FIG. 2;

FIG. 6 is an explanatory diagram of another camera control of the embodiment shown in FIG. 2;

FIG. 7 is an explanatory diagram of still another camera control of the embodiment shown in FIG. 2;

FIG. 8 is a schematic functional block diagram of a modified embodiment of the present invention.

FIG. 9 is an operation flowchart of the modified embodiment shown in FIG. 9;

[Explanation of symbols]

 10: Camera 12: Imaging device 14: Optical parameter control circuit 16: Direction control circuit 20: Camera control circuit 22: CPU 24: RAM 26: ROM 28: I / O 30: Secondary storage device 32: Bus 34: Microphone 36 : A / D converter 40: Camera direction specifying processing unit 42: Camera direction specifying unit 44: Speaker direction specifying processing unit 46: Speaker direction storing unit 48: Relative direction calculating unit 50: Relative direction storing unit 52: Voice recognition Processing unit 54: Recognition result storage unit 56: Relative expression analysis processing unit 56a: Relative expression analysis processing unit 58: Azimuth control parameter storage unit 60: Azimuth control parameter output processing unit 62: Viewpoint extraction processing unit 64: Viewpoint storage unit

Claims (15)

[Claims] 1. A camera control device for controlling a camera capable of externally controlling a shooting direction, comprising: camera direction specifying means for obtaining current direction information of the camera; and speaker direction specifying means for obtaining a direction of a speaker. A relative azimuth calculating means for calculating a relative azimuth between the camera and the speaker; a voice input means; and a voice recognition process for recognizing utterance contents related to azimuth control of the camera from an input voice of the voice input means. Means for calculating an azimuth control parameter for controlling the azimuth of the camera from the viewpoint of the speaker from the relative azimuth calculated by the relative azimuth calculating means and the recognition result of the voice recognition processing means. Relative expression analysis processing means, and azimuth control parameter output processing for outputting information indicating the azimuth control parameter calculated by the relative expression analysis processing means to the camera The camera control apparatus characterized by a means. 2. The apparatus according to claim 1, further comprising: viewpoint extracting means for extracting a command for designating a viewpoint from a recognition result of said speech recognition processing means; and viewpoint retaining means for retaining viewpoint information. Camera control device. 3. The camera control device according to claim 1, further comprising viewpoint setting means for setting a viewpoint. 4. An apparatus according to claim 1, further comprising: a table holding unit for holding an instruction operation table in which an instruction and an operation related to the relative direction are set; and a table updating unit for updating the instruction operation table based on the recognition result and the relative azimuth. The camera control device according to claim 1, wherein: 5. A camera azimuth holding means for holding camera azimuth information specified by the camera azimuth specifying means, and a speaker azimuth holding means for holding speaker azimuth information specified by the speaker azimuth specifying means. Relative orientation holding means for holding the relative orientation calculated by the relative orientation calculation means; recognition result holding means for holding the recognition result of the voice recognition processing means; and the orientation calculated by the relative expression analysis processing means. The camera control device according to claim 1, further comprising: an azimuth control parameter holding unit that holds a control parameter. 6. A camera control method for controlling a camera capable of externally controlling a photographing direction, comprising: a camera direction specifying step for extracting current direction information of the camera; and a speaker direction specifying step for obtaining a direction of a speaker. A relative azimuth calculating step of calculating a relative azimuth of the camera and the speaker; a voice recognition processing step of recognizing utterance content related to azimuth control of the camera from an input voice of voice input means; A relative expression analysis processing step of calculating an azimuth control parameter for performing azimuth control of the camera when viewed from the viewpoint of the speaker, based on the relative azimuth calculated by the calculation step and the recognition result of the voice recognition processing step; An azimuth control parameter for outputting information indicating the azimuth control parameter calculated in the relative expression analysis processing step to the camera; And a data output processing step. 7. The apparatus according to claim 6, further comprising: a viewpoint extracting step of extracting a command for designating a viewpoint from a recognition result of the voice recognition processing step; and a viewpoint holding step of holding viewpoint information. Camera control method. 8. The camera control method according to claim 6, further comprising a viewpoint setting step of setting a viewpoint. 9. A table holding step for holding an instruction operation table in which an instruction and an operation related to a relative direction are set, and a table updating step for updating the instruction operation table based on a recognition result and a relative azimuth. The camera control method according to any one of claims 6 to 8, wherein: 10. A camera azimuth holding step for holding camera azimuth information specified by the camera azimuth specifying step, and a speaker azimuth holding step for holding speaker azimuth information specified by the speaker azimuth specifying step. A relative orientation holding step of holding the relative orientation calculated by the relative orientation calculation step; a recognition result holding step of holding the recognition result of the voice recognition processing step; and an orientation calculated by the relative expression analysis processing step The camera control method according to claim 6, further comprising: an azimuth control parameter holding step of holding a control parameter. 11. A camera control method for controlling a camera whose shooting direction can be controlled externally, comprising: a camera direction specifying step for extracting current direction information of the camera; and a speaker direction specifying step for obtaining a direction of a speaker. A relative azimuth calculating step of calculating a relative azimuth of the camera and the speaker; a voice recognition processing step of recognizing utterance content related to azimuth control of the camera from an input voice of voice input means; A relative expression analysis processing step of calculating an azimuth control parameter for performing azimuth control of the camera when viewed from the viewpoint of the speaker, based on the relative azimuth calculated by the calculation step and the recognition result of the voice recognition processing step; An azimuth control parameter for outputting information indicating the azimuth control parameter calculated in the relative expression analysis processing step to the camera; A storage medium storing program software for implementing a camera control method having a meter output processing step so as to be externally readable. 12. The apparatus according to claim 11, further comprising: a viewpoint extracting step of extracting a command for designating a viewpoint from a recognition result of the speech recognition processing step; and a viewpoint holding step of holding viewpoint information. Storage media. 13. The storage medium according to claim 11, further comprising a viewpoint setting step of setting a viewpoint. 14. A method according to claim 1, further comprising: a table holding step for holding an instruction operation table in which an instruction and an operation related to the relative direction are set; and a table updating step for updating the instruction operation table based on the recognition result and the relative azimuth. The storage medium according to any one of claims 11 to 13, wherein: 15. A camera azimuth holding step for holding camera azimuth information specified by the camera azimuth specifying step, and a speaker azimuth holding step for holding speaker azimuth information specified by the speaker azimuth specifying step. A relative orientation holding step of holding the relative orientation calculated by the relative orientation calculation step; a recognition result holding step of holding the recognition result of the voice recognition processing step; and an orientation calculated by the relative expression analysis processing step The storage medium according to claim 11, comprising: an azimuth control parameter holding step of holding a control parameter.