JP6631166B2 - Imaging device, program, and imaging method - Google Patents

Description

  The present invention relates to a photographing device, a program, and a photographing method.

  2. Description of the Related Art There is a system in which a participant who speaks at a conference is tracked and photographed by a camera, and a video of the conference is distributed to the outside.

  For example, a microphone array is arranged on a camera installed in a conference room, and the direction of a participant who speaks is detected by the microphone array. The direction of the camera is controlled according to the detected direction of the participant, and the speaking participant is photographed. When another participant utters, the direction of the camera is changed to the direction of the uttering participant. The captured video (image) of the conference is distributed to the terminal of the viewer via the network.

  For example, a technology has been disclosed in which the direction of a speaker is detected using a microphone array configured by arranging a plurality of microphones, and the direction of a camera is controlled in the detected direction (for example, Patent Document 1).

  However, the conventional technique has a problem that clear audio cannot be recorded during shooting.

  For example, if the speaker is seated far away from the camera, the voice may be too low to record clear voice. Also, if the speaker is seated too close to the camera, the sound may be too loud and the sound may be broken.

  Thus, the disclosed technology aims to record clearer audio.

In the embodiment, in a photographing apparatus for photographing a person, a detection unit that detects a direction of the person who spoke and a volume at the time of speech, a control unit that controls a photographing direction in the detected direction, and a detection unit that is detected by the detection unit A measurement unit for measuring the size of the person in the image in the direction, wherein the volume is smaller than the minimum value of the first range according to the predetermined range , and the size of the person on the image is the minimum value of the second range. And a notifying unit for notifying the person when the distance is smaller than the threshold value.

  More clear audio can be recorded.

FIG. 1 is a diagram illustrating an example of the overall configuration of a video distribution system. FIG. 3 is a diagram illustrating a hardware configuration of a distribution terminal. FIG. 3 is a diagram illustrating an example of a functional configuration of a distribution terminal according to the first embodiment. FIG. 4 is a diagram illustrating a relationship between a camera position coordinate and an angle. It is a figure showing the relation between the position of a participant, and the message notified. FIG. 3 is a diagram illustrating a control flow according to the first embodiment. FIG. 9 is a diagram illustrating an example of a functional configuration of a distribution terminal according to a second embodiment. FIG. 4 is a diagram illustrating an example of a data structure of a user table. FIG. 10 is a diagram illustrating a control flow according to the second embodiment. It is a figure showing the flow of authentication processing. FIG. 14 is a diagram illustrating a first hardware configuration of a distribution terminal according to a third embodiment. FIG. 14 is a diagram illustrating a second hardware configuration of the distribution terminal according to the third embodiment. It is a figure showing an example of the appearance of a stereo camera. It is a figure showing an example of the appearance of a distance sensor. FIG. 13 is a diagram illustrating a control flow according to the third embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the specification and the drawings, components having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted.

(Embodiment 1)
FIG. 1 is a diagram illustrating an example of the overall configuration of the video distribution system 1. The video distribution system 1 includes a server 2, a distribution terminal 3, user terminals 4a to 4n, and a display device 6. The server 2 has a communication unit 21. The distribution terminal 3 includes a communication unit 31, a processing unit 32, a data acquisition unit 33, a data output unit 34, and a storage unit 35. The server 2, the distribution terminal 3, and the user terminals 4a to 4n are connected via a communication network 5. The display device 6 is connected to the data output unit 34. The display device 6 may be provided in the distribution terminal 3.

  The data acquisition unit 33 acquires, for example, image data and audio data in a conference room. The communication unit 31 transmits the acquired image data and audio data to the server 2 via the communication network 5. The server 2 distributes image data and audio data to the user terminals 4a to 4n via the communication network 5.

  FIG. 2 is a diagram illustrating a hardware configuration of the distribution terminal 3. As shown in FIG. 2, the distribution terminal 3 stores a program used for driving the CPU 101 such as a CPU (Central Processing Unit) 101 and an IPL (Initial Program Loader) for controlling the operation of the entire distribution terminal 3. A ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103 used as a work area of the CPU 101, a flash memory 104 for storing various data such as a terminal program, image data, and audio data, and under the control of the CPU 101. An SSD (Solid State Drive) 105 that controls reading or writing of various data to or from the flash memory 104, a media drive 107 that controls reading or writing (storage) of data to or from a recording medium 106 such as a flash memory, and a destination of the distribution terminal 3. Operation buttons operated when selecting And a network I / F (Interface) 111 for performing data transmission using the communication network 5.

  The distribution terminal 3 includes a built-in camera 112 that captures an image of a subject under the control of the CPU 101 to obtain image data, an image sensor I / F 113 that controls driving of the camera 112, a built-in microphone 114 that inputs audio, A built-in speaker 115 that outputs sound, a sound input / output I / F 116 that processes input / output of a sound signal between the microphone 114 and the speaker 115 under the control of the CPU 101, and image data on an external display 120 under the control of the CPU 101. , A device interface 117 for connecting various external devices, and an address bus and data for electrically connecting the above-described components as shown in FIG. A bus line 110 such as a bus is provided.

  The display 120 is a display unit configured by a liquid crystal or an organic EL that displays an image of a subject, operation icons, and the like. The display 120 is connected to the display I / F 117 by a cable 120c. The cable 120c may be a cable for an analog RGB (VGA) signal, a cable for a component video, an HDMI (registered trademark) (High-Definition Multimedia Interface), or a DVI (Digital Video). (Interactive) signal cable.

  The camera 112 includes a lens and a solid-state imaging device that converts light into electric charge to convert an image (video) of a subject into electrons. As the solid-state imaging device, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled) is used. Device) An image sensor or the like is used.

  External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I / F 118 via a USB (Universal Serial Bus) cable or the like. When an external camera is connected, the external camera operates prior to the built-in camera 112 under the control of the CPU 101. Similarly, when an external microphone is connected or an external speaker is connected, the external microphone or the internal speaker 115 is given priority over the built-in microphone 114 or the built-in speaker 115 under the control of the CPU 101. Drive external speakers.

  Note that the recording medium 106 is configured to be detachable from the distribution terminal 3. In addition, as long as it is a non-volatile memory that reads or writes data under the control of the CPU 101, an EEPROM (Electrically Erasable and Programmable ROM) may be used instead of the flash memory 104.

  Further, the terminal program may be recorded in a computer-readable recording medium such as the recording medium 106 as an installable or executable file, and distributed. Further, the terminal program may be stored in the ROM 102 instead of the flash memory 104.

  FIG. 3 is a diagram illustrating an example of a functional configuration of the distribution terminal 3 according to the first embodiment. The data acquisition unit 33 has a voice acquisition unit 33a and an imaging unit 33b. The voice obtaining unit 33a obtains voice data in the conference room.

  The imaging unit 33b acquires the image data of the speaker. The imaging unit 33b stores the acquired image data of the speaker in the storage unit 35. Further, the communication unit 31 transmits image data and audio data in the conference room to the server 2.

  The processing unit 32 includes a detection unit 32a, a control unit 32b, a measurement unit 32c, and a notification unit 32d. The detection unit 32a detects the direction of the participant speaking in the conference room using the voice acquisition unit 33a. Specifically, for example, the detection unit 32a calculates an average value of the voice from the voice data acquired by the voice acquisition unit 33a, and determines that the speaker exists when the average value of the voice is equal to or more than a predetermined value. I do.

Subsequently, the detection unit 32a detects the direction of the speaker based on the audio data obtained from each microphone constituting the microphone array. For example, when the direction in which the reference point (coordinates (0, Y ₀ )) in the conference room is imaged from the origin having coordinates (0, ₀ ) to 0 degrees, the position coordinates ( X ₀ , Y ₀ ), and the photographing angle θ at the time of photographing the participant based on the audio data.

Hereinafter, data relating to the shooting direction corresponding to the position of the speaker is referred to as direction data. The direction data is represented, for example, as (X ₀ , Y ₀ , θ) by the position coordinates (X ₀ , Y ₀ ) of the imaging unit 33 b and the shooting angle θ when the participant was shot.

FIG. 4 is a diagram illustrating a relationship between the position coordinates and the angle of the imaging unit 33b. The direction data B (0,0,0) indicates the position coordinates (0,0) and the imaging angle of 0 degrees of the imaging unit 33b when the reference point β (0, Y ₀ ) is imaged from the origin (0,0). Show.

The direction data A (X ₁ , Y ₁ , θ ₁ ) indicates the position coordinates (X ₁ , Y ₁ ) of the imaging unit 33 b and the shooting angle θ ₁ of the imaging unit 33 b corresponding to the direction in which the speaker is detected. . The photographing angle theta ₁ is origin (0, 0) from the reference point β in the case of (0, _{Y 0)} is 0 degrees imaging angle of the photographed image pickup unit 33b and position coordinates _{(X 1, Y} 1) This shows the photographing angle when the speaker is photographed from. In the case of fixing the imaging unit 33b to the origin (0,0) may represent the direction data imaging angle theta ₁ only.

Further, the detection unit 32a may detect the speaker using the plurality of imaging units 33b. For example, when a speaker is detected using a plurality of imaging units 33b, the identification number n of the imaging unit 33b that detected the speaker, and the data (X ₀ , Y ₀ , θ) of the position and imaging angle of the imaging unit 33b. based on the bets, the detection unit 32a, the direction data _{(n, X 0, Y 0} , θ) , including an identification number n of cameras may be acquired.

  Referring back to FIG. The control unit 32b controls the imaging direction so that the imaging unit 33b faces the speaker. For example, the control unit 32b turns the imaging unit 33b based on the direction data so that the speaker is at the center of the image. Subsequently, the control unit 32b acquires the image data of the speaker captured by the imaging unit 33b. The control unit 32b extracts the image data of the face part from the acquired image data and stores the image data in the storage unit 35.

  The extraction of the image data of the face portion is executed, for example, based on the face shape and the skin color area. For example, the control unit 32b converts the RGB values of the pixels of the image data into the YCC color system, and extracts skin color pixels whose Cr value and Cb value are within predetermined ranges. The control unit 32b extracts a face image by specifying a region where skin color pixels are gathered from the image data.

  The measuring unit 32c measures the size of the participant's face included in the image data stored in the storage unit 35. The measurement unit 32c measures the face size (inch) by specifying the left end, right end, upper end, and lower end of the area where the skin color pixels are gathered.

  When the participant is seated near the imaging unit 33b, the size of the face part is relatively large, and when the participant is seated at a position away from the imaging unit 33b, the size of the face part is relatively small. Become.

  The notifying unit 32d determines whether the voice of the participant obtained by the voice obtaining unit 33a is larger than the maximum value of the predetermined range and whether the voice of the participant is smaller than the minimum value of the predetermined range. . In addition, the notifying unit 32d makes a determination to compare with the predetermined range, for example, by using an average value of the volume of the voice data of the speaker. Further, whether the size of the face part of the participant measured by the measuring unit 32c is larger than the maximum value of the predetermined range, and whether the size of the face part of the participant is smaller than the minimum value of the predetermined range, judge. The voice of the participant is expressed in units such as decibels “dB”, and the size of the participant's face is expressed in units such as inches “in”.

  The notifying unit 32d notifies the participant of a message prompting the user to leave the microphone when the voice of the participant is larger than the maximum value of the predetermined range and the size of the face of the participant is larger than the maximum value of the predetermined range. I do. When the voice of the participant is smaller than the minimum value of the predetermined range and the size of the face of the participant is smaller than the minimum value of the predetermined range, the notification unit 32d transmits a message prompting the participant to approach the microphone. Notify

  As described above, when the volume of the voice data of the speaker is larger than the maximum value of the predetermined range or smaller than the minimum value of the predetermined range, the notification unit 32d further determines the voice data based on the size of the face of the speaker. Then, it is determined whether or not to notify the speaker. For example, the notifying unit 32d does not notify the speaker of a message when the speaker is far from the microphone but can obtain a clear voice with a large voice volume. Thereby, it is possible to avoid notifying the speaker of the message even when the recording of the sound is not affected.

  In the present embodiment, when either the voice of the speaker or the size of the face part is within the predetermined range, the notification unit 32d does not target the speaker, but the present invention is not limited thereto. For example, the notification unit 32d may set the speaker as a notification target when either the voice of the speaker or the size of the face part is out of the predetermined range.

  The notification unit 32d notifies the participant of the message by displaying a telop on the display device 6 via the data output unit 34, for example. The notification unit 32d may notify the participant of the message by outputting voice data to the outside via the data output unit 34. The method of notifying the participant of the message is not limited to the above.

  FIG. 5 is a diagram showing the relationship between the position of the participant and the message to be notified. In FIG. 5A, the participant β is seated opposite to the data acquisition unit 33 (voice acquisition unit 33a) via the table, and is located farther from the data acquisition unit 33 than (b) and (c). I'm sitting. In such a case, the notification unit 32d may display "Please approach the microphone because the voice of the participant is smaller than the minimum value of the predetermined range and the size of the face of the participant is likely to be smaller than the minimum value of the predetermined range." Is displayed on the display device 6.

  In addition, in FIG. 5B, the participant β is seated near the data acquisition unit 33 and is seated at a position closer to the data acquisition unit 33 than in (a) and (c). In such a case, the notification unit 32d may indicate that the participant's voice is larger than the maximum value in the predetermined range and the size of the face of the participant is smaller than the maximum value in the predetermined range. Is displayed on the display device 6.

  In FIG. 5C, the participant β is seated at a position closer to the data acquisition unit 33 than to the data acquisition unit 33 and farther from the data acquisition unit 33 than to FIG. . In such a case, the notification unit 32d does not display the telop and the like on the display device 6 because the voice of the participant is within the predetermined range.

  Note that the content of the message notified to the participant is not limited to the above.

  FIG. 6 is a diagram illustrating a control flow according to the first embodiment. When distribution of a video conference is started (step S10), the detection unit 32a determines whether or not a speaker exists (step S11). The detection unit 32a determines that there is a speaker, for example, when the average value of the volume of the audio data acquired by the audio acquisition unit 33a is equal to or greater than a predetermined value. When there is no speaker (step S11: No), the detection unit 32a determines again whether or not there is a speaker after a predetermined time has elapsed (step S11). On the other hand, when there is a speaker (Step S11 Yes), the detection unit 32a detects the direction of the speaker by the voice acquisition unit 33a (Step S12). Subsequently, the control unit 32b controls the shooting direction of the imaging unit 33b in the direction of the speaker (Step S13). Subsequently, the measurement unit 32c measures the size of the speaker's face based on the image data of the speaker captured by the imaging unit 33b (step S14).

  The notifying unit 32d determines whether or not the volume of the voice data of the speaker acquired from the voice acquiring unit 33a is smaller than the minimum value in the predetermined range (Step S15). When the volume of the voice data of the speaker is lower than the minimum value of the predetermined range (Yes in Step 15), the notification unit 32d determines that the face size of the speaker measured by the measurement unit is the minimum value of the predetermined range. It is determined whether it is smaller than (Step S16). When the size of the face of the speaker is equal to or larger than the minimum value of the predetermined range (No in step S16), the detection unit 32a determines again whether or not the speaker is present after a predetermined time elapses by the timer (step S21) (step S11). ). On the other hand, when the size of the speaker's face is smaller than the minimum value of the predetermined range (Step S16 Yes), the notification unit 32d notifies the speaker to approach the microphone (Step S17). The message is notified by, for example, displaying a telop on the display device 6. Subsequently, after a predetermined time elapses with the timer (step S21), the detection unit 32a again determines whether or not there is a speaker (step S11).

  In the determination of the volume of the voice data of the speaker (step S15), when the volume of the voice data of the speaker is equal to or more than the minimum value of the predetermined range (No in step 15), the notification unit 32d determines that the volume of the voice data of the speaker is predetermined. The process proceeds to the determination of whether the value is equal to or less than the maximum value of the range (step S18). Subsequently, when the volume of the voice data of the speaker is equal to or lower than the maximum value of the predetermined range (No in step S18), the detection unit 32a determines whether or not the speaker is present again after a predetermined time has elapsed by the timer (step S21). Is performed (step S11). On the other hand, when the volume of the voice data of the speaker is larger than the maximum value of the predetermined range (Yes at Step S18), the notification unit 32d determines whether the face size of the speaker is larger than the maximum value of the predetermined range (Step S18). S19). When the size of the face of the speaker is equal to or smaller than the maximum value (No at Step S19), the detection unit 32a determines again whether or not there is a speaker (Step S11) after a predetermined time has elapsed with a timer (Step S21). Execute. On the other hand, when the size of the face of the speaker is larger than the maximum value (Yes in step S19), the notification unit 32d notifies the speaker to move away from the microphone (step S20). Subsequently, after a predetermined time elapses with the timer (step S21), the detection unit 32a again determines whether or not there is a speaker (step S11).

(Embodiment 2)
FIG. 7 is a diagram illustrating an example of a functional configuration of the distribution terminal 7 according to the second embodiment. The distribution terminal 7 includes a communication unit 31, a processing unit 32, a data acquisition unit 33, a data output unit 34, and a storage unit 35. The storage unit 35 has a user table 35a.

  The data acquisition unit 33 has a voice acquisition unit 33a and an imaging unit 33b. The processing unit 32 includes a detection unit 32a, a control unit 32b, a measurement unit 32c, a notification unit 32d, and a storage unit 32e. The storage unit 32e is connected to the user table 35a. The notification unit 32d is connected to the data output unit 34.

  FIG. 8 is a diagram illustrating an example of the data structure of the user table 35a. The user table 35a is a table for identifying a notified participant, and associates a user ID, a face image, and the presence or absence of a notification. “User ID” is a number that uniquely identifies a user. The distribution terminal 7 identifies a participant by the “user ID”. “Face image” indicates image data of a user's face portion. For example, “face image” is image data obtained by cutting out a face portion from image data of the entire user. For example, it is stored in an image format such as JPEG or GIF. The “presence / absence of notification” indicates whether or not a notification to urge the participant (user) to move the position is made. If "Yes" is stored, it indicates that the participant has already notified the participant such as "Please approach the microphone." When "absence" is stored, it indicates that the participant has not been notified yet. For example, the user table 35a indicates that the face image of the user ID “0101” is “image a” and has not yet been notified. Further, the face image of the user ID “0103” is “image c”, which indicates that the notification has already been performed.

Referring back to FIG. The detection unit 32a detects the direction of the participant speaking in the conference room using the voice acquisition unit 33a. For example, when the direction in which a reference point (coordinates (X ₀ , Y ₀ )) in the conference room is imaged from the origin having coordinates ( ₀ , ₀ ) to 0 degrees, the position of the imaging unit 33b is determined. The coordinates (X ₁ , Y ₁ ) and the photographing angle θ at the time of photographing the participant are acquired.

  The control unit 32b controls the shooting direction of the imaging unit 33b in the direction of the speaker. For example, based on the direction data detected by the detection unit 32a, the control unit 32b turns the imaging unit 33b so that the speaker becomes the center of the captured image.

  The measuring unit 32c measures the size of the participant's face included in the image data stored in the storage unit 35.

  The notifying unit 32d determines that the volume of the voice data of the speaker is larger than the maximum value of the predetermined range and the face size of the speaker is larger than the maximum value of the predetermined range, or the volume of the voice data of the speaker is predetermined. When the face size of the speaker is smaller than the minimum value of the predetermined range, the image data of the speaker is stored in the built-in memory. The image data of the speaker may be an image of the entire speaker or an image obtained by cutting out the face of the speaker. The notifying unit 32d ends the process when the volume of the voice data of the speaker is within the predetermined range and when the face size of the speaker is within the predetermined range.

  Subsequently, the notifying unit 32d specifies the user ID of the speaker by comparing with the image of the user stored in the user table 35a and performing face authentication. For example, the notification unit 32d performs face authentication using a face authentication algorithm such as a unique face. The storage unit 32e refers to “presence or absence of notification” corresponding to the specified user ID in the user table 35a, and determines whether or not the speaker has been notified in the past. Subsequently, when the notification is not performed, the notifying unit 32d notifies the speaker of the position change.

  For example, when the volume of the voice data of the speaker is larger than the maximum value in the predetermined range and the face size of the speaker is larger than the maximum value in the predetermined range, the notifying unit 32d utters a message prompting the user to move away from the microphone. Notify others. On the other hand, when the volume of the voice data of the speaker is smaller than the minimum value of the predetermined range and the face size of the speaker is smaller than the minimum value of the predetermined range, the notification unit 32d utters a message prompting the user to approach the microphone. Notify others.

  When the message is notified to the speaker by the notifying unit 32d, the storage unit 32e stores “present” indicating that the notification has been made in “presence or absence of notification” corresponding to the user ID in the user table 35a.

  FIG. 9 is a diagram illustrating a control flow according to the second embodiment. When distribution of the video conference is started (step S30), the detection unit 32a determines whether or not a speaker exists (step S31). When there is no speaker (Step S31 No), the detection unit 32a determines again whether or not there is a speaker after a predetermined time has elapsed (Step S31). On the other hand, when there is a speaker (Step S31 Yes), the detection unit 32a detects the direction of the speaker by the voice acquisition unit 33a (Step S32). Subsequently, the control unit 32b controls the shooting direction of the imaging unit 33b in the direction of the speaker (Step S33). Subsequently, the measuring unit 32c measures the size of the speaker's face (step S34).

  Subsequently, the notification unit 32d determines whether or not the volume of the voice data of the speaker acquired from the voice acquisition unit 33a is smaller than a minimum value in a predetermined range (Step S35). When the volume of the voice data of the speaker is smaller than the minimum value of the predetermined range (step 35 Yes), the notifying unit 32d determines whether the size of the face of the speaker measured by the measurement unit is smaller than the minimum value of the predetermined range. Is determined (step S36). When the size of the face of the speaker is equal to or larger than the minimum value of the predetermined range (No at Step S36), the detection unit 32a determines again whether or not there is a speaker (Step S31) after a predetermined time has elapsed by the timer (Step S41). ). On the other hand, when the size of the speaker's face is smaller than the minimum value in the predetermined range, the notifying unit 32d executes an authentication process (step S37). The authentication process will be described with reference to FIG. Subsequently, after a predetermined time elapses with a timer (step S41), the detection unit 32a again determines whether or not there is a speaker (step S31).

  In the determination of the volume of the speaker's voice data (step S35), when the volume of the speaker's voice data is equal to or greater than the minimum value in the predetermined range (step 35No), the notifying unit 32d determines that the volume of the speaker's voice data is predetermined. The process proceeds to the determination of whether the value is equal to or less than the maximum value of the range (step S38). Subsequently, when the volume of the voice data of the speaker is equal to or less than the maximum value in the predetermined range (No in step S38), the detection unit 32a determines whether or not the speaker is present again after a predetermined time has elapsed by the timer (step S41). Is performed (step S31). On the other hand, when the volume of the voice data of the speaker is larger than the maximum value of the predetermined range (Yes at Step S38), the notification unit 32d determines whether the face size of the speaker is larger than the maximum value of the predetermined range (Step S38). S39). When the size of the speaker's face is equal to or smaller than the maximum value (No at Step S39), the detection unit 32a determines again whether or not a speaker exists (Step S31) after a predetermined time has elapsed with a timer (Step S41). Execute. On the other hand, when the size of the speaker's face is larger than the maximum value (Yes at Step S39), the notifying unit 32d executes an authentication process (Step S40). Subsequently, after a predetermined time has elapsed with the timer (step S21), the detection unit 32a determines again whether or not there is a speaker (step S31).

  FIG. 10 is a diagram illustrating a flow of the authentication process. The authentication processing in FIG. 10 corresponds to (Step S37) and (Step S40) in FIG. The notifying unit 32d stores the image data of the speaker in the built-in memory (Step S50). Subsequently, the notifying unit 32d specifies the user ID of the speaker by comparing the image data of the user's face stored in the user table 35a and performing face authentication (step S51). Subsequently, the storage unit 32e refers to the user table 35a and determines whether or not the position change has been notified to the speaker in the past (step S52). The notifying unit 32d terminates the process when notifying the speaker in the past (Step S52 No). If the speaker has not been notified yet (step S52 Yes), the notifying unit 32d notifies the speaker that the position has been moved (step S53).

  For example, when the volume of the voice data of the speaker is smaller than the minimum value of the predetermined range and the face size of the speaker is smaller than the predetermined range (when shifting from step S37), the notification unit 32d approaches the microphone. Is notified to the speaker. On the other hand, when the volume of the voice data of the speaker is larger than the maximum value of the predetermined range and the face size of the speaker is larger than the maximum value of the predetermined range (when the process proceeds from step S40), the notifying unit 32d sets the microphone. Notify the speaker of a message prompting you to leave.

  Subsequently, the storage unit 32e stores “Yes” in “Presence or absence of notification” in the user table 35a (Step S54).

  As described above, by determining in advance whether or not the user has been notified of the position movement in the past, it is possible to repeatedly notify when there is a situation where the user cannot move, such as when the conference room is full. Can be avoided.

(Embodiment 3)
The distance from the speaker may be measured by a method other than measuring the distance from the face size as in the first or second embodiment. For example, the measuring unit 32c may measure the distance to the speaker using a distance sensor. Examples of the distance sensor include a stereo camera, an ultrasonic sensor, an infrared sensor, and the like. Note that the stereo camera may acquire the image data of the speaker in parallel with the measurement of the distance to the speaker.

  FIG. 11A is a diagram illustrating a first hardware configuration of the distribution terminal 8 according to the third embodiment. In the first hardware configuration, the distance from the speaker is measured using the stereo camera 50. The distribution terminal 8 differs from the distribution terminal 3 of the first embodiment in that a stereo camera 50 is connected to the image sensor I / F 113, and the other hardware configuration is the same as that of the first embodiment. Note that a configuration may be adopted in which a camera 112 for photographing and a stereo camera 50 for distance measurement are connected to the image sensor I / F 113, respectively.

  FIG. 11B is a diagram illustrating a second hardware configuration of the distribution terminal 8 according to the third embodiment. In the second hardware configuration, the distance from the speaker is measured using the infrared sensor 51. The distribution terminal 8 differs from the distribution terminal 3 of the first embodiment in that an infrared sensor 51 or an ultrasonic sensor 52 is connected via a sensor I / F 122 connected to a bus 110, and other hardware configurations are not implemented. Same as in the first embodiment.

  In addition, the distribution terminal 8 has the same functional configuration as that of FIG.

  FIG. 12 is a diagram illustrating an example of the appearance of the stereo camera 50. The stereo camera 50 is a device that measures a distance from a speaker using a plurality of cameras installed in parallel. The shooting direction of each camera is independently controlled. In the example of FIG. 12, two cameras are installed close to each other, but the distance between the cameras may be increased. In the example of FIG. 12, two cameras are used, but three or more cameras may be used.

  A method of calculating the distance from the speaker using the stereo camera 50 will be described. Based on the distance A [m] between the cameras, the focal length B [m] of the cameras, and the difference C [m] between the positions of the speakers imaged by each camera, the measuring unit 32c calculates the following formula: To calculate the distance D [m] from the speaker.

(Equation 1)
D = A × B / C
The greater the distance A [m] between the cameras, the higher the distance measurement accuracy is. However, when the distance A between the cameras is increased, the distance measurement time increases. This is because if the distance between the cameras is large, the time from the start of the search for the speaker to the capture of the speaker by each camera becomes long. When the distance A [m] between the cameras is increased, the time required for the search can be reduced by setting the distance range to be the target measured by the stereo camera 50 narrow.

  For example, the target distance range measured by the stereo camera 50 is set according to, for example, the size of the room in which the image is taken. By setting in advance the distance range to be measured by the stereo camera 50, the time for measuring the distance to the speaker can be shortened.

  Further, the difference C between the positions of the speakers is, for example, when the position of the speaker imaged by one camera and the position of the speaker imaged by the other camera are shifted left and right by 5 cm, for example. The position difference C [m] is 0.05.

  Further, the stereo camera 50 may be used as the camera 112 in FIG. 2 other than the measuring unit 33c. For example, when used as the camera 112, the stereo camera 50 may generate image data by combining images captured by a plurality of cameras. Further, the stereo camera 50 may transmit an image captured by one camera to the server 2 as image data. In addition, the stereo camera 50 may be used exclusively for measuring the distance to the speaker, and the distribution terminal 3 may include a separate camera 112 for acquiring image data of the speaker.

  FIG. 13 is a diagram illustrating an example of an external appearance of a camera 112 having the infrared sensor 51. For example, as shown in FIG. 13, the infrared sensor 51 is arranged in parallel with the camera 112 so as to face the same direction as the shooting direction of the camera 112. After the imaging direction of the camera 112 is controlled to the direction of the speaker detected by the detection unit 32a, the infrared sensor 51 starts measuring the distance to the speaker. When the ultrasonic sensor 52 is used as a distance sensor, the ultrasonic sensor 52 is arranged in parallel with the camera 112 similarly to the infrared sensor 51 in FIG.

  Further, when the infrared sensor 51 is a distance sensor, the infrared sensor 51 irradiates the speaker with infrared rays, for example, and determines the distance to the speaker based on the principle of triangulation based on the position of the light receiving element that has detected reflected light. measure.

  Specifically, the infrared sensor 51 irradiates the speaker with infrared light, and receives reflected light from the speaker using a position sensing element PSD (Position Sensing Device). Since the position of the position detecting element that detects the reflected light changes according to the distance from the speaker, the measuring unit 32c converts the position of the position detecting element that has detected the reflected light into a distance, thereby obtaining a communication with the speaker. The distance can be calculated.

  Note that the element used as the infrared sensor 51 is not limited to the PSD, and another type of element such as OES (Opto Elektronischer Schaltkreis) may be used.

  When the ultrasonic sensor 52 is used, the measuring unit 32c controls the direction of the speaker detected by the detecting unit 32a, transmits an ultrasonic wave to the detected speaker, measures a reflected wave, Alternatively, the distance to the speaker is measured by irradiating the speaker with infrared rays and measuring the reflected light.

  For example, the measuring unit 32c measures the distance from transmitting the ultrasonic wave to the speaker using the ultrasonic sensor to receiving the reflected wave, thereby measuring the distance from the speaker. For example, assuming that the time from transmitting an ultrasonic wave to a speaker to receiving a reflected wave is t [s] and the sound speed is c [m / s], the measuring unit 32c utters the speech based on the following equation. The distance L to the person is calculated.

(Equation 2)
L = c × t / 2
FIG. 14 is a diagram illustrating a control flow according to the third embodiment. When distribution of the video conference is started (step S60), the detection unit 32a determines whether or not there is a speaker (step S61). When there is no speaker (step S61 No), the detection unit 32a determines again whether or not there is a speaker after a predetermined time has elapsed (step S61). On the other hand, when there is a speaker (Step S61 Yes), the detection unit 32a detects the direction of the speaker by the voice acquisition unit 33a (Step S62). Subsequently, the control unit 32b controls the shooting direction of the imaging unit 33b in the direction of the speaker (Step S63). Subsequently, the measuring unit 32c measures the distance from the speaker based on the stereo camera 50, the infrared sensor 51, or the ultrasonic sensor 52 (Step S64).

The notifying unit 32d determines whether or not the volume of the speaker's voice data acquired from the voice acquiring unit 33a is smaller than the minimum value in the predetermined range (Step S65). When the volume of the voice data of the speaker is smaller than the minimum value of the predetermined range (Step 65Yes), the notifying unit 32d determines whether the distance to the speaker measured by the measurement unit is larger than the maximum value of the predetermined range. Is determined (step S66). When the distance from the speaker is equal to or less than the maximum value of the predetermined range (No in step S66), the detection unit 32a determines again whether or not there is a speaker (step S61) after a predetermined time elapses with a timer (step S71). Execute On the other hand, when the distance to the speaker is larger than the maximum value in the predetermined range (Yes at Step S66), the notification unit 32d notifies the speaker to approach the microphone (Step S67). Subsequently, after a predetermined time elapses with the timer (step S71), the detection unit 32a again determines whether or not there is a speaker (step S61).

In determining whether or not the volume of the voice data of the speaker is lower than the minimum value of the predetermined range (step S65), the notifying unit 32d determines that the volume of the voice data of the speaker is equal to or higher than the minimum value of the predetermined range (step S65). 65No), and it is determined whether or not the volume of the voice data of the speaker is equal to or less than the maximum value in the predetermined range (step S68). Subsequently, when the volume of the voice data of the speaker is equal to or lower than the maximum value of the predetermined range (No in step S68), the detection unit 32a determines whether or not the speaker is present again after a predetermined time has elapsed by the timer (step S71). (Step S61) is executed. On the other hand, when the volume of the voice data of the speaker is larger than the maximum value of the predetermined range (Yes at Step S68), the notification unit 32d determines whether the distance to the speaker is smaller than the minimum value of the predetermined range (Step S69). ). When the distance from the speaker is equal to or greater than the minimum value (No at Step S69), after a predetermined time has elapsed with the timer (Step S71), the detection unit 32a again determines whether or not there is a speaker (Step S61). I do.

On the other hand, when the distance to the speaker is smaller than the minimum value (step S69 Yes), the notification unit 32d notifies the speaker to move away from the microphone (step S70). Subsequently, after a predetermined time elapses with the timer (step S71), the detection unit 32a again determines whether or not there is a speaker (step S61).

  In the third embodiment, the authentication processing described in the second embodiment may be performed. In such a case, in (Step S67) and (Step S70), the authentication processing shown in FIG. 10 is performed.

  In the first to third embodiments, the functions of the distribution terminal 3, the distribution terminal 7, or the distribution terminal 8 can be realized by executing a program stored in a computer medium.

  As described above, the distribution terminal 3 has been described in the first embodiment, the distribution terminal 7 has been described in the second embodiment, and the distribution terminal 8 has been described in the third embodiment. However, the present invention is not limited to the above-described embodiment. Various modifications and improvements are possible within.

  Note that, in the present embodiment, the distribution terminal 3 and the distribution terminal 7 are examples of a photographing device. The storage unit 35 is an example of a storage unit. The detection unit 32a is an example of a detection unit. The control unit 32b is an example of a control unit. The measurement unit 32c is an example of a measurement unit. The notification unit 32d is an example of a notification unit. The storage unit 32e is an example of a storage unit. The user ID is an example of identification information.

2 server 3, 7, 8 distribution terminal 4 user terminal 5 communication network 6 display device 31 communication unit 32 processing unit 32a detection unit 32b control unit 32c measurement unit 32d notification unit 33 data acquisition unit 33a audio acquisition unit 33b imaging unit 34 data Output unit 35 Storage unit 100 Video distribution system

JP 2008-103824 A

Claims (8)

In a photographing device for photographing a person,
A detection unit that detects the direction of the uttered person and the volume at the time of utterance;
A control unit that controls a shooting direction to the detected direction,
A measurement unit that measures the size of the person in the direction of the direction detected by the detection unit,
A notification unit that notifies the person when the volume is smaller than the minimum value of the first range according to the predetermined range and the size of the person on the image is smaller than the minimum value of the second range ;
An imaging device having   The photographing apparatus according to claim 1, wherein the notification unit notifies the person when the volume is higher than a maximum value in a predetermined range. The measuring unit measures a distance to the person in the direction detected by the detecting unit ,
The notification unit, the volume is smaller than the minimum value before Symbol first range, and when the distance between the detected persons is larger than the maximum value of the third range, according to claim 1, a notification to the person Shooting equipment. Prior Symbol notification unit, wherein the volume before Symbol greater than the maximum value of the first range and when the size of the image of the person is larger than the maximum value of the second range, the notification to the person claimed Item 3. An imaging device according to Item 2. The measuring unit measures a distance to the person in the direction detected by the detecting unit ,
The notification unit, the volume is greater than the maximum value before Symbol first range, and when the distance between the detected persons is smaller than the minimum value of the third range, according to claim 2, the notification to the person Shooting equipment. A storage unit that stores the identification information of the person corresponding to the face image extracted from the captured image in the storage unit in association with the presence or absence of the notification,
The notification unit, when not subjected to notification to the photographed person photographing apparatus according to any one of claims 1 to 5, the notification. A program to be executed by a photographing device for photographing a person,
Detects the direction of the person who spoke and the volume at the time of speech,
Controlling the shooting direction to the detected direction,
Measure the size of the person in the detected direction on the image,
When the sound volume is smaller than the minimum value of the first range related to the predetermined range and the size of the person on the image is smaller than the minimum value of the second range , a process of notifying the person is performed on the photographing apparatus. Program to let. A method performed by a photographing device for photographing a person,
Detects the direction of the person who spoke and the volume at the time of speech,
Controlling the shooting direction to the detected direction,
Measure the size of the person in the detected direction on the image,
When the sound volume is smaller than the minimum value of the first range related to the predetermined range and the size of the person on the image is smaller than the minimum value of the second range , the photographing device executes a process of notifying the person. how to.

Images