JP2013201642A - Electronic device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic device with excellent handleability, capable of storing sound data at appropriate timing.SOLUTION: An electronic device 100 includes: a sound collection device 30 for collecting sound; and a control part 70 for storing, in a storage part 41, the sound collected by the sound collection device after a face of an object person is detected by a detector 20 and from prescribed time before the mouth of the object person is moved. While there is the "risk" of being late for the first vocalization timing when sound recording is started after the object person moves or opens the "mouth", by starting the sound recording from 10 seconds before for instance, the risk of missing the recording of the first vocalization is eliminated.

Description

  The present invention relates to an electronic device.

  It has been proposed to ask other people to shoot scenes that cannot be taken by themselves or to use a fixed-point camera. Further, in recording sound during photographing, it has been proposed to record sound data in a storage medium in association with image data when face detection is performed (for example, see Patent Document 1).

JP 2009-177740 A

  However, the usability of the conventional imaging apparatus (electronic device) has not always been good.

  An electronic apparatus according to the present invention includes a sound collecting device that collects sound, and a sound collected by the sound collecting device after a predetermined time after the subject's mouth moves, after the detection device's face is detected. And a control unit that stores the information in the storage unit.

  According to the present invention, a user-friendly electronic device can be realized.

It is a block diagram of the imaging device by a first embodiment. It is a flowchart explaining the video recording process which a control part performs. It is a conceptual diagram of the imaging system by 2nd embodiment. FIG. 4 is a block diagram illustrating a configuration of the imaging system in FIG. 3. It is a block diagram which illustrates the composition of a movie image pick-up part. It is a flowchart explaining the video recording process which the control part of a master performs.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram of an imaging apparatus 100 according to the first embodiment. The imaging apparatus 100 records a sound (for example, music, song, dialogue, reading, etc.) relating to the subject, and takes a moving image. The imaging requester is registered in the imaging apparatus 100 in advance. Information indicating the face of the person who is the requester is stored in a flash memory 43 described later.

  In FIG. 1, the imaging apparatus 100 includes an imaging unit 10, a detection unit 20, a sound collection unit 30, a recording unit 40, a display unit 50, a timer unit 60, and a control unit 70. The imaging unit 10 includes a first imaging unit 11, a driving unit 12, and a second imaging unit 13. The first imaging unit 11 includes a plurality of lenses, an image sensor such as a CCD or a CMOS, and an image processing circuit. In the present embodiment, the first mouth 11 mainly captures a person's “mouth” (for example, a moving image) and generates image data. To do.

  The drive unit 12 is configured by an actuator that employs a linear motor, a voice coil motor, or the like, and adjusts the posture (imaging direction) of the first imaging unit 11. In the present embodiment, the drive unit 12 adjusts the posture (imaging direction) of the first imaging unit 11 according to the imaging result (image) of the second imaging unit 13. Note that the imaging optical axis of the first imaging unit 11 at the initial position is substantially parallel to the imaging optical axis of the second imaging unit 13. When the imaging direction is changed by driving the first imaging unit 11 by the driving unit 12, the imaging optical axis of the first imaging unit 11 intersects the imaging optical axis of the second imaging unit 13.

  Note that the first imaging unit 11 may be configured to be detachable from the housing of the imaging device 100, and the first imaging unit 11 may be attached to an arbitrary position of the housing of the imaging device 100. Further, the first imaging unit 11 may be arranged away from the imaging device 100.

  The second imaging unit 13 includes a plurality of lenses capable of wide-angle shooting with a wider angle of view than the lens of the first imaging unit 11, an imaging device such as a CCD or a CMOS, and an image processing circuit. In the present embodiment, the second imaging unit 13 performs wide-angle imaging (for example, a moving image) that includes the imaging area of the first imaging unit 11 to generate image data.

  The detection unit 20 includes a face detection unit 21 and a face recognition unit 22. The face detection unit 21 detects a face by pattern matching with a captured image using main parts (eyes, eyebrows, nose, mouth) of the face as a reference pattern, or determines whether a certain image is a face. Thus, detection using a learning method based on boosting for detecting a face can be performed. The face detection unit 21 also detects whether or not the “mouth” is open (moving) by the above-described pattern matching or boosting method.

  The face recognition unit 22 recognizes who the face detected by the face detection unit 21 is. Specifically, face recognition is performed using a known recognition algorithm (elastic bunch graph matching, hidden Markov model, etc.). This recognition algorithm includes a method of directly comparing visual features geometrically and a method of statistically digitizing an image and comparing the numerical value with a template, and either method can be used.

  The sound collection unit 30 includes a first microphone 31, a drive unit 32, a second microphone 33, and a sound processing circuit (not shown). The first microphone 31 is composed of a microphone having directivity, and collects sound input from a specific direction with high sensitivity. For example, a super-directional dynamic microphone, a super-directional condenser microphone, or the like can be used.

  The drive unit 32 includes an actuator that employs a linear motor, a voice coil motor, or the like, and adjusts the sound collection direction of the first microphone 31. In the present embodiment, the sound collection direction of the first microphone 31 is adjusted according to the imaging result (image) of the first imaging unit 11. Note that the first imaging unit 11 and the first microphone 31 may be integrated (unitized). When unitized, either the drive unit 12 or the drive unit 32 can be omitted.

  The second microphone 33 is composed of an omnidirectional microphone and collects a sound in a wider range than the range in which the first microphone 31 collects sound. The recording unit 40 includes a buffer memory 41, a recording I / F (interface) 42, and a flash memory 43.

  The audio signals collected by the first microphone 31 and the second microphone 33 are amplified by an audio processing circuit (not shown). The amplified audio signal is converted into digital audio data by an A / D conversion circuit (not shown).

  The buffer memory 41 temporarily stores image data picked up by the first image pickup unit 11 and the second image pickup unit 13, or is used as a work memory of the control unit 70 described later. A plurality of buffer memories 41 may be provided, and the buffer memory 41a for the first imaging unit 11 and the buffer memory 41b for the second imaging unit 13 may be configured separately. Further, the buffer memory 41 temporarily stores image data in the pre-process and post-process of image processing by the above-described image processing circuit. As the buffer memory 41, a volatile semiconductor memory or the like can be appropriately selected and used.

  The recording I / F 42 is formed with a connector for connecting a storage medium (for example, an SD card) 99. The recording I / F 42 executes data writing / reading with respect to the storage medium 99 connected to the connector. The recording I / F 42 of the present embodiment records (saves) a part of an image captured by the second imaging unit 13 in the storage medium 99, and captures an image captured by the first imaging unit 11 (in this example, “ The “mouth” image) is not recorded (saved) in the storage medium 99 (recording in the storage medium 99 is prohibited).

  The storage medium 99 is composed of a hard disk, a memory card with a built-in semiconductor memory, or the like. In this embodiment, a memory card is used. The recording medium 99 is used for storing image data and audio data.

  The flash memory 43 is configured by a nonvolatile memory. In the present embodiment, image data (referred to as reference image data) representing a shape for detecting a state where the “mouth” is open or moving (referred to as reference image data), face data registered in advance for face recognition ( Face data), acoustic data (sound information) prepared in advance for detecting sounds of musical instruments, whistles, and athletic meet pistols, and the like are stored in the flash memory 43.

  The display unit 50 is configured by a liquid crystal panel, for example, and displays an image, an operation menu screen, and the like. In the present embodiment, the display unit 50 displays the image captured by the second imaging unit 13 and does not display the image captured by the first imaging unit 11 (in this example, the “mouth” image). A transparent touch panel may be laminated on the surface of the display unit 50. In this case, the user can select the coordinates of the touch operation and the operation menu displayed corresponding to the coordinates by touching the touch panel while viewing the operation menu of the display unit 50. Become.

  The timer unit 60 is constituted by a crystal oscillation circuit, for example, and performs a timer process. In the present embodiment, the time information before and after the first imaging unit 11 captures an image of opening (moving) of the “mouth” is output to the control unit 70 described later. In other words, the time measuring unit 60 outputs time stamp information based on the time point when the first imaging unit 11 images the opening (moving) of the “mouth”.

  The control unit 70 has a CPU (not shown) and controls the entire imaging apparatus 100. In the present embodiment, imaging control is performed according to the movement of the “mouth” of the subject.

<Recording process>
Recording processing executed by the control unit 70 of the imaging apparatus 100 configured as described above will be described with reference to the flowchart of FIG. For example, when a recording switch (not shown) is turned on, the control unit 70 activates a program for performing the recording process shown in FIG. In step S1 of FIG. 2, the control unit 70 starts imaging by the second imaging unit 13, sequentially stores moving image data in the buffer memory 41, and proceeds to step S2. In step S2, the control unit 70 starts recording to sequentially store the audio data collected through the second microphone 33 in the buffer memory 41, and proceeds to step S3.

  In step S3, the control unit 70 starts timing by the timing unit 60 together with the start of the imaging and recording, and proceeds to step S4. In step S <b> 4, the control unit 70 uses the face detection unit 21 to determine whether a face has been detected from the image captured by the second imaging unit 13. The control unit 70 makes a positive determination in step S4 when the face captured in the image captured by the second imaging unit 13 proceeds to step S5, and makes a negative determination in step S4 when the face is not included. To repeat the process.

  In step S5, the control unit 70 determines whether the face detected in step S4 has been recognized using the face recognition unit 22. The control unit 70 makes an affirmative determination in step S5 when the face information of the photographing client (hereinafter referred to as a target person) stored in the flash memory 43 matches the face detected in step S4, and makes a determination in step S6. Proceed to The process proceeds to step S6 when a target person registered in advance is recognized. On the other hand, when the face information of the subject stored in the flash memory 43 does not match the face detected in step S4, the control unit 70 makes a negative determination in step S5 and returns to step S4. That is, the processes in steps S4 and S5 are repeated until face recognition is completed for a plurality of faces detected in step S3.

  The controller 70 that recognizes the face of the subject causes the position adjustment of the first imaging unit 11 and the first microphone 31 (attitude control for adjusting the shooting direction and the sound collection direction) in step S6. Specifically, the position of the “mouth” is detected or estimated from the identified face (the face recognized as the subject), and the first imaging unit 11 and the first microphone 31 are opposed to the “mouth”. The control unit 70 controls the drive unit 12 and the drive unit 32, respectively. The estimation means obtaining the position of the “mouth” based on the position of the “eye” or the position of the “nose” when “eyes” or “nose” is detected instead of detecting the “mouth”.

  In step S7, the control unit 70 that has adjusted the positions of the first imaging unit 11 and the first microphone 31 causes the first imaging unit 11 to start imaging the “mouth” detected or estimated in step S5. Data is sequentially stored in the buffer memory 41, and the process proceeds to step S8.

  In step S8, the control unit 70 starts recording to sequentially store the audio data collected through the first microphone 31 in the buffer memory 41, and proceeds to step S9. Here, at the time when recording by the first microphone 31 is started in step S <b> 8, the person of the “mouth” imaged by the first imaging unit 11 may not emit sound. However, as will be described later, if recording by the first microphone 31 is started after the person imaged by the first imaging unit 11 moves or opens the “mouth”, the recording start is the first utterance timing. Therefore, recording is started from step S8 before the movement of the “mouth” is detected, and the audio data is stored in the buffer memory 41.

  In step S <b> 9, the control unit 70 detects whether or not the “mouth” being imaged by the first imaging unit 11 has moved (whether it has been opened). Also in this case, the control unit 70 refers to the reference image data stored in the flash memory 43 and detects whether or not the “mouth” has moved (opened) by the detection unit 20. When the movement of the “mouth” is detected, the control unit 70 makes a positive determination in step S9 and proceeds to step S10. When the movement of the “mouth” is not detected, the control unit 70 makes a negative determination in step S9 and returns to step S7. Thereby, the processing from step S7 to step S9 is repeated until the movement of the “mouth” is detected.

  In step S10, the control unit 70 that has detected the movement of the “mouth” confirms the elapsed time measured in step S3 and instructs the storage to proceed to step S11. For example, when 10 seconds have elapsed from the start of timing in step S3, the control unit 70 sends an instruction to the storage I / F 42, and a predetermined time before the movement of the “mouth” is detected (in this example, 1 second before) In other words, after 9 seconds from the start of timing, the image data captured by the second imaging unit 13 stored in the buffer memory 41 is recorded (saved) in the storage medium 99, and the timing starts 9 After 2 seconds, the voice data collected via the first microphone 31 and the second microphone 33 stored in the buffer memory 41 is recorded (saved) in the storage medium 99. Image data and audio data are included in one file, for example.

  Note that the audio data recorded in the storage medium 99 may be any one of the audio data collected via the first microphone 31 and the second microphone 33. Of the audio data collected through two different microphones, for example, audio data in a predetermined sound pressure level range is stored, or audio data including (or not including) a predetermined frequency band is stored. Thus, it is possible to record unnecessary sound while eliminating unnecessary sound recording.

  Further, the control unit 70 may move the zoom optical system (not shown) of the second imaging unit 13 to the telephoto side at the timing when the movement of the “mouth” is detected, or may zoom up by electronic zoom processing. Thereby, for example, the second imaging unit 13 can image the identified face (that is, the face of the subject) up. In addition to the scaling operation or instead of the scaling operation, the frame rate at which the second imaging unit 13 captures an image may be increased. Thereby, the responsiveness with respect to a subject's motion improves.

  Further, the control unit 70 may change the sensitivity (gain) of the first microphone 31 in accordance with the way the “mouth” moves. This is because it is expected that a loud voice will be produced as the “mouth” is opened wide, so that the sensitivity (gain) of the first microphone 31 is lowered according to the size of the “mouth”, thereby increasing the voice input. It is for preparing. By performing such gain control, the dynamic range of the sound collection unit 30 can be expanded. The change of the sensitivity (gain) according to the opening size of the “mouth” may be applied to the second microphone 33. The change of the gain can be realized by changing the amplification gain in the sound processing circuit (not shown) of the sound collection unit 30, for example.

  The “mouth” of the subject imaged by the second imaging unit 13 moves or does not move. For this reason, the control unit 70 detects whether or not the subject's “mouth” has not moved (closed) for a predetermined time in step S11. Here, it is assumed that the predetermined time can be arbitrarily set. In the present embodiment, the predetermined time is, for example, 5 seconds. If the movement of the “mouth” is not detected by the detection unit 20 for 5 seconds, the control unit 70 makes a positive determination in step S11 and proceeds to step S12. When the movement of the “mouth” is detected by the detection unit 20, the control unit 70 makes a negative determination in step S11 and repeats the determination process.

  That is, while the movement of the “mouth” of the subject is detected, the control unit 70 makes a negative determination in step S11 and repeats the determination process, so that the image is captured by the second imaging unit 13 and stored in the buffer memory 41. The recorded image data is recorded (saved) in the storage medium 99, and the audio data collected through the first microphone 31 and the second microphone 33 and stored in the buffer memory 41 is recorded (saved) in the storage medium 99. )

  In step S12, when there is no movement of the “mouth” for a predetermined time, the control unit 70 determines whether or not to end the imaging. For example, when the scene in which the subject is imaged is a choir or theater performance at a school, there is a big applause at the end, and an announcement is made to tell the end. For this reason, the control part 70 of this embodiment complete | finishes the imaging by the 1st imaging part 11 and the 2nd imaging part 13 when a big applause and the word "end" are input via the 2nd microphone 33. Judge that it should.

  The sound data indicating “big applause” and the voice dictionary data indicating the word “end” may be stored in advance in the flash memory 43 as the sound information. In addition, when shooting an athletic meet or the like, the competition may be terminated by a pistol signal, so that acoustic data indicating the “pistol signal” may be stored in the flash memory 43 as sound information.

  If the determination is affirmative in step S12, the control unit 70 performs a predetermined off process and ends the process of FIG. In the off process, the image pickup by the first image pickup unit 11 and the second image pickup unit 13 is ended, the recording by the first microphone 31 and the second microphone 33 is ended, and the image data and audio data accumulated in the buffer memory 41 The process of recording (storing) in the storage medium 99 is also terminated.

  On the other hand, if the control unit 70 makes a negative determination in step S12, the control unit 70 proceeds to step S13. In step S <b> 13, the control unit 70 determines whether the face of the subject can be recognized using the face recognition unit 22. When the face information of the subject stored in the flash memory 43 matches the face included in the image captured by the second imaging unit 13, the control unit 70 makes a positive determination in step S13 and proceeds to step S9. Return. When returning to step S9, imaging is continued and the above-described processing is repeated.

  When the face information of the subject stored in the flash memory 43 is not included in the image captured by the second imaging unit 13, the control unit 70 makes a negative determination in step S13 and proceeds to step S14. In step S14, the control unit 70 ends the imaging by the first imaging unit 11, and proceeds to step S15. In step S15, the control unit 70 ends the recording by the first microphone 31 and returns to step S3.

According to the first embodiment described above, the following operational effects can be obtained.
(1) The image pickup apparatus 100 is picked up by the sound collecting unit 30 after the subject's face is detected by the sound collecting unit 30 and the detection unit 20 and before a predetermined time before the “mouth” of the subject moves. And the control unit 70 for storing the stored sound in the buffer memory 41, the sound data can be stored at an appropriate timing, and the usability is improved. Specifically, if the subject starts to record after moving or opening his / her mouth, there is a possibility that the recording will not be in time for the first utterance timing, but for example, recording the sound from 10 seconds before By starting, you can eliminate the risk of failing to record the first utterance.

(2) Since the imaging device 100 of (1) includes the drive unit 32 that adjusts the direction of the sound collection unit 30 based on the detection result by the detection unit 20, for example, the sound from the subject is appropriately collected. I can sound.

(3) In the imaging device 100 of (1) or (2), the control unit 70 that adjusts the gain of the sound collection unit 30 based on the detection result by the detection unit 20 is provided. For example, the dynamic range of the sound collection unit 30 can be expanded by reducing the gain of the first microphone 31 and preparing for a large voice input as the “mouth” of the subject is greatly opened.

(4) In the imaging device 100 of (1) to (3) above, the control unit 70 finishes storing the sound in the buffer memory 41 after the detection unit 20 no longer detects the subject's face. Therefore, it is possible to automatically stop without unnecessarily continuing recording, thereby improving usability.

(5) In the imaging device 100 of (1) to (3) above, the control unit 70 ends the storage of the sound in the buffer memory 41 according to a predetermined sound. Since recording can be automatically stopped according to the situation, usability can be improved.

(6) In the imaging device 100 of (1) to (5) above, the sound collection unit 30 includes the first microphone 31 having directivity, and therefore selectively records sound from a specific direction. You can also.

(7) In the imaging apparatus 100 of (6), the sound collection unit 30 includes the second microphone 33 having a wider sound collection range than the first microphone 31, and the first is later than the sound collection start by the second microphone 33. Since the controller 70 for starting the sound collection by the microphone is further provided, for example, it is possible to control the sound collection in a specific direction after collecting a wide range of sounds.

(8) Since the imaging apparatus 100 according to (1) to (7) above includes the flash memory 43 that stores data related to the shape of the “mouth”, the “mouth” is appropriately compared to the case where the data is not provided. "And its movement can be detected.

(Modification 1)
In the first embodiment described above, by interposing the buffer memory 41, a part of the image captured by the second imaging unit 13 is recorded (saved) in the storage medium 99, and the first imaging unit 11 captures the image. The image (in this example, the mouth image) is not recorded (saved) in the storage medium 99. Instead, the image data from the first imaging unit 11 and the second imaging unit 13 and the audio data from the first microphone 31 and the second microphone 33 are recorded in the storage medium 99, and the time measurement data confirmed in step S10 is recorded. It may be configured to be recorded in the storage medium 99 together with the metadata, and to edit the image data and the sound data based on the time data on a personal computer or the like later (determining whether to save). .

(Second embodiment)
Next, a second embodiment will be described with reference to FIGS. FIG. 3 is a conceptual diagram of an imaging system 200 according to the second embodiment. The imaging system 200 records a sound related to the subject (for example, music, song, dialogue, reading, etc.) as a subject and photographs a moving image.

  In FIG. 3, the imaging system 200 includes a motion detection imaging unit 210 that detects a “mouth” of a subject, and a plurality of video imaging units 220 that capture a video of the subject based on the detection result of the motion detection imaging unit 210 ( In this example, 220-1 to 220-3). The photographing client is registered in advance, and face information indicating the face of the requesting person is stored in the flash memory 43 in each of the moving image capturing units 220-1 to 220-3. In the present embodiment, the moving image capturing unit 220 includes three moving image capturing units 220-1 to 220-3, and the basic configuration is the same. The number of moving image capturing units 220 is not limited to three, and may be five or ten.

  FIG. 4 is a block diagram illustrating the configuration of the imaging system 200 of FIG. FIG. 5 is a block diagram illustrating the configuration of one moving image capturing unit 220-n among the three moving image capturing units 220-1 to 220-3. 4 and 5, the same constituent elements as those of the imaging apparatus 100 described in the first embodiment are the same among the constituent elements constituting the motion detection imaging unit 210 and the plurality of moving image imaging units 220-n. The description is omitted.

  4 and 5, the motion detection imaging unit 210 corresponds to the imaging unit 10 in the first embodiment. The motion detection imaging unit 210 includes a first imaging unit 11, a first microphone 31, a driving unit 211, a communication unit 212, and a control unit 213. The first imaging unit 11 mainly performs imaging for detecting the movement of the “mouth” of the subject. The drive unit 211 drives the first imaging unit 11 and the first microphone 31 that are unitized, for example, and adjusts the posture (shooting direction) of the first imaging unit 11 and the sound collection direction of the first microphone 31.

  The communication unit 212 includes a wireless communication unit, and performs wireless communication with the communication unit 221-n of the moving image capturing unit 220-n. In addition, the communication unit 212 transmits the audio data collected through the first microphone 31 to the moving image capturing unit 220-n. The control unit 213 includes a CPU (not shown), and controls the entire motion detection imaging unit 210 and performs cooperative control with the moving image imaging unit 220-n.

  Hereinafter, the details of the moving image capturing unit 220-n will be described with reference to FIG. In FIG. 5, the moving image capturing unit 220-n corresponds to a block other than the image capturing unit 10 in the first embodiment. The moving image imaging unit 220-n includes a second imaging unit 13, a detection unit 20, a second microphone 33, a recording unit 40, a display unit 50, a time measuring unit 60, a communication unit 221, and a control unit 222. And have. The communication unit 221 includes a wireless communication unit, and performs wireless communication with the communication unit 212 of the motion detection imaging unit 210. The communication unit 221 also communicates with the communication unit 221 of the other moving image capturing unit 220.

  The control unit 222 includes a CPU (not shown), and controls the entire moving image capturing unit 220-n and performs cooperative control with the control unit 213 of the motion detection imaging unit 210. In the present embodiment, the control unit 222-1 of the moving image capturing unit 220-1 (FIG. 4) serves as a master control unit that controls the entire imaging system 200, and the control unit 222 of the moving image capturing unit 220-2 (FIG. 4). -2, The control part 222-3 of the moving image imaging part 220-3 (FIG. 4) demonstrates as what becomes a control part of a slave, respectively.

  Recording processing executed by the control unit 222-1 of the moving image imaging unit 220-1 (master) in the imaging system 200 configured as described above will be described with reference to the flowchart of FIG. For example, when a recording switch (not shown) is turned on, the control unit 222-1 activates a program for performing the recording process in FIG.

  In step S101 in FIG. 6, the control unit 222-1 (master) instructs the moving image capturing units 220-2 and 220-3 to perform image capturing via the communication unit 221-1, and performs its second image capturing. The imaging is instructed to the unit 13-1, and the process proceeds to step S102. Accordingly, the second imaging units 13-1 to 13-3 of the moving image capturing units 220-1 to 220-3 start capturing moving images, respectively. Note that it is not always necessary to instruct imaging by all the moving image imaging units 220-1 to 220-3, and the number of the imaging units may be set as appropriate.

  In step S102, the control unit 222-1 (master) detects whether or not the moving image capturing unit 220-1 itself, the moving image capturing unit 220-2, and 220-3 have captured the face. Specifically, the detection unit 20-1 detects whether a face is included in the imaging result of the second imaging unit 13-1. Further, the control unit 222-1 (master) further determines whether or not a face is included in the imaging results of the second imaging units 13-2 and 13-3 with respect to the moving image imaging units 220-2 and 220-3 serving as slaves, respectively. Are detected by the detection units 20-2 and 20-3, and the detection results are transmitted from the communication units 221-2 and 221-3 to the moving image capturing unit 220-1 as a master, respectively. The control unit 222-1 (master) has its own second imaging unit 13-1, the second imaging unit 13-2 of the moving image capturing unit 220-2, and the second imaging unit 13-3 of the moving image imaging unit 220-3. Step S102 is repeated until at least one face is detected.

  In step S103, the control unit 222-1 (master) determines whether or not the face detected in step S102 has been recognized. Specifically, while instructing its own detection unit 20-1 to perform face recognition, the detection units 20-2 and 20-3 perform face recognition on the moving image capturing units 220-2 and 220-3 serving as slaves, respectively. Instruct. This face recognition is performed in each of the moving image capturing units 220-1 to 220-3 by the same method as in the first embodiment. By performing face recognition in each of the moving image capturing units 220-1 to 220-3, it is possible to reduce the time required for face recognition as compared with the case where face recognition processing is performed collectively by one moving image capturing unit.

  The control unit 222-1 (master) makes an affirmative decision in step S103 when the subject (photographer) recognizes the face, proceeds to step S104, and denies step S103 if the subject is not recognized. Determine and return to step S102. That is, the processes in steps S102 and S103 are repeated until face recognition is completed in at least one of the moving image capturing units 220-1, 220-2, and 220-3.

  In step S104, the control unit 222-1 (master) selects a device that captures a moving image from the moving image capturing units 220-1 to 220-3. When all the moving image capturing units 220-1 to 220-3 recognize the target person's face, the control unit 222-1 (master) continues image capturing with all the moving image capturing units 220-1 to 220-3. Let However, not all the moving image capturing units 220-1 to 220-3 always recognize the face of the subject. Therefore, in the present embodiment, a case where the subject's face is recognized only by the moving image capturing unit 220-3 in FIG. 3 will be described as an example.

  In this case, the control unit 222-1 (master) selects the second imaging unit 13-2 of the adjacent moving image capturing unit 220-2 in addition to the second imaging unit 13-3 of the moving image capturing unit 220-3. Then, the imaging is continued and the imaging by the second imaging unit 13-1 of the moving image imaging unit 220-1 is stopped (turned off). This is because when the target person moves, the moving image capturing unit 220-2 adjacent to the moving image capturing unit 220-3 may capture the target person, whereas the moving image capturing unit 220-3 This is because the distant moving image capturing unit 220-1 has a low possibility of capturing the subject.

  In addition, when the subject's face is recognized by the moving image capturing unit 220-2 adjacent to both of the moving image capturing units 220-1 and 220-3, based on the image captured by the moving image capturing unit 220-2. What is necessary is just to detect a motion vector of a subject and select the moving image capturing unit 220-1 or 220-3 based on the motion vector. This is because when there are a plurality of moving image capturing units that are likely to capture a moving target person, a moving image capturing unit that is more likely to be captured is selected.

  In addition, when the target person's face is recognized by the moving image capturing unit 220-3 in FIG. 3, for example, when the target person is not expected to move, such as a scene in which the target person is singing (the magnitude of the motion vector is For less than a predetermined value, the imaging in the second imaging unit 13-2 of the moving image capturing unit 220-2 adjacent to the moving image capturing unit 220-3 may be stopped (turned off).

  The control unit 222-1 (master) in the case of stopping (turning off) the imaging of the second imaging unit in the moving image capturing unit 220-1 or the moving image capturing unit 220-2 is further provided with the moving image capturing units 220-1 and 220-2. The user may be prompted to move to a position where the subject can be imaged. For example, a message “Let's shoot close to the subject” is displayed on the display unit 50-1 of the moving image capturing unit 220-1, or “the approaching subject is approached” to the moving image capturing unit 220-2 via the communication unit 221-1. Or send a message saying "Let's shoot." The control unit 222-2 of the moving image capturing unit 220-2 that has received this message causes the display unit 50-2 to display a message “Let's shoot near the subject”.

  Furthermore, the moving image capturing units 220-1 and 220-2 are fixed cameras that automatically shoot without user operation, and the moving image capturing units 220-1 and 220-2 include the second image capturing units 13-1 and 13-. In the case where driving members (not shown) for controlling the posture (photographing direction) 2 are provided in advance, the moving image photographing unit 220-1 is arranged in such a direction that the subject can be imaged by driving these driving members. , 220-2 may be automatically controlled.

  In step S105 in FIG. 6, the control unit 222-1 (master) sends an instruction from the communication unit 221-1 to the moving image capturing units 220-3 and 220-2 selected in step S104, and the moving image capturing unit 220- 3 and the second microphone 33-3 of the moving image capturing unit 220-2 and recording are started by the second microphone 33-3, and the process proceeds to step S106.

  In step S106, the control unit 222-1 (master) sends an instruction from the communication unit 221-1 to the moving image capturing units 220-3 and 220-2 selected in step S104, the time measuring unit 60-3, and the time measuring The timing start by the unit 60-2 is started, and the process proceeds to step S107.

  In step S107, the control unit 222-1 (master) sends an instruction from the communication unit 221-1 to the motion detection imaging unit 210, and adjusts the positions of the first imaging unit 11 and the first microphone 31 (imaging direction and sound collection direction). (Attitude control for adjusting). Specifically, the control unit 213 controls the driving unit 211 so that the first imaging unit 11 and the first microphone 31 are opposed to the position of the “mouth” detected or estimated from the face recognized as the target person. Let The position information of “mouth” is acquired by, for example, a moving image capturing unit (220-3 in this example) that recognizes the face, and the result is transmitted from the communication unit 221-3 to the motion detection imaging unit 210. The estimation means obtaining the position of the “mouth” based on the position of the “eye” and the position of the “nose” as described above.

  In addition, when the face of the subject is detected by the plurality of moving image capturing units 220, the control unit 213 performs the first operation based on the imaging result by the moving image capturing unit 220 that captures the face in front of the subject. The posture control of the imaging unit 11 and the first microphone 31 may be performed. Instead of this, the control unit 213 estimates the front position of the face of the subject based on the imaging results of the plurality of moving image imaging units 220, and performs posture control of the first imaging unit 11 and the first microphone 31. You may make it perform. When based on the imaging results of the plurality of moving image capturing units 220, for example, the respective imaging results are transmitted to the motion detection imaging unit 210 from a plurality of moving image capturing units (for example, 220-2 and 220-3) whose faces are recognized. .

  In step S108, the control unit 222-1 (master) sends an instruction from the communication unit 221-1 to the motion detection imaging unit 210, and the first imaging unit 11 captures the “mouth” detected or estimated in step S107. Let it begin. The moving image data is transmitted from the communication unit 212 of the motion detection imaging unit 210 to the moving image imaging unit 220-1, and is sequentially stored in the buffer memory 41-1 in the moving image imaging unit 220-1.

  In step S <b> 109, the control unit 222-1 (master) sends an instruction from the communication unit 221-1 to the motion detection imaging unit 210, and starts recording audio data collected via the first microphone 31. The audio data is transmitted from the communication unit 212 of the motion detection imaging unit 210 to the moving image imaging unit 220-1, and is sequentially stored in the buffer memory 41-1 in the moving image imaging unit 220-1.

  In step S <b> 110, the control unit 222-1 (master) detects whether the “mouth” imaged by the first imaging unit 11 of the motion detection imaging unit 210 has moved (whether opened). Specifically, the same reference image data as in the first embodiment is stored in advance in the flash memory 43 in the moving image capturing unit 220-1, and the detection unit 20 is referenced with reference to this reference image data. -1 detects whether the "mouth" has moved (opened).

  When the motion of the “mouth” is detected from the moving image data transmitted from the motion detection imaging unit 210 and sequentially stored in the buffer memory 41-1, the control unit 222-1 (master) makes a positive determination in step S110 and performs step Proceeding to S111, if the movement of the “mouth” is not detected, a negative determination is made in step S110, and the process returns to step S108. Thus, the processing from step S108 to step S110 is repeated until the movement of the “mouth” is detected.

  In step S111, the control unit 222-1 that has detected the movement of the “mouth” confirms the elapsed time measured in step S106 and instructs the storage to proceed to step S112. For example, the control unit 222-1 (master) in the case where 10 seconds have elapsed from the start of timing in step S106 instructs the moving image capturing units 220-3 and 220-2 via the communication unit 221-1 ( Including the time information at which the movement of the “mouth” is detected), and the buffer memories 41-3 and 41-2 are sent from the time when the movement of the “mouth” is detected to a predetermined time (in this example, 1 second before). The stored image data by the second imaging units 13-3 and 13-2 are recorded (saved) in the storage media 99-3 and 99-2, respectively.

  In addition, when the movement of the subject's mouth is detected in step S110, the control unit 222-1 (master) sends an instruction to the moving image capturing unit 220-3, and zooms (not illustrated) of the second imaging unit 13-3. The optical system (or electronic zoom) may be moved to the telephoto side, or an instruction is sent to one of the moving image capturing units 220-3 and 220-2, and the second image capturing unit 13-3 or 13-2 The zoom optical system (or electronic zoom) may be moved to the telephoto side. When one zoom optical system (or electronic zoom) is moved to the telephoto side, the other zoom optical system (or electronic zoom) may be maintained without change, or the zoom optical system (or electronic zoom) may be maintained. ) May be moved to the wide-angle side. As a result, it is possible to capture the subject's face up with one moving image capturing unit or to capture the wide angle with the other moving image capturing unit.

  The “mouth” of the subject imaged by the moving image capturing units 220-3 and 220-2 moves or does not move. Therefore, the control unit 222-1 (master) detects whether or not the subject's “mouth” has not moved (closed) for a predetermined time in step S <b> 112. Here, it is assumed that the predetermined time can be arbitrarily set. In the present embodiment, the predetermined time is, for example, 5 seconds. When the movement of the “mouth” is not detected by the detection unit 20-1 for 5 seconds, the control unit 222-1 (master) makes an affirmative determination in step S112 and proceeds to step S113. When the movement of the “mouth” is detected by the detection unit 20-1, the control unit 222-1 (master) makes a negative determination in step S112 and repeats the determination process.

  That is, while the movement of the “mouth” of the subject is detected, the control unit 222-1 (master) makes a negative determination in step S112 and repeats the above determination process. 220-2 records (saves) the image data by the second imaging units 13-3 and 13-2 stored in the buffer memories 41-3 and 41-2 respectively in the storage media 99-3 and 99-2. ) Also, the audio data collected via the second microphones 33-3 and 33-2 and stored in the buffer memories 41-3 and 41-2 are recorded (stored) in the storage media 99-3 and 99-2, respectively. )

  In step S <b> 113, the control unit 222-1 (master) determines whether or not to end imaging when there is no movement of the “mouth” for a predetermined time, as in the case of the first embodiment. The control unit 222-1 (master) performs a predetermined off process when the determination in step S113 is affirmative, and ends the process of FIG. In the off process, the imaging and recording (sound collection) by all the moving image imaging units 220-1 to 220-3 and the motion detection imaging unit 210 are ended.

  On the other hand, if the control unit 222-1 (master) makes a negative determination in step S113, the process proceeds to step S114. In step S114, the control unit 222-1 (master) determines whether or not the subject's face can be recognized. Similar to the case of step S103, the detection units 20-2 and 20-3 respectively instruct the face recognition to the moving image capturing units 220-2 and 220-3 that are the devices being imaged.

  The control unit 222-1 (master) makes a positive determination in step S114 when the subject (photographer) recognizes the face, returns to step S110, and denies step S114 if the subject is not recognized. Determine and proceed to step S115. When returning to step S110, imaging is continued and the above-described processing is repeated.

  In step S115, the control unit 222-1 (master) sends an instruction from the communication unit 221-1, ends the imaging by the motion detection imaging unit 210, and proceeds to step S116. In step S116, the control unit 222-1 (master) sends an instruction from the communication unit 221-1, ends the recording by the motion detection imaging unit 210, and returns to step S102.

According to the second embodiment described above, the following operational effects can be obtained.
(1) The imaging system 200 starts from the second microphone 33, the first microphone 31, and the detection unit 20 after detecting the subject's face and from the second before the subject's “mouth” moves. And a control unit 222 that stores the sound collected by the microphone 33 in the buffer memory 41, so that audio data can be accumulated at an appropriate timing.

(2) Since the imaging system 200 of (1) includes the drive unit 211 that adjusts the direction of the first microphone 31 based on the detection result by the detection unit 20, for example, the sound from the subject is appropriately collected. I can sound.

(3) In the imaging system 200 of (1) or (2) above, the control unit 222 finishes storing the sound in the buffer memory 41 after the detection unit 20 no longer detects the subject's face. Therefore, it is possible to automatically stop without unnecessarily continuing recording, thereby improving usability.

(4) In the imaging system 200 of the above (1) or (2), the control unit 222 ends the storage of the sound in the buffer memory 41 according to a predetermined sound. Since recording can be automatically stopped according to the situation, usability can be improved.

(5) In the imaging system 200 of (1) to (4) above, the sound collection unit includes the first microphone 31 having directivity, so that sound from a specific direction can be selectively recorded. it can.

(6) In the imaging system 200 of (5) above, the sound collection unit includes the second microphone 33 having a wider sound collection range than the first microphone 31, and the first microphone is later than the sound collection start by the second microphone 33. Since the control unit 222 for starting the sound collection by 31 is further provided, for example, the sound collection of the sound can be started from a specific direction after the target person is recognized.

(7) Since the imaging system 200 of the above (1) to (6) includes the flash memory 43 that stores data related to the shape of the “mouth”, the “mouth” is appropriately compared with the case where the data is not provided. "And its movement can be detected.

(Modification 2)
In the second embodiment described above, a case where one target person is imaged by a plurality of moving image capturing units 220-n has been described as an example. However, a plurality of target persons may be imaged. It may be. When imaging a plurality of subjects, it may be assigned which subject is to be imaged by which moving image imaging unit 220. For example, the moving image capturing unit 220 may capture a subject that is easy to capture.

  In the above description, the imaging device 100, the motion detection imaging unit 210, and the moving image imaging unit 220 are illustrated as examples of electronic devices. However, the imaging device 100 and the imaging system 200 are configured using a multi-function mobile phone or a tablet computer. May be.

  The above description is merely an example, and is not limited to the configuration of the above embodiment. Needless to say, the first embodiment and the second embodiment described above may be appropriately combined.

DESCRIPTION OF SYMBOLS 10 ... Imaging part 11 ... 1st imaging part 12, 32, 211 ... Drive part 13 ... 2nd imaging part 20 ... Detection part 21 ... Face detection part 22 ... Face recognition part 30 ... Sound collecting part 31 ... 1st microphone 33 ... Second microphone 40 ... recording unit 41 ... buffer memory 42 ... recording I / F
43 ... Flash memory 50 ... Display unit 60 ... Timing unit 70, 213, 222 ... Control unit 99 ... Storage medium 100 ... Imaging device 200 ... Imaging system 210 ... Motion detection imaging unit 212, 221 ... Communication unit 220 (220-1 to 220-1) 220-3) ... Moving image capturing unit

Claims (8)

A sound collector that collects sound,
A control unit that stores sound collected by the sound collection device in a storage unit after a predetermined time after the subject's face is detected by the detection device and before the subject's mouth moves;
An electronic device characterized by comprising: The electronic device according to claim 1,
An electronic apparatus comprising: a posture control unit that adjusts a direction of the sound collecting device based on a detection result by the detection device. The electronic device according to claim 1 or 2,
An electronic apparatus comprising a gain control unit that adjusts a gain of the sound collector based on a detection result of the detection device. The electronic device according to any one of claims 1 to 3,
The electronic device according to claim 1, wherein the control unit terminates the storage of the sound in the storage unit after the detection apparatus no longer detects the face of the subject. The electronic device according to any one of claims 1 to 3,
The electronic device according to claim 1, wherein the control unit terminates the storage of the sound in the storage unit according to a predetermined sound. The electronic device according to any one of claims 1 to 5,
The sound collecting device includes a first microphone having directivity. The electronic device according to claim 6,
The sound collection device includes a second microphone having a wider sound collection range than the first microphone,
An electronic apparatus, further comprising: a sound collection control unit for starting sound collection by the first microphone later than the sound collection start by the second microphone. In the electronic device as described in any one of Claim 1 to 7,
An electronic apparatus comprising a storage member that stores data relating to a mouth shape.

Images