JP2017005322A - Report function for camera photographing utilizing wearable terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus system which enables a user to communicate only an audio signal from a desired wireless microphone, even in the case where a plurality of wireless microphones are detected, by utilizing information of face detection or AF/AE focusing.SOLUTION: The present invention relates to an imaging apparatus comprising reception means for receiving audio signals that are transmitted from a wearable audio collection device by radio communications. The imaging apparatus comprises: storage means for storing information associating the wearable audio collection device with a specific subject; imaging means which acquires an image signal by capturing an optical image within a predetermined angle of photographing view; estimation means which estimates whether the specific subject is included within the predetermined angle of photographing view of the imaging means; selection means which selects a subject that should be used for audio collection while paying attention thereto in accordance with image information if there are a plurality of specific subjects; the means for receiving the audio signal output that is obtained from the audio collection device, in accordance with an estimation result of the estimation means; and recording means by which audio signal input obtained from an audio collection device within the imaging apparatus and the image signal obtained by the imaging means are combined and recorded on a recording medium in association with each other.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus capable of processing an audio signal provided by an external microphone attached to a wearable terminal equipped with a detachable microphone and a display unit.

  In recent years, many systems have been proposed in which a camera is connected to a wearable wireless device such as a smartphone or a wearable terminal to perform image acquisition and camera control, and there is an increasing demand for cooperation.

  Conventionally, an imaging apparatus records an audio signal obtained by a wireless microphone together with an image signal. Some of these imaging apparatuses receive an audio signal acquired and transmitted by a wireless microphone, process and record the level of the audio signal. In Patent Document 1, the distance to the subject is estimated based on the received signal strength of wireless communication from the wireless microphone, and the level of the audio signal received from the wireless microphone is controlled.

  However, in this case, an audio signal is received even though the wireless microphone is not within the angle of view. In Patent Document 2, in addition to the received signal strength of wireless communication, the angle of view depends on the presence of face detection and the magnification of the camera. The sound level received from the wireless microphone was controlled in the state where it was detected. Further, when a plurality of wireless microphones are detected, recording is performed by changing the magnification of the volume level based on the received signal strength of face detection / wireless communication.

JP 2006-314078 A JP 2011-130134 A

  However, in the related art disclosed in the above-mentioned patent document, when a plurality of wireless microphones are detected, control is performed for all communications and the level of the audio signal is controlled, so that the audio is congested. In addition, a single imaging device is configured to receive data from a plurality of wireless microphones, but the amount of information is enlarged in order to receive audio data from a plurality of wireless microphones, all of which communicate with the camera at the same time. Depending on the communication state, the communication speed becomes slow, and the transmission / reception speed of the audio data becomes slow at the time of moving image shooting.

  Therefore, an object of the present invention is to select only a desired wireless microphone and detect only the audio signal of the desired wireless microphone when a plurality of wireless microphones are detected using face detection or imaging information at the time of focusing. An object of the present invention is to provide an imaging apparatus system that can perform recording while emphasizing a desired audio signal by communication, and can reduce the amount of communication.

  In order to achieve the above object, an imaging apparatus according to the present invention is an imaging apparatus having receiving means for receiving an audio signal transmitted by radio communication from a wireless apparatus, wherein the wireless apparatus and a specific subject are connected to each other. Storage means for storing the associated information, imaging means for capturing an optical image within a predetermined shooting angle of view to obtain an image signal, and the specific subject within the predetermined shooting angle of view of the imaging means Estimation means for estimating whether or not the image is detected, means for selecting a desired subject among the detected subjects according to the estimation result of the estimation means, and audio output from the subject selected by the selection means A recording unit that records and mixes the signal and the audio signal obtained by the imaging unit and records the recording unit in association with the image signal obtained by the imaging unit And wherein the door.

  According to the imaging device of the present invention, when recording using an external microphone attached to a wearable wireless device with a display unit, it is estimated whether or not there is a subject to be focused on based on the face information of the subject. The recording target is determined and the result is returned to the display unit of the wearable wireless device. Then, according to the estimation result, the voice signal received from the external microphone is controlled after selecting the most notable terminal from the plurality of wearable wireless devices, so the voice data from the wireless device is selected until the selection is determined. Since the image pickup apparatus does not communicate with each other, useless communication can be prevented, and only desired sound can be collected without mixing communication.

  In addition, since the recording status can be confirmed with the wearable wireless device on the transmission side at the time of shooting, the user can select the desired wearable wireless device because the shooting operation can be performed while confirming the recording. It becomes possible.

It is a block diagram which shows schematic structure based on 1st Embodiment. It is a flowchart which shows the registration procedure of the wearable wireless apparatus based on 1st Embodiment. It is a registration information table of the wearable wireless device recorded on the imaging device according to the first embodiment. It is a flowchart which shows the procedure which determines the wearable wireless apparatus made to communicate with an imaging device based on 1st Embodiment. It is a supplementary figure explaining the state of face detection, and the communication object of a wearable type wireless device. It is a flowchart which shows the communication condition display procedure of an imaging device and a mounting | wearing type wireless apparatus based on 1st Embodiment. It is a supplementary figure of the display method of a wearing type wireless device.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The technical scope of the present invention is determined by the claims, and is not limited by the following individual embodiments. In addition, not all combinations of features described in the embodiments are essential to the present invention.

[Example 1]
In the first embodiment, a system will be described as an example of an imaging system including an external microphone attached to a wearable wireless device and an imaging device. FIG. 1 is a block diagram illustrating a schematic configuration of the imaging device 100 and the wearable wireless device 119 according to the first embodiment. The imaging apparatus 100 can record a file such as a captured moving image or still image on the memory card 110. The image sensor 101 is a CMOS sensor that generates and outputs charges corresponding to the amount of incident light by photoelectric conversion. In addition to reading out signals of all pixels, the image sensor 101 adds charges of specific pixels and thins out charges every specific row or column. Can also be read.

  The image sensor driving circuit 102 can drive the image sensor 101 in the moving image mode or the still image mode based on the control by the CPU 114. The image processing circuit 103 performs A / D conversion on the image signal from the image sensor 101 and performs predetermined image processing such as gamma correction, color balance adjustment, and color conversion. The image memory 104 is used for temporarily storing moving image or still image image data from the image processing circuit 103 for display and recording processing. The microphone 105 takes in external sound and outputs an audio signal.

  The audio processing circuit 106 performs A / D conversion on the audio signal from the microphone 105 and encodes it using the PCM encoding method. In addition, the audio signal transmitted from the communication unit 121 of the wearable wireless device 118 and received through the communication unit 111 of the imaging device 100 can be combined with the audio from the microphone 105 of the imaging device. The PCM buffer 107 stores the PCM audio data from the audio processing circuit 106 for recording processing.

  At the time of moving image recording, the image conversion circuit 115 multiplexes the moving image data stored in the image memory 104 and the PCM audio data stored in the PCM buffer 107 to generate stream data in a predetermined format. The generated stream data is stored in the stream buffer 108.

  The stream buffer 108 has a memory area that can store stream data for a certain period. Data in the stream buffer 108 is written to the memory card 109 unless otherwise specified. The write controller 110 controls the connection state of the memory card 110 and controls access to the memory card 110.

  The communication unit 111 performs communication between the imaging device 100 and an external device, and transmits and receives data such as audio signals and image signals. For example, an audio signal transmitted from the communication unit 121 of the wearable wireless device 117 is received and sent to the audio processing circuit 106. In addition, control signals related to shooting and other information are transmitted and received. The communication unit 111 is a wireless communication module such as an infrared communication module or a wireless LAN communication module.

  The operation unit 112 includes a release button, a moving image recording button, a mode switching dial for switching between a still image continuous shooting mode and a moving image recording mode, and the like, and is used by a user to input various instructions to the imaging apparatus 100. The display unit 113 includes a liquid crystal display panel, displays an image captured by the image sensor 101 during recording, and displays a reproduced image during reproduction. The display unit 113 also displays information such as a setting menu for the imaging apparatus 100, a recording time, a remaining battery level, and the like.

  The CPU 114 is a central processing unit and controls each unit of the imaging device 100. The memory 118 is used as a work area for the CPU 114 and the file system 116. The file system 116 provides information necessary for writing and reading a file to the memory card 110.

  The wearable wireless device 117 includes a display unit 123 that displays information, a microphone 118 that inputs sound, and a communication unit 121 that communicates with a camera. Similarly to the microphone 105, the microphone 118 collects an audio signal around the wearable wireless device using the built-in microphone, performs analog-digital data conversion by the audio processing circuit 119, and is temporarily stored in the PCM buffer 120. Transmission is performed to an external device through the communication unit 121.

  Similar to the communication unit 111, the communication unit 121 performs communication between the wearable wireless device 117 and an external device, and transmits audio signal data stored in the PCM buffer 120. In addition, a control signal related to the start and end of shooting, control signals for shooting, and other control signals are transmitted and received. The communication unit 121 is a wireless communication module such as an infrared communication module or a wireless LAN communication module.

  The operation unit 122 is used for the power source of the wearable wireless device 117 and for the user to input various instructions to the wearable wireless device 117. The display unit 124 includes a liquid crystal display panel and displays a communication status with the imaging device 100. The display unit 124 also displays information such as a setting menu for the wearable wireless device 117, a recording time, a remaining battery level, and the like. The memory 125 is used as a work area for the CPU 124 and the display unit 123.

  Here, the normal operation of the imaging system of the present embodiment will be described. If the user performs an operation for starting the shooting mode from the operation unit 112, the CPU 114 is instructed to start the shooting mode and enters a shooting standby state. In a shooting standby state, when the user operates the recording start button of the operation unit 112 to transmit support for shooting start, the CPU 114 transmits a shooting start control signal to each block when a shooting start instruction signal is transmitted. Is controlled to perform the following operations.

  The optical image of the subject captured by the image sensor 101 is converted into an image signal and converted into a digital signal, the image processing circuit 103 is processed based on the set value, and is transmitted to the display unit 113. The display unit 113 displays a video related to the received image signal. The image signal is temporarily stored in the image memory 104. An analog audio signal input from the microphone 105 is converted to digital by the audio processing circuit 106, appropriate audio characteristics are processed, and temporarily stored in the PCM buffer 107. Thereafter, the image signal stored in the image memory 104 and the audio signal data stored in the PCM buffer 107 are multiplexed and temporarily stored in the stream buffer 108.

  At this time, in the imaging apparatus 100 of this embodiment, if the communication unit 119 receives the digital audio signal transmitted from the wearable wireless device 117, the digital audio signal is transmitted from the communication unit 119 to the audio processing circuit 106. The Then, the audio processing circuit 105 mixes the audio data obtained from the microphone 105 and the audio signal transmitted from the communication unit 119, and temporarily stores them in the PCM buffer 107 as one audio signal. Then, the write control unit 117 reads the stream stored in the stream buffer 108 and records it on the memory card 109 through the recording medium control circuit 111 as one moving image file.

  The above operation is continued during shooting. Then, when a shooting end instruction signal is sent by the user operating the operation button of the operation unit 112, the control unit transmits it to each block. If data remains in the stream buffer 108 and the PCM buffer 107 as in the case of shooting, encoding and writing to the memory card are continued, and when there is no data, the moving image file is completed and the recording operation is stopped. When the recording operation is completed, the camera returns from the shooting state to the shooting standby state.

  As illustrated in FIG. 2, the imaging apparatus 100 according to the present exemplary embodiment can perform communication permission information registration with the wearable wireless apparatus 117 at an arbitrary timing other than the recording state. In S201 in the figure, the communication unit 111 of the imaging device 100 and the communication unit 121 of the wearable wireless device are activated, and a transmission / reception operation is performed to determine whether the terminal can be registered as a communication method. If it is determined that the terminal can communicate (Yes in S202), the image pickup device 101 of the image pickup apparatus 100 is activated.

  The face of the person wearing the wearable wireless device 117 is photographed by the operation of the operation unit 112 to acquire facial feature information, and pairing information of the wearable wireless device 117 is generated, and registration data (S205) shown in S204 is obtained. ) Is stored in the memory 118. The pairing information includes the unique ID of the wearable wireless device 117 shown in S301 and the feature information of the photographed face. This information is used when the wearable wireless device 117 determines communication permission.

  In this series of shooting operations, the sound mixing determination process will be described with reference to FIGS. 4, 5, 6, and 7.

  FIG. 4 is a diagram for explaining a determination flow of whether or not the wearable wireless device 117 and the imaging device 100 communicate with each other and an audio signal is transmitted from the wearable wireless device 117 to the imaging device 100. FIG. These are the assumption figures of the spectacles-type wearable wireless apparatus 117 provided with the assumed display part 123, the microphone 118, and the communication part 111. FIG.

  FIG. 6 is a flowchart of processing of display data to be transmitted to the wearable wireless device 117 when communication or voice data transmission permission is determined, and FIG. 7 is an actual display example of the display unit 122. This process may be executed as long as the wearable wireless device 117 and the imaging device 100 can communicate with each other in the recording standby state and the recording state.

  When the recording device is set to the recording standby state and the recording state, in S401, the image data obtained from the image sensor 101 is analyzed to determine whether a face image is present in the image captured by the image processing circuit 102. . If a face image is detected (Yes in S401), it is analyzed in S402 whether an image signal correlated with the face image 301 corresponding to the registered wearable wireless device 117 is included. When a face image highly correlated with the face image 301 is detected (Yes in S402), a plurality of face image detection analyzes can be performed.

  For example, in the state of 502 in FIG. 5, it is determined that there is one subject having a registered ID face image. In the case of 503, it is determined that there are two subjects having a registered ID face image. Since two face images are detected but there is only one registered subject, it is determined that there is one detected subject.

  Therefore, when only one face image having a high correlation is found (states 502 and 504 in FIG. 5), the one wearable wireless device 117 is mounted within the angle of view of the imaging device 100. Assuming that the wireless device is a wireless device, communication with the wearable wireless device having the wearable wireless device ID 302 is attempted. When communication can be started (S403: Yes), an audio signal is transmitted from the communication unit of the wearable wireless device 117 to the imaging device 100, and the audio processing circuit 107 performs synthesis processing.

  When a plurality are found (state 503 in FIG. 5), the face image and age information registered with the ID of the wearable wireless device with which communication was possible are extracted. Priorities are given by weighting the subject distance obtained from the size of the face image divided by age and the focus position shown in 505 and 506. For example, the face image with ID0001 is 35% of the angle of view (weight: 2.5), the focusing distance is 1.5 m (weight: 4), and the face image of ID0002 is 30% of the angle of view (weight: 3). If the in-focus distance is 2.5 m (weight: 3), the weights are summed and compared (ID0001: 6.5, ID0002: 6). Since ID0001 is higher, the subject of interest is the subject of ID0001. Become.

  ID0001 is determined to be the most noticeable subject, and the wearable wireless device 117 is estimated to be a wearable wireless device within the angle of view of the imaging device 100, and an audio signal is transmitted from the communication unit of the wearable wireless device 117 to the imaging device 100. Then, the voice processing circuit 107 performs synthesis processing. These operations are continued during the photographing operation and the photographing standby operation.

  In addition, when the user presses the operation start button on the operation unit 112 in order to operate the communication unit 111 when the imaging apparatus 100 is in the shooting standby operation and when the imaging operation is performed, the communication standby mode is set. Unless this mode is set, communication is not performed even in a state where communication with the imaging apparatus 100 is possible. When the communication mode is entered, the display unit 130 is in a state of waiting for communication with the imaging device 100 to indicate to the subject wearing the wearable wireless device 117 that it is a start-up. To do. In the embodiment, as indicated by reference numeral 701, “active” is displayed.

  Next, when it is determined in S403 that communication is possible, the display unit 130 displays that the recording is on standby. In the embodiment, as indicated by reference numeral 702, “standby” is displayed. Next, when it can be determined in S404 that there are not a plurality of objects, the object is immediately registered as a recording object and communication is started. In that case, the display unit 130 displays that recording is in progress. In the embodiment, as indicated by reference numeral 703, “recording” is displayed.

  Here, as described above, the image processing circuit 114 of the present embodiment analyzes the image signal in order to determine whether the detected face information is a specific subject corresponding to the wearable wireless device 117 in which the detected face information is registered. Is going. Then, it is detected whether an image highly correlated with the registered subject is included. Here, a technique for discriminating (detecting) a specific person's face is representative of a technique for discriminating (detecting) a specific person from deviations from positions of the average person's face contour, eyes, nose, mouth, and the like. There are various techniques, and since it is not the main line of the present invention, the description is omitted here.

  As described above, even if there are a plurality of wearable wireless devices, it is possible to select a wearable wireless device that is appropriately focused and to communicate audio data of a specific subject to the imaging device. With this configuration, it is possible to receive audio from the wearable wireless device that is securely within the angle of view of the imaging device, and even when multiple subjects are detected, the audio from the subject of interest Therefore, it is possible to perform a shooting operation with appropriate audio data without mixing communication with a plurality of wireless devices or mixing audio data. In addition, since the audio signal data is transmitted / received by the imaging apparatus only from one wireless device after selection is determined, the amount of received data of the imaging apparatus can be reduced.

  In the present invention, when a plurality of wireless microphones are detected by using face detection or focusing information, a desired wireless microphone is selected and then only desired audio signals of the desired wireless microphone are communicated. By making it possible to record with emphasis on the signal, it is possible to shoot moving images while preventing noise caused by mixing of a plurality of audio data and a decrease in the speed of wireless communication.

100 imaging device, 101 imaging device, 102 imaging device driving circuit,
103 image processing circuit, 104 image memory, 105 microphone,
106 audio processing circuit, 107 PCM buffer, 108 stream buffer,
109 memory card, 110 write control unit, 111 communication unit, 112 operation unit,
113 display unit, 114 CPU, 115 file system, 116 memory,
117 Wearable wireless device, 118 microphone, 119 sound processing circuit,
120 PCM buffer, 121 communication unit, 123 operation unit, 124 display unit,
125 CPU, 126 memory

Claims (7)

An image pickup apparatus having receiving means (111, 121) for receiving an audio signal transmitted from an external sound collecting apparatus by wireless communication, and storing means for storing information relating the sound collecting apparatus to a specific subject ( 117), an image pickup means (101, 102, 103, 104) for picking up an optical image within a predetermined shooting angle of view and acquiring an image signal, and the specific shooting angle of view within the predetermined shooting angle of view of the imaging means. An estimation means (114) for estimating whether or not a subject is present; a selection means (114) for selecting a subject to be used for sound collection by paying attention to image information when there are a plurality of specific subjects;
According to the selection result of the selection means, the means (111) for receiving the sound signal output obtained from the sound collecting device and the sound signal input obtained from the sound collecting device in the imaging device are synthesized. An image pickup apparatus comprising: means (106); and recording means (104, 107, 108, 109, 110) for associating and recording the image signal obtained by the image pickup means on a recording medium. 2. The imaging apparatus according to claim 1, wherein the storage unit captures the subject associated with the sound collection apparatus with the imaging apparatus, stores the face information, and stores the information together with the ID and age information of the sound collection apparatus. The estimation means determines whether the specific image within the predetermined shooting angle of view of the imaging means depends on whether an image highly correlated with the face image of the specific subject is detected in the image signal obtained by the imaging means. The imaging apparatus according to claim 1, wherein it is estimated whether or not a subject is detected. When it is estimated that the specific subject is detected within the imaging field angle, the selection unit detects the ratio of all the estimated subjects to the imaging field angle, and calculates the degree of attention according to the ratio. The imaging apparatus according to claim 1, wherein a subject with the highest degree of attention is selected. When it is estimated that the specific subject is detected within the imaging angle of view, the selection unit calculates focus information for all the estimated subject positions, calculates the degree of attention according to the in-focus distance, and calculates the most attention. The imaging apparatus according to claim 1, wherein an object having a high degree is selected. The degree of attention in the selection means is calculated by changing the weighting based on the age information stored in the storage means of the subject estimated to be detected within the imaging angle of view, and the subject with the highest degree of attention according to the result is calculated. 6. The imaging device according to claim 4, wherein the imaging device is selected. The receiving means receives information transmitted by a plurality of identifiable wireless devices and selects the wireless device in the selected selecting means according to claim 5. In accordance with the selection result, only the audio signal of the determined wireless device is received, and the recording unit synthesizes the audio signal received from the wireless device and the audio signal of the imaging device.