JP2007312181A - Imaging sound pickup signal reproduction system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To attain an imaging sound pickup signal reproduction system with which recognition and judgment of an abnormality by a viewer and listener are surely performed and a video is output by being switched to a video at a monitoring point where the abnormality is generated. <P>SOLUTION: An imaging sound pickup device 1 is constituted by providing an imaging sound pickup part 11 having a cylindrical part 116 with a plurality of cameras 113a-113d and a plurality of microphones 112a-112h which pick up binaural sound and a selection part 12 which outputs a sound signal of a video signal set as a main video signal from among a plurality of video signals obtained from the plurality of cameras as a selected sound signal, a playback device 3 has a video signal output part 33 which sets the main video signal to be output to a display means 42 and a crosstalk canceler 34 which processes a crosstalk signal so as to be canceled by using a filter in which a head-related transfer function measured by using a cylindrical sound receiving object 64 is stored and outputs the selected sound signal to right and left speakers. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention has a plurality of imaging cameras for shooting a plurality of subjects and a plurality of binaural sound pickup microphones for binaural sound pickup with the shooting direction of each imaging camera as the front, while binaural sound pickup is performed. The present invention relates to a picked-up image pickup signal reproduction system that reproduces audio signals with left and right speakers arranged in front of a viewer.

Recently, the performance of surveillance cameras has improved, and surveillance cameras have been used in many places, such as surveillance of factory equipment, surveillance of cash register departments at convenience stores, etc., surveillance of merchandise stores, etc. .
On the other hand, if one surveillance camera pans the surroundings where the camera is installed, surveillance can be performed effectively with a wide viewing angle. When binaural sound pickup is performed in the monitoring area, the monitoring area can be effectively monitored by a high-quality monitoring video and a realistic reproduction sound field. Further, when an abnormality is found during monitoring, the monitoring camera can be immediately switched to a subject that is abnormal and the monitoring area can be monitored. It is preferable that the sound to be binaurally picked up can be switched in the sound collecting direction as needed by switching the monitoring camera.

Patent Document 1 discloses a multipoint remote monitoring system that makes it easy to specify a monitoring area where an abnormality has occurred while monitoring a plurality of monitoring targets with a plurality of monitoring cameras. A virtual sound space having a two-dimensional spread corresponding to the monitoring points A, B, and C of the monitored station is defined around 360 degrees around the user in the monitoring room, and the sound information is located around 360 degrees around the user. A multi-point remote monitoring system is disclosed in which sound image localization processing is performed so that localization is performed, the point of actual sound information is notified to the user from the sound image localization position, and the delay of abnormality detection is improved.
JP 10-51759 A

  However, in the multipoint remote monitoring method disclosed in Patent Document 1, a plurality of monitoring points are rearranged as a virtual space in the monitoring room, and the direction in which sound can be heard in the monitoring room is recognized as a point where an abnormality has occurred. I am trying to let you. After the recognition, the video and audio of the monitoring area recognized by the user are switched and output to the display to search for an abnormality in the monitoring area. In this multipoint remote monitoring method, after the user recognizes the abnormal sound, the video and the sound are switched and output according to the recognition, so when the recognition and judgment are ambiguous, the discovery of the cause of the abnormality is delayed. . It has not been possible to realize a multi-point remote monitoring system that can immediately switch and output the monitoring video and the monitoring sound to the monitoring point where the abnormality has occurred after the abnormality has occurred.

  Therefore, the present invention has been made to solve the above-described problems, and it is possible to reliably recognize and judge the abnormality by the viewer. It is an object of the present invention to provide an image pickup / acquisition signal reproduction system capable of switching to a generated monitoring point and outputting it.

  According to a first aspect of the present invention, there is provided an image pickup device that outputs a video signal picked up by a camera and an audio signal picked up by a microphone, and the video signal supplied from the image pickup device. In the imaging sound pickup signal reproduction system, the image pickup sound collection device includes: a reproduction device that outputs the sound signal supplied from the image pickup sound collection device to the left and right speakers. When a plurality of cameras and one of the plurality of cameras is used as a shooting camera, the pair of cameras are arranged at a predetermined angle leftward and rightward with respect to the shooting camera and have the same height from the bottom surface. Binaural sound picked up by left and right microphones is picked up as binaural sound for each of the plurality of cameras as a sound signal accompanying a video signal picked up by the shooting camera. An image pickup / sound pickup unit having a cylindrical portion having a plurality of microphones, and an audio accompanying a video signal set as a main video signal on the playback device side from a plurality of video signals obtained from the plurality of cameras A selection unit that outputs a signal as a selected audio signal, wherein the playback device sets one of the plurality of video signals as the main video signal and displays the video signal set as the main video signal A video signal output unit for generating a main video signal for output to the display means, and a signal reproduced from the left and right speakers when the selected audio signal is supplied and the selected audio signal is reproduced by the left and right speakers A crosstalk canceller that processes to cancel the crosstalk signal viewed at the right and left ears of the viewer and outputs to the left and right speakers The crosstalk canceller includes a filter that stores a head-related transfer function obtained in advance, and the head-related transfer function converts the measurement signal into an audio signal collected by a pair of microphones attached to a cylindrical structure. An imaging sound pickup signal reproduction system characterized by being a head-related transfer function measured based on the above is provided.

  According to the present invention, the imaging and sound collection device is arranged in a left direction and a right direction at a predetermined angle with respect to the photographing camera when one of the plurality of cameras is a photographing camera, and A plurality of microphones for collecting binaural sound for each of a plurality of cameras as an audio signal accompanying a video signal obtained by photographing a pair of left and right microphones having the same height from the bottom surface with a photographing camera. An imaging sound collection unit having a cylindrical portion provided, and an audio signal accompanying a video signal set as a main video signal on the playback device side among a plurality of video signals obtained from a plurality of cameras is output as a selection audio signal A reproducing unit configured to set one of a plurality of video signals as a main video signal and display from the video signal set as the main video signal. A video signal output unit that generates a main video signal for output to the display means, and a selection audio signal is supplied, and when the selection audio signal is reproduced by the left and right speakers, the signals reproduced from the left and right speakers are A crosstalk canceller that processes to cancel the crosstalk signal viewed at the right and left ears and outputs it to the left and right speakers, and the crosstalk canceller stores the head-related transfer function obtained in advance. Since the head-related transfer function is a head-related transfer function measured based on the audio signal collected by the pair of microphones mounted on the cylindrical structure, the head-related transfer function has a filter. An imaging sound pickup signal reproduction system that makes it possible to make a reliable judgment and immediately switch and output the monitoring video and the monitoring sound to the monitoring point where the abnormality has occurred after the abnormality has occurred. It can be realized Temu.

Hereinafter, an image pickup and reproduction system according to an embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a block diagram illustrating a configuration example of an imaging sound collection / reproduction system according to an embodiment of the present invention. FIG. 2 is a diagram (part 1) illustrating a configuration example of a main part of the image pickup and reproduction system according to the embodiment of the present invention. FIG. 3 is a diagram for explaining a configuration example of a main part of the image pickup and reproduction system according to the embodiment of the present invention. FIG. 4 is a diagram (part 2) illustrating a configuration example of a main part of the image pickup and reproduction system according to the embodiment of the present invention. FIG. 5 is a diagram showing a display example of a reproduced image according to the embodiment of the present invention. FIG. 6 is a diagram (No. 3) illustrating a configuration example of a main part of the image pickup and reproduction system according to the embodiment of the present invention. FIG. 7 is a diagram illustrating a configuration example of a filter characteristic measuring apparatus of a crosstalk canceller according to an embodiment of the present invention. FIG. 8 is a diagram illustrating a configuration example of the binaural sound receiving unit used when the filter characteristics are measured according to the embodiment of the present invention. FIG. 9 is a diagram illustrating an example of impulse response characteristics of the crosstalk canceller according to the embodiment of the present invention. FIG. 10 is a diagram illustrating an example of frequency response characteristics of the crosstalk canceller according to the embodiment of the present invention. FIG. 11 is a diagram showing an example of impulse response characteristics of a conventional crosstalk canceller for reference. FIG. 12 is a diagram showing an example of frequency response characteristics of a conventional crosstalk canceller.

  The image pickup / acquisition signal reproduction system can surely recognize and judge the abnormality by the viewer, and after the abnormality occurs, it is possible to immediately switch and output the monitoring video and the monitoring sound to the monitoring point where the abnormality has occurred. For the purpose of realizing an image pickup / acquisition signal reproduction system, an image pickup / collection device has a plurality of cameras and a left angle of a predetermined angle with respect to the image pickup camera when one of the plurality of cameras is set as an image pickup camera. Binaural audio picked up by a pair of left and right microphones arranged in the right direction and having the same height from the bottom is collected as binaural audio for each of the multiple cameras as an audio signal accompanying the video signal shot by the shooting camera. An image pickup / sound pickup unit having a cylindrical portion with a plurality of sounding microphones and a plurality of video signals obtained from a plurality of cameras. A selection unit that outputs an audio signal accompanying the video signal set as the main video signal as the selected audio signal, and the playback device sets one of the plurality of video signals as the main video signal, A video signal output unit that generates a main video signal for display from the video signal set as the video signal and outputs it to the display means, and a selection audio signal is supplied, and when the selection audio signal is reproduced by the left and right speakers, A crosstalk canceller that processes the signal reproduced from the speaker so as to cancel the crosstalk signal that is viewed by the right and left ears of the viewer and outputs the signal to the left and right speakers. The head-related transfer function has a filter that stores the obtained head-related transfer function. The head-related transfer function is a voice signal obtained by collecting a measurement signal with a pair of microphones attached to a cylindrical structure. It was achieved as is HRTFs measured based on.

The configuration of the image pickup sound signal reproduction system will be described.
An image pickup sound collection signal reproduction system 10 shown in FIG. 1 includes an image pickup sound collection device 1 including an image pickup sound collection unit 11, a selection unit 12, a compression unit 13, and a transmission unit 14, a transmission path 2, a reception unit 31, and expansion. The playback device 3 includes a unit 32, a signal control unit 33, and a crosstalk canceller 34, and the display unit 4 includes speakers 41 and 43 and a monitor 42.
The imaging sound collection unit 11 illustrated in FIG. 2 includes an auricle 111a to 111d, microphones 112a to 112d, imaging cameras 113b and 113c, and a structure 117.
The imaging sound collection unit 11a illustrated in FIG. 4 includes an auricle 111a to 111h, microphones 112a to 112h, imaging cameras 113a to 113d, and a cylindrical structure 116.
The crosstalk canceller 34 shown in FIG. 6 includes filters 341 to 344, adders 345 and 346, and filters 347 and 348.
The filter characteristic measuring apparatus 6 shown in FIG. 7 includes a personal computer 61, amplifiers 62, 65, 66, a speaker 63, and a cylindrical sound receiver 64.
A cylindrical sound receiving body 64 shown in FIGS. 8A and 8B includes microphones 641 and 642 and a cylindrical structure 649, and a human head type sound receiving body 69 shown in FIG. 8C includes microphones 691 and 692. , Ear canals 693 and 694, auricles 695 and 696, and a human head structure 699.

  First, the imaging / sound collecting unit 11 of the imaging / sound-collecting apparatus 1 installed in the monitoring area captures a monitoring image with the four directions of the imaging / sound collecting unit 11 in front, left, rear, and right as monitoring areas. In addition, a plurality of binaural sounds having a plurality of directions as front directions are collected. The selection unit 12 is controlled by the signal control unit 33 of the playback device 3 to recognize a front image as a main image and other left, rear, and right three-direction images as sub-images, and to capture and collect images. Of the plurality of binaural sounds picked up by the unit 11, the binaural sound picked up with the direction in which the main image is taken as the front is selected and output as a sound signal. A plurality of images and binaural sound collected by the imaging sound collection unit 11 are compression-coded by the compression unit 13 and sent from the sending unit 14 to the transmission path 2. The receiving unit 31 of the reproduction apparatus 3 receives the image signal and the audio signal sent from the imaging / acquisition / collection apparatus 1, and decodes and obtains a plurality of images and binaural audio that have been compression-encoded by the expansion unit 32. The signal control unit 33 generates a multi-image including a main image and a sub-image and supplies it to the monitor 42 for display. The obtained binaural sound is cross-talked by the cross-talk canceller 34. Canceling processing is performed, and the sound is supplied to the left and right speakers 41 and 43 to be sounded. The viewer views the multi-image displayed on the monitor 42 by binaural sound collected with the main image as the front direction.

  The binaural sound is subjected to crosstalk cancellation processing by a crosstalk canceller 34, which will be described later, and is produced from the speakers 41 and 43. The binaural sound that has been subjected to the crosstalk cancellation processing is a signal in which a cancellation signal for canceling the crosstalk sound generated by the left (right) speaker and received by the viewer's right (left) ear is added in advance. Is supplied to the speaker and pronounced. Accordingly, the viewer can listen to the binaural audio signal as rich binaural sound similar to the case of listening to the binaural audio signal with headphones while reproducing the speaker.

  Next, for example, when an abnormal sound is received from the left side of the listening sound, the listener performs an operation on the signal control unit 33 to set the image captured in the left direction as the main image. The signal control unit 33 transmits a control signal having the left direction image as the main image to the selection unit 12. The selection unit 12 recognizes the left image as a main image and the other three images as sub-images, obtains binaural sound collected with the left side of the imaging sound collection unit 11 as the front, and obtains an imaging sound collection device in the same manner as described above. Send from 1. The monitor 42 displays a main image in front of the left direction, and binaural sounds with the left direction in front are reproduced from the speakers 41 and 43. The monitoring area can be monitored by binaural sound with the direction in which the abnormal sound is generated as the main image and the direction of the main image as the front.

Next, this will be described in detail.
The imaging sound collection unit 11 of the imaging sound collection device 1 will be described with reference to FIG.
FIG. 4A is an elevation view of the image pickup sound collecting unit 11, and each of the four image pickup cameras 113 a to 113 d provided around the structure 117 performs image pickup of four monitoring areas. The structure 117 has an acoustic signal shielding characteristic that is substantially equivalent to that of the viewer's head. Around the structure 117, auricles 111 a to 111 d and microphones 112 a to 112 f are provided, and two auricles and microphones that are arranged to face each other are used. Two binaural sounds are picked up with two of the directions in front.
FIG. 4B is a front view of the imaging sound pickup unit 11. When the imaging camera 113c is set as a main image capturing camera, binaural sounds are collected using the auricles 111b and 111d and the microphones 112b and 112f. The binaural sound to be collected is binaural sound with the median plane in the direction in which the imaging camera 113c is directed.

With reference to FIG. 3, the auricle of the imaging sound pickup unit 11 will be further described.
FIG. 2A shows only the auricle and microphone used for binaural sound collection when the imaging camera 113b is set as the main image capturing camera, as compared with FIG. 2A. The direction in which the photographing camera 113b is directed is the front. The auricles 111a and 111c and the microphones 112a and 112c provided on the side surface of the structure 117 perform binaural sound collection using sound emitted from a subject imaged in front of the imaging camera 113b as front sound. The auricles 111a and 111c are located on the left side of the figure. Sounds emitted from the subject located on the right side of the structure 117 are collected by the auricles 111a and 111c and input to the microphones 112a and 112c. The sound emitted from the left side (rear side) of the structure 117 is input to the microphones 112a and 112c after the high frequency components are attenuated by the auricles 111a and 111c.

  FIG. 3C shows the movement of the auricle 111c when the main image capturing camera is switched from the imaging camera 113b to the imaging camera 113d. The auricle 11c has a shape obtained by dividing a sphere into four parts, and the auricle 11c rotates 90 degrees around a center line indicated by a one-dot chain line in the figure. In the state where the pinna 11c is rotated 90 degrees, the left and right of the binaural sound output from the microphones 112a and 112c are switched. The auricles 111a and 111c and the microphones 112a and 112c perform binaural sound collection using a sound emitted from a subject imaged by the imaging camera 113d as a front sound.

An application example of the imaging sound collection unit will be described with reference to FIG.
The image pickup / sound pickup unit 11a shown in the figure has the auricles 111a to 111d and the microphones 112a to 112d on the circumference of the upper part of the cylindrical structure 116, the image pickup cameras 113a to 113d on the circumference of the center, and Auricles 111e to 111h and microphones 112e to 112h are provided on the lower circumference. Here, the auricles 111a to 111d and the auricles 111e to 111h are directed in opposite directions.

  The auricle of the imaging sound collection unit 11 shown in FIG. 2 is movable and reverses the sound collection direction, whereas the auricle of the imaging sound collection unit 11a shown in FIG. Since four pairs of auricles and microphones are provided, binaural audio can be switched by simply switching audio signals output from the respective microphones. The imaging sound collection unit 11 can collect binaural sound in four directions only by using the four auricles 111a to 111d and the microphones 112a to 112d. The imaging sound collecting unit 11a collects binaural sounds in four directions using eight auricles 111a to 111h and microphones 112a to 112h. Since the imaging sound collecting unit 11a further includes four microphones and the like as compared with the imaging sound collecting unit 11, it is not necessary to rotate the auricles 111a to 111h when switching the sound collecting direction, and a pinna switching mechanism is provided. Without switching, the switching is instantaneous and no frictional sound is generated when the pinna is switched.

A multi-image displayed on the monitor 42 will be described with reference to FIG.
FIG. 5A has (a) a large screen display area and (b) to (d) small screen display areas. Video signals output from the imaging cameras 113a to 113d are displayed in the display areas (a) to (d), respectively. This is a display example when the video signal output from the imaging camera 113a is designated as a main image and the video signals of the imaging cameras 113b to 113d are selected as sub-images. In FIG. 5B, the large screen display area is (b), and the small screen display areas are (a), (c), and (d). This is a display area setting example when a video signal output from the imaging camera 113b is designated as a main image and a video signal output from another imaging camera is selected as a sub-image.

The crosstalk canceller will be described with reference to FIG.
First, of the binaural audio signal, the left channel signal P _L (t) is input to the filters 341 and 342, and the right channel signal P _R (t) is input to the filters 343 and 344. The filters 341 to 344 sequentially have electroacoustic conversion characteristics of characteristics h _rs (t), −h _lo (t), −h _ro (t), and h _ls (t) related to the head-related transfer functions described later. Give these characteristics to the input signal. The adder 345 adds the signals output from the filters 341 and 343, and the filter 347 gives a characteristic of d (t) to the added signal. The adder 346 adds the signals output from the filters 342 and 344, and the filter 348 gives a characteristic of d (t) to the added signal. Signals output from the filters 347 and 348 are output signals of the crosstalk canceller 34. These output signals are amplified by an amplifier (not shown), supplied to speakers 41 and 43, and sounded.

  A signal sounded from the speaker 41 is received by the viewer's 48 left ear, and part of the sounded signal is received by the viewer's 48 right ear as a first crosstalk signal indicated by a broken line. The crosstalk canceller 34 generates a first crosstalk cancel signal for canceling the received first crosstalk signal and causes the speaker 43 to generate a sound. The crosstalk signal received by the right ear is canceled (attenuated) by the first crosstalk cancellation signal. Similarly, a second crosstalk cancellation signal for canceling the second crosstalk signal, which is generated by the speaker 43 and is received by the viewer 48 with the left ear of the viewer 48, is generated from the speaker 41.

Note that the first crosstalk cancellation signal generated from the speaker 43 is further received as a crosstalk component by the viewer's 48 left ear. The crosstalk component of the first crosstalk cancellation signal is viewed with the left ear together with the binaural audio signal P _L (t) of the left channel. Since both are similar signals, there is little effect on listening quality. The crosstalk component of the second crosstalk cancellation signal generated from the speaker 41 is viewed with the right ear together with the binaural audio signal P _R (t) of the right channel. Similarly, the influence of the crosstalk component of the second crosstalk cancellation signal on the listening quality is small.

The filter characteristic measuring apparatus 6 for obtaining the head-related transfer function characteristics stored in the filters 341 to 344, 347, and 348 will be described with reference to FIG.
First, a measurement signal such as an impulse sound having a raised cosine waveform is generated by the personal computer 61. The measurement signal is amplified by the amplifier 62. The measurement signal generated from the speaker 63 is received by the left and right microphone units 641 and 642 attached to the cylindrical sound receiver 64. The left and right signals obtained by receiving the sound are amplified by the amplifiers 66 and 67 and input to the personal computer 61. The personal computer 61 compares the generated measurement signal with the received signal, and the head of the signal generated by the speaker 63 and received by the left and right microphone units 641 and 642 attached to the cylindrical sound receiver 64. Transfer functions h _ls (t) and h _lo (t) are obtained. h _ls (t) is a characteristic in which a signal generated from the left speaker is received by the left microphone unit, and h _lo (t) is a signal generated by the left speaker received by the right microphone unit. This is a characteristic related to crosstalk.

Here, only the left speaker 63 is shown in FIG. 7, but the right speaker is placed in a symmetrical position and measured in the same manner, and the head related transfer functions h _ro (t) and h _rs are measured. (T) is determined. h _rs (t) is a characteristic generated by the right speaker and heard by the right microphone unit, and h _ro (t) is a characteristic generated by the right speaker and received by the left microphone unit.

The cylindrical sound receiver 64 will be described with reference to FIG. (A) is a top view and (B) is a perspective view.
Microphone units 641 and 642 are provided on both sides of the cylindrical sound receiving body 64 shown in FIG. Each microphone unit 641 and 642 does not have an auricle or an external auditory canal. The diaphragms (not shown) of the microphone units 641 and 642 are disposed on the surface of the cylindrical sound receiving body 64. (C) shown as a reference is a top view of a so-called human head type sound receiver. The human head type sound receiving body 69 is provided with pinna members 695 and 696 and ear canals 693 and 694 on both sides of the human head structure 699, and microphones 691 and 692 are provided on the back side of the ear canal. It has been. The microphones 691 and 692 are provided at positions corresponding to the positions of the human eardrum, and attempt to pick up sound signals that approximate the sound that a person listens to.

  The cylindrical sound receiver 64 to which the microphone units 641 and 642 shown in (A) and (B) are attached does not have an auricle or an external auditory canal compared to the human head-type sound receiver 69 shown in (C). . The sound receiving characteristics of the microphone units 641 and 642 attached to the cylindrical structure 649 are not affected by the characteristic difference given by the pinna or the ear canal having different shapes, and the head-related transfer function can be measured. It is a characteristic. The characteristics measured by using the microphone units 641 and 642 are that the sound wave generated from the speaker 63 is shielded by the cylindrical structure 649, but diffracts along the surface of the cylindrical structure 649 and is reflected on the microphone unit 642. It is measured as a characteristic of the incoming sound wave. By using the cylindrical structure 649, it is possible to obtain a head-related transfer function having an average head shielding characteristic with respect to different head shielding characteristics of a plurality of head-shaped viewers.

FIG. 9 shows respective waveforms related to the head-related transfer function obtained by measurement using the impulse sound by the acoustic signal transfer characteristic measuring device 6. (A)-(D) are waveform diagrams of h _ls (t), h _lo (t), h _rs (t), and h _ro (t) in this order. (E) is a waveform diagram of d (t). The vertical axis represents the amplitude of the signal voltage normalized with a predetermined output voltage, and the horizontal axis represents the time indicated by the number of samples when the measured signal is sampled at 48 kHz.
10A to 10E show frequency characteristics obtained by Fourier analysis of the signal shown in FIG. In the figure, the frequency positions of 100 Hz, 1 kHz, and 10 kHz are indicated by broken lines. The gain difference between the broken line and the broken line in the response characteristic is 10 dB.

Characteristic of the filter _{341~344 h rs (t), -} h lo (t), - h ro (t), h ls (t) is obtained by measuring an acoustic signal transfer characteristics measuring device 6 characteristic h _ls ( The characteristics (reverse polarity) obtained by inverting the signs of t) and h _rs (t) and h _lo (t) and h _ro (t) are used.
The characteristic d (t) of the filters 347 and 348 is given by
d (t) = {h _ls (t) * h _rs (t) −h _lo (t) * h _ro (t)} ⁻¹
Here, “*” is an arithmetic symbol indicating multiplication.
Each filter characteristic obtained as described above is stored in a storage unit (not shown) provided in the filter, and a predetermined filter characteristic is given by convolving the characteristic read from the storage unit with an input signal.

FIG. 11 and FIG. 12 show characteristics measured using the human head-type sound receiver 69 shown in FIG. 8C instead of the microphone units 641 and 642 attached to the cylindrical structure 649.
FIG. 11 shows characteristics obtained by measurement in the same manner as in FIG. The impulse response waveform measured using the human head-type sound receiver 69 is closer to the raised cosine, which is the measurement input signal, than the waveform obtained using the microphone units 641 and 642, compared to the waveform containing a lot of vibration. It is a waveform. The negative voltage portion has a small amplitude, and the subsequent vibration waveform has a small amplitude.

  FIG. 12 shows frequency response characteristics corresponding to FIG. 11 measured using the human head type sound receiver 69. The characteristics obtained by using the microphone units 641 and 642 attached to the cylindrical sound receiving body 64 are less flat in the frequency characteristics and are almost flat. Each characteristic of (A) to (E) shown in FIG. 12 is enhanced and attenuated between frequencies of 1.5 to 7 kHz, and the characteristic shown in FIG. 10 is more enhanced and attenuated. The level is small. This is because there is no disturbance in characteristics caused by the pinna or the external auditory canal. When the microphone units 641 and 642 are used, a part of the sound wave generated by the speaker 63 is reflected by the auricle, and the reflected sound wave and the directly arriving sound wave are combined in the same phase to be enhanced or reversed phase. It has no effect of being synthesized and attenuated, and is obtained as a small characteristic that is further enhanced or attenuated at a specific frequency due to the effects of resonance and anti-resonance of the ear canal.

A description will be given by comparing the difference in the crosstalk cancellation characteristics obtained by the difference in the characteristics of the filters 341 to 344.
The crosstalk cancellation characteristic was measured using the binaural audio speaker reproduction system shown in FIG. The characteristics of the filters 341 to 344 and the filters 347 and 348 of the crosstalk canceller 34 in the system are the characteristics obtained using the microphone units 641 and 642 attached to the cylindrical sound receiver 64, and the human head sound receiver. A viewing test by a plurality of viewers was performed while exchanging the characteristics obtained by using 69. The viewer inserted a thin small microphone into the ear canal and measured that the sound received by the small microphone was the sound received by the viewer. When the characteristics of the filters 341 to 344 and the filters 347 and 348 are measured using the characteristics obtained by the microphone units 641 and 642 attached to the cylindrical sound receiving body 64, the separation performance (crossover) exceeding 100 dB at 100 Hz to 2 kHz. Compared to the case where the talk canceling effect is obtained, when the characteristics obtained with the human head type sound receiving body 69 are used, only a separation degree of about 12 dB can be obtained.

Obtaining a sufficient degree of separation means that the crosstalk cancellation signal is operating effectively, and that the crosstalk cancellation signal is not viewed as a reverse phase component. The binaural signal is picked up by a viewer with a suitable presence. When an abnormal sound occurs, the viewer can recognize the approximate occurrence location even if the location is a location that is not displayed on the main screen displayed on the monitor 42.
Here, it has been described that the multi-screen shown in FIG. 5 is generated by the signal control unit 33 of the playback apparatus 3. The multi-screen may be generated by the selection unit 12 of the imaging sound pickup apparatus 1. In that case, since the multi-screen is compression-encoded by the compression unit 13 and transmitted through the transmission path 2, the code amount for transmitting the sub-screen can be reduced.
Further, if the image pickup / recording apparatus 1 has a video signal recording function and the sub-screen image picked up by the image pickup / collection unit 11 is recorded without being reduced, the recorded image is caused by the subsequent generation of abnormal noise. Useful for analysis.

  As described above, according to the imaging / acquisition / acquisition signal reproduction system shown in the present embodiment, the imaging / acquisition / acquisition apparatus 1 uses a plurality of cameras 113a to 113d and one of the plurality of cameras as a shooting camera. Audio that accompanies the video signal captured by the camera for binaural sound picked up by a pair of left and right microphones arranged at a predetermined angle left and right with respect to the camera for shooting and having the same height from the bottom Image pickup and pickup unit 11 having cylindrical portion 116 having a plurality of microphones 112a to 112h for collecting binaural sound for each of a plurality of cameras as a signal, and a playback device from a plurality of video signals obtained from the plurality of cameras And a selection unit 12 that outputs an audio signal accompanying the video signal set as the main video signal on the side as a selected audio signal. 3 is a video signal output unit 33 that sets one of a plurality of video signals as a main video signal, generates a main video signal for display from the video signal set as the main video signal, and outputs the generated main video signal to the display means 42. When the selected audio signal is supplied and the selected audio signal is reproduced by the left and right speakers, the signal reproduced from the left and right speakers cancels the crosstalk signal viewed by the viewer's right and left ears. And a crosstalk canceller 34 for outputting to the left and right speakers, and the crosstalk canceller 34 has filters 341 to 344 storing the head-related transfer functions obtained in advance. Is a head-related transfer function measured on the basis of audio signals picked up by a pair of microphones 641 and 642 attached to a cylindrical structure 649. Recognition and judgment reliably performed abnormality, the abnormality can be realized an image picked-up sound signal reproducing system 10 to allow the monitoring image and switch to the resulting monitoring point monitoring voice immediately abnormal output from occurring.

It is a block diagram which shows the structural example of the imaging sound collection / reproducing system which concerns on implementation of this invention. It is FIG. (1) which shows the structural example of the principal part of the imaging sound collection / reproducing system which concerns on implementation of this invention. It is a figure for demonstrating the example of a structure of the principal part of the imaging sound collection / reproducing system based on implementation of this invention. It is FIG. (2) which shows the structural example of the principal part of the imaging sound-collecting / reproducing system which concerns on implementation of this invention. It is a figure which shows the example of a display of the reproduction | regeneration image which concerns on implementation of this invention. It is FIG. (3) which shows the structural example of the principal part of the imaging sound collection / reproduction system which concerns on implementation of this invention. It is a figure which shows the structural example of the filter characteristic measuring apparatus of the crosstalk canceller which concerns on implementation of this invention. It is a figure which shows the structural example of the binaural sound receiving part used at the time of the filter characteristic measurement which concerns on implementation of this invention. It is a figure which shows the example of an impulse response characteristic of the crosstalk canceller which concerns on implementation of this invention. It is a figure which shows the example of a frequency response characteristic of the crosstalk canceller which concerns on implementation of this invention. It is the figure which showed for reference the example of the impulse response characteristic of the conventional crosstalk canceller. It is the figure which showed the example of the frequency response characteristic of the conventional crosstalk canceller for reference.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging sound collection apparatus 2 Transmission path 3 Reproducing apparatus 4 Display part 6 Filter characteristic measurement apparatus 10 Imaging sound collection signal reproduction system 11, 11a Imaging sound collection part 12 Selection part 13 Compression part 14 Sending part 31 Receiving part 32 Expansion part 33 Signal Control unit 34 Crosstalk canceller 41, 43, 63 Speaker 42 Monitor 61 Personal computer 62, 65, 66 Amplifier 64 Cylindrical receiver 69 Human head receiver 111a-111h Auricle 112a-112h Microphone 113a-113d Imaging camera 116 Cylindrical structure 117 Structures 341 to 344, 347, 348 Filters 345, 346 Adders 641, 642, 691, 692 Microphone 649 Cylindrical structure 693, 694 Ear canal 695, 696 Auricle 699 Human head structure

Claims (1)

An image pickup device that outputs a video signal picked up by a camera and an audio signal picked up by a microphone, and outputs the video signal supplied from the image pickup sound pickup device to a display device In an imaging / acquisition / acquisition signal reproduction system including a reproduction device that outputs the audio signal supplied from the imaging / acquisition / acquisition apparatus to left and right speakers,
The imaging sound pickup device is:
When a plurality of cameras and one of the plurality of cameras is used as a shooting camera, the pair of cameras are arranged at a predetermined angle leftward and rightward with respect to the shooting camera and have the same height from the bottom surface. An imaging receiver having a cylindrical portion having a plurality of microphones that pick up binaural sound for each of the plurality of cameras as a sound signal accompanying a binaural sound picked up by the left and right microphones as a video signal picked up by the photographing camera. The clef,
A selection unit that outputs, as a selection audio signal, an audio signal associated with the video signal set as a main video signal on the playback device side among the plurality of video signals obtained from the plurality of cameras;
The playback device
A video signal output unit that sets one of the plurality of video signals as the main video signal, generates a main video signal for display from the video signal set as the main video signal, and outputs the main video signal to a display unit;
When the selected audio signal is supplied and the selected audio signal is reproduced by the left and right speakers, the signal reproduced from the left and right speakers cancels the crosstalk signal viewed by the viewer's right and left ears. A crosstalk canceller for processing and outputting to the left and right speakers,
The crosstalk canceller has a filter that stores a head-related transfer function obtained in advance, and the head-related transfer function is measured based on an audio signal collected by a pair of microphones mounted on a cylindrical structure. An image pickup sound signal reproduction system characterized by having a head related transfer function.