JP5533282B2 - Sound playback device - Google Patents

Description

  The present invention relates to a sound reproducing device that localizes a sound image at an arbitrary position.

  Conventionally, as shown in FIG. 1A, sound based on an audio signal is emitted from five speakers L, C, R, SL, and SR installed around a viewer U, so that the listener is surrounded. There has been a three-dimensional sound image control device that localizes a sound image (see, for example, Patent Document 1).

JP 2002-44799 A

  As shown in FIG. 1B, the viewer U usually installs five speakers L, C, R, SL, SR connected to an audio amplifier near the wall K of the room R. And since the viewer U generally views at a certain distance from the speaker, the viewer U not only receives the direct sound from the speaker but also the reflected sound (indirect sound) from the ceiling, floor and wall of the room at the same time. listening. In general, humans judge the distance to the sound source as the main clue to the volume ratio of the direct sound and the indirect sound. Therefore, when the sound is emitted from each speaker, the viewer U uses the distance to the sound source as the distance to the sound source. I feel the real distance.

  Therefore, as shown in FIG. 1B, the sound source is localized inside the region T surrounded by the five speakers L, C, R, SL, SR (for example, the left front of the viewer U). Even if an attempt is made to simulate the sound source S with a conventional audio amplifier, the viewer U cannot reproduce the ratio of the direct sound and the indirect sound when the sound source is actually placed at that position. For this reason, it is difficult for the viewer U to perceive a sense of distance as if there is a sound source at that location.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an acoustic reproduction apparatus that can localize a sound image at an arbitrary position around a viewer.

  The sound reproducing device of the present invention includes coordinate information acquisition means, sound emission means, supply means, and sound image control means. The coordinate information acquisition means acquires coordinate information in the viewing environment of the video displayed by the display device. The sound emitting means arranges the first sound source around the viewing position, and arranges the second sound source closer to the viewing position than the first sound source. The supply means supplies an audio signal corresponding to the video to the first sound source and the second sound source, thereby causing the same sound to be emitted from the first sound source and the second sound source. The sound image control means adjusts the gain ratio of the audio signal supplied to the first sound source and the second sound source according to the coordinate information.

  When the sound reproducing device is installed in, for example, a cubic room and the viewer's viewing position is set at the center of the room, the second sound source is installed closer to the viewing position than the first sound source. The sound pressure of the direct sound abruptly attenuates in inverse proportion to the inverse square of the distance, whereas the sound pressure of the indirect sound does not change much as the distance from the sound source changes. Therefore, the viewer perceives the sound emitted by the first sound source as a clue as a sound with a high ratio of indirect sounds, that is, a sound heard from a distance. On the other hand, since the distance from the second sound source to the viewer is smaller than the distance from the first sound source to the viewer, the proportion of the direct sound in the audible volume is larger than that from the first sound source. Therefore, the viewer at the viewing position perceives the sound emitted by the second sound source as a sound with a high direct sound ratio, that is, a sound heard from nearby. Therefore, by emitting the same sound from the first sound source and the second sound source and adjusting the gain of both sound signals, the ratio of the direct sound and the indirect sound that can be heard by the viewer at the viewing position can be adjusted. The sound image can be localized at an arbitrary distance between the second sound sources. For example, if the second sound source is positioned close to the viewing position, the sound emitted from the second sound source is almost directly heard by the viewer, so that the sound source can be localized close to the viewer's viewing position. it can.

  Specifically, the first sound source and the second sound source can be realized as follows. For example, the sound emission means includes a first speaker and a second speaker, the first speaker emits a sound signal of the first sound source, and the second speaker is installed closer to the viewing position than the first speaker. This can be realized by emitting the sound signal of the second sound source.

  In the configuration of the present invention, actually, in a form such as a 5.1 surround system, for example, six speakers corresponding to the conventional 5.1 channel are used as the first speakers, and speakers newly added to this configuration. Can be realized by using the second speaker for installation near the viewer. As a form of the second speaker, one or a plurality of speakers are installed in front of the viewer, or are installed behind the viewer, for example, on the headrest portion of the sofa, or are used for viewing stereoscopic images. It can be installed on a frame. With such a configuration, a sound source can be installed in the vicinity of the ear of the viewer, and the ratio of direct sound and indirect sound can be controlled with the first speaker.

  Further, the first sound source and the second sound source can also be realized as follows. For example, the sound emitting means includes a directivity control speaker that combines a speaker array and a signal processing means, and a normal speaker that does not perform directivity control by the signal processing or special structure.

  In this configuration, the sound signal of the second sound source is converted into a sound beam by the speaker array and emitted, so that the direct sound ratio at the viewing position can be increased as compared with a speaker without directivity control. Therefore, the second sound source can be localized near the viewing position without actually installing a speaker near the viewing position. As a result, the second sound source speaker can be installed at the same position as the first sound source speaker, or the first sound source speaker and the second sound source speaker can be integrated. Further, in the above-described configuration, a loudspeaker whose directivity is structurally controlled such as a flat speaker may be used as the directivity control loudspeaker instead of the loudspeaker array.

  Moreover, in the environment where the speaker having the above-described configuration is installed, the sound of the room itself such as a studio may be intentionally controlled. In such an environment, since the reverberation is less than in a normal room due to a sound absorbing wall or the like, the direct sound ratio of the sound reaching the viewing position from the first sound source is high, and as a result, directly and indirectly with the second sound source. It may be difficult to control the sense of distance using the sound ratio. In the present invention, in order to reproduce a sufficient effect even in such a special environment, it is possible to add a reverberation component to the audio signal emitted by the first sound source, and increase or decrease the added amount according to the coordinate information of the video. it can.

  In this configuration, by adding a reverberation component to the audio signal emitted by the first sound source, even if the reverberation is low, the viewer can listen as if the ratio of indirect sound from the first sound source is high. be able to. When a viewer hears a sound with a high ratio of indirect sound from the first sound source, the viewer feels that the sound has been emitted from a distance farther than when no reverberation is added. The difference in feeling can be reproduced. The amount of reverberation component added here may be determined as follows. The reverberation time of the room is measured in advance, and when the reverberation time is less than the predetermined time, the reverberation addition amount is increased, and when the reverberation time is large, the reverberation addition amount is decreased. Thus, when the amount of reverberation added to the first sound source is determined according to the reverberation measurement value, a certain effect can be reproduced even in different environments. Further, when the amount of added reverberation is larger than the appropriate amount determined by the measured value or the like, the sense of distance of the first sound source is intentionally increased, and the coordinates of the stereoscopic video are in the depth direction of the TV screen. However, it is possible to reproduce the sense of distance according to the stereoscopic image, and to make the sound source localization position control more effective. When the stereoscopic video has a depth effect and a pop-out effect at the same time, the depth direction is expressed by adding reverberation to the first sound source, and at the same time, the localization in the pop-out direction is expressed by the second sound source. In this case, since the ratio of the direct sound and the indirect sound of the first sound source that can be heard by the viewer changes due to the addition of reverberation, the gain ratio between the first sound source and the second sound source is changed according to the amount of reverberation (the amount of reverberation added). (2), the gain ratio of the second sound source at the same localization position is increased, and when the amount of reverberation is reduced, the gain ratio of the first sound source at the same localization position is increased. You can maintain the correct orientation.

  In other words, depending on the stereoscopic video, the background spreads in the depth direction, and there is an expression that the video portion to be emphasized, such as a person, protrudes in the opposite direction. In such a case, the reverberation is generated in the first sound source to express the depth. It is necessary to increase the gain to the second sound source in order to add pop-up and express pop-up. At this time, if a fixed distance-gain ratio between sound sources is used, the direct-indirect sound ratio of the first sound source changes due to the addition of reverberation and cannot be correctly localized at the calculated image position. Therefore, the sound image control means can be operated so as to always maintain the correct sound localization position by varying the distance-sound source gain ratio according to the amount of reverberation components. Specifically, the gain ratio of the second sound source at the same localization position is increased as the amount of added reverberation increases, and conversely, the gain ratio of the second sound source at the same localization position is decreased as the amount of added reverberation decreases. .

  In the sound reproducing device of the present invention, the sound image control means adjusts the frequency characteristics of the audio signals of the first sound source and the second sound source so as to suppress a change in the frequency characteristics of the predetermined band in the audio signal corresponding to the video. You can also

  In order to express the state where the sound image is approaching from a distance as one of the methods of the acoustic effect, to gradually increase the high frequency component of the sound of the sound image and to express the state where the sound image is moving away, There is a method of gradually reducing the high frequency component of the sound of the sound image. On the other hand, the sound reproducing device of the present invention arranges the first sound source around the viewing position, arranges the second sound source closer to the viewing position than the first sound source, and determines the first sound source and the sound source according to the coordinate information. The sound image is actually moved by adjusting the gain of the audio signal of the second sound source. Therefore, if the above-described acoustic effect is added to the audio signal itself of the 3D content, the high frequency component is excessive for the audio from the second sound source that is actually reproduced in the vicinity of the viewing position, resulting in sound quality that is not suitable for viewing. there is a possibility. Therefore, in the present invention, the sound effect reproduced from the second sound source is ensured by suppressing the acoustic effect according to the front-rear coordinates of the video. That is, as the video approaches the viewing position, frequency components of a predetermined band, for example, high frequency components, in the audio signals of the first sound source and the second sound source are reduced, and as the video moves away from the viewing position, the first sound source and the second sound source. The frequency component of a predetermined band, for example, a high frequency component is recovered in the audio signal. This makes it possible to make the quality of the sound reproduced from the second sound source appropriate even if the content side has an acoustic effect that emphasizes the effect of the video jumping out in the viewing position direction.

  According to the present invention, a sound field with higher presence is reproduced by outputting sound corresponding to coordinate information in a video viewing environment from a sound source added in the vicinity of the viewing position with respect to a conventional surround system. can do.

FIG. 1A is a layout diagram of speakers connected to a conventional audio amplifier, and FIG. 1B shows a speaker connected to a conventional audio amplifier indoors, and a sound source near the viewer. It is a figure for demonstrating the case where is localized. It is a block diagram showing a schematic structure of a 3D system provided with a sound reproduction device concerning a 1st embodiment of the present invention. FIG. 3 is a diagram showing the relationship between the video displayed on the monitor screen and the stereoscopic video perceived by the viewer. 3A shows a case where a stereoscopic video is perceived deeper than the monitor screen, FIG. 3B shows a case where a stereoscopic video is perceived on the monitor screen, and FIG. This is a case of perceiving in front of the screen. FIG. 4 is an XY plan view showing the distance relationship between the viewer and the monitor image and the stereoscopic image. 4A shows a case where the stereoscopic video is located behind the monitor screen, FIG. 3B shows a case where the stereoscopic video is located on the monitor screen, and FIG. This is a case of being positioned in front of the screen. FIG. 5 is a diagram for explaining a localization position of a sound image. 5A shows the case where the sound image is localized at the viewing position, FIG. 5B shows the case where the sound image is localized between the attach speaker and the surround speaker, and FIG. 5C shows the position of the surround speaker (Cch). When the sound image is localized, FIG. 5D is a case where the sound image is localized behind the surround speaker (Cch), and FIG. 5E is a case where the sound image is localized by a plurality of speakers. It is a figure which shows the state which installed the presence speaker indoors. It is a graph which shows the frequency characteristic according to the distance of a sound source. It is a block diagram which shows schematic structure of the 3D system provided with the sound reproduction apparatus which concerns on 2nd Embodiment. It is a figure which shows the sound emission state of the sound beam of a speaker array.

  Details of the sound reproducing device will be described below.

[First Embodiment]
FIG. 2 is a block diagram showing a schematic configuration of a 3D system including the sound reproducing device according to the first embodiment of the present invention. As shown in FIG. 2, the 3D system 1 includes a video playback device 2 and an audio playback device 3. The video playback device 2 includes a content signal receiving unit 21, a content playback unit 22, a video processing unit 23, a video signal output unit 25, and an audio signal output unit 26.

  The sound reproduction device 3 includes a video signal input unit 31, a video processing unit 32 (corresponding to coordinate information acquisition means), an audio signal input unit 33, a sound image processing unit 34, a first sound source signal output unit 35, and a second sound source signal output unit. 36 and a video signal output unit 37. The sound image processing unit 34 includes a mix level calculation unit (corresponding to a sound image control unit) 341, a waveform adjustment unit (corresponding to a supply unit) 343, a waveform adjustment unit (corresponding to a supply unit) 345, a reverberation adding unit 347, and a monitoring unit 348. It has. A monitor (corresponding to a display device) 38 is connected to the video signal output unit 37. As an example, five speakers 41 to 45 are connected to the first sound source signal output unit 35. Each of the speakers 41 to 45 corresponds to sound emitting means (first speaker). In the following description, an L (front left) ch speaker 41, a C (front middle) ch speaker 42, and an R (front right) ch speaker 43 are used. , SL (rear left) ch speaker 44, and SR (rear right) ch speaker 45. In the following description, the Lch speaker 41, the Cch speaker 42, the Rch speaker 43, the SLch speaker 44, and the SRch speaker 45 are collectively referred to as a surround speaker 40. In FIG. 2, the surround speaker 40 is connected to the ITU-R BS. In this example, the monitor 38 is arranged in the vicinity of the Lch speaker 41, the Cch speaker 42, and the Rch speaker 43. In the 5.1ch surround system, a subwoofer is installed, but in the following description, description of the processing of the audio signal of the subwoofer is omitted.

  An attach speaker 47 is connected to the second sound source signal output unit 36. The attach speaker 47 corresponds to sound emitting means (second speaker) and emits the sound output from the second sound source signal output unit 36.

  In the first embodiment, in order to localize the second sound source of the present invention in the vicinity of the viewer U, the attach speaker 47 is installed (placed) in the immediate vicinity of the viewer U. Further, in order to localize the first sound source around the viewer U, a surround speaker 40 is installed (arranged), and a sound image corresponding to a stereoscopic image is localized at an arbitrary position between the surround speaker 40 and the attach speaker 47. . The sound image processing unit 34 of the sound reproduction device 3 calculates the ratio (mix level) of the audio signals output to the surround speaker 40 and the attach speaker 47 based on the three-dimensional coordinate information of the stereoscopic video output by the video processing unit 32. Based on this mix level, the amplitude (gain) of the audio signal output to the surround speaker 40 and the attach speaker 47 is adjusted. Then, the sound image processing unit 34 outputs a sound signal to the first sound source signal output unit 35 and the second sound source signal output unit 36 and emits sound from the surround speaker 40 and the attach speaker 47.

  The video playback device 2 outputs the video signal and audio signal of the 3D content to the sound playback device 3. In this example, the content signal receiving unit 21 of the video reproduction device 2 receives a broadcast wave, outputs a video signal of 3D content to the video processing unit 23, and outputs an audio signal of 3D content to the audio signal output unit 26. Output.

  The content playback unit 22 plays back a storage medium (not shown), outputs a video signal of 3D content to the video processing unit 23, and outputs an audio signal of 3D content to the audio signal output unit 26.

  The video processing unit 23 outputs the video signal of the 3D content input from the content signal receiving unit 21 or the content reproduction unit 22 to the video signal output unit 25. The video signal output unit 25 outputs a video signal of 3D content to the sound playback device 3, and the audio signal output unit 26 outputs a plurality of channels of 3D content audio signals to the sound playback device 3.

  The video signal input unit 31 of the sound reproduction device 3 outputs the input video signal of the 3D content to the video processing unit 32 and the video signal output unit 37. The video processing unit 32 analyzes the input video signal, acquires the three-dimensional coordinates of the stereoscopic video with respect to the viewing position 90, and outputs it to the sound image processing unit 34. The video signal output unit 37 outputs the video signal input from the video signal input unit 31 to the monitor 38. The monitor 38 displays a stereoscopic video of 3D content.

  For example, the video playback device 2 plays back video of 3D content created so that two videos provided with parallax (shift between left and right videos) can be seen separately by the viewer's left eye and right eye using special glasses. Therefore, the viewer perceives this video as a three-dimensional video.

  The video processing unit 32 of the sound reproduction device 3 localizes the sound image at a position corresponding to the stereoscopic video, and the left-eye video (hereinafter referred to as the left video) and the right-eye video (hereinafter, the right video) of the 3D content. And the coordinates indicating how much the stereoscopic video is projected (retracted) and where the stereoscopic video is located on the left, right, top and bottom on the screen.

  FIG. 3 is a diagram showing the relationship between the video displayed on the monitor screen and the stereoscopic video perceived by the viewer. 3A shows a case where a stereoscopic video is perceived deeper than the monitor screen, FIG. 3B shows a case where a stereoscopic video is perceived on the monitor screen, and FIG. 3C shows a case where the stereoscopic video is monitored. It is a case where it perceives before this screen.

  The human eye has left and right parallax (binocular parallax), and the left eye and the right eye look different because the object is viewed at different positions, and the human brain is a three-dimensional object where the line of sight intersects Perceive that exists. Also, humans perceive that the larger the parallax is, the closer the object is, and the smaller the parallax is, the farther the object is. 3D content is an application of such human characteristics. By making two images with parallax (the difference between the left and right images) visible to the left and right eyes separately using special glasses, for example. The viewer is made to perceive this video as a stereoscopic video. In the 3D content, the difference (parallax) between the left video and the right video is changed to perceive that the stereoscopic video is popping out or retracted from the screen.

  As shown in FIG. 3A, when the left image 131L is shifted to the left side of the monitor 38 and the right image 131R is shifted to the right side of the monitor 38, the place where the line of sight intersects is behind the screen. Therefore, the viewer U perceives the stereoscopic video 131 as if it was retracted into the back of the screen. Further, as shown in FIG. 3B, when the left video 132L and the right video 132R are displayed at the same position on the monitor 38 without deviation, the place where the line of sight intersects is on the screen. The image 132 is perceived as being at the same position as the screen. Also, as shown in FIG. 3C, when the left video 133L is displayed shifted to the right side of the monitor 38 and the right video 133R is displayed shifted to the left side of the monitor 38, the place where the line of sight intersects is closer to the screen. Therefore, the viewer U perceives the stereoscopic video 133 as if it jumped out in front of the screen.

  In this way, the viewer U perceives the position of the stereoscopic video differently depending on the difference (parallax) between the left video and the right video, so that the position of the parallax and the stereoscopic video is in the direction perpendicular to the screen of the monitor 38. A proportional relationship holds. Therefore, the video processing unit 32 analyzes the shift (parallax) of the stereoscopic video between the left video and the right video of the 3D content and the position on the screen where the shift occurs, and the position of the stereoscopic video with respect to the screen, That is, three-dimensional coordinates for the viewing position 90 of the stereoscopic video are calculated and output to the sound image processing unit 34. The coordinates of the stereoscopic image are calculated as follows, for example.

  FIG. 4 is an XY plan view showing the distance relationship between the viewer and the monitor image and the stereoscopic image. 4A shows a case where the stereoscopic video is located behind the monitor screen, FIG. 4B shows a case where the stereoscopic video is located on the monitor screen, and FIG. This is a case of being positioned in front of the screen. In FIG. 4, the viewing position 90 (midpoint of both eyes) of the viewer U is the origin O (0, 0), and the distance in the Y-axis direction from the viewing position 90 (origin O) of the viewer U to the screen of the monitor 38 Is Ls, and the distance in the Y-axis direction from the viewing position 90 (the origin O) of the viewer U to the stereoscopic video 131 is Ld.

For example, the parallax value can be determined by calculating the coordinate difference between the left video 131L and the right video 131R. As shown in FIG. 4A, when the stereoscopic image 131 (cube as an example in the figure) 131 is located at a point P (0, Ld) at the back of the screen of the monitor 38, this point P is the left image 131L. Is the coincidence point P _L (x _L , Ls), and the right image 131R is the coincidence point P _R (x _R , Ls). In this case, the video processing unit 32 performs template matching on the left video 131L and the right video 131R, for example, and obtains a coincidence point between the objects in the two videos. For example, the left video is divided into an arbitrary number of blocks, and this block is regarded as a template, and a block similar to the template is searched while being shifted pixel by pixel from the right video. Since the left image and the right image have parallax, they are not exactly the same, and actually the block most similar to the template is searched for. The video processing unit 32 determines a block having the maximum correlation value with the template of the left video in the right video as a matching point.

  When obtaining the correlation as described above, as shown in FIGS. 4 (A) and 4 (C), when the deviation of the coincidence point of the right image with respect to the left image is a positive value, the stereoscopic image is monitored. When the shift is a negative value, the stereoscopic image is positioned in front of the monitor screen.

The video processing unit 32 has determined the point _{P R} of _{P L} and the right image 131R points left image 131L and match point, the deviation of the point _{P L} and the point _{P R,} i.e. using the disparity, the coordinates of the three-dimensional image 131, That is, the position coordinates on the screen plane of the stereoscopic video are calculated from the distance from the viewing position 90 to the stereoscopic video 131 and the position of the block of the coincidence point.

For example, the video processing unit 32 calculates the distance (coordinates in the Y-axis direction) from the viewing position 90 to the stereoscopic video 131 as follows. As shown in FIG. 4A, the distance between the viewer's left eye UL and right eye UR is E, and the shift (parallax) between the left video 131L and the right video 131R displayed on the screen of the monitor 38 is D (= x _{If R} −x _L ), the following proportional relationship is established, and thus Equation 1 is obtained.

Ld: (Ld−Ls) = E: D
Ld = Yp = (E / ED) Ls (Formula 1)
Similarly, since the proportional relationship is also established for the stereoscopic images 132 and 133 shown in FIGS. 4B and 4C, the distance to the stereoscopic image 131 can be calculated by Equation 1.

  In addition, as for the constant E of Formula 1, it is preferable to input the distance between both eyes of a person into the sound reproducing device 3 in advance. Further, the constant Ls of Equation 1 may be input to the sound reproduction device 3 in advance according to the viewing position 90.

Further, the localization position of the stereoscopic video is estimated from the parallax D (= x _R −x _L ) between the left video 38L and the right video 38R without performing the calculation of Equation 1 using the constants (E, Ls), You may make the parallax D correspond to a coordinate as it is. For example, D = 3 (cm) and Ld = 1 (m) are set.

  In the above description, the case where the coordinates of the stereoscopic video are calculated in the front-rear direction (Y-axis direction: the back direction and the pop-out direction) with respect to the screen of the monitor 38 has been described. Similarly, the coordinates of the stereoscopic video can be calculated in the vertical direction (Z-axis direction) and the horizontal direction (X-axis direction).

  The audio level output to the surround speaker 40 and the attach speaker 47 is performed by the mix level calculation unit 341 of the sound reproduction device 3 illustrated in FIG. 2 based on the three-dimensional position information with respect to the viewing position 90 of the stereoscopic video output by the video processing unit 32. Signal distribution (mix level information) is calculated. Then, the mix level calculation unit 341 outputs the mix level information to the waveform adjustment unit 343 and the waveform adjustment unit 345. The mix level information includes information on the gain ratio of the audio signal of the same source distributed to the surround speaker 40 and the attach speaker 47 in order to localize the sound image at a position corresponding to the stereoscopic video, and the audio signal of which channel the source is. Contains information that indicates whether it is present. When the coordinate P of the stereoscopic video is, for example, between the Cch speaker 42 and the attach speaker 47 in front of the screen, the audio signal corresponding to this stereoscopic video (in this case, the audio signal of Cch) is the Cch speaker 42 and the attach speaker 47. To be reproduced with gain distribution according to the coordinate position. Also, when the coordinate P of the stereoscopic video is between the screen and the viewing position on the left side of the screen, it is reproduced from the Lch speaker 41, the Cch speaker 42, and the attach speaker 47, for example, with gain distribution according to the coordinate position.

  Note that the above gain distribution method is based on the premise that the coordinate P of the stereoscopic video is between the surround speaker 40 and the attach speaker 47, but depending on the position of the attach speaker, the calculated coordinate may be viewed from the attach speaker. It is conceivable that it is close to. In this case, the attached speaker may be set to the maximum volume with the position of the attached speaker as a limit. Further, the actual coordinates may be scaled so as to be expressed by the distance between the attached speaker and the surround speaker. That is, the coordinate position is always scaled (converted) so as to be between the attached speaker and the surround speaker. For example, when the calculated coordinate position is in the middle of the screen and the viewer, it is localized in the middle of the surround speaker and the attached speaker.

  Based on the input mix level information, the waveform adjustment unit 343 selects an audio signal corresponding to a stereoscopic image from the Lch, Cch, Rch, SLch, and SRch audio signals input from the audio signal input unit 33, The gain of the selected audio signal is adjusted and output to the first sound source signal output unit 35.

  Similarly to the waveform adjustment unit 343, the waveform adjustment unit 345 also selects and selects an audio signal corresponding to the stereoscopic video (the same audio signal selected by the waveform adjustment unit 343) based on the input mix level information. The gain of the sound signal thus adjusted is adjusted and output to the second sound source signal output unit 36.

  In addition, the waveform adjustment unit 343 and the waveform adjustment unit 345 adjust the phase (adjust the delay amount) so that the sound emitted from the surround speaker 40 and the sound emitted from the attach speaker 47 reach the viewing position 90 at the same timing. )I do.

  For example, when a sound image is localized between the Cch speaker 42 and the attach speaker 47, the waveform adjustment unit 343 and the waveform adjustment unit 345 perform the following processing. As shown in FIG. 2, the attach speaker 47 is installed in the immediate vicinity of the viewer U, and the Cch speaker 42 is installed in the vicinity of the monitor 38, so that the waveform adjustment unit 345, for example, emits sound from the attach speaker 47. The timing is delayed with respect to the sound emission timing of the Cch speaker 42 by the time that the sound propagates the distance (for example, 1 to 2 m) from the monitor 38 to the viewing position. Thereby, it can adjust so that an audio | voice may propagate to the viewing position 90 substantially simultaneously. The distance between the Cch speaker 42 and the attach speaker 47 with respect to the viewing position 90 is measured in advance, and the waveform adjustment unit 343 and the waveform adjustment unit 345 adjust the sound emission timing of both speakers according to this distance, The sound can be propagated to the viewing position 90 more accurately at the same time. Thereby, it is possible to prevent the audio signals emitted from the surround speaker 40 and the attach speaker 47 from canceling at the viewing position 90.

  The reverberation adding unit 347 adds a reverberation component in an amount corresponding to the distance Ld to the surround speaker 40 and expresses the depth direction when the calculated coordinates of the stereoscopic video are located in the depth direction of the screen. This is because the indirect sound ratio of the sound increases when reverberation is added to the sound, so that a person perceives the sound as being heard from a distance farther than the actual speaker position. In addition, the reverberation adding unit 347 has a small difference in the direct sound-indirect sound ratio between the speakers, such that the reverberation of the room in which the speakers are installed is remarkably small or the installation positions of the attached speakers and the surround speakers are close to each other. Even if it is difficult to express the localization in the depth direction, reverberation is added to make a difference between the direct sound and indirect sound ratios so that a certain effect can be obtained regardless of the installation environment. The amount of added reverberation in this case is determined by the result of measuring the reverberation characteristics of a room installed in advance.

  Depending on the 3D image, the background may expand in the depth direction, and the video portion that you want to emphasize, such as a person, may appear to protrude in reverse. In such a case, reverberation is added to the surround speaker 40 to express the depth. In order to express popping out, the gain to the attached speaker must be increased. At this time, if a fixed distance-gain ratio between sound sources is used, the direct-indirect sound ratio of the surround speaker 40 changes due to the addition of reverberation and cannot be correctly localized at the calculated position of the stereoscopic image. The mix level calculation unit 341 operates so as to always maintain a correct sound localization position by changing the distance-sound source gain ratio according to the amount of reverberation component added to the sound signal. Specifically, as the amount of reverberation component to be added increases, the gain ratio of the attached speaker at the same localization position is increased, and conversely, as the amount of reverberation component to be added decreases, the gain ratio of the attach speaker at the same localization position is decreased.

  FIG. 5 is a diagram for explaining a localization position of a sound image. 5A shows the case where the sound image is localized at the viewing position, FIG. 5B shows the case where the sound image is localized between the attach speaker and the surround speaker, and FIG. 5C shows the position of the surround speaker (Cch). When the sound image is localized, FIG. 5D is a case where the sound image is localized behind the surround speaker (Cch), and FIG. 5E is a case where the sound image is localized by a plurality of speakers. FIG. 6 is a diagram illustrating a state in which the presence speaker is installed in the room. In the following description, the viewing position is set as the origin O as in FIG.

  (1) As shown in FIG. 5A, when the coordinate P of the stereoscopic video is the origin O (Ld = 0), the waveform adjusting unit 343 and the waveform adjusting unit 345 adjust the gain of the audio signal corresponding to the stereoscopic video. Then, the sound is set to be emitted only from the attached speaker 47 and not emitted from the Cch speaker 42. At this time, since the attached speaker is installed in the immediate vicinity of the viewer, the sound perceived from the attached speaker is mainly a direct sound, and as a result, it is perceived that the sound can be heard from the viewer.

  (2) As shown in FIG. 5B, when the coordinate P of the stereoscopic video is between the origin O and the Cch speaker 42 (0 <Ld <Ls), the waveform adjusting unit 343 and the waveform adjusting unit 345 are the stereoscopic video. By adjusting the gain of the audio signal corresponding to, and adjusting the volume of the sound emitted from the attached speaker 47 and the Cch speaker 42, the viewer perceives the sound localization at the midpoint 55 position. Sound with a sound ratio can be played.

    (3) As shown in FIG. 5C, when the coordinate P of the stereoscopic video is the sound emission surface (Ld = Ls) of the Cch speaker 42, the waveform adjusting unit 343 and the waveform adjusting unit 345 are audio corresponding to the stereoscopic video. The gain of the signal is adjusted so that sound is not emitted from the attached speaker 47, and the sound is reproduced only from the Cch speaker 4. Accordingly, the viewer U perceives a sense of distance determined by the direct sound-indirect sound ratio of the sound from the Cch speaker 42.

    (4) As shown in FIG. 5D, when the coordinate P of the stereoscopic video is behind the Cch speaker 42 (Ld> Ls), the Cch speaker 4 alone or a plurality including Cch is provided as in (3). The reverberation component according to the distance from the screen to the coordinate P is added to the surround speaker sound by the reverberation adding unit 347 at the same time. The reverberation component added to the surround speaker sound is increased or decreased by the reverberation adding unit 347 according to the distance from the screen to the coordinate P (coordinate information of the stereoscopic video). As a result, the viewer U perceives a sound field extending to the depth of the screen.

  Thus, by changing the volume of the attached speaker 47 and the amount of reverberation added to the surround speaker 40, the balance of the sound between the attach speaker 47 and the surround speaker 40 changes, and the sound heard by the viewer U can be changed. The ratio of direct sound and indirect sound changes. People feel a sense of localization of the sound image due to the ratio of direct sound and indirect sound, frequency characteristics, phase difference, etc., but in the present invention, the ratio of direct sound and indirect sound is controlled to mainly adjust the sense of distance of the sound image. ing. As a result, the sound image can be moved following the stereoscopic video that moves by jumping out or withdrawing from the screen of the monitor 38.

  In the above description, the case where the sound image is localized mainly using only the Cch speaker 42 and the attach speaker 47 has been described. However, in the actual content, audio having a high correlation with Cch is transmitted to other channels (for example, L / Rch). Therefore, if only the Cch is extracted and the effect of the present invention is applied, there is a possibility that a sense of discomfort is produced in terms of sound quality. Therefore, in practice, a plurality of channels including Cch can be selected and reproduced. Furthermore, in the present invention, the sound image can be localized at a position other than the front by adjusting the audio gain of each channel of the surround speaker 40 and the attach speaker 47 according to the left and right positions on the 3D video screen.

  For example, by adjusting the sound gain of the Lch speaker 41, the SLch speaker 44, and the attach speaker 47 according to the left and right positions on the 3D video screen, as shown in FIG. The sound image 57 can be localized in a triangular region formed by a line connecting the Lch speaker 41, the SLch speaker 44, and the attach speaker 47.

  Also, as shown in FIG. 6, a presence left channel (PLch) speaker 51 and a presence right channel (PRch) speaker 52 are installed in front of the viewing position 90 in the front left and right directions. Then, according to the vertical position on the screen of the stereoscopic video, for example, by adjusting the sound gain of the Rch speaker 43, the PRch speaker 52, and the attach speaker 47 and releasing the sound, the sound image is also displayed in the vertical direction of the room 91. Can be localized. That is, as shown in FIG. 6, the sound image 58 can be localized in a triangular region formed by a line connecting the Rch speaker 43, the PRch speaker 52, and the attach speaker 47.

  In the sound image processing unit 34, the monitoring unit 348 monitors the movement of coordinates with respect to the viewing position 90 of the stereoscopic video by the video processing unit 32. The monitoring unit 348 causes the waveform adjusting units 343 and 345 to output a control signal, and adjusts the frequency characteristics of the audio signal of the attach speaker 47 according to the coordinate information of the stereoscopic video and the installation position of the attach speaker.

  FIG. 7 is a graph showing frequency characteristics according to the distance of the sound source. To express the state where the sound image approaches from a distance as one of the acoustic effect methods, to gradually increase the high frequency component of the sound of the sound image and express the state where the sound image moves away from the near In addition, there is a method of gradually reducing the high frequency component of the sound of the sound image. In this method, as shown in FIG. 7, for example, when a certain level of sound is emitted from a low range to a high range, if the sound source is near the viewing position, the high frequency component of the sound is reached without attenuation. This simulates the phenomenon in which the high frequency component of the sound attenuates as it moves farther away from the sound source viewing position. This method is hereinafter referred to as a contact / separation effect. In a general sound reproducing device, a viewer can experience the contact / separation of a sound image by reproducing an audio signal to which the above contact / separation effect is added.

  On the other hand, the sound reproducing apparatus 3 of the present invention can actually make a sound image close to and away from the sound image. For this reason, if the contact / separation effect is added to the audio signal of the 3D content in advance, if the playback is performed as it is from the attached speaker, the high frequency component is excessive in terms of sound quality for the viewer, and the comfortable viewing is disturbed or the sound is reproduced. There is a possibility that the sound image cannot be accurately localized at the localization position of the stereoscopic video intended by the device 3. Therefore, in the present invention, the monitoring unit 348 monitors the movement of the coordinates with respect to the viewing position 90 of the stereoscopic video by the video processing unit 32, and the frequency characteristics according to the movement of the coordinates and the installation position of the attached speaker (distance from the viewer). To suppress the excess of high-frequency components and the contact / separation effect. In other words, when the stereoscopic image moves to the front from the screen, the amount of decrease in the high frequency component of the audio signal increases as the viewing position is approached, and when the stereoscopic image moves from the front to the screen direction, the audio signal increases as the distance increases. The high frequency component is increased and the original frequency characteristic is completely restored at the position of the Cch speaker. Further, when the stereoscopic video is behind the screen, the frequency characteristics are not adjusted.

  The attached speaker 47 mentioned so far does not necessarily have to be a speaker that can reproduce the entire frequency band. A speaker that can emit sound in a frequency band in which the viewer U can perceive the localization of the sound image and the movement of the sound image, for example, a small attached speaker having a minimum resonance frequency f0 of 500 Hz or more may be used. Such a small attached speaker has a high degree of freedom in the installation position and method, and can be attached to, for example, glasses used for viewing 3D video, and the speaker can be arranged in the vicinity of the ear of the viewer U. Therefore, it is possible to let the viewer hear the voice having a very high direct sound ratio, and the effect of the present invention can be maximized.

[Second Embodiment]
Next, a sound reproducing device according to the second embodiment of the present invention will be described. FIG. 8 is a block diagram illustrating a schematic configuration of a 3D system including the sound reproducing device according to the second embodiment. FIG. 9 is a diagram showing a sound emission state of the sound beam of the speaker array. In the following description, the same components as those in the first embodiment are denoted by the same reference numerals and description thereof is omitted.

  A 3D system 6 illustrated in FIG. 8 includes a video playback device 2 and an audio playback device 8. In the 3D system 6, unlike the 3D system 1 shown in FIG. 1, the speaker 47 of the sound reproduction device 3 is not installed in the immediate vicinity of the viewing position 90, but is focused at the point 50 closest to the viewing position 90 by the speaker array 49. A sound beam (sound signal of the second sound source) is connected. The other configuration is the same as that of the 3D system 1, and the sound signal of the first sound source is emitted from the surround speaker 40 and the sound image is localized (arranged) at an arbitrary position of the viewing position 90. As an example, the speaker array 49 is installed in the vicinity of the monitor 38 in the room 91 (the front side of the room 91). In the speaker array 49, a plurality of speaker units 49-1 to 49-N are arranged in a predetermined arrangement.

  The sound reproduction device 8 has partially changed the configuration of the sound reproduction device 3, and the sound image processing unit 34B includes a phase adjustment unit 349 subsequent to the waveform adjustment unit 345. The phase adjustment unit 349 corresponds to a signal processing unit, and as shown in FIG. 9, the delay amount of the audio signal (the audio signal of the second sound source) output to each speaker unit 49-1 to 49-N of the speaker array 49. And the sound beam focused at the point 50 closest to the viewing position 90 (for example, the front) is emitted by the speaker array 49. When the sound beam as described above is emitted from the speaker array 49, the viewer U perceives that the sound source (second sound source) is localized near the point 50. This is because the sound beam has a higher direct sound ratio due to its sharp directivity compared to sound from speakers that do not perform signal processing at the same position as the speaker array or directivity control by a special structure. is there. Therefore, the viewer U receives the impression that the sound source is near the place where the speaker array is installed. Thereby, it is not necessary to install the attach speaker in the immediate vicinity of the viewing position 90, and the degree of freedom of configuration and arrangement and convenience can be improved. Further, the same effect as described above may be realized by a method other than the speaker array. Attached speakers only need to be able to output sound with a higher direct sound ratio than normal speakers that do not perform directivity control. For example, parametric array speakers that use ultrasonic waves, flat baffle speakers that can reproduce highly straight sound, etc. May be used.

  As described above, in the present invention, by arranging a plurality of speakers (surround speakers 40) around the viewing position 90 (positions away from the viewing position 90), the sound emitted by these speakers is not viewed at the viewing position. Since the ratio of indirect sound is increased, the viewer can perceive the sound as being audible from a distance. In addition, by arranging the attach speaker 47 in the immediate vicinity of the viewing position 90 of the viewer U, the sound emitted by this speaker has a high direct sound ratio at the viewing position, so that the sound that can be heard from near by the viewer can be obtained. It can be perceived. Further, if the second sound source is arranged in the immediate vicinity of the viewing position, the sound emitted from the second sound source is almost directly heard by the viewer, so that the sound source can be localized at the viewer's viewing position. . Therefore, the sound image can be localized at any position by adjusting the mix level of the surround speaker and the attach speaker.

  In the first embodiment, the example in which the five surround speakers 40 are arranged around the viewing position 90 of the viewer U is shown, but the present invention is not limited to this, and other methods can be used. It is. For example, instead of a surround speaker, a speaker array is installed, a five-channel audio signal is input to the speaker array, the sound emission timing of each speaker unit is controlled, five sound beams are generated, and sound is generated on the wall of the room. By reflecting the beam, a sound field similar to that in which five speakers are arranged around the viewing position 90 can be generated. Therefore, it is also possible to install the attach speaker in the immediate vicinity of the viewing position 90 and emit sound simulating the surround speaker 40 by the speaker array.

  In addition, two speaker arrays are installed around the viewing position 90, and a sound beam focused in the immediate vicinity of the viewing position 90 is emitted from one speaker array to simulate an attached speaker. It is also possible to have a configuration that emits sound simulating the surround speaker 40.

  In addition, although the example which has arrange | positioned one attachment speaker in front of the viewer U in FIG. 2 was shown, this invention is not limited to this. For example, two attached speakers may be arranged in front or side of the viewer U so as to emit stereo sound, or a person who easily misjudges before and after localization with respect to sound in the median plane direction. An attached speaker may be installed on the sofa immediately behind the viewer's head or on the headrest of a movie theater seat using the auditory characteristics.

  In either case, the sound image can be localized at any position around the viewer by adjusting the volume balance between the attached speaker and the surround speaker.

  FIG. 8 shows an example in which one audio beam is focused in front of the viewer U (nearest), but the present invention is not limited to this. For example, the speaker array includes two audio beams (stereo). (Sound beam) may be output (sound emission) and set so that the left and right in front of the viewer U are focused. By doing in this way, the same effect as the case where two attached speakers are arranged can be obtained.

  In the above description, the audio playback device 3 has been described as analyzing the video signal output from the video playback device and calculating the video coordinates. You may give the coordinate information of an image | video from the outside of the sound reproduction apparatus 3, such as using coordinate information. In that case, the mix level may be calculated based on the coordinate information without analyzing the video signal. Thereby, the structure of the sound reproduction apparatus 3 can be simplified. In addition, the coordinate information is not necessarily acquired from the 3D content. For example, video data obtained by converting 2D content into 3D video by a playback device may be used, or analysis of time-series correlation of 2D video images or analysis of the linkage between images and audio / volume changes. Coordinate information created in anticipation of the above may be used.

  Specifically, in order to acquire the coordinates of a moving object from a 2D image, the following method can be considered. By analyzing the time-series correlation, an object image is extracted from consecutive frames on the time axis, and it is assumed that an object whose image is enlarged in the next frame (in a similar figure) is approaching. You can also get coordinate information (changes) from the relationship between the background and the size of the object of interest (perspective), or use the fact that the outline gradually becomes clear when approaching from the fog. Good. In the above-described example, the relative relationship in which an extracted object approaches / descends is mainly extracted. The absolute distance can also be determined by combining the above-mentioned perspective movement when the absolute distance that can be calculated by using a scene discrimination method such as whether the video is an indoor scene or the outdoors is approximated. In addition, it may be difficult to determine whether an object that is enlarged on the image is approaching or whether the object itself is expanding like when a balloon is inflated. In such a case, an audio signal may be used. For example, if there is an audio signal component whose volume increases as the apparent size (view angle) of the object increases, it is determined that the object is approaching. Alternatively, the determination can be made in consideration of the panning state of the sound image of the multi-channel signal as described later. In addition, a method of estimating the back-and-forth movement based on a change in the distance between face parts (for example, the interval between eyes) using the face detection frequently used in digital cameras or the like in recent years can be considered.

  Further, the present invention can be implemented by extracting the coordinate information of the sound image from the sound source of the source without necessarily using image processing. Some content audio tracks are mixed using a plurality of audio channels such as 5.1ch and 7.1ch with the intention of moving the sound image. For example, consider a case where a sound source such as a helicopter passing over the head (directly above) moves back and forth. At this time, first, the signal (monaural) of the sound image is emitted only from the center speaker, then is distributed to the center + the same volume of LR, and then the same volume of LR, + the same volume of SR, SL (the volume before and after this time) Next, the sound image is moved from the front to the rear in the same manner as SR + SL, and (if there is a back speaker, the sound volume is further distributed to the back). In the case of such a source, the sound reproduction device analyzes the audio track mixed in reverse and analyzes how the in-phase (monaural) component is distributed to the channels, thereby producing the sound image intended by the manufacturer. You can guess the movement. Furthermore, the direct sound and the indirect sound are used as expressions for making the content creator feel a sense of distance by using the sense of distance and the human perception of sound related to the sound as described in [Problems to be Solved by the Invention]. It is usual to change the ratio. As described above, the ratio of the direct sound and the indirect sound can be calculated from the audio track, and the coordinates of the sound image intended by the manufacturer can be estimated from the calculated value. By using these, position information of the sound image can be obtained and used for moving the sound image of the present invention.

  As described above, the coordinate information of the sound image can be acquired by various methods, and any method may be used, and it is conceivable to improve the coordinate information acquisition accuracy by combining a plurality of methods.

  DESCRIPTION OF SYMBOLS 1,6 ... 3D system 2 ... Video reproduction apparatus 3,8 ... Sound reproduction apparatus 21 ... Content signal receiving part 22 ... Content reproduction part 23 ... Video processing part 25 ... Video signal output part 26 ... Audio signal output part 31 ... Video signal Input unit 32 ... Video processing unit 33 ... Audio signal input unit 34 ... Sound image processing unit 35 ... First sound source signal output unit 36 ... Second sound source signal output unit 37 ... Video signal output unit 37 ... Video signal output unit 38 ... Monitor 41 ... Lch speaker 42 ... Cch speaker 43 ... Rch speaker 44 ... SLch speaker 45 ... SRch speaker 47 ... Attached speaker

Claims (6)

Coordinate information acquisition means for acquiring coordinate information in the viewing environment of the video displayed by the display device;
A sound emitting means for disposing a first sound source around the viewing position and disposing the second sound source closer to the viewing position than the first sound source;
Supply means for supplying an audio signal corresponding to the video to the first sound source and the second sound source;
Sound image control means for adjusting a gain ratio of an audio signal supplied to the first sound source and the second sound source in accordance with the coordinate information;
A sound reproducing device comprising:   The sound emitting means is a first speaker that emits an audio signal of the first sound source, and a second speaker that is installed closer to the viewing position than the first speaker and emits an audio signal of the second sound source. The sound reproducing device according to claim 1, further comprising a speaker. The sound emitting means is
A speaker array in which a plurality of speaker units are arranged;
A signal processing unit that delays and distributes the audio signal of the second sound source to the plurality of speaker units, and emits an audio beam focused at a position where the second sound source is localized;
A speaker for emitting an audio signal of the first sound source;
The sound reproducing device according to claim 1, further comprising:   The sound reproduction device according to claim 1, wherein the sound image control unit adds a reverberation component to an audio signal emitted by the first sound source, and increases or decreases the addition amount according to coordinate information of the video.   5. The sound image control unit according to claim 4, wherein the sound image control means varies a gain ratio of an audio signal supplied to the first sound source and the second sound source in accordance with a reverberation component amount added to the audio signal emitted by the first sound source. Sound playback device.   The said sound image control means adjusts the frequency characteristic of the audio | voice signal of a said 2nd sound source so that the change of the frequency of the predetermined band in an audio | voice signal may be suppressed according to the coordinate information of the said image | video. The sound reproducing device described in 1.

Images