JP2009017438A - Information transmission apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information transmission apparatus capable of obtaining presence. <P>SOLUTION: A projector 15 captures a moving image and audio and displays the moving image on a screen 11. When the relevant moving image is displayed, an object detection section 13 detects an object from the relevant moving image and based on a detection result of the object detection section 13, an audio control section 14 controls a plurality of audio output sections 12. The plurality of audio control sections 12 are disposed so as to correspond to different positions on the screen 11. An audio control section 14 is controlled so as to output the audio of the object only from an audio output section 12 corresponding to a position of the relevant object, for example. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an information transmission device that transmits moving images and sound.

  In an information transmission device such as a television, a computer terminal, and a videophone, a technique for giving a sense of reality is desired.

  Conventionally, in order to give a sense of realism, visually, the screen (screen) on which moving images are displayed is enlarged, or auditoryly, a loud sound is output from a low-frequency output speaker, or a home theater system. In some cases, multiple speakers are used.

  Moreover, the prior art for producing such a sense of presence is disclosed by patent document 1 and patent document 2, for example.

  However, these devices have a problem that a sense of reality is lowered because the screen and the speaker are located in a spatially separated place.

  As a conventional technique in view of such problems, Patent Document 3 is characterized in that sound is generated from substantially the same surface as the moving image display surface, and each portion (partial sound generating element) of the moving image display surface is characterized in that Discloses an information transmission system that controls sound localization.

However, in the technique of Patent Document 3, since an audio signal is assigned to each partial audio generating element in advance, the audio signal must be divided in advance, and when a moving image such as a monaural audio signal or a 2ch stereo audio signal is viewed. Can not get a high sense of realism.
JP-A-6-237458 JP 2003-274314 A JP 2001-78282 A

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an information transmission device capable of providing a sense of reality.

  In order to achieve the above object, the present invention adopts the following configuration.

  An information transmission apparatus according to the present invention includes a screen on which a moving image is displayed, a plurality of audio output units arranged corresponding to different positions on the screen, and an object detection for detecting an object from the moving image. And a control means for controlling each of the plurality of sound output units based on the detection result of the object detection means.

In the information transmission apparatus according to the present invention, a plurality of controllable sound output units are arranged corresponding to different positions on the screen, and each sound output unit is controlled based on the detection result of the object detection means. The For example, when a certain object is shown on a part of the screen, the viewer can hear the sound from the direction of the object by controlling the sound output unit to output the sound corresponding to the position of the object. Can be heard, so that it is possible to obtain a feeling that the object is producing sound. Furthermore, when the target object moves, the viewer can continue to obtain a feeling that the target object emits sound by switching the sound output unit that outputs the sound in accordance with the movement. In other words, the viewer can get a three-dimensional feeling as if what is actually shown in the moving image is there, and as a result, it feels as if he is in the space on the screen. A feeling can be obtained.

  The information transmission apparatus according to the present invention further includes voice extraction means for extracting the voice of the target detected by the target detection means from the input voice, and the control means is extracted by the extraction means. It is preferable that the sound output is output from a sound output unit corresponding to the display position of the object. As a result, even if the input sound is mixed with sound other than the sound emitted by the object (actually, the moving image such as a movie or TV program also includes sound other than the sound emitted by the object) In many cases, the viewer can obtain a feeling that only the sound emitted from the object can be heard from the direction of the object.

  The information transmission apparatus according to the present invention preferably further includes an input unit for inputting the moving image and sound, and a moving image display unit for displaying the input moving image on the screen. Thereby, the usefulness of the information transmission device is improved.

  Preferably, the control means determines the volume of the sound extracted by the extraction means based on the size of the object. In reality, there is usually a difference in the magnitude of the sound produced by a distant object and the sound produced by a nearby object (for example, the sound produced by a distant object sounds smaller). . In addition, since moving images tend to appear smaller as an object is farther away, a situation closer to reality can be reproduced by determining the volume based on the size of the object.

  In addition, the information transmission apparatus according to the present invention further includes an independent sound output unit arranged at a position independent of the screen, and the control means outputs at least a part of the sound other than the sound extracted by the extraction means as an independent sound output. It is preferable to output from the section. As a result, for example, the voice or BGM of a person who is not shown in the moving image can be heard from outside the screen, and the three-dimensional effect and the sense of presence of the sound are further enhanced.

  Preferably, the control means outputs at least a part of the sound other than the sound extracted by the extracting means from the sound output unit corresponding to all positions on the screen. As a result, BGM and narration can be heard from the entire screen. Furthermore, it is possible to output sound other than the object without providing an additional device.

  Preferably, the control means outputs at least a part of the sound other than the sound extracted by the extracting means from an audio output unit corresponding to a partial area of the screen. This can reduce the number of audio output units that output audio among all the audio output units, thereby reducing power consumption.

  In the information transmission apparatus according to the present invention, it is possible to obtain a sense of reality for any moving image (including sound) by controlling the sound output unit based on the result of the object detection means.

  Exemplary embodiments of the present invention will be described in detail below with reference to the drawings.

<Device configuration>
FIG. 1 is a block diagram showing a functional configuration of an information transmission apparatus according to an embodiment of the present invention. This information transmission device is a device that transmits moving images and sound.

  The information transmission device includes a screen 11, a plurality of audio output units 12, an object detection unit 13, and an audio control unit 14 (see FIG. 1).

  Any technology may be applied to the screen 11 as long as it can display a moving image such as a projection screen or a video screen of a display device. The screen 11 in this embodiment is a projection screen. In the present embodiment, a commercially available projector 15 (projector) shown in FIG. 1 acquires a moving image and sound, and displays the moving image on the screen 11. Note that the projector 15 may acquire a moving image and sound from a commercially available DVD (movie or the like) using a DVD player or the like, or acquire a moving image (including sound) stored in a hard disk or the like. Also good.

  The sound output unit 12 may apply any technique as long as it can output sound, such as a cone speaker or a flat panel speaker. The audio output unit 12 in this embodiment is a flat panel speaker. The flat panel speaker has a smaller sound wave diffusion / attenuation than the cone speaker, so that the user can more accurately recognize the position where the sound can be heard. That is, by using a flat panel speaker as the audio output unit 12, the user can obtain a higher sense of realism.

  Further, the plurality of audio output units 12 are arranged corresponding to different positions on the screen 11. The plurality of audio output units 12 in the present embodiment are arranged in a matrix of m × n on the back surface of the screen 11 as shown in FIG. The plurality of audio output units 12 need not be arranged as shown in FIG. If the audio output unit 12 is transparent, the audio output unit 12 may be disposed on the front surface of the screen 11. You may arrange | position radially instead of a matrix form. The number of audio output units 12 may be any number (for example, 100 or 2). The arrangement and number of the audio output units 12 may be arranged in any manner according to the purpose of the user.

  The object detection unit 13 has a function of detecting an object from a moving image (the object detection unit 13 in the present embodiment acquires a moving image from the projector 15 as shown in FIG. 1). Any existing technique may be applied to the object detection processing by the object detection unit 13. The target object may be anything such as a sound-generating object or a moving object. For example, a person, a vehicle (a train, an automobile, an airplane, etc.) may be used as the object. In addition, what has a movement may have a change in display position itself, and may have a movement with respect to surrounding movements (background etc.). The detection result of the object detection unit 13 may be any amount as long as it represents the object, such as the type, size, and position of the object (in this embodiment, the detection result of the object The size and position (position coordinates of the center of the object) are output).

  The audio control unit 14 has a function of controlling the respective audio output units 12 based on the detection result of the object detection unit 13 (the audio control unit 14 in the present embodiment includes a projector 15 as shown in FIG. Get audio). In addition, the audio | voice control part 14 may be provided for every audio | voice output part 12, and you may control all the audio | voice output parts 12 with the one audio | voice control part 14. FIG. An example of the control method of the audio output unit 12 will be described in detail later.

<Information transmission function>
The function and processing flow of the information transmission apparatus will be described with reference to the flowchart of FIG.

  When the information transmission function is activated, the projector 15 acquires a moving image and its sound (step S101). In the present embodiment, it is assumed that a moving image in which only the person 41 is captured and its sound (sound only of the voice of the person 41) is acquired as shown in FIG.

  Next, the moving image acquired in step S101 is displayed on the screen 11 (step S102).

  Simultaneously with step S102 (simultaneously with the moving image being displayed on the screen 11), the following steps are performed.

  The object detection unit 13 detects the object from the moving image acquired in step S101 (step S103). In the present embodiment, a human face is considered as the object (that is, the face of the person 41 is detected in the example of FIG. 4).

  And the audio | voice control part 14 controls each audio | voice output part 12 based on the detection result of the target object detection part 13 (step S104). In the example of FIG. 4, the sound control unit 14 controls each sound output unit 12 (including the sound output unit 42) so that sound is output only from the sound output unit 42 corresponding to the position of the face of the person 41. To do. Thereby, the sound can be heard only from the direction of the person 41. The sound may be output from one sound output unit 42 corresponding to the position of the face (object), or may be output from a plurality of sound output units 42 centered on the position. (For example, the number of audio output units 42 that output the audio may be determined according to the size of the face).

  Even when the object moves (for example, even if the person 41 is a moving image moving from right to left on the screen), the sound corresponding to the position of the object moves along with the movement of the object. Since the output unit 42 is sequentially switched, it is possible to maintain a state where sound can be heard only from the direction of the person 41 (see FIG. 4).

  As described above, in the present embodiment, it is possible to obtain a sense of realism by controlling the plurality of audio output units 12 according to the detection result of the object (in the example of FIG. 4, the person 41 actually You can get a sense of presence as if you were there.)

<Modification 1>
In the first modification, a case where voices other than the target object (for example, BGM) are mixed in the voice will be described. Note that descriptions of functions and processes similar to those described above are omitted.

<Device configuration>
FIG. 5 is a block diagram illustrating a functional configuration of the information transmission device according to the first modification. This information transmission device further includes a voice extraction unit 51. In addition, the same code | symbol is attached | subjected about the function similar to FIG.

  The voice extraction unit 51 has a function of extracting the voice of the target detected by the target detection unit 13. In the first modification, the voice extraction unit 51 extracts and distinguishes the voice of the target object from the voices acquired from the projector 15 and outputs the voice to the voice control unit 14. Note that any existing technique may be applied to the voice extraction processing in the voice extraction unit 51. As an example, there is a method of performing frequency analysis of sound and extracting only the frequency range of the sound of the object.

<Information transmission function>
The function and processing flow of the information transmission apparatus according to the first modification will be described with reference to the flowchart of FIG.

The processes in steps S201 to S203 are the same as the processes in steps S101 to S103 in FIG. However, in the modification 1, the sound other than the object is mixed in the sound.

  Following step S203, the voice extraction unit 51 extracts the voice of the object from the voice acquired in step S201 (step S204). In the example of FIG. 4, the voice (voice) of the person 41 is extracted.

  And the audio | voice control part 14 controls each audio | voice output part 12 based on the detection result of the target object detection part 13 (step S205). In the example of FIG. 4, the audio extracted in step S <b> 204 is controlled so as to be output only from the audio output unit 42 corresponding to the face position of the person 41. Thereby, even if voices other than the person 41 are mixed in the voice, the voice of the person can be heard only from the direction of the person 41.

  As described above, in the first modification, the information transmission apparatus further includes the voice extraction unit 51, and extracts the voice of the object from the acquired voice. As a result, even when voices other than the target are mixed in the voice, the voice of the person can be heard only from the direction of the target, and a sense of reality can be obtained.

<Modification 2>
In the second modification, the loudness is determined according to the size of the object. In fact, the sound emitted by nearby objects is louder than the sound emitted by distant objects. In addition, moving images tend to appear smaller as an object is farther away. For this reason, it is possible to reproduce a more realistic situation by determining the volume of the sound according to the size of the object.

  For example, as shown in FIG. 7, in a moving image in which a person 71 approaches, the size (size) gradually appears larger as the person 71 approaches. Therefore, as the size of the person 71 increases, the volume of the sound of the person 71 (the sound output from the sound output unit 72 corresponding to the position of the person 71) is gradually increased. Thereby, the viewer can obtain a stereoscopic effect as if the person 71 is really approaching. Note that the volume may be increased, or the volume may be increased in the case where the volume can be increased by increasing the number of audio output units 72 that output the sound.

<Modification 3>
In Modification 3, as shown in FIG. 8, the information transmission apparatus of Modification 1 further includes an independent audio output unit 81.

  As with the audio output unit 12, the independent audio output unit 81 may be any device that can output audio. However, the independent audio output unit 81 is arranged at a position independent of the screen 11.

  In Modification 3, as shown in FIG. 9, the sound other than the object in Modification 1 (the remaining sound extracted by the sound extraction unit 51 (the sound of the object)) is output from the independent sound output unit 81. (In the example of FIG. 9, it is assumed that a person 91 is detected as an object, and the sound of the person 91 is output from the sound output unit 12 corresponding to the position of the person 91). Thereby, sound (BGM or the like) uttered from other than the object (person 91) can be heard from outside the screen 11, and the three-dimensional effect and the sense of reality of the sound are further enhanced.

<Modification 4>
In Modification 4, as shown in FIG. 10, control is performed so that sound other than the object is output from all the sound output units 12 (in the example of FIG. 10, the person 101 is detected as the object). The voice of the person 101 is output from the voice output unit 12 corresponding to the position of the person 101). As a result, a sound obtained by combining the sound extracted by the extracting means and the remaining sound (that is, the sound acquired by the projector 15) is output from the sound output unit 12 corresponding to the position of the object, and the sound other than the object is output. The sound is output from the sound output unit 12 other than the sound output unit 12.

  In the modified example 4, by eliminating the sound of the object from a position other than the object, the sound other than the object can be heard from the entire screen 11, and the sound of the object can be heard only from the direction of the object. Thereby, the viewer can get a sense of reality.

<Modification 5>
In the fifth modification, control is performed so that sound other than the object is output from a partial area of the screen 11.

  In the example of FIG. 11, the sound other than the object is output from the sound output unit 12 that does not correspond to the position of the object (that is, the sound output unit 12 that corresponds to a region other than the region surrounded by the dotted line in FIG. 11) (FIG. 11). In the example, it is assumed that the person 111 is detected as an object, and the sound of the person 111 is output from the sound output unit 12 corresponding to the position of the person 111). By outputting the sound separately from the position of the object, the viewer can obtain a sense of reality.

  In the example of FIG. 12, a sound other than the object is output from the sound output unit 12 corresponding to the center position of the screen 11 (in the example of FIG. 12, assuming that the person 121 is detected as the object, the sound of the person 121 is It is assumed that the sound is output from the sound output unit 12 corresponding to the position of the person 121). By always outputting the sound other than the object from the center position of the screen 11, the viewer can distinguish the sound from the sound of the object (the object sometimes appears in the center position of the screen 11. However, since the sound other than the object is always output from the center position, it can be distinguished).

  In the modified example 5, by outputting the sound other than the target object limited to a part of the region, the power consumption can be suppressed low, and the viewer can get a sense of reality.

  Modifications 1 to 5 may be combined with each other. In that case, the audio other than the object may be divided into a plurality. For example, a part of the sound other than the object may be output from the sound output unit 12, and the remaining sound may be output from the independent sound output unit 81 (rain sound or the like is output from the sound output unit 12, and BGM For example, output from the independent voice output unit 81). Further, the method is not limited to the above method as long as the method performs sound control using image analysis such as object detection processing.

  Note that the processing order shown in FIGS. 3 and 6 may be changed as much as possible. For example, instead of performing processing such as object detection processing and audio control after acquisition by the projector 15, the object detection unit 13 and the audio control unit 14 acquire moving images and audio, respectively, and perform various processes. It may be output to the projector 15 later (the audio may be output directly to the audio output unit 12 without being output to the projector 15). Thereby, even if the processing load such as object detection and voice control is large (even if the processing time is long), the output of the voice and the display of the moving image can be made at the same timing.

  The function of the projector 15 may be included in the constituent elements of the information transmission device. FIG. 13 is a block diagram showing a functional configuration of the information transmission apparatus. The information transmission apparatus further includes an input unit 131 and a moving image display unit 132.

  The input unit 131 is for inputting a moving image and sound, and the moving image display unit 132 is for displaying the input moving image on the screen (that is, the input unit 131 and the moving image display unit 132). Are combined to play the role of the projector 15).

  Although the moving image is not described in detail in the present embodiment, the moving image may not be a movie stored on a DVD (as exemplified in the present embodiment). For example, it may be a television program received by an antenna, or a game screen obtained from a game machine or the like. Any moving image that can detect the object may be used.

  In this embodiment, the case where a moving image is displayed on the projection screen (screen 11) using the projector 15 (example applied to a screening system such as a movie theater) has been described. Alternatively, a configuration may be adopted in which a video screen such as a television device is used for the screen 11 and the television device acquires a moving image from an antenna, a DVD playback device, or the like. That is, the information transmission device of the present invention is not limited to a screening system, but can be applied to a television device, a computer terminal, a video phone, and the like.

FIG. 1 is a block diagram illustrating a functional configuration of the information transmission apparatus. FIG. 2 is a diagram illustrating an example of an installation mode of the audio output unit 12. FIG. 3 is a flowchart showing a flow of processing of the information transmission apparatus. FIG. 4 is a diagram illustrating an example of a control method of the voice control unit 14. FIG. 5 is a block diagram illustrating a functional configuration of the information transmission device according to the first modification. FIG. 6 is a flowchart illustrating a processing flow of the information transmission apparatus according to the first modification. FIG. 7 is a diagram illustrating an example of a control method of the voice control unit 14. FIG. 8 is a block diagram illustrating a functional configuration of an information transmission device including the independent audio output unit 81. FIG. 9 is a diagram illustrating an example of a control method of the voice control unit 14. FIG. 10 is a diagram illustrating an example of a control method of the voice control unit 14. FIG. 11 is a diagram illustrating an example of a control method of the voice control unit 14. FIG. 12 is a diagram illustrating an example of a control method of the voice control unit 14. FIG. 13 is a block diagram illustrating a functional configuration of an information transmission apparatus including the input unit 131 and the moving image display unit 132.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Screen 12,42,72 Audio | voice output part 13 Object detection part 14 Audio | voice control part 15 Projector 41,71,91,101,111,121 Person 81 Independent audio | voice output part 131 Input part 132 Moving image display part

Claims (3)

A screen on which moving images are displayed;
A plurality of audio output units arranged corresponding to different positions on the screen;
Object detection means for detecting an object from the moving image;
Control means for controlling each of the plurality of sound output units based on the detection result of the object detection means;
An information transmission device comprising: Voice extraction means for extracting the voice of the object detected by the object detection means from the input voice;
The information transmission apparatus according to claim 1, wherein the control unit outputs the voice extracted by the extraction unit from a voice output unit corresponding to a display position of the object. An input unit for inputting the moving image and sound;
A moving image display unit for displaying the input moving image on the screen;
The information transmission apparatus according to claim 1, further comprising:

Images