JPH04237288A - Audio signal output method for plural-picture window display - Google Patents

Abstract

PURPOSE:To display plural participants simultaneously in a video conference system and to easily identify each talker. CONSTITUTION:In the video conference system interconnecting points A, B, C, D, a multi-window system is adopted for a monitor 21 for a participant 11 at the pint A and windows 25, 26, 27 are arranged laterally to the screen 22 and images 16, 19, 28 of participants of the points B, C, D are displayed respectively onto the windows 25, 26, 27. Speakers 42, 43 are provided respectively to the right and left of the monitor 21 and a voice of the participant at the point B corresponds to the horizontal position of the participant image 16 on the screen 22 and an output level ratio of the speakers 42, 43 is selected to be 100:0, then as if the voice were generated from the participant image 16 and a voice of the participant at the point C corresponds to the horizontal position of the participant image 19 on the screen 22 and an output level ratio of the speakers 42, 43 is selected to be 50:50, then as if the voice were generated from the participant image 19.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] This invention is applicable to, for example, when a plurality of windows are provided on one screen of a TV conference device or a parent-child screen system, and each window displays a different image, the sound and music associated with the image are displayed. This invention relates to a sound signal output method.

0002

2. Description of the Related Art Conventionally, displays in a type of video conference in which a plurality of people converse at the same time using a video conference system or a video telephone connected between multiple points have been displayed by switching to a single display device. For example, points A, B, C
, D, when switching to one monitor for display in a video conference system connected between four points, the participant 11 at point A is imaged by the camera 12 as shown in FIG. 5, and the image output is sent to the control device. 13. Points B and C via communication line 14
, D to each communication partner. Furthermore, the participant 11 at point A can use the display monitor 15 to view the image 16 of the participant at point B, for example. Here, points C and D
If you want to see the participants, you can do so by switching the images displayed on the display monitor 15 to images from points C and D via the control device 13. Further, the audio at points B, C, and D is mixed by the control device 13 and output to the speaker 17. In this method, since the voices input from points A, B, C, and D are output by mixing, there is a drawback that it is not possible to recognize which participant's voice is being outputted.

[0003] Also, in the past, there has been a system in which a plurality of monitors are provided at one point and images from each different point are displayed. FIG. 6 shows a method of displaying images of communication partners on two monitors in a video conference system connected between four points A, B, C, and D. At point A, a display monitor 18 is used in addition to the display monitor 15. They are arranged side by side, and the image signal from point C is displayed on this display monitor 18. In this way, points B and C
Images 16 and 19 of the participants can be seen.

[0004] When it is desired to see the image of the participant at point D, the image on the display monitor 16 or 18 is switched to display the image of the participant at point D. Furthermore, the audio from points B, C, and D is mixed by the control device 13 and output from the speaker 17 . In this conventional method, the audio input from points A, B, C, and D is output by mixing, similar to when switching and displaying on one monitor. It has the same drawback of not being able to recognize whether it is a voice or not.

[0005] Furthermore, conventionally, an entire screen is treated as one window and an image is displayed on it, and the screen (
In the so-called parent-child screen display method, in which another small window is provided as a part of a window) and another image is displayed in this small window, the sound signals such as audio or music that are attached to one of the windows are switched. Either the two sound signals associated with both windows are simply mixed and output from the speaker, making it impossible to distinguish between the two.

[0006]

[Means for Solving the Problems] According to the present invention, a plurality of speakers are distributed around the screen of a display device that is provided with a plurality of windows and displays different images on one screen at the same time. , and output the sound signal by distributing the sound signal associated with the window in inverse proportion to the distance from these to the window.

[0007]

Embodiment FIG. 1 shows an embodiment in which the present invention is applied to a video conference system connected between four points A, B, C, and D. In other words, FIG. 1 shows the device installed at point A,
The display monitor 21 is of a so-called multi-window display type, and a plurality of video windows 2 are displayed on the screen 22.
3, 24, 25 are provided, and these windows 23, 2
Images 16 of each participant at points B, C, and D are shown at 4 and 25, respectively.
19 and 28 are displayed.

In the control device 13, a video/audio input/output control section 31 connects a communication line 14, a television camera 12, a multi-window enlargement/reduction section 32, and an audio level detection section 3.
3, and a multi-window enlargement/reduction section 32, a multi-window display control section 34, a multi-window movement processing section 35, a mouse input processing section 36, and a sound image localization control section 37 are connected to the central control section 38. The detection output of the audio level detector 33 is output to the audio level controller 39, and the output of the audio level controller 39 is output to the sound image localization controller 37. A mouse 41 is connected to the mouse input processing section 36. Speakers 42 and 43 are arranged on both the left and right sides of the screen 22.

The image of the participant 11 at point A captured by the camera 12 is transmitted to the communication partner via the video/audio input/output control section 31 and the communication line 14. Also, communication line 1
The multiplexed video and audio of the three points inputted to the control device 13 via the video/audio input/output control section 31 are separated into the video and audio of the three points, and the video is sent to the multi-window enlargement/reduction section. Multi-window display control unit 3 via 32
4, and the sound is input to the sound image localization control section 37 via the sound level detection section 33 and the sound level output control section 39. Here, the videos at points B, C, and D are designated as video channels 1, 2, and 3, respectively, and the audio accompanying video channels 1, 2, and 3 are designated as audio channels 1, 2, and 3, respectively. The multi-window display control section 34 displays the participant image 16 at point B under the instructions of the central control section 38, for example, as displayed on the screen 22 of the multi-window display monitor 21.
That is, video channel 1 is placed in the video window 25 on the left side of the screen 22, participant image 19 at point C, ie, video channel 2, is placed in the video window 26 in the center of the screen 22, and participant image 28 at point D, ie, the video channel. 3 to screen 2
2 in the video window 27 on the right side.

In the present invention, the audio accompanying the window 25, that is, the audio channel 1, is transmitted to the speakers 42, 43 in inverse proportion to the distance from the window 25 to the speakers 42, 43.
43. The sounds associated with windows 26 and 27, respectively, audio channels 2 and 3, are similarly distributed to speakers 42 and 43. For this reason, the central control unit 38
The display center horizontal coordinate position of each of the windows 25, 26, and 27 and the audio channel number associated with the video channel displayed in the window are input to the sound image localization control unit 37. The sound image localization control unit 37 controls the distribution level of the audio output of the audio channel associated with the video channel displayed in that window to the speakers 42 and 43 based on the input display center horizontal coordinate position of each window. .

[0011] When the possible audio level range is set to five levels,
As shown in FIG. 2A, the screen 22 is divided horizontally into five regions ■ to ■, and the region ■ is located on the left speaker 42 side. Accordingly, a distribution ratio for distributing the audio accompanying the image (window) to the speakers 42 and 43 is determined. This determination is made as shown in FIG. 2B, so that a louder level is supplied to the closer speaker and a lower level is supplied to the farthest speaker. For example, the audio accompanying the window located in area ■ is distributed to the speakers 42 and 43 at a ratio of 0.75:0.25, and the ratio of the outputs of the speakers 42 and 43 is in this relationship. It sounds like the sound is coming from area ■. Therefore, screen 22 in Figure 1
Since the display center horizontal coordinate positions of the video windows 25, 26, and 27 displayed in the video windows 25, 26, and 27 are in the areas ■, ■, and ■, respectively, the video displayed in the video window 25, that is, the audio channel 1 associated with the video channel 1 The audio level ratio of speaker 42:speaker 43 is 1:0. Similarly, audio channels 2 and 3 associated with video channels 2 and 3 displayed in video windows 26 and 27
The audio level ratio of speakers 42:speakers 43=0.5:0.5 and speakers 42:speakers 43=0:1, respectively. The sound image localization control section 37 changes and outputs the sound level output to the speakers 42 and 43 based on this ratio. In this way, the voice of the audio channel 1 accompanying the video image displayed on the video window 25, that is, the voice of the participant at point B, can be heard by the participant 11 as if it were coming from the left side of the screen 22. . Similarly, the sounds of audio channels 2 and 3 accompanying the moving images displayed on the video windows 26 and 27, that is, the sounds of each participant at points C and D, are respectively transmitted from the center and right side of the screen 22. It sounds like it's being spoken. In this way, the display position of the conference participant displayed in the multi-window matches the apparent voice output position, so compared to conventional technology, the output voice can be heard from which participant. This makes it easier to recognize whether the
Good video conferencing becomes possible.

As shown in FIG. 3A, if a speaker 44 is provided in the center of the speakers 42 and 43, that is, three speakers are arranged horizontally below the screen 22, and the audio level variable range is set to seven levels, , the screen 22 is horizontally divided into seven regions ■ to ■, and the audio accompanying the image of the window in each region is output from the speakers 42, 43, and 44 at a distribution ratio as shown in FIG. 3B. Bye. In this case as well, the audio level output to the speakers 42, 43, and 44 is changed and outputted according to the display center horizontal coordinate position, as in FIG. 2A. By using 3 speakers and setting the audio level variable range to 7 levels, more detailed control is possible, and as in the case described above, it is easier to recognize the speaker of the output audio, making it possible to achieve good results. Video conferencing will be possible.

Next, a case will be described in which the mouse 41 is used to move the moving image multi-window displayed on the multi-window display monitor 21. mouse 41
To move a window, for example, operate a switch on the mouse 41 to display a function menu window on a part of the screen 22, and select "Move" among the items (enlargement, reduction, movement, deletion, etc.) with a marker. Click, then click in the same way on the window you want to move, then move the marker to the position you want to move and click. Operate the mouse 41 to open window 2, for example.
5 is moved to near the center of the screen 22. FIG. 2C shows a state in which the video window 25 has been moved to near the center. First, the window to be moved is selected using the mouse 41, and then the mouse is operated to move the marker to a desired position. As a result, the data input using the mouse 41 is input to the central control unit 38 as movement coordinate values by the mouse input processing unit 36. The central control unit 38 inputs the movement coordinate value to the multi-window movement processing unit 35, and the multi-window movement processing unit 35 performs movement processing of the video window based on the movement coordinate value. This moved video window is input to the multi-window display control unit 34, and the video window 25 is displayed at the marker position.

The central control unit 38 simultaneously inputs the movement coordinate value to the sound image localization control unit 37, and the sound image localization control unit 37 adjusts the moving video window 2 based on the movement coordinate value.
The supply/distribution ratio of the audio accompanying 5 to the speakers 42 and 43 is changed according to FIG. 2B. For example, when the video window 25 is moved to the position shown in FIG. Become. In this way, the display position of the moving image displayed in the multi-window can be moved to an arbitrary position, and the speaker 42,
43, and by changing the audio output level ratio, the display position of the conference participant displayed in the multi-window and the audio output position are matched and output. This corresponds to moving seats in a real meeting, and it also simulates the movement of seats in a video conference, creating a more realistic TV than with conventional technology. Meetings will be possible.

Next, the case of enlarging/reducing the moving image window will be explained. When a desired video window is selected using the mouse 41 and an operation is performed to change the frame of the window to a desired size, the central control unit 38 uses the size information of the video window determined using the mouse 41 to perform multi-window enlargement/ Reduction section 32, audio level control section 39
Transfer to. In the multi-window enlargement/reduction section 32,
Based on this video window size information, the size of the video window is enlarged or reduced, and transferred to the multi-window display control unit 34 for display. The audio level control unit 39 increases or decreases the level of the audio accompanying the video window based on the transferred video window size information. As a result, the audio of the video window that is reduced in size is output at a low volume, and the audio of the video window that is enlarged is output at a high volume, so the audio of the video window that is enlarged to attract the user's attention is automatically output. In addition, the audio in the video window that displays a reduced video image that the user thinks is not very relevant to the meeting is automatically reduced, allowing the user to focus on other important audio. This makes it possible to conduct video conferences that attract attention and have a good user interface.

[0016] Conversely, a case will be explained in which the corresponding video window is enlarged or reduced in accordance with the received audio level. The received audio level is detected by the audio level detection unit 33, and a video window with an audio level above a certain threshold value is displayed in an enlarged manner. Furthermore, by reducing the size of a video window with an audio level below a certain threshold, it becomes possible to conduct video conferences and video calls with a good user interface. In other words, for example, if a participant at point B makes a loud noise to draw the attention of other participants,
The sound level detecting section 33 determines that the sound level is higher than the threshold value, and the window 25 for displaying the participants at point B is enlarged and displayed. Further, for example, if the participant at point C does not participate in the conversation for a while and remains silent for more than a certain period of time, the video window 26 is displayed in a reduced size.

[0017] As a system for displaying two moving images in a window format, there is a system that displays them in a parent-child screen format. An example in which the present invention is applied to this parent-child screen system will be described with reference to FIG. 4. The screen 22 of the parent-child screen display monitor 51 has a window (main screen) that fills the entire screen.
For example, an image of a certain channel is displayed on the screen 22.
In other words, a part of the window 52 has a window (
A sub-screen) 53 is provided, on which images of other channels are displayed. The parent and child screen display control section 54 for this parent and child screen display monitor 51 includes a video/audio input control section 55, a parent and child screen display control section 56, a child screen movement processing section 57, an audio level output control section 58, and a remote controller input processing section. 59 and a central control section 61 are provided and connected to each other. The video and audio input to the video/audio input unit control 55 are input to a parent-child screen display control unit 56 and an audio level output control unit 58, respectively. The screen viewer instructs the display position of the child screen 53 using the remote controller 62, and the instructed data is transferred to the remote controller input processing section 59. The remote controller input processing section 59 inputs the instructed sub-screen display position information to the central control section 61 . The central control unit 61 transfers the input child screen display position information to the parent and child screen display control unit 56 , and the parent and child screen display control unit 56 displays the child screen 53 in the parent screen 52 .
is embedded and displayed on the parent-child screen display monitor 51. At the same time, the central control unit 61 inputs the child screen display position information to the audio level output control unit 58. Audio level output control section 58
Now, the audio output levels of the parent screen and the child screen are controlled based on the child screen display position information. The method of controlling the audio output level is similar to the embodiment shown in FIG. For example, the parent and child screen is shown in Figure 4.
When the screen 22 is displayed as shown in FIG. Ru.

In this way, when two moving images are displayed simultaneously using the parent and child screen method, the audio levels of the parent and child screens output to the two speakers 42 and 43 are controlled according to the display position of the child screen 53. Therefore, it is possible to listen to the voices of the parent screen and the child screen at the same time, and there is an advantage that it is possible to easily recognize which voice is coming from the parent screen or the child screen. Note that in this parent-child screen method, the sound signal accompanying the image (window) is not limited to audio, but may also be music. Although moving images are displayed in the above description, the present invention can also be applied to displaying still images. In the above, multiple speakers were arranged horizontally (horizontally) and the level ratio of the sound output from them was controlled, but if the windows are arranged vertically, the speakers should be arranged vertically (vertically). Then, the sound image position may be localized in the vertical direction in correspondence with the window by controlling the level ratio of the sounds output from these. Furthermore, the speakers may be distributed throughout the periphery of the screen, and the sound image may be localized in correspondence with the secondary position of the window within the screen.

[0019]

Effects of the Invention As explained above, in a system that can display different images in multiple windows using a multi-window display method, a parent-child screen method, etc., multiple videos of multiple images can be displayed depending on the display position of the multiple images being displayed. Since the level of the sound accompanying the image is changed and output to n speakers, as a result, the sound is heard from the position where the image is displayed, so the sound output is lower than that of conventional technology. Since it becomes easier to recognize the speaker, it becomes possible to have a good video conference, or to identify and listen to the sound of the child screen in parent and child screen display.

[0020] Furthermore, when enlarging or reducing a window in order to focus on one of the multiple windows displayed,
By increasing or decreasing the level of the sound signal accompanying the window according to the enlargement/reduction ratio, a video conference with a better user interface becomes possible. Conversely, by enlarging or reducing the display of the window depending on the received audio level, if you want to attract attention, you can call out loud and enlarge the display, and if you leave your seat for a while during the meeting. By displaying the window in a reduced size, the display screen can be used effectively and a video conference with a good user interface can be held.

Further, when the present invention is applied to a parent and child screen display method, there is an advantage that the sound of the parent and child screens can be heard at the same time, and it is possible to easily recognize which voice is coming from the parent screen or the child screen. There is.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment in which the present invention is applied to an audio output method in a display device for a video conference.

2A is a diagram showing divided areas of the screen when the audio level variable range is set to five levels in FIG. 1, and B is a diagram showing the output level ratio of audio from speakers 42 and 43 associated with each area. , C is a diagram showing a state in which the window 25 has been moved from the state in FIG. 1 to the center.

[Fig. 3] A is a diagram showing the divided areas of the screen when there are three speakers and the audio level variable range is set to seven levels, and B is the output from the audio speakers 42, 43, and 44 associated with each area. A diagram showing level ratios.

FIG. 4 is a block diagram showing an embodiment in which the present invention is applied to a sound output method in a parent-child screen system.

FIG. 5 is a block diagram showing a monitor device of a conventional video conference system.

FIG. 6 is a block diagram showing another monitor device of the conventional video conference system.

Claims (2)

[Claims] [Claim 1] Provide multiple windows on one screen,
Regarding a display device that displays different images in each of these windows, in a method of outputting a sound signal accompanying the image, a plurality of speakers are distributed around the screen,
A sound signal output method for displaying a plurality of image windows, characterized in that the sound signals accompanying the images of the windows are distributed to these speakers in inverse proportion to the distance from these speakers to the window, and the sound signals are outputted. . Claim 2: Depending on the scaling of the window,
2. The method of outputting a sound signal for displaying a plurality of image windows according to claim 1, further comprising controlling the level of the sound signal associated with the window.