JP2006270425A - Method and program for outputting sound, and information processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound output method for easily specifying which application an output sound belongs to, and to provide a sound output program, and an information processor. <P>SOLUTION: The sound output method is the one for allowing a plurality of application programs to respectively display a plurality of pictures on a display and outputting the sound corresponding to the respective pictures from a plurality of sound output devices. The method includes: a step for acquiring picture display information which indicates the display positions of the pictures on the display; a step for acquiring acoustic environmental information which indicates respective position relations between the display and the sound output devices; and a step for outputting the sound corresponding to the picture display information from the respective sound output devices, based on the acoustic environmental information. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an audio output method, an audio output program, and an information processing apparatus, and more particularly to an audio output method, an audio output program, and an information processing for performing audio output corresponding to an installation position of a program display device and an installation position of an audio output device. Relates to the device.

Conventionally, general-purpose personal computers (hereinafter abbreviated as PCs) have a function of inputting and outputting voice and images. In recent years, a video conference system via a network has been developed using such a PC. For example, a PC is used in a video conference system that exchanges video and audio of a plurality of people in real time via a digital communication network.
When a plurality of users participate in this video conference system, a plurality of screens (windows) for outputting the images and sounds of each user are displayed on the display of each user. A technique for managing the plurality of screens is disclosed in Patent Document 1. In the multi-window system disclosed in Patent Document 1, the auditory information output is localized at a position corresponding to the window position on the screen.

  The PC operates a plurality of applications on the same PC device that have audio output and applications that simultaneously receive and display video and audio used in the video conference system from each communication partner. In this case, if voices are output almost simultaneously from a plurality of applications or communication partners, it is difficult for the user to understand from which application the output is from only those voices. Therefore, the user needs to check the screen display and determine the application that is outputting the sound.

In addition, the speaker is installed asymmetrically with respect to the display, or is installed at a physically very distant position. Alternatively, the user may use headphones instead of speakers. In such a case, the position of the voice or sound image of each communication partner and the display position on the other party's display may be inconsistent, and the user may feel inconvenience.
JP-A-2-272625

As described above, the conventional PC has a problem in that it is difficult to easily identify which application the output sound belongs to because the user's vision and hearing are different.
The present invention has been made to solve such problems, and provides an audio output method, an audio output program, and an information processing apparatus that can easily identify which application the output audio belongs to. For the purpose.

  The audio output method according to the present invention is a method in which each of a plurality of application programs displays a plurality of screens on a display, and outputs a sound corresponding to each of the plurality of screens from a plurality of audio output devices, Acquiring screen display information indicating display positions of the plurality of screens on the display; acquiring acoustic environment information indicating a positional relationship between the display and each of the plurality of audio output devices; and the acoustic environment information. And a step of outputting a sound corresponding to the screen display information from each of the plurality of sound output devices.

  Furthermore, the step of outputting the sound includes a step of controlling an output volume of the sound output from each of the sound output devices based on the acoustic environment information.

  Furthermore, the step of outputting the sound includes a step of controlling the phase of the sound output from each of the sound output devices based on the acoustic environment information.

  The acoustic environment information includes device arrangement information indicating a positional relationship between the display and each of the plurality of audio output devices, and audio output by the plurality of audio output devices is based on the device arrangement information. Controlled.

As an optimal example, the output volume of the sound can be controlled so that the output volume of the sound output by the sound output device having a small distance from the display is small.
As another optimal example, the phase of the sound can be controlled such that the phase of the sound output by the sound output device having a small distance from the display is delayed.

  The audio output method according to the present invention comprises a step of storing the screen display information. Further, the screen display information is screen position information indicating a position of the screen on the display, and in the step of outputting the sound, each of the plurality of sound output devices is configured according to the screen position relationship. Output audio.

  As an optimal example, when the screen is displayed on the display near the predetermined audio output device, the output sound of the audio output by the predetermined audio output device is increased. The output sound of the sound can be controlled.

Furthermore, the audio output method according to the present invention comprises a step of storing the acoustic environment information.
As an optimal example, the step of storing the acoustic environment information includes a step of displaying a display icon indicating the display on the display, and a sound output device icon indicating the sound output device on the display. And storing device arrangement information indicating the positional relationship between the display and the audio output device in accordance with the positional relationship between the display icon and the audio output device icon on the display. Can have.

  As another optimal example, the step of storing the acoustic environment information includes a step of displaying an input screen for inputting the acoustic environment information on the display, and the display and the audio on the input screen. A step of inputting device arrangement information indicating a positional relationship with the output device; and a step of storing a numerical value indicating the positional relationship between the display and the audio output device according to the input device arrangement information.

  The program according to the present invention causes a computer to realize that each of a plurality of application programs displays a plurality of screens on a display and outputs a sound corresponding to each of the plurality of screens from a plurality of sound output devices. A step of acquiring screen display information indicating display positions of the plurality of screens on the display, and a step of acquiring acoustic environment information indicating a positional relationship between the display and each of the plurality of audio output devices. And outputting a sound corresponding to the screen display information from each of the plurality of sound output devices based on the acoustic environment information.

  An information processing apparatus according to the present invention is an information processing apparatus including the display and the plurality of audio output devices, and further includes the plurality of application programs and the program as described above. .

  ADVANTAGE OF THE INVENTION According to this invention, the audio | voice output method, the audio | voice output program, and information processing apparatus which can specify easily what application an output audio | voice belongs can be provided.

The present invention changes a sound image position according to a display position / size of an application screen on a screen when a plurality of applications are activated simultaneously in a general application operating on a PC. . As an optimal example, the present invention can change the sound image position according to the display position of the application display unit on the screen when two or more types of applications with audio output are operated simultaneously in parallel. Thus, when the present invention is used, it is possible to make the user hear the sound as if the sound is coming from the display portion, regardless of the positional relationship between the display and the sound output device.
Further, when the present invention is used, the sound image position can be changed in accordance with each screen display position and size even in the case of an application that displays a plurality of screens with the same application. As a result, the present invention can be applied to, for example, a video conference or a web conference in which the other party's video and audio are received at the same time and audio communication is performed while each video is displayed on the screen. Can be realized.

Hereinafter, the best mode for carrying out the present invention will be described.
Embodiment 1 of the Invention
In the first embodiment of the present invention, a case will be described in which the present invention is applied to a case where a plurality of applications are simultaneously operated on a general-purpose PC.

First, the main internal configuration of the general-purpose PC according to the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing an example of a main configuration of the internal configuration of the PC.
As shown in FIG. 1, the PC 10 includes a CPU 11, a memory 12, an external storage device 13, a communication control unit 14, a keyboard / mouse 15, a display 16, a left speaker 17, a right speaker 18, a left microphone 19, a right microphone 20, and a camera. 21.
The CPU 11 is an example of a control unit and performs processing control of various operations of the PC 10. The memory 12 is an example of a storage unit, and includes a RAM or the like.

The external storage device 13 is an example of a storage unit, and includes an HDD or the like. The external storage device 13 stores an application for operating the PC 10. The stored application may be an application having a screen display such as an image or a video or an arbitrary audio output. Further, this application may be an application that itself does not perform screen display or audio output.
The communication control unit 14 is an example of a communication unit that controls connection of the PC 10 to the network 140. The PC 10 is connected to the network 140 via the communication control unit 14 and performs processing via the network 140 at the time of execution.

The keyboard / mouse 15 is an example of an input unit, and is used for user input. The keyboard / mouse 15 is controlled by the user input control unit 150.
The display 16 is an example of a display unit, and may be a liquid crystal display device, a CRT, or the like. The display 16 is controlled by the screen display control unit 160.
The left speaker 17 and the right speaker 18 are examples of audio output means, and are stereo speakers arranged on the left and right sides of the display 16. These speakers 17 and 18 are controlled via the audio control unit 170. As the audio output device, a speaker having three or more output channels or headphones can be used.
The microphones 19 and 20 are examples of audio input means, and are stereo microphones that are mounted via the audio control unit 170. The camera 21 is an example of an imaging unit and is mounted via the capture device 210.

As shown in FIG. 1, the PC 10 has two database screen display information DB 22 and an acoustic environment information DB 23.
The screen display information DB 22 is a database that holds screen display information related to the screen displayed on the display 16. For example, the screen display information DB 22 stores application display position information related to the display position of an application on the display 16 and the display position of another application being executed. This display position information includes, for example, the display position and size of the screen.

  The acoustic environment information DB 23 stores sound output environment information indicating information on the relative relationship between the distance between the display 16 and the sound output device. As an example of the audio output environment information, there is a relative positional relationship from the end of the display 16 to the left and right speakers 17 and 18 with respect to the horizontal width of the display 16. Specifically, as the audio output environment information, the right speaker 18 is only 10 cm away from the display 16 while the left speaker 17 is 2 meters away from the display 16 having a width of 30 cm. Can be used as information.

  Furthermore, the acoustic environment information DB 23 is a database that stores acoustic environment information related to the audio output device of the PC 10. For example, the acoustic environment information DB 23 stores audio output device information indicating the type of audio output device, such as whether the device outputting audio is a speaker or headphones, the type of speakers or headphones, or the number of audio channels. .

As another example of the audio output environment information, when the audio output device supports output in the vertical direction and the depth direction, there is a relative positional relationship with the display 16 for each.
These audio output device information and acoustic environment information may be quantitative fixed values input in advance on the screen, or desired numerical values may be selected on the screen from numerical examples created in advance. Alternatively, the user may directly input a desired fixed value. Further, for example, the input can be performed by a qualitative setting method for setting the arrangement state of the display 16 and the speakers 17 and 18 using a GUI (Graphical User Interface). A setting method using this GUI will be described later.

  When such a PC 10 performs an operation based on various applications, the CPU 11 reads the application from the external storage device 13, expands it on the memory 12, and executes it. As a result, the application is activated, and various processes controlled by the CPU 11 are executed by the activated application. Specifically, the activated application reads or generates various data, or stores the various data in various databases such as the memory 12, the communication control unit 14, the screen display information DB 22, and the acoustic environment information DB 23. Or enter.

Next, the processing operation of the PC 10 according to the present invention will be described.
First, the overall processing of the PC 10 according to the present invention will be described with reference to FIG. Here, it demonstrates, referring FIG. 3 suitably. FIG. 2 is a flowchart showing an example of the processing operation of the PC 10 according to the present invention.
The user first activates a predetermined application on the PC 10 (S101). The activated application program reads setting values (screen display information) such as the screen display position and display size of the video from the screen display information DB 22 (S102). The application program inputs the read setting value to the screen display control unit 160. The screen display control unit 160 performs display control of the display 16 based on the input set value, and displays video.

となる。
従って、アプリケーション・プログラムは、音声出力値を決定するとき、画面３０１〜３０３に対応する音声Ｌ_１、Ｒ_１、Ｌ_２、Ｒ_２、Ｌ_３、Ｒ_３を取得し、式（１）、式（２）から左右のスピーカ１７，１８それぞれの出力音量Ｌ、Ｒを算出する。 Based on the read setting value, the application calculates and determines an audio output value to be output by combining the left and right speakers 17 and 18 (S103). The audio output value is determined as follows.
FIG. 3 shows an example of a display screen when the audio output value is determined. In FIG. 3, three applications are activated, and a screen 301, a screen 302, and a screen 303 that are divided into three from the left are displayed beside the display portion 300 of the display 16. For example, a screen 301 displays a word processor application, a screen 302 displays a video playback application, and a screen 303 displays a video conference application. The audio output by these applications is stereo audio, and the audio corresponding to the screen 301 is L ₁ , R ₁ , and the audio corresponding to the screen 302 and the screen 303 is also L ₂ , R ₂ , L ₃ , R ₃ . . At this time, when the left and right speakers 17 and 18 are immediately adjacent to the display 16 of the PC 10, the respective output volumes L and R are, for example,

It becomes.
Therefore, the application program acquires the voices L ₁ , R ₁ , L ₂ , R ₂ , L ₃ , and R ₃ corresponding to the screens 301 to 303 when determining the voice output value, and the expressions (1) and (3) are obtained. From (2), the output volumes L and R of the left and right speakers 17 and 18 are calculated.

  In this way, the sound image position of the application displayed on the screen 301 is determined within a width corresponding to the screen 301. Therefore, it does not overlap with the sound image position of the application on the screen 302 or the screen 303, and it becomes easy to determine which application the sound being heard belongs to. Moreover, if it does as mentioned above, the stereo effect of the audio | voice to output is not impaired and a sense of reality can be increased.

In FIG. 3, three screens 301 to 303 obtained by dividing the display portion 300 into three are displayed. However, when the display portion 300 is divided into n, L and R can be obtained as follows. it can.

  Thereafter, the application program reads acoustic environment information such as the positional relationship between the display 16 and the speakers 17 and 18 from the acoustic environment information DB 23 (S104), and based on the read acoustic environment information, the actual speakers 17 and 18 are read. Is controlled (S105). That is, the application program controls so that the output volume actually output by the output volume from the left speaker 17 and the output volume from the right speaker 18 becomes the audio output value determined based on the set value. Specifically, the application program generates control data for controlling the output amount of sound based on the read acoustic environment information, and inputs the control data to the sound control unit 170. The voice control unit 170 inputs voice control data to the speakers 17 and 18 based on the input control data. The speakers 17 and 18 output sound with an appropriate output amount based on the sound control data (S106).

After the sound is output from the speakers 17 and 18, the application program checks whether or not there is a request to end the application (S107). If there is no request, the application program loops and returns to the processing of S101. Further, the application program ends various processes when there is a termination request.
Through such a series of processes S101 to S107, for example, each time the user enlarges / reduces the size of the display screen of the application or moves the display position of the display screen from right to left, the output volume And the sound image position can follow the screen display.

Next, the screen display information setting process of the PC 10 according to the present invention will be described with reference to FIG. FIG. 4 is a flowchart showing an example of the screen display information setting process.
The user changes the display position and display size of the application displayed on the screen of the display 16 using the keyboard / mouse 15 (S111). Then, the application program acquires the display position and display size of the application from the user input control unit 150 (112), and stores them in the screen display information DB 22. Thereby, the screen display information DB 22 is updated each time (S113), and the setting of the screen display information is completed.
At the same time, the application program acquires data indicating the display position and display size of the changed application from the user input control unit 150. The application program inputs the acquired display position and display size to the screen display control unit 160, and the screen table control unit 160 changes the display screen of the display 16 based on these (S114).

  Here, the display screen is changed in the last S114. However, the present invention is not limited to this, and the display screen may be changed in the middle of S111 to S114. For example, immediately after an instruction to change the display position / size of the application in S111 is performed, the display screen is changed in S114, the display position / size information is acquired in S112, and the screen display information DB 22 in S113 is acquired. Changes may be made.

Furthermore, when the user moves the display position of the video within the application, the same processing as that shown in FIG. 4 is performed. The application program acquires the display position of the video from the user input control unit 150 and stores it in the screen display information DB 22. Thereby, the screen display information DB 22 is updated, and the setting of the screen display information is completed.
At the same time, the application program acquires data indicating the changed display position from the user input control unit 150. The application program inputs the acquired display position to the screen display control unit 160, and the screen table control unit 160 changes the display screen of the display 16 based on these.

  The same applies to the case where the user starts another application. That is, in response to the activation of this other application, the program of this other application acquires data indicating the activation of the application and stores it in the screen display information DB 22. Thereby, the screen display information DB 22 is updated, and the setting of the screen display information is completed. At the same time, the separately activated application program inputs data indicating the display screen of the application to the screen display control unit 160. Based on this, the screen table control unit 160 displays the activated application screen.

Subsequently, the acoustic environment information setting process of the PC 10 according to the present invention will be described with reference to FIG. Here, description will be made with reference to FIGS. 1 and 6 as appropriate. FIG. 5 is a flowchart showing an example of the acoustic environment information setting process. Here, a setting method using the above-described GUI will be described.
The user inputs the acoustic environment on the screen of the display 16 using the keyboard / mouse 15 (S121). FIG. 6 shows an example of a display screen displayed on the display 16. As shown in FIG. 6, a display icon 411, a left speaker icon 412, and a right speaker icon 413 are prepared in advance in the icon field 410 below the display portion 310. The user uses the keyboard / mouse 15 to move the icons 411 to 413 to the arrangement field 420 at the top of the display portion 310 by drag and drop or the like. Then, the application program automatically reads the sizes and display positions of the icons 411 to 413 via the user input control unit 150. The application program calculates the positional relationship between the display 16 and the speakers 17 and 18 based on the read data (S122).

  The application program stores data indicating the calculated positional relationship in the acoustic environment information DB 23, and the acoustic environment DB 23 is updated (S123). Thereby, the setting of the acoustic environment information is completed, and the relative position between the display 16 and the speakers 17 and 18 is set. The icons 411 to 413 may be enlarged or reduced so that the relative position between the display 16 and the speakers 17 and 18 can be set more intuitively. Further, the icons 412 and 413 of the left and right speakers 17 and 18 are not divided into right and left, and the speaker icon on the left of the display icon 411 is automatically discriminated as the left speaker and the speaker icon on the right is automatically discriminated as the right speaker. You may do it.

The setting input of the acoustic environment information DB 23 may be configured so that the database can be directly updated by user input even when the acoustic output processing application according to the present invention is not activated. Furthermore, representative values may be set as initial settings in case the user does not input them.
Further, when the acoustic environment information DB 22 is updated by direct user input, the same processing is performed. In this case, the application need not be running.

  As described above, according to the present invention, the display position of each application displayed on the display 16 is stored in the screen display information DB 22. Therefore, the volume output from each of the left and right speakers 17 and 18 can be adjusted based on the stored screen display information. Therefore, even when a plurality of applications are activated on the display 16, the sound image position can be associated with the display position of each application displayed on the display 16. Therefore, the user can intuitively discriminate by simply listening to the sound without confirming which application the output sound is the sound output from.

  In particular, when the same beep sound is output in a console application or the like, the user can determine the direction of the beep sound that can be heard, and thus the above-described effect becomes remarkable. In addition, even when the left and right speakers 17 and 18 are arranged far from the screen or when there is no screen at the center of the left and right speakers 17 and 18, the screen display position and the sound image position correspond to each other. Effects similar to the above effects can be obtained.

  Further, acoustic environment information such as the position of the display 16 and the positions and types of the speakers 17 and 18 is stored in the acoustic environment information DB 23. At the same time, when changing the output volume, the distance between the display 16 and each of the speakers 17 and 18 is used as a parameter for determining the output volume. Therefore, the output volume from the speakers 17 and 18 can be changed based on this acoustic environment information. Therefore, the sound can be output from the sound image position corresponding to the position of the display 16 without depending on the positional relationship between the display 16 and the speakers 17 and 18.

  For monaural output applications, the present invention can be applied by determining a representative position that determines audio output. In addition, when the number of output channels of the audio output device is 2, the relative relationship between the horizontal width of the display and the distance from the left and right ends of the display 16 to the audio output device can be used as a parameter.

  Furthermore, the acoustic environment information DB 32 can be changed / updated at any time regardless of the activation status of the application. At the same time, the contents of the acoustic environment information DB 23 are referred to every time the output volume is determined. Therefore, regardless of whether the application is running or not, changing the type and position of the audio output device can follow the change in the output volume at any time. Therefore, the user can always listen to the sound in an optimal acoustic environment.

  Thus, since the screen display information of the screen display information DB 22 and the acoustic environment information of the acoustic environment information DB 23 are read, even if the display position of the application screen changes or the number of running applications increases or decreases, The sound image position follows. Therefore, the user can change the video arrangement to his / her preference at any time with peace of mind.

  Further, since the relative position between the display 16 and the speakers 17 and 18 is used as a parameter, there is no need to grasp a quantitative distance. Therefore, the user can easily specify the sound image positions of the display and the speakers 17 and 18 using a user interface such as a GUI without inputting a set value as a quantitative value.

上記の画面３１１，３１２の数を２箇所ではなくｎ箇所とした場合は、以下のように表すことができる。

図８に、音声出力値の決定時における表示画面の他の一例が示されている。この図８は、アプリケーションからの音声出力がモノラルである場合の表示画面である。図８は図７と同様に、２つのアプリケーションを起動し、それぞれを画面に表示している。各画面に対する音声出力の基準点をあらかじめ決定しておく。基準点は通常、ディスプレイ１６の表示部分における横方向の中央などにする。図８では、表示部分３１０の左端を０とし、そこから各基準点までの長さを、それぞれｘ_Ｍ１，ｘ_Ｍ２し、画面３１１の左端から右端までの長さを図７と同様にｘ_ｄｉｓｐとする。また、画面３１１に表示されているアプリケーションのモノラル音声をＭ_１、画面３１２の音声を同様にＭ_２とする。音声を再生するステレオスピーカＬとＲは、画面の左端、右端に配置されている。
このとき、Ｌ、Ｒから出力される音声は例えばそれぞれ以下のようになる。

上記の画面３１１，３１２の数を２箇所ではなくｎ箇所とした場合は、以下のように表すことができる。

Embodiment 2 of the Invention
In the first embodiment of the invention, the display screen at the time of determining the audio output value is divided equally into the screens 301 to 303. However, the present invention is not limited to this, and the application startup screen is displayed at an arbitrary location. Can do.
FIG. 7 shows another example of the display screen when the audio output value is determined. In FIG. 7, two applications are activated, and two screens 311 and 312 are displayed on the display portion 310. The left end of the display portion 310 is set to 0, the lengths from there to the right end and the left end of each screen 311, 312 are respectively x _L1 , x _R1 , x _L2 , x _R2 , and the length from the left end to the right end of the screen 311 is Let x _disp be. Further, the stereo sound of the application displayed on the screen 311 is assumed to be L ₁ and R ₁ , and the sound on the screen 312 is similarly assumed to be L ₂ and R ₂ . The speakers L and R for reproducing stereo sound are arranged at the left end and the right end of the display portion 310 of the display 16.
At this time, sounds output from L and R are as follows, for example.

When the number of the screens 311 and 312 is n instead of two, it can be expressed as follows.

FIG. 8 shows another example of the display screen when the audio output value is determined. FIG. 8 shows a display screen when the audio output from the application is monaural. As in FIG. 7, FIG. 8 starts two applications and displays them on the screen. A reference point for audio output for each screen is determined in advance. The reference point is usually set at the horizontal center of the display portion of the display 16 or the like. In FIG. 8, the left end of the display portion 310 is set to 0, the lengths from there to the respective reference points are x _M1 and x _M2 , respectively, and the length from the left end to the right end of the screen 311 is set to x _disp as in FIG. And In addition, it is assumed that the monaural sound of the application displayed on the screen 311 is M ₁ , and the sound of the screen 312 is similarly M ₂ . Stereo speakers L and R for reproducing sound are arranged at the left and right ends of the screen.
At this time, sounds output from L and R are as follows, for example.

When the number of the screens 311 and 312 is n instead of two, it can be expressed as follows.

  Further, depending on the application, there may be cases where the monaural audio output and the stereo audio output are mixed and executed. In this case, an appropriate output value can be obtained by dividing Formula (7), Formula (8), Formula (11), and Formula (12) into L and R and adding them.

  When these methods are used, even when the applications on the display 16 are arranged apart from each other on the left and right, or on the contrary, they are heard at a sound image position adapted to each screen display position. Can do. That is, no matter how the screen display positions are arranged, the respective sound image positions can be flexibly associated. In addition, when the sound image position such as up and down and depth can be expressed, such as when there are three or more audio output devices, the position difference in the vertical direction of the video may be calculated.

Embodiment 3 of the Invention
In the first to third embodiments of the present invention, the sound volume output from the left and right speakers 17 and 18 is determined from the application screen, but may be determined from the physical arrangement relationship between the PC 10 and the speakers 17 and 18. it can.
FIG. 9 shows an example of an arrangement relationship between the PC 10 and the speakers 17 and 18. The positions of the left speaker 17 and the right speaker 18 are separated from the display 16, the distance from the left speaker 17 to the left end of the display 16 is D _L , the distance from the left end to the right end of the display 16 is D _disp , and the right end of the display 16 is _Let the distance to the right speaker 18 be DR. In addition, the speaker output values when optimized at a position adjacent to the display 16 are L _disp and R _disp respectively. At this time, the output volume balances L and R from the left and right speakers 17 and 18 can be expressed as follows, for example.

  Thus, by using the above formulas (13) and (14), the sound volume is adjusted so that the sound image position is suitable for the position of the display 16 even when the display 16 is not in the center of the speakers 17 and 18. Balance can be determined. Further, when the sound output device can express the sound image position such as up and down and depth, the sound output device may be separately calculated so that the sound image position matches the display position.

Embodiment 4 of the Invention
In the first to third embodiments of the present invention, the case where a speaker is used as an audio output device has been described. However, headphones can be used instead of the speaker. Also in this case, the internal configuration of the PC 10 is the same, and the headphones are connected to the audio output control unit 170 of the PC 10.

FIG. 10 shows an arrangement state when headphones are used as an audio output device. As shown in FIG. 10, a display 51 is disposed in front of the user 50, and the user 50 wears headphones 52. In this case, the user 50 can hear the voice of L _disp from the position on the left and the voice of R _disp from the position on the right. In this case, the sound is heard by the user 50 with a spread of 180 degrees, but the video is displayed on the front. For the user 50, there is a discrepancy between visual and auditory senses, which may cause uncomfortable feelings or confusion.

When the headphones 52 are used, for example, as shown in FIG. 11, a check box 414 indicating use of the headphones 52 is prepared in the icon field 410. When the check box 414 is checked, the left and right speaker icons 412 and 413 are associated with the left and right audio output devices of the headphones 52.
Specifically, when the user checks the check box 414, the application program acquires data indicating that the check box 14 is checked, and stores this data in the acoustic environment information DB 23. The application program reads the acoustic environment information from the acoustic environment information DB 23 together with the data relating to this check, and the same processing as that when the speakers 17 and 18 are used is performed. Accordingly, the audio output amount output from the left and right audio output devices to the headphones 52 is calculated using information on a predetermined relative position.

  Even when the headphones 52 are used, the setting method of the audio output device information and the acoustic environment information is not limited to the qualitative setting method using the GUI as described above. As an example of a quantitative setting method, a user may directly input a quantitative fixed value desired on the screen in advance, or may select a desired numerical value on the screen from numerical examples created in advance. In this way, the user can simulate the environment in a pseudo manner by inputting a specific numerical value.

  Further, in the arrangement field, the situation can be created in a pseudo manner by calculating by arranging the left and right speaker icons 412 and 413 far apart from the horizontal width of the display. If the output adjustment method is used in this way, the position of the sound image that can be heard can be shifted to a desired position such as the center portion in accordance with the position where the display can be seen. As a result, the position of the screen that the user 50 is actually looking at and the position of the sound image that is heard are relatively matched, so that discomfort can be reduced.

  In FIG. 10, the arrangement of the virtual left and right audio output devices in the headphone 52 is determined. However, when the audio output device has three or more output channels, the vertical direction can also be expressed. Yes, depth can be expressed with 5.1ch system. In such a case, it is possible to set the relative positions of the virtual left and right audio output devices in the headphones 52 using the same GUI as in FIG.

That is, even when the headphone 52 is used as an audio output device, the relative position between the display 16 and the virtual left and right audio output devices is used as a parameter. Therefore, even if a quantitative distance cannot be input, it is possible to make the display position on the screen correspond to the sound image position. Therefore, the same effects as those of the first to third embodiments of the present invention can be obtained according to the display position contents of the screen, the viewing angle of the display from the user, and the like.
As described above, since the screen display information of the screen display information DB 22 and the acoustic environment information of the acoustic environment information DB 23 are read each time audio is output to the headphones 52, the audio output device is connected to the speakers 17 and 18 during execution of the application. Even if the headphone 52 is changed, the sound image position follows. Therefore, the user can change the video arrangement to his / her preference at any time with peace of mind.

Embodiment 5 of the Invention
In the first to fourth embodiments of the invention, a case has been described in which when a plurality of applications that simultaneously perform screen display and audio output are executed on the general-purpose PC 10, the display position of each application and the sound image position are matched.
In the present embodiment, a case will be described in which there are a plurality of screen display units and corresponding audio outputs in the same application / system (hereinafter collectively referred to as an application). In this embodiment, a video conference system that receives, displays, and reproduces video and audio from a plurality of communication partners will be described as an example.
In the present embodiment, the video conference application is executed by executing the video conference application on the general-purpose PC. However, the present invention is a system that realizes the video conference function using dedicated hardware and system. Can be applied to.

As shown in FIG. 12, the general-purpose PCs PC 101, PC 102, and PC 103 are connected to the network 140. These PCs 101 to 103 are similarly configured. Here, only the configuration of the PC 101 will be described, and the configuration of the other PCs 102 and 103 will be omitted. FIG. 13 shows the main internal configuration of the PC 101 in this embodiment.
As illustrated in FIG. 13, the PC 101 includes left and right speakers 17 and 18 that are stereo speakers, microphones 19 and 20 that are stereo microphones, and a camera 21. Further, the speaker may not be a stereo speaker but may be a speaker having an output of three or more channels, or a headphone can be used instead of the speaker. A monaural microphone may be used as the microphone. Further, when the video is not transmitted, the camera 21 may not be provided. In this case, it is assumed that a screen or the like that can be used as a substitute for the video is displayed for the recipient of the communication partner. The other configurations of the PC 101 are the same devices having the same reference numerals as those in the configuration of the PC 10, and the description thereof is omitted here.

As shown in FIG. 13, the PC 101 has a received video display information DB 221 instead of the screen display information DB 22.
The received video display information DB 221 is a database that holds screen display information related to a screen on which data received by the PC 101 is displayed on the display 16. For example, the received video display information DB 221 stores the display position of an application that displays the video of the communication partner on the display 16. This display position information includes, for example, the display position and size of the screen.
The received video display information DB 221 is a database having the same function as the screen display information DB 22, and the display position of each video is changed or updated as needed by direct user input to the database.

FIG. 14 is a flowchart showing an example of the overall processing of the PC 101 in the video conference system.
As shown in FIG. 14, the PC 101 activates the video conference application (S131), and starts communication with the other PCs 102 and 103 (S132). As a result, the PC 101 is communicating with the PC 20 and the PC 30 via the network 28, and transmits and receives the respective video and audio. At this time, the application program inputs a control signal for executing connection with the network 140 to the communication control unit 14. Based on this control signal, the communication control unit 140 performs communication control and establishes a connection with the network 140.
Here, the communication connection method differs depending on the system. For example, in the case of the P2P (Peer to Peer) method, in order to transmit and receive video and audio from each terminal, communication is established with each of the PCs 101 to 103 participating in the conference. That is, for example, when communicating with 5 terminals other than its own terminal, the PC 10 establishes 5 communications. In addition, when the participant of a meeting is 1 terminal other than an own terminal, it will become 1 to 1 communication. When using a server such as an MCU (Multipoint Control Unit), communication is generally established only between the PC and the MCU. However, there may be a connection form having both of the above properties, such as an audio only MCU system and a video P2P system.

The TV conference application program reads the setting values (received video display information) related to the display position information of each received video from the received video display information DB 126 and displays them on the screen (S133). The application program inputs the read setting value to the screen display control unit 160. The screen display control unit 160 performs display control of the display 16 based on the input set value, and displays video.
The video conference application program calculates and determines an audio output value to be output by combining the left and right speakers 17 and 18 based on the read setting value (S134). As in the first to third embodiments of the present invention, the sound output value is determined by Expression (7) and Expression (8) when the received sound is stereo, and Expression (11) when the received sound is monaural. Using a method such as Expression (12), the volume is adjusted so that the sound image position corresponds to the display position of the video.

  Thereafter, the video conference application program reads the acoustic environment information from the acoustic environment information DB 23 (S135), and actually controls the output volume of the speakers 17 and 18 based on the read acoustic environment information (S136). The speakers 17 and 18 output sound with an appropriate output amount based on the acoustic environment information (S137). After the sound is output from the speakers 17 and 18, the application program checks whether there is an end request (S138), and if there is no request, loops and returns to the processing of S131. Further, the video conference application program terminates various processes when requested to terminate.

  When the PC 101 in this embodiment sets the received video display information, the display position and display size of the video conference application on which the communication partner using the PCs 102 and 103 is displayed are changed. The PC 101 performs the same setting process as the PC 10 in accordance with the change of the display position / size.

  As described above, the present invention can also be used in a system that simultaneously displays and reproduces a plurality of videos and sounds in an application, such as a video conference system. By applying the present invention, it is possible to listen to the sound at the sound image position that matches the screen position regardless of the type and position of the sound output device. For this reason, the display position of the video and the sound image position of the sound coincide with each other, so that the user can easily determine which video the person is listening to.

  In addition, even if the display size and display position of the video are changed during the conference, the sound image position can be tracked at any time, so that the user can change the video size and position with peace of mind. For example, if the position of the image of the person who wants to hear at the center is moved to the center of the display, the sound can be heard from the center and concentrated.

  Alternatively, if the display size is increased, a wider voice output can be obtained from the left and right, and the person's speech can be heard more easily. In addition, even when audio is received in stereo, audio can be output without impairing stereo effects such as left and right volume differences. As described above, when the present invention is applied to the video conference system, more realistic communication can be realized.

Other Embodiments of the Invention
In the first to fifth embodiments of the present invention, the display position and the sound image position of the application screen are adjusted by adjusting the sound volume of the sound output devices (speakers 17, 18 and headphones 52) on the left and right based on the contents of the sound environment information DB23. And made it correspond. On the other hand, the display position of the application screen and the sound image position can be made to correspond by controlling the phases of the left and right output sounds. In this embodiment, a method by this phase adjustment will be described. Here, description will be made with reference to FIG.

  With the activation of the application, the activated application program reads setting values (screen display information) such as the screen display position and display size of the video from the screen display information DB 22. The application program inputs the read setting value to the screen display control unit 160. The screen display control unit 160 performs display control of the display 16 based on the input set value, and displays video.

  The application program calculates and determines the phase of the sound output from the left and right speakers 17 and 18 based on the read setting value. The application program reads the acoustic environment information such as the positional relationship between the display 16 and the speakers 17 and 18 from the acoustic environment information DB 23. The application program controls the phase of the sound actually output from the speakers 17 and 18 based on the read acoustic environment information.

Specifically, the application program generates control data for controlling the phase of the voice based on the read acoustic environment information, and inputs the control data to the voice control unit 170. The audio control unit 170 inputs phase control data to the speakers 17 and 18 based on the input control data. The speakers 17 and 18 output sounds having different phases on the left and right based on the phase control data. That is, sounds having different times to reach the user are output from the left and right speakers 17 and 18.
The application program confirms whether there is an end request after the sound is output from the speakers 17 and 18, and executes a loop or end process.
Thus, the same effects as those of the first to fifth embodiments can be obtained also by using the phase of the output sound.

It is a schematic diagram which shows an example of the internal structure of PC concerning this invention. It is a flowchart which shows an example of processing operation of PC concerning the present invention. It is a schematic diagram which shows an example of the display screen of PC concerning this invention. It is a flowchart which shows an example of processing operation of PC concerning the present invention. It is a flowchart which shows an example of processing operation of PC concerning the present invention. It is a schematic diagram which shows an example of the display screen of PC concerning this invention. It is a schematic diagram which shows an example of the display screen of PC concerning this invention. It is a schematic diagram which shows an example of the display screen of PC concerning this invention. It is a schematic diagram which shows an example of the arrangement | positioning state of PC which concerns on this invention. It is a schematic diagram which shows an example of the arrangement | positioning state of PC which concerns on this invention. It is a schematic diagram which shows an example of the display screen of PC concerning this invention. It is a schematic diagram which shows an example of the arrangement | positioning state of PC which concerns on this invention. It is a schematic diagram which shows an example of a structure of PC concerning this invention. It is a flowchart which shows an example of processing operation of PC concerning the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... PC, 11 ... CPU, 12 ... Memory, 13 ... External storage device, 14 ... Communication control part,
15 ... Keyboard / mouse, 16 ... Display, 17 ... Left speaker, 18 ... Right speaker,
19 ... Left microphone, 20 ... Right microphone, 21 ... Camera,
22 ... screen display information DB, 23 ... acoustic environment information DB
140 ... network, 150 ... user input control unit, 160 ... image display control unit,
170: Voice control unit, 210 ... Capture device

Claims (14)

Each of a plurality of application programs displays a plurality of screens on a display, and outputs a sound corresponding to each of the plurality of screens from a plurality of sound output devices,
Obtaining screen display information indicating display positions of the plurality of screens on the display;
Obtaining acoustic environment information indicating a positional relationship between the display and each of the plurality of audio output devices;
An audio output method comprising a step of outputting audio corresponding to the screen display information from each of the plurality of audio output devices based on the acoustic environment information.   2. The audio output method according to claim 1, wherein the step of outputting the audio includes a step of controlling an output volume of audio output from each of the audio output devices based on the acoustic environment information.   2. The sound output method according to claim 1, wherein the step of outputting the sound includes a step of controlling a phase of sound output from each of the sound output devices based on the acoustic environment information. The acoustic environment information includes device arrangement information indicating a positional relationship between the display and each of the plurality of audio output devices,
The audio output method according to claim 2 or 3, wherein audio output by the plurality of audio output devices is controlled based on the device arrangement information.   5. The audio output method according to claim 2, wherein the audio output volume is controlled so that an audio output volume output by an audio output device having a small distance from the display is reduced.   5. The audio output method according to claim 3, wherein the phase of the audio is controlled so that the phase of the audio output by the audio output device having a small distance from the display is delayed.   The audio output method according to claim 1, further comprising a step of storing the screen display information. The screen display information is screen position information indicating a position of the screen on the display,
8. The audio output method according to claim 7, wherein in the step of outputting the audio, each of the plurality of audio output devices outputs the audio according to the screen positional relationship.   When the screen is displayed on the display on the side close to the predetermined audio output device, the output audio of the audio is set so that the output audio of the audio output by the predetermined audio output device is increased. 9. The audio output method according to claim 8, wherein control is performed.   The audio output method according to claim 1, further comprising a step of storing the acoustic environment information. Storing the acoustic environment information comprises displaying a display icon indicating the display on the display;
Displaying an audio output device icon indicating the audio output device on the display;
Storing device arrangement information indicating a positional relationship between the display and the audio output device according to a positional relationship between the display icon and the audio output device icon on the display. The audio output method according to claim 10. Storing the acoustic environment information includes displaying an input screen for inputting the acoustic environment information on the display;
Inputting device arrangement information indicating a positional relationship between the display and the audio output device on the input screen;
11. The audio output method according to claim 10, further comprising a step of storing a numerical value indicating a positional relationship between the display and the audio output device according to the input device arrangement information. Each of the plurality of application programs is a program for causing a computer to display a plurality of screens on a display and to output a sound corresponding to each of the plurality of screens from a plurality of sound output devices,
Obtaining screen display information indicating display positions of the plurality of screens on the display;
Obtaining acoustic environment information indicating a positional relationship between the display and each of the plurality of audio output devices;
An audio output program for causing a computer to include a step of outputting audio corresponding to the screen display information from each of the plurality of audio output devices based on the acoustic environment information. The display;
An information processing apparatus having the plurality of audio output devices,
The plurality of application programs;
An information processing apparatus further comprising the program according to claim 13.

Images