CN109804644B

CN109804644B - Sound output method, computer device and computer readable recording medium

Info

Publication number: CN109804644B
Application number: CN201680089777.7A
Authority: CN
Inventors: 崔源晙
Original assignee: Line Corp
Current assignee: Aiwanwai Co ltd; From Zhongzhong Global Co ltd
Priority date: 2016-10-13
Filing date: 2016-10-13
Publication date: 2021-06-01
Anticipated expiration: 2036-10-13
Also published as: CN109804644A; WO2018070564A1

Abstract

The invention provides a sound output method and device for a plurality of displays. The sound output method of a computer device including a plurality of display devices or connected to a plurality of the display devices may include: a step of determining a first coordinate of a specific object on a whole screen displayed by a plurality of display devices; determining a display device corresponding to the specific object among the plurality of display devices based on the determined first coordinates; and outputting a sound set for the specific object through the sound output device included in the determined display device, when the determined display device includes the sound output device.

Description

Sound output method, computer device and computer readable recording medium

Technical Field

The following description relates to a sound output method and apparatus for a plurality of displays, and a computer program stored in a computer-readable recording medium for executing the sound output method in a computer by being combined with the computer.

Background

There are various conventional techniques for outputting sound in relation to a screen displayed by a display device. For example, korean laid-open patent No. 10-2008-0108839 relates to a screen sound source localization device, a speaker group information generation method for screen sound source localization, and a playback method for a localized screen sound source, and discloses a technique for selecting a specific speaker group among a plurality of speakers set and provided on the periphery of a display device so as to obtain the same Gain (Gain) in order to localize a virtual screen sound source in synchronization with an image object existing at a specific position on the screen of the display device.

However, the above-described conventional technique relates to a technique for setting a virtual sound source in relation to one screen displayed by one display device, and as described above, the conventional technique only considers a case where a sound is output in relation to one screen displayed by one display device, and does not consider a case where a plurality of screens (for example, a plurality of windows) exist by a plurality of displays. Therefore, there is a problem that the directionality of the sound outputted in relation to the specific object by the plurality of displays cannot be accurately provided, and an unnatural sound is outputted depending on which display device among the plurality of display devices the content such as a game or a video is displayed.

Disclosure of Invention

Technical problem

The present invention provides a sound output method and system that can output a sound set for a specific object according to which position of which display device the specific object exists when a computer device including or connected to a plurality of display devices displays the specific object through the plurality of display devices, and a computer program stored in a computer-readable recording medium in order to execute the sound output method in a computer by being combined with the computer.

The present invention provides a sound output method and system that can output a sound set for a specific object according to where the specific object exists on a display device in consideration of a sound output device included in the display device, and a computer program stored in a computer-readable recording medium for executing the sound output method on a computer in conjunction with the computer.

Means for solving the problems

The present invention provides a sound output method for a computer device including a plurality of display devices or connected to the plurality of display devices, the sound output method including: a step of determining a first coordinate of a specific object on a whole screen displayed by a plurality of display devices; determining a display device corresponding to the specific object among the plurality of display devices based on the determined first coordinates; and outputting a sound set for the specific object through the sound output device included in the determined display device, when the determined display device includes the sound output device.

The present invention provides a computer program stored in a computer-readable recording medium for executing a sound output method in a computer device including a plurality of display devices or connected to a plurality of display devices by being combined with the computer device, the sound output method including: a step of determining a first coordinate of a specific object on a whole screen displayed by a plurality of display devices; determining a display device corresponding to the specific object among the plurality of display devices based on the determined first coordinates; and outputting a sound set for the specific object through the sound output device included in the determined display device, when the determined display device includes the sound output device.

The present invention provides a computer device for executing a sound output method, the computer device including at least one processor for executing computer readable instructions, the at least one processor determining first coordinates of a specific object on an entire screen being displayed by a plurality of display devices included in the computer device or a plurality of display devices connected to the computer device, the at least one processor determining a display device corresponding to the specific object among the plurality of display devices based on the determined first coordinates, and in a case where the determined display device includes an acoustic output device, outputting an acoustic set for the specific object through the acoustic output device included in the determined display device.

ADVANTAGEOUS EFFECTS OF INVENTION

The sound set for the specific object may be output according to which position of which display device the specific object exists when the specific object is displayed through the plurality of display devices by a computer device including the plurality of display devices or connected to the plurality of display devices.

The sound output device included in the display device may be considered to output sound set for a specific object according to which position of which display device the specific object exists.

Drawings

Fig. 1 is a diagram showing an example of a sound output environment according to an embodiment of the present invention.

Fig. 2 is a diagram showing an example of sound output in an embodiment of the present invention.

Fig. 3 is a block diagram for explaining the internal structure of a computer device according to an embodiment of the present invention.

Fig. 4 is a block diagram showing an example of structural elements that a processor of a computer device may include in one embodiment of the present invention.

Fig. 5 is a flowchart illustrating an example of a sound processing method executable by a computer device according to an embodiment of the present invention.

Fig. 6 is a flowchart for explaining an example of outputting sound by the sound output device included in the specified display device in one embodiment of the present invention.

Fig. 7 is a flowchart for explaining an example of outputting sound in a case where the determined display device does not include a sound output device in one embodiment of the present invention.

Fig. 8 is a diagram for explaining the first coordinates and the second coordinates of the object in one embodiment of the present invention.

Fig. 9 is a diagram showing an example of determining coordinates of a plurality of layered objects in one embodiment of the present invention.

Fig. 10 is a diagram showing an example of determining the coordinates of a minimized window object in an embodiment of the present invention.

Fig. 11 is a diagram showing an example of an object located outside the screen of the display device in an embodiment of the present invention.

Fig. 12 is a diagram showing an example of determining the first coordinate of the object beyond the screen by the virtual coordinate system in the embodiment of the present invention.

Fig. 13 is a diagram showing an example in which a plurality of display devices display screens connected to each other in one embodiment of the present invention.

Fig. 14 is a diagram showing an example in which a plurality of display devices display mutually independent screens, respectively, in one embodiment of the present invention.

Detailed Description

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

The sound output method of the embodiment of the present invention may be performed by a computer device directly including a plurality of display devices or connected to a plurality of display devices through a network or the like. For example, a computer program (for example, an operating system or an application program) according to an embodiment of the present invention may be installed and driven in a computer device, and the computer device may execute the sound output method according to an embodiment of the present invention according to the control of the driven computer program. The computer program may be stored in a computer-readable recording medium for executing the sound output method in the computer device by being combined with the computer device.

Fig. 1 is a diagram showing an example of a sound output environment according to an embodiment of the present invention. Fig. 1 shows an example in which a computer device 100 is connected to a plurality of

display devices

111, 112, and 113 and a plurality of

speakers

121 and 122.

In this case, the specific object (for example, a computer window (window) object) output by the computer device 100 may be displayed by one or more of the plurality of

display devices

111, 112, and 113. In other words, with the use of

multiple display devices

111, 112, 113, there will be multiple possibilities that a particular object may be displayed.

In addition, in the case where at least one of the plurality of

display devices

111, 112, and 113 includes an acoustic output device, the plurality of

speakers

121 and 122 may be omitted. The plurality of

speakers

121 and 122 may be provided independently of whether or not at least one of the plurality of

display devices

111, 112, and 113 includes an acoustic output device. Furthermore, the number of the acoustic output devices included in the plurality of

display devices

111, 112, and 113 may be two or more, instead of one. In other words, there is a greater possibility that sound can be output than in the case where the plurality of

speakers

121, 122 are simply used through the plurality of

display devices

111, 112, 113.

As described above, the plurality of display environments can make various paths on which a specific object can be displayed, and also, various paths on which set sounds are output for the specific object. Therefore, the computer device 100 outputs the sound set for the specific object so as to correspond to the plurality of situations, and thereby, from the viewpoint of the user 130, the sound corresponding to the specific object displayed on the screen can be output.

Fig. 2 is a diagram showing an example of sound output in an embodiment of the present invention. Fig. 2 shows an example of displaying an object 210 by a third display device 113 of the plurality of

display devices

111, 112, 113. In this case, if the sound set for the object 210 is output only through the plurality of

speakers

121 and 122 or through the sound output device included in the first display device 111 or the second display device 112, the user will feel unnatural to the output sound. Therefore, the computer device 100 can cause the sound set for the object 210 to be output through the sound output device included in the third display device 113 on which the object 210 is displayed.

Furthermore, as shown in fig. 2, when the object 210 is displayed on the right side of the third display device 113 and when the sound set for the object 210 is multi-channel sound, the computer device 100 can output multi-channel sound so as to generate a virtual sound source for the object 210 displayed on the right side of the third display device 113 by adjusting the left-right balance of the multi-channel sound to be output by the sound output device included in the third display device 113. The prior art of imparting directionality to multi-channel sound is well known, and one of ordinary skill in the art can readily appreciate that the balancing of multi-channel sound described above can be handled using one of the known prior art techniques.

As described above, in the multi-display environment, the sound set for the object 210 is output by the sound output device of the appropriate display device (the third display device 113) according to the display position of the object 210, whereby more natural sound can be provided to the user 130.

Fig. 3 is a block diagram for explaining the internal structure of a computer device according to an embodiment of the present invention. Fig. 3 shows the physical internal structural elements for implementing the computer device 100, showing the memory 310, the processor 320, the communication module 330, and the input-output interface 340.

The memory 310 is a computer-readable recording medium and may include a permanent mass storage device (persistent mass storage device) such as a Random Access Memory (RAM), a Read Only Memory (ROM), and a disk drive. Wherein, the ROM and the permanent mass storage device can be separated from the memory 310 and disposed as an additional permanent storage device in the computer device 100. The memory 310 may store an operating system and/or at least one program code (for example, a computer program that is stored in a recording medium included in the computer apparatus 100 and controls the computer apparatus 100 in a manner of executing the sound output method according to an embodiment of the present invention). Such software structural elements may be loaded into the memory 310 from a computer-readable recording medium distinguished from the memory 310. The computer-readable recording medium may include, among others, a floppy disk drive, a magnetic disk, a magnetic tape, a DVD/CD-ROM drive, a memory card, and the like. In another embodiment, the software structural elements may be loaded into the memory 310 through the communication module 330, not through the computer-readable recording medium.

The processor 320 may be configured to perform basic computations, logic, and input-output operations, thereby processing computer program-based instructions. The instructions may be provided to the processor 320 through the memory 310 or the communication module 330. For example, the processor 320 may be configured to execute instructions received from the memory 310 in accordance with program code loaded in the memory 310. As a more specific example, the processor 320 can control the computer apparatus 100 in such a manner that the sound output method is executed by sequentially executing instructions based on the codes of the computer program loaded in the memory 310.

The communication module 330 may optionally be included in the computer device 100 and may provide functionality for communicating with other physical devices over an actual computer network. As an example, data processed by processor 320 of computer device 100 may be sent to computer device 100 from other physical devices on a computer network through computer network and communication module 330 and transferred to memory 310 or processor 320. Conversely, the processing result of the operation on the data received by the computer device 100 may be transmitted to other physical devices through the communication module 330 and the computer network.

The input/output interface 340 may be an interface with the input/output device 350. For example, in the input/output device 350, the input device may include a keyboard, a mouse, or the like, and the output device may include a plurality of the above-described

display devices

111, 112, 113 or a plurality of

speakers

121, 122. In this case, the input-output interface 340 may include an input-output port for connecting with the input-output device 350. In fig. 3, the input/output device 350 is a device distinguished from the computer device 100, but according to an embodiment, the computer device 100 may be implemented by the computer device 100 including the input/output device 350.

Also, in another embodiment, computer device 100 may include more structural elements than those in FIG. 3. However, most of the prior art structural elements need not be explicitly shown. For example, computer device 100 may also be implemented to include various structural elements such as various physical buttons or sensors.

Fig. 4 is a block diagram illustrating an example of a structural element that a processor of a computer device may include in an embodiment of the present invention, and fig. 5 is a flowchart illustrating an example of a sound processing method that may be performed by the computer device in an embodiment of the present invention. Fig. 4 shows components that the processor 320 of the computer device 100 may include, and may include a coordinate determination section 410, a display device determination section 420, and an acoustic output control section 430. The processor 320 and the structural elements of the processor 320 may perform the steps (steps 510 to 550) included in the sound output method shown in fig. 5. In this case, the processor 320 and the structural elements of the processor 320 may be implemented as instructions (instructions) that execute code based on an operating system included in the memory 310 and/or code based on at least one computer program. The structural elements of the processor 320 may be expressions of different functions (differential functions) of the processor 320 executed by the processor 320 according to control instructions provided by codes of a computer program stored in the computer apparatus 100. For example, the coordinate determination unit 410 may be used to control the processor 320 of the computer device 100 to express the function of the processor 320 so that the processor 320 determines the coordinates of a specific object within the screen.

In step 510, the coordinate determination unit 410 may determine the first coordinate of the specific object on the entire screen displayed by the plurality of display devices. The specific object may be one of the structural elements displayed on the screen and individually recognizable, such as a skin of a computer window or a video player, and a button or an area included in the skin of the computer window or the video player.

Referring to the above examples in fig. 1 and 2, the coordinate determination unit 410 can determine the coordinates of the object to be displayed on the entire screen displayed by the plurality of

display devices

111, 112, and 113. As a specific example, the first coordinates for the specific object may be specified in a virtual coordinate system (or a coordinate system for identifying pixels of the entire screen displayed by the plurality of

display devices

111, 112, and 113) having the leftmost upper end of the first display device 111 as the origin (0, 0). In this case, the first coordinate may be determined as a specific coordinate point in the coordinates formed by the specific object, such as a center point or the leftmost upper coordinate in the coordinates formed by the specific object. As another example, the first coordinate may be a coordinate pair including a leftmost coordinate and a rightmost coordinate among coordinates formed by the specific object. As another example, the first coordinates may be a set of coordinates including leftmost and rightmost coordinates and uppermost and lowermost coordinates among the coordinates formed by the specific object. Such first coordinates are used to determine which display device among the plurality of

display devices

111, 112, 113 the corresponding object is displayed by, and may be used without limitation as long as the coordinates or the set of coordinates are used for determination of such a display device.

In step 520, the display device determination part 420 may determine a display device corresponding to the specific object among the plurality of display devices based on the determined first coordinates. For example, the display device identification unit 420 may identify a portion corresponding to each display device in the coordinate system for the entire screen, and may identify a display device corresponding to the specific object according to which portion the first coordinate is included in.

In step 530, the acoustic output control unit 430 checks whether the determined display device includes an acoustic output device, and if the determined display device includes an acoustic output device, step 540 is executed, and if the determined display device does not include an acoustic output device, step 550 is executed. For example, the operating system of the computer apparatus 100 may know whether the corresponding display apparatus includes an audio output apparatus from an Interface such as a High Definition Multimedia Interface (HDMI) or a display port. For example, the operating system may acquire information such as whether an external device such as a display device includes an additional acoustic output device through handshaking (handshaking) with information of the external device. Accordingly, the processor 320 of the computer device 100 can know whether the connected display device includes the acoustic output device through the operating system. In the case where the acoustic output control section 430 performs a control operation according to a specific application, the corresponding application can acquire information on whether or not the corresponding display device includes an acoustic output device from the operating system. In the event that the operating system is unable to obtain information from the display device regarding the absence or presence of an audio output device, the computer device 100 may receive or store additional data regarding whether an audio output device is included per display device. In this case, whether the corresponding display device includes the acoustic output device may be confirmed by searching for the received or stored additional data through an identifier of the display device or a device name or a model name, etc.

In step 540, the acoustic output control unit 430 may output the acoustic set for the specific object through the acoustic output device included in the specified display device. In this case, since the sound set for the specific object is output by the sound output device of the display device corresponding to the current specific object, a more natural sound can be output to the user. When the position of the object is changed and the display device corresponding to the object is changed, the sound set for the corresponding object can be output by the sound output device of the other display device that is changed.

In step 550, the sound output control unit 430 may output a sound set for the specific object through a speaker device included in the computer device 100 or connected to the computer device 100.

The above steps (steps 510 to 550) describe the case of outputting monaural sound, but in the case where sound to be output is multi-channel sound, sound can be output by more methods based on conditions, whereby more natural sound can be output to the user.

Fig. 6 is a flowchart for explaining an example of outputting sound by the sound output device included in the specified display device in one embodiment of the present invention. The steps shown in fig. 6 (steps 610-630) may be included in step 540 previously described.

In step 610, when the set sound is multi-channel sound, the sound output control unit 430 may check the second coordinates of the specific object in the virtual coordinate system associated with the first screen being displayed on the specified display device. The virtual coordinate system related to the first screen may be a virtual coordinate system having the leftmost upper end of the first screen as an origin (0, 0). For example, the coordinate determination unit 410 may determine the second coordinate in the virtual coordinate system related to the determined screen of the display device, but may not determine the second coordinate in the virtual coordinate system of the entire screen, and the acoustic output control unit 430 may confirm the second coordinate of the object determined by the coordinate determination unit 410. In other words, the second coordinates may include coordinates in the screen for the determined one display device, and not coordinates in the entire screen for the plurality of display devices, and such second coordinates may be used to determine the position of the object within the determined display device.

In step 620, the acoustic output control unit 430 may balance the multi-channel sound based on the relative position of the specific object based on the confirmed second coordinates. For example, the acoustic output control unit 430 may balance the multi-channel sound so that the multi-channel sound output from the acoustic output device generates a virtual sound source at a virtual position indicated by the second coordinates, in order to balance the multi-channel sound. As described above, the step of balancing the multi-channel sound for generating a virtual sound source at a specific position may be implemented using one of the existing technologies.

In step 630, the acoustic output control unit 430 may output the balanced multi-channel acoustic through the acoustic output device. Therefore, the user can receive natural sound relating to the position of the object by not only receiving natural sound relating to the position of the object in the screen displayed by the plurality of display devices through sound output from the sound output device of the display device corresponding to the specific object, but also receiving balanced multi-channel sound in order to produce the same effect as sound produced by the virtual sound source generated corresponding to the position of the object in the selected display device, thereby receiving more natural sound.

The balance can be used also in the case of outputting multi-channel sound through a general speaker device. In this case, instead of using the second coordinates which are the coordinates of the object in the virtual coordinate system for the screen of the specified display apparatus, the first coordinates which are the coordinates of the object in the virtual coordinate system for the entire screen of the plurality of display apparatuses are used.

Fig. 7 is a flowchart for explaining an example of outputting sound in a case where the determined display device does not include a sound output device in one embodiment of the present invention. The steps shown in fig. 7 (steps 710 and 720) may include step 550 previously described.

In step 710, the acoustic output control unit 430 may balance the multi-channel sound so that the multi-channel sound output from the speaker device generates a virtual sound source at a virtual position indicated by the first coordinates of the specific object. Accordingly, the sound output control section 430 can deal with the balance of multi-channel sound so that the effect of generating multi-channel sound in a virtual sound source can be produced. According to the embodiment, the computer device 100 may previously store data on the configuration of the display device and data on how to balance the multi-channel sound according to the configuration of the display device, and may also determine the position of the virtual sound source by the stored data and the position information acquired by the data on the configuration of the current display device and the second coordinates which are relative coordinates acquired in the respective displays. As described above, the balance of multi-channel sound can be handled using one of known prior art (e.g., a 3D sound source generation method such as a Head-Related Transfer Function (HRTF)).

In step 720, the sound output control unit 430 may output the balanced multi-channel sound through the speaker device. Therefore, when sound is output by the speaker device, the position of the virtual sound source is determined not based on the position of the object in one display device but based on the positions of the objects in the entire screen for the plurality of display devices, and multi-channel sound is balanced and displayed so that the same effect as that of outputting the sound source by the determined virtual sound source can be obtained, and therefore, more natural sound can be output to users using the plurality of display devices.

In the above-described embodiment, the first coordinates may be determined as two-dimensional coordinates of the specific object on the entire screen regardless of the depth of the object layered on the entire screen for display. For example, even if a first object is occluded by a second object and is not displayed on the screen, the first coordinates may be determined as two-dimensional coordinates on the entire screen regardless of the depth of the first object in the screen. Similarly to the first coordinates, the second coordinates may also be determined as two-dimensional coordinates regardless of the determined depth on the screen of the display device.

Also, according to an embodiment, the specific object may include a computer window (window) object. For example, it may be a window in which a particular game is being executed or a window in which a video is being played. Such a window can disappear from the screen by minimization, and even in this case, the sound of the game or video being executed or being played can be continuously output. In this case, the first coordinates may include coordinates at which the window object is finally displayed on the entire screen. Similarly, the second coordinates may also include coordinates at which the window object is last displayed on the determined screen of the display device. Therefore, even if a specific window object is not displayed on the screen by being minimized, the user can feel that sound is output from the virtual sound source at the position where the window object is displayed last, and thus can receive more natural sound.

Fig. 8 is a diagram for explaining the first coordinates and the second coordinates of the object in one embodiment of the present invention. Fig. 8 shows an example in which a virtual coordinate system is set on the entire screen 810 for a plurality of display devices. In this case, in the virtual coordinate system, assuming that the horizontal axis is an x axis and the vertical axis is a y axis, the coordinates of the virtual coordinate system are represented by (x, y), the coordinates of the leftmost upper end position of the entire screen 810 are (0, 0), and the coordinates of the rightmost lower end are (30, 7). Fig. 8 shows an example in which the virtual coordinate system set on the entire screen 810 is applied to the

individual screens

820, 830, and 840 of the plurality of display devices. In this case, the first coordinates of the object 850 displayed on the second screen 830 may be determined as (17, 6) by the virtual coordinate system for the entire screen 810. In contrast, since the second coordinate is a position in the coordinate system for the second screen 830, it can be determined as (7, 6). In other words, the first coordinates may be determined using a virtual coordinate system of the entire screen, and the second coordinates may be determined using a virtual coordinate system for the corresponding display device. As an example of determining such coordinates, those skilled in the art can easily understand that any method that can relatively express the object position in the entire screen 810 and the object position in the

individual screens

820, 830, 840, such as using the pixel coordinate systems of the

individual screens

820, 830, 840 as the virtual coordinate system, can be used.

Fig. 9 is a diagram showing an example of determining coordinates of a plurality of layered objects in one embodiment of the present invention. Fig. 9 shows a second object 930 superimposed with the first object 920 in the screen 910 of one display apparatus. In fig. 9, the first object 920 is presented in the form of a crypt, but this is for convenience of illustration only, and in essence, it is assumed that the first object 920 is completely obscured from display by the second object 930 in the screen 910. In this case, although the first object 920 is not displayed to the user, the computer device 100 knows the existence of such a first object 920 and can determine the first coordinates or the second coordinates for the first object 920. Even if the first object 920 is not displayed but it is necessary to output the sound set for the first object 920, the computer device 100 may output the sound using the determined first coordinate or second coordinate.

Fig. 10 is a diagram showing an example of determining the coordinates of a minimized window object in an embodiment of the present invention. Fig. 10 shows an example in which a window object 1011 is displayed on the first screen 1010 of one display device. In this case, the dashed circle 1012 represents an example of a user interface that can minimize the window object 1011. If the user selects the corresponding user interface (e.g., by mouse clicking on an area where the corresponding user interface is displayed), the first screen 1010 may be changed to a state where the window object 1011 disappears, as with the second screen 1020. In this case, the computer device 100 may determine the first coordinate or the second coordinate through the coordinate displayed last by the window object 1011. For example, in the case of playing a video through the window object 1011, even if the window object 1011 disappears like the second screen 1020 by minimization, the computer apparatus 100 can output the sound of the video with the coordinates displayed last by the window object 1011 as a reference, thereby naturally providing the sound of the video to the user even if the window object 1011 suddenly disappears.

In the above embodiment, the case where the object is located within the screen of the display device is described. However, the object may be located outside the screen of the display device.

Fig. 11 is a diagram showing an example of an object located outside the screen of the display device in one embodiment of the present invention. Fig. 11 shows an example in which a part of the window object 1120 is located outside the screen 1110 according to the configuration of the window object 1120, as an example of the screen 1110 of the display device. In this case, the window object 1120 may include the a object 1130 beyond the screen 1110. As described above, in order to output a set sound with respect to the a object 1130 extending beyond the screen 1110, the computer device 100 needs to specify the first coordinates and the second coordinates of the position outside the screen 1110 in the virtual coordinate system.

Fig. 12 is a diagram showing an example of determining the coordinates of an object beyond a screen by a virtual coordinate system in an embodiment of the present invention. Fig. 12 shows an example of a virtual coordinate system 1240 inside the

screens

1210, 1220, 1230 of a plurality of display devices. In this case, it is assumed that the horizontal axis is the x axis and the vertical axis is the y axis in the virtual coordinate system 1240, the coordinates of the virtual coordinate system 1240 are represented by (x, y), the origin 1250 of the virtual coordinate system 1240 is set to be the coordinates of the leftmost upper end of the first screen 1210 in fig. 12, and the coordinates (0, 0) are set. In this case, the coordinates of the first point 1260 shown in fig. 12 may be (8, -2). And, the coordinates of the second point 1270 may be (18, 9). Similarly, the coordinates of the third point 1280 may be (29, -3).

In this case, it may be determined that an object located at the first point 1260 corresponds to the first screen 1210, an object located at the second point 1270 corresponds to the second screen 1220, and an object located at the third point 1270 corresponds to the third screen 1230.

As a more specific example, the computer device 100 may determine a display device providing the second screen 1220 among the plurality of display devices using the coordinates (18, 9) of the second point 1270, and may determine whether the determined display device includes an acoustic output device. Therein, the coordinates (18, 9) may be determined as the first coordinates of the second point 1270. If the display device providing the second screen 1220 does not include an audio output device, the computer device 100 can output an audio set for the object using the speaker device. When the sound to be output is multi-channel sound, the multi-channel sound can be balanced so that a virtual sound source is generated at the position of the virtual coordinates (18, 9) in accordance with the conventional 3D sound source generation method.

In addition, if the display device providing the second screen 1220 does not include an acoustic output device, the computer device 100 may output an acoustic set for the object through the acoustic output device of the identified display device. In this case, in the case where the sound to be output is multi-channel sound, the computer device 100 may also determine the second coordinates for the corresponding object. In this case, the virtual coordinate system for the second screen 1220 may have the leftmost upper end of the second screen 1220 as the origin, and in this case, the second coordinate of the second point 1270 may be (8, 9). The computer device 100 can balance the multi-channel sound by the sound output device included in the specified display device so as to generate a virtual sound source at the position of the virtual second coordinates (8, 9) according to the existing 3D sound source generation method.

In the above embodiment, the example in which the screens of the plurality of display devices are connected to each other has been described. However, a single independent screen in which screens of a plurality of display devices are not connected to each other may be displayed.

Fig. 13 is a diagram showing an example of screens in which a plurality of display devices are connected to each other in one embodiment of the present invention, and fig. 14 is a diagram showing an example in which a plurality of display devices display screens independent of each other in one embodiment of the present invention.

First, fig. 13 is a diagram showing an example of screens in which the first screen 1310 and the second screen 1320 are connected to each other. In this case, the display device that outputs the object 1330 may be determined by various methods such as a center point of the object 1330 or a scale of displaying the object 1330. In the example of fig. 13, as the first coordinate determined as the center point of the object 1330 exists in the second screen 1320, the computer device may determine the display device outputting the object 1330 as the display device displaying the second screen 1320. In this case, the sound set for the object 1330 may be output by the sound output device of the display device displaying the second screen 1320.

On the other hand, as shown in fig. 14, the first screen 1410 and the second screen 1420 may also display screens independent of each other. In this case, the same is true as in the case where the object 1430 is displayed by one display which is not a plurality of displays. In other words, the object 1430 is displayed by the display device displaying the first screen 1410, and the sound set for the object 1430 can be output by the sound output device of the display device displaying the first screen 1410.

In the case where the display device displaying the first screen 1410 does not include the acoustic output device, the acoustic can be output through a separate speaker device. However, in this case, it is difficult to deal with the balance of multi-channel sound. For this reason, as described above, the computer device 100 may have data on the configuration of the display device and data on how to handle the balance according to the configuration of the display device in advance, and the multi-channel sound may be balanced using the data stored in advance in the computer device 100. It can be easily understood by those skilled in the art that if the position of a virtual sound source to be generated can be grasped by the arrangement of the display device, the multi-channel sound can be balanced in such a manner that a virtual sound source is generated at a corresponding position by using one of the conventional techniques of the three-dimensional sound source generation method described above. Further, if information on the arrangement of the display devices is grasped in advance, the position at which the virtual sound source is to be generated can be grasped based on the absolute position of the specific display device and the second coordinate which is the relative coordinate in the corresponding display device.

As described above, according to the embodiments of the present invention, when a computer device including or connected to a plurality of display devices displays a specific object through the plurality of display devices, it is possible to output a sound set for the specific object according to which position of which display device the specific object exists. Further, in consideration of the acoustic output device included in the display device, it is possible to output an acoustic set for a specific object according to which position of which display device the specific object exists.

The system or apparatus described above can be realized by hardware components, software components, or a combination of hardware components and software components. For example, the devices and components described in the embodiments, such as a processor, a controller, an Arithmetic Logic Unit (ALU), a digital signal processor (digital signal processor), a microcomputer, a Field Programmable Gate Array (FPGA), a Programmable Logic Unit (PLU), a microprocessor, or any other device capable of executing instructions (instructions) and responding thereto, may be implemented by using more than one general-purpose computer or special-purpose computer. The processing device may execute an Operating System (OS) and one or more software applications executing on the OS. Also, the processing device may access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, there is a case where one processing device is used as an example for description, but a person having ordinary skill in the art to which the present invention pertains can understand that a processing device may include a plurality of processing elements (processing elements) and/or a plurality of types of processing elements. For example, the processing device may include multiple processors or a processor and a controller. Further, the processor may be another processing configuration (processing configuration) such as a parallel processor (parallel processor).

The software may comprise a computer program (computer program), code, instructions (instructions) or a combination of one or more of these, constituting the processing means or command processing means either alone or in combination in a manner to perform the required operations. Software and/or data may be embodied, permanently or temporarily, in some type of machine, component, physical device, virtual device, computer storage medium or device for parsing by or providing instructions or data to a processing device. The software may also be distributed over network coupled computer systems and stored or executed via distributed methods. The software and data may be stored in more than one computer-readable recording medium.

The methods of the embodiments are implemented in the form of program instructions executable by various computer units and stored on a computer readable medium. The computer-readable media may contain one or a combination of program instructions, data files, data structures, etc. The program instructions recorded on the medium may be those specially designed and constructed for the purposes of the embodiments, or may be those commonly used by those skilled in the computer software art. Examples of the computer-readable recording medium include magnetic media (magnetic media) such as hard disks, floppy disks, and magnetic tapes, optical recording media (optical media) such as CD-ROMs and DVDs, magneto-optical media (magnetic-optical media) such as floppy disks, and specially configured hard disk devices that store and execute program instructions, such as Read Only Memories (ROMs), Random Access Memories (RAMs), and flash memories. Examples of the program instructions include machine code formed by a compiler and high-level language code executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited embodiments and the accompanying drawings, various modifications and variations can be made by those skilled in the art to which the present invention pertains, based on the above description. For example, even if the described techniques are performed in a different order than the described methods, and/or structural elements of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than the described methods, or replaced or substituted by other structural elements or equivalents, appropriate results can be achieved.

Accordingly, other examples, embodiments, and equivalents to the claims are intended to be within the scope of the claims.

Claims

1. A sound output method for a computer device including a plurality of display devices or connected to a plurality of the display devices, the sound output method comprising:

a step of determining a first coordinate of a specific object on a whole screen displayed by a plurality of display devices;

determining a display device corresponding to the specific object among the plurality of display devices based on the determined first coordinates;

confirming whether the determined display device comprises an acoustic output device; and

and outputting the sound set for the specific object through the sound output device included in the determined display device, when the determined display device includes the sound output device.

2. The audio output method according to claim 1, wherein the step of outputting the audio set for the specific object through the specified audio output device included in the display device includes:

confirming a second coordinate of the specific object in a virtual coordinate system associated with a first screen displayed on the specified display device when the set audio is a multi-channel audio;

balancing the multi-channel sound based on the relative position of the specific object based on the confirmed second coordinates; and

and outputting the balanced multi-channel sound through the sound output device.

3. The sound output method according to claim 2, wherein in the step of balancing the multi-channel sound, the multi-channel sound is balanced such that the multi-channel sound output by the sound output device generates a virtual sound source at a virtual position indicated by the second coordinates.

4. The audio output method according to claim 2, wherein when the specific object is located outside the first screen, the second coordinate is determined as a virtual coordinate outside the first screen in a virtual coordinate system related to the first screen.

5. The sound output method according to claim 1, characterized by further comprising: and a step of outputting the sound set for the specific object through a speaker device included in the computer device or a speaker device connected to the computer device when the determined display device does not include a sound output device.

6. The sound output method according to claim 5, wherein in the step of outputting the sound set for the specific object by the speaker device, when the set sound is a multi-channel sound, the multi-channel sound is output by the speaker device, and the multi-channel sound is balanced and output such that a virtual sound source is generated at a virtual position indicated by the first coordinate of the specific object by the multi-channel sound output by the speaker device.

7. The audio output method according to claim 1, wherein the first coordinate is determined as a two-dimensional coordinate of the specific object on the entire screen regardless of depths of a plurality of objects hierarchically displayed on the entire screen.

8. Sound output method according to claim 1,

the above-mentioned specific object includes a window object,

in a case where the window object disappears from the entire screen by the minimization of the window object, the first coordinates include coordinates at which the window object is finally displayed on the entire screen.

9. The audio output method according to claim 1, wherein when the specific object is located outside the entire screen, the first coordinates are determined as virtual coordinates outside the entire screen in a virtual coordinate system related to the entire screen.

10. A computer-readable recording medium storing a computer program for executing a sound output method at a computer apparatus by being combined with the computer apparatus including a plurality of display apparatuses or being connected with a plurality of the display apparatuses, the computer-readable recording medium characterized in that the sound output method comprises:

11. The computer-readable recording medium according to claim 10, wherein the step of outputting the sound set for the specific object through the sound output means included in the determined display means includes:

12. The computer-readable recording medium according to claim 11, wherein in the step of balancing the multi-channel sound, the multi-channel sound is balanced such that the multi-channel sound output by the sound output device generates a virtual sound source at a virtual position indicated by the second coordinates.

13. The computer-readable recording medium of claim 10, wherein the sound output method further comprises: and a step of outputting the sound set for the specific object through a speaker device included in the computer device or a speaker device connected to the computer device when the determined display device does not include a sound output device.

14. The computer-readable recording medium of claim 13, wherein in the step of outputting sound set for the specific object through the speaker device, when the set sound is multi-channel sound, the multi-channel sound is output through the speaker device, and the multi-channel sound output through the speaker device is balanced and output such that a virtual sound source is generated at a virtual position indicated by the first coordinate of the specific object by the multi-channel sound output through the speaker device.

15. The computer-readable recording medium according to claim 10, wherein in a case where the specific object is located outside the entire screen, the first coordinates are determined as virtual coordinates outside the entire screen in a virtual coordinate system related to the entire screen.

16. A computer device for performing a sound output method, said computer device characterized in that,

comprising at least one processor embodied to execute computer-readable instructions,

at least one of the processors determines a first coordinate of a specific object on an entire screen being displayed through a plurality of display devices further included in the computer device or a plurality of display devices connected to the computer device,

at least one of the processors determines a display device corresponding to the specific object among the plurality of display devices based on the determined first coordinates,

confirming whether the determined display device comprises an acoustic output device,

when the specified display device includes an acoustic output device, the acoustic output device included in the specified display device outputs an acoustic set for the specific object.

17. The computer device of claim 16,

when the sound set for the specific object is output by the sound output device included in the specified display device, if the set sound is a multi-channel sound,

at least one of the processors confirms a second coordinate of the specific object in a virtual coordinate system associated with a first screen being displayed through the determined display device,

balancing the multi-channel sound based on the confirmed relative position of the specific object based on the second coordinate,

18. The computer device according to claim 16, wherein in a case where the determined display device does not include an audio output device, at least one of the processors outputs an audio set for the specific object through a speaker device included in the computer device or a speaker device connected to the computer device.

19. The computer device of claim 18,

when the set sound is a multi-channel sound, the at least one processor outputs the multi-channel sound through the speaker device, wherein the multi-channel sound output through the speaker device is balanced and output such that a virtual sound source is generated at a virtual position indicated by the first coordinate of the specific object.