JP2006245936A - Sound processing apparatus, method of controlling sound processing apparatus, program, integrated circuit, and reproducing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound processing apparatus for achieving localization of click sound associated with a button without increasing the memory capacity of a sound buffer. <P>SOLUTION: A graphic controller 105 writes information of a size of video screen, display positions and sizes of buttons in an interactive graphic stream to a display information buffer 102, a processing section 110 reads the written information, reads a monaural sound of a sound ID made to correspond to a button subjected to UO (user operation) 170 from the sound buffer 101 and provides an output delay and a strength difference between right and left sounds to the read monaural sound thereby processing the monaural sound into stereo sound data localized at the position of the button and provides an output of the data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a sound processing apparatus for reproducing a BD-ROM or the like, and more particularly to a technique for realizing a click sound.

  In the reproduction of digital content, the user can select whether or not to display subtitles by operating an object called a button indicating a choice displayed on the screen. However, in the conventional DVD (Digital Versatile Disk), when the user presses the button, there is no provision to output a click sound that is synchronized with the button operation, so the user can confirm that the button has been pressed by voice. Can not. On the other hand, the BD-ROM (Blu-ray Disc-Read Only Memory) specification for high definition (HD) content defines a click sound, so that the button is pressed. Can be confirmed.

This click sound is recorded as a sound file on the BD, and is stored in a memory area called a sound buffer in the playback device before the playback of the content of the BD. When a button displayed on the screen is selected or executed by a user operation during reproduction, a sound associated with the button is extracted from the sound buffer and output, and a click sound is realized. Non-Patent Document 1 discloses BD-ROM data specifications including definition of sound data and buttons.
"System Description Blu-ray Disc Read-Only Format Part3"

  Sound data usually contains several monaural sounds, and when a certain button is selected, the corresponding sound is output. At that time, even if there are multiple speakers, for example, left and right, monaural sound is output as it is from each speaker. It will be. Therefore, when the button is at the edge of the screen, the user feels uncomfortable.

  If all stereo, that is, sounds for each of the left and right speakers are prepared so that each button is localized at the position of the button, this sense of incongruity can be eliminated, but the problem is that the memory capacity of the sound data increases. However, since the size of the sound buffer is limited, it is practically impossible to prepare sound data in stereo.

The present invention has been made in view of the above problems, and provides a sound processing device that generates a click sound with a sense of orientation according to the arrangement of operated buttons without increasing the volume of sound data. Objective.
In order to solve the above-described problems, the sound processing apparatus according to the present invention is associated with an object specified by a user among a plurality of objects included in a graphics stream, as a digital stream including the graphics stream is reproduced. A sound processing device that processes and outputs sound,
A sound buffer that records sound data associated with the object;
Extraction means for extracting display information comprising the size of the video screen and the display position of each object on the video screen from the graphics stream;
Based on the extracted display information, the sound data associated with the object is read from the sound buffer and processed so that the sound image is localized at the display position of the specified object, and the processed sound data is output. Machining output means.

  With the above configuration, for example, when a plurality of objects are arranged in the left, right, up, and down directions on the screen, even if the click sound associated with each object is the same monaural sound, it corresponds to the arrangement of each object. A click sound with a sense of orientation can be realized. Also, even if the click sound associated with each object is different, it is sufficient to associate a different monaural click sound with each object, and there is no need to prepare a stereo click sound with localization, There is an effect that it is not necessary to increase the buffer capacity for storing the click sound. A button is also an object.

Further, the digital stream to be played back is recorded on a BD-ROM, the graphics stream includes an interactive graphics stream, and the display information is recorded in units of pixels, and the extraction is performed. The means extracts the display information from the interactive graphics stream.
As a result, a click sound corresponding to an object defined on the BD-ROM can be obtained.

  In addition, the extraction unit extracts display information including a width of the video screen and a horizontal display position of each object, and the processing output unit extracts, from the display information, a horizontal direction with respect to the width of the video screen by the object. Calculate the division ratio, process the monaural sound data associated with the object based on the calculated division ratio into a stereo sound, and specify the sound image by adjusting at least one of the difference between the left and right sound intensity and the delay It is supposed that the object is positioned at the horizontal display position of the object.

Thereby, it is possible to realize a click sound having a feeling of localization according to the arrangement of objects in the horizontal direction of the video screen.
In addition, the extraction unit extracts display information including a height of a video screen and a vertical display position of each object, and the processing output unit includes a sound image of the display information extracted by the extraction unit. In order to give the viewer an impression according to the vertical position of the object, different filters are applied to the sound data associated with the object according to the ratio of the vertical position of the object.

Thereby, it is possible to realize a click sound that can give the viewer an impression according to the arrangement of objects in the vertical direction of the video screen.
The processing means applies a high-pass filter when the vertical position of the object is on the upper half of the screen, and applies a low-pass filter when it is on the lower half.

As a result, for the buttons on the upper part of the screen, the treble part is emphasized to give a light impression to the viewer. When the buttons are on the lower part of the screen, the bass part is emphasized. By processing the click sound so as to give a heavy impression to the viewer, an image related to the vertical position of the object can be given to the viewer.
The plurality of objects are all buttons.

Thereby, a click sound corresponding to the button is obtained.
The information processing apparatus further includes storage means for storing, as additional information, information on the size of the display for reproduction display and the positions of the left and right speakers, and the processing output means is designated based on the display information and the additional information. The sound data associated with the object is processed so as to be localized at the position of the object.

Thereby, since the position of the button between the left and right speakers can be accurately grasped based on the information on the size of the display and the positions of the left and right speakers, a stereo sound localized at the position of the button can be obtained.
A control method for a sound processing apparatus according to the present invention includes a sound buffer that records sound data associated with an object, and a plurality of digital streams included in the graphics stream accompanying reproduction of the digital stream including the graphics stream. A method of controlling a sound processing apparatus that processes and outputs a sound associated with an object designated by a user among objects, and displays display information including a size of a video screen and a display position of each object on the video screen. Based on the extraction step of extracting from the graphics stream and the extracted display information, the sound data associated with the object is read from the sound buffer so that the sound image is localized at the display position of the specified object. Processed and processed And a processing output step of outputting Undodeta.

By this control method, a click sound localized at the button display position can be obtained.
The program of the present invention has a sound buffer that records sound data associated with an object, and the user plays a digital stream including the graphics stream by a user among a plurality of objects included in the graphics stream. An extraction step for extracting, from the graphics stream, display information including a size of a video screen and a display position of each object on the video screen, to a sound processing device that processes and outputs a sound associated with a specified object; Based on the extracted display information, the sound data associated with the object is read from the sound buffer and processed so that the sound image is localized at the display position of the specified object, and the processed sound data is output Machining output To perform the steps.

By this program, a click sound localized at the button display position can be obtained.
The integrated circuit according to the present invention is an integrated circuit that processes and outputs a sound associated with an object designated by a user among a plurality of objects included in the graphics stream in accordance with the reproduction of the digital stream including the graphics stream. Thus, each means of the sound processing apparatus according to claim 1 is constituted.

With this integrated circuit, a click sound localized at the button display position can be obtained.
A playback device according to the present invention is a playback device for a digital stream including a graphics stream, and includes the sound processing device according to claim 1.
With this playback device, a click sound localized at the button display position can be obtained.

Hereinafter, embodiments of a sound processing apparatus according to the present invention will be described with reference to the drawings.
(Embodiment 1)
In the first embodiment, the position of the button in the horizontal direction is localized. That is, a monaural sound is processed so that a sound source is localized at a horizontal display position of a button for performing a click sound, and a stereo sound is created and output.

FIG. 1 is a diagram illustrating a configuration of a sound processing apparatus. The sound processing device is incorporated in the playback device. FIG. 1 also shows a main part of a playback apparatus related to the sound processing apparatus.
The components of the sound processing apparatus will be described with reference to FIG.
The sound processing apparatus 100 includes a sound buffer 101, a display information buffer 102, a graphics controller 105, and a processing unit 110.

  The sound buffer 101 stores sound (speech) data used when a button or the like as an interactive object is pressed. The sound data is recorded as a sound data file called sound.bdmv on a Blu-ray disc called a BD-ROM, and the sound data is stored in the sound buffer 101 by the playback device before starting playback of content such as a movie.

This will be described with reference to FIG. 2 and FIG.
FIG. 2 is a diagram illustrating an example of an overview of a playback system including a BD-ROM. In FIG. 2, a BD-ROM 210 that is a portable recording medium is used for supplying movie works and the like to a system formed by a playback device 220, a television 230, and a remote controller 240. The playback device 220 also includes the sound processing device 100.

  FIG. 3 is a diagram showing the structure of a directory in the BD-ROM 105 that manages files including sound data files. Details of the configuration 301 of the BD-ROM 105 are described in Non-Patent Document 1. In FIG. 3, the BD-ROM has a BDMV directory under the ROOT directory, and under the BDMV directory, an index file (index.bdmv) in which an Index Table that constitutes a title is defined, a dynamic scenario. A file (MovieObject.bdmv) containing a movie object to be defined, a STREAM directory for storing an AV clip (01000.m2ts), and a playlist file (00000.mpls) defining a logical playback path (playlist) in the AV clip There are a PLAYLIST directory that stores the clip information, a CLIPINF directory that stores the clip information file (01000.clpi) that stores the management information of the AV clip, an AUXDATA directory, and the like.

sound.bdmv is stored under the AUXDATA directory.
When the BD-ROM 105 is loaded, the playback device reads sound.bdmv 120 in the AUXDATA directory and stores it in the sound buffer 101.
FIG. 4 is a diagram showing the structure of sound data. The sound data includes a soundIndex 411 for managing a plurality of sounds and soundData 412 for storing each sound. The soundIndex 411 stores the attribute of each sound such as the number of channels indicating mono / stereo, and the address and length of the sound in the soundData 412. Each sound of the soundData 412 is composed of 48 kHz, 16-bit LPCM (Linear Pulse Code Modulation) samples. In this embodiment, each sound is monaural.

Both sound.bdmv 120 and sound buffer 101 hold sound data in the form of sound data 120. The sound buffer 101 has a fixed-length size of 2 MB, so that sound.bdmv is smaller than this size. The number of sounds that can be registered is up to 128, and the ID of each sound is assigned in order from 0 in the storage order.
Returning to FIG. 1, the display information buffer 102 stores display information such as button positions, is updated by the graphics controller 105, and is referenced by the processing unit 110.

FIG. 5 is a diagram showing the contents of display information stored in the display information buffer 102. The display information includes a video width 502 of the video scene, a video height 503, the number of buttons 504, and button information 511.
The video width 501 and the video height 502 of the video screen are equal to the width and height of the interactive graphics plane 151, respectively, and the unit is pixels.

The button information 511 includes a button ID 512, a horizontal position 513 of the button, a vertical position 514 of the button, a button width 515, and a button height 516, all of which are in pixels.
FIG. 6 is a diagram showing the relationship between the screen and buttons, and shows the video screen and buttons shown in the example of FIG. In the display information, the display position (256, 148) of the button 1011 on the video screen 1001 is that of the upper left point of the button rectangle, and the display position (320, 180) at the center of the button is the button width which is the size of the button. The half value of each of 515 and button height 516 is obtained by adding to the display position (256, 148).

In this embodiment, when the button is localized, the button is localized at the center position of the button.
In the present embodiment, it is also assumed that a stereo left speaker and a right speaker (not shown) are at the left end 1015 and the right end 1016 of the screen. Therefore, in the example shown in FIG. 6, the ratio of the distance from the left edge 1015 of the screen to the button 1011 and the distance from the button 1011 to the right edge 1016 is 320: 960, that is, the position of 1: 3 from the left edge of the screen width. Has a button.

The ratio a: b at which the button divides the width of the screen as described above is also the ratio at which the left and right speakers are divided in the present embodiment where it is assumed that there are left and right speakers at both ends of the screen. The division ratio of a: b is used as a parameter in sound processing performed by the processing unit 110 described later to localize the sound source at the button position.
Returning to FIG. 1 again, the graphics controller 105 is also a part of the interactive decoder 150 that reproduces the interactive fixture stream. The graphics controller 105 searches the composition buffer (hereinafter referred to as “CB”) 155 and the decoded object buffer (Decoded Object Buffer) 156, and corresponds to the display information in the display information buffer 102 included in the interactive fixture story. However, if they are recorded in the CB 155 or the decoded object buffer 156, they are read out and written in the display information buffer 102.

Before describing which items of the digital story correspond to which items of display information, the digital stream of the BD-ROM will be briefly described next.
FIG. 7 shows the contents of a digital stream called an AV clip. As shown in FIG. 7, the AV clip is a stream obtained by multiplexing an interactive graphics stream, a presentation graphics stream, a video stream, and an audio stream. Is a stream consisting of The interactive graphics TS packet 701 is obtained by dividing a PES packet 702, and the PES packet 702 is obtained by dividing an interactive graphics segment. Similarly, other streams are composed of layers of TS packets, PES packets, and segments in order from the bottom. Interactive graphics segments include ICS (Interactive Composition Segment) and ODS (Object Definition Segment).

FIG. 8 shows the contents of the ICS.
In FIG. 8, ICS 801 stores control information of an interactive graphics stream. FIG. 8 shows video_width 802 and video_height 803 indicating the size of the video, number_of_butons 804 indicating the number of buttons, and information 811 for each button. In the button information 811, a button ID (button_id) 812 is an identifier of the button, and when there is a UO (user operation), the graphics controller 105 identifies which button operation is the button ID (button_id) 812. To do.

  The horizontal position (button_horizontal_position) 813 and the vertical position (button_vertical_position) 814 of the button indicate the display position of the button, and this indicates, for example, the display position of the upper left corner of the rectangular area of the button 1011 shown in FIG. Further, in normal_start_object_id_ref825, when the button i in the normal state is drawn with an animation, the first number is described in the normal_start_object_id_ref825 among the serial numbers added to the plurality of ODSs constituting the animation. In normal_end_object_id_ref 826, when the button i in the normal state is drawn with animation, the last number is described in the normal_end_object_id_ref 826 among the consecutive object_IDs added to the plurality of ODSs constituting the animation. When the ID indicated by the normal_end_object_id_ref 826 is the same as the ID indicated by the normal_start_object_id_ref 825, the still image of the graphics object indicated by the ID becomes the pattern of the button i. The selected_state_sound_id_ref 817 is the sound ID of the sound used when the button is selected, the selected_start_object_id_ref 818 is the object ID of the object displayed first when the button is selected, and the selected_end_object_id_ref 819 is the object ID of the object displayed last. .

  The size of the selected state of the button is the size of the object indicated by selected_start_object_id_ref 818. Similarly, activated_state_sound_id_ref 820 activated_start_object_id_ref 721 and activated_start_object_id_ref 822 that are referred to when activated are defined, and the meanings of these are also the same. Note that the size of the object pointed to by selected_start_object_id_ref 818 is the same as the size of the object pointed to by activated_start_object_id_ref 822.

In addition, number_of_navigation_commands 823 and navigation_commands 824 are the number of commands and commands executed by the graphics controller 105 when the button is activated.
FIG. 9 shows the contents of ODS.
ODS defines one or more objects. In the example shown in FIG. 9, the ODS 901 is the head ODS of two objects defining a certain button object. The ODS 901 describes object_id 911, last_in_sequence_flag 912, object_data_fragment 915, and the like.

  object_id 911 is an object ID of this object, and last_in_sequence_flag 912 is a flag indicating whether or not the ODS is the last segment defining the object. In the object_data_fragment 915, object_data_length 922 indicates the length defined by the ODS 901 in the compressed object 925 that stores the display image of the object in the run-length compression format.

object_width 923 and object_height 924 indicate the width and height of the object, and the button width and button height of the button indicated by the button information 811 in FIG. is there.
Next, referring to FIG. 1, an outline from when the digital story is processed by the playback device to output will be described, and then the details of the interactive graphics processing and the display stored in the display information buffer 102 will be described. The location of each item of information will be described sequentially.

MPEG2-TS packets 121 are sequentially input to the PID filter 122 and distributed to a transport buffer (TB) corresponding to the PID of each TS packet. For example, in the case of an interactive graphics stream, 0x1400 to 141F is given as the PID of the TS packet.
For example, in the case of a presentation graphics stream, the presentation graphics decoder 130 extracts TS packets from the transport buffer 123, decodes them, and outputs presentation graphics data. The presentation graphics plane 131 receives the data and expands it. The CLUT 132 performs color display processing and outputs the video.

In the case of a video stream, the video decoder 140 extracts and decodes a video TS packet from the TB 124 and outputs video data. The video plane 141 inputs the data, expands it, and outputs the video.
In the case of audio, the audio decoder 160 extracts and decodes TS packets from the TB 126, and outputs audio data. The mixer 161 inputs the data and outputs it together with the sound data from the sound processing apparatus 100.

  Finally, in the case of an interactive graphics stream, the interactive graphics decoder 150 extracts the TS packet from the transport buffer 123, decodes it, and outputs object data. The interactive graphics plane 151 expands the object. The developed object is color-displayed by the CLUT 152 and output as a video.

Next, the processing of the interactive graphics decoder 150 will be described in detail.
The interactive graphics decoder 150 includes an EB (coded data buffer) 153, a stream graphics processor 154, a CB 155, and a decoding object buffer 156.

  At the time of transfer from TB 125 to EB 153, the TS packet and each header of the PES packet are deleted and stored as segment data. The stream graphics processor 154 transfers the segment stored in the EB 153 to the CB 155 if the segment is the ICS 801, and transfers it to the decoding object buffer 156 if the segment is the ODS. At this stage, all segments necessary for display are stored in the corresponding buffer.

  At this time, the graphics controller 105 reads the video_width 802 and video_height 803 from the ICS 801 stored in the CB 155 and writes them as the video width 502 and the video height 503 in the display information buffer 102 shown in FIG. Similarly, number_of_butons 804 of ICS 801 is written in the display information buffer 102 as the number of buttons 504. For each button, button_id 812 is set as button ID 512, and button_horizontal_position 813 and button_vertical_position 814 are written in the display information buffer 102 as button horizontal position 513 and button vertical position 514. Also, the ODS 901 in the decryption object buffer 156 is searched for an ODS 901 (or the first ODS if there is more than one) that matches the object ID of the selected_start_object_id_ref 818, and the object_width 923 and object_width 923 are displayed as the button width 515 and button height 516 of the button, respectively. Write to the information buffer 102.

Then, when the time of PTS (Presentation Time Stamp: not shown) comes among the items described in the ICS 801 segment, the graphics controller 105 converts the button object stored in the decoding object buffer 156 to the interactive graphics plane. 151 is output.
When the user performs UO (user operation) 170 by selecting (selecting) or activating (pressing the ENTER key) on the button displayed on the screen, the graphics controller 105 displays the button ID of the button and the button. Is identified based on the ICS 801 of the CB 155 and the ODS of the decoding object buffer 156, and is notified to the processing unit 110.

  Here, the sound ID of the click sound associated with the button specified by the graphics controller 105 is selected_state_sound_id_ref 817 of the button_id 812 that matches the button ID of the ICS 801 shown in FIG. 8 when the UO 170 selects the button. Further, when the UO 170 activates a button, activated_state_sound_id_ref 820 is obtained.

FIG. 10 is a diagram illustrating an example of button display and a remote controller in the UO 170.
FIG. 10A shows an example of button display. The screen 1001 shows two buttons, “with caption” and “without caption”. When displaying for the first time, only the button 1011 is displayed as a button indicating “with caption”.
The user operates the up / down / left / right keys of the remote controller 240 shown in FIG. 10B to select, for example, the “with caption” button. The button 1013 indicating that the graphic controller 105 has been selected is overwritten on the button 1011 and displayed. At this time, on the screen 1001 in FIG. 10, it seems that a button or the like indicating selection is displayed at a different position, but it is displayed at the same position as the button 1011 instead. When the user further presses the Enter key on the remote controller 240, the currently selected button is activated, and the graphics controller 105 overwrites the button 1013 with the button 1014 indicating activation and displays the button.

Returning to FIG. 1, the processing unit 110 processes monaural sound data into stereo sound data, and outputs the stereo sound data.
When the processing unit 110 receives a notification from the graphics controller 105 including the button ID of the button operated by the user and the sound ID associated with the button, the processing unit 110 transmits the sound data indicated by the sound ID from the sound buffer 101. The read sound data is processed into a stereo sound so as to be localized at the button display position indicated by the button ID, and is output to the mixer 161. Here, the display position of the button refers to the display position in the horizontal direction at the center of the button.

FIG. 11 is a plan view of a display 1111 for displaying buttons and the like and a viewer 1115 as viewed from above, and shows various parameters used in sound processing. With reference to FIG. 11, first, parameters used in processing sound data by the processing unit 110 will be described.
In FIG. 11, the left speaker 1112 is at the left end of the display 1111 and the right speaker 1113 is at the right end. Further, it is assumed that the display position in the horizontal direction at the center of the button 1114 is that the display 1111 displaying the pixel width W, that is, the video width W is divided into a: b. When the horizontal position of the center of the button 1114 obtained above is c, a: b = c / W: (Wc) / W.

  A viewer 1115 who is an operator of the button has a distance between both ears h and is located at a distance d from the display 1111 on a vertical line passing through the center of the display 1111. The distances from the button 1114 to the viewer's 1115 right ear and left ear are r and l, respectively. Here, the processing unit 110 reads and uses the video width 502 stored in the display information buffer 102 of FIG. 7 as the video width W of FIG.

  Next, mathematical formulas used for processing sound data by the processing unit 110 will be described. Here, if the speed of sound is v, the squares of l and r are expressed by equations (1) and (2), respectively.

これらの式からｌとｒをもとめ、次の式（３）に代入し、ステレオサウンドのRに対するLの遅延時間を求める。ここで、ｗ、ｄ、ｈ、ｖには、たとえば、それぞれ０．６、２、０．２、３４０（メートル単位）の数値を代入する。ただし、ｄ、ｈ、ｖの値の設定は可変であり、上述の値に限られない。

From these equations, l and r are obtained and substituted into the following equation (3) to obtain the L delay time for R of stereo sound. Here, numerical values of 0.6, 2, 0.2, and 340 (meter units) are substituted for w, d, h, and v, respectively. However, the settings of d, h, and v are variable and are not limited to the above values.

また、L（左）の音の強度を、R（右）の音の強度に基づく式（４）により求める。

Further, the intensity of the sound of L (left) is obtained by Expression (4) based on the intensity of the sound of R (right).

加工部１１０はグラフィックスコントローラ１０５から指定のあったサウンドIDが示すモノラルのサウンドデータをステレオサウンドのRのサウンドとしてそのまま出力し、Lのサウンドは、式（４）で強度の加工を施したサウンドを、式（３）で示す時間だけ、Rに遅延して出力する。

The processing unit 110 outputs the monaural sound data indicated by the sound ID designated from the graphics controller 105 as it is as the R sound of the stereo sound, and the sound of L is the sound that has been subjected to strength processing according to Equation (4). Is output after being delayed to R by the time indicated by the expression (3).

Next, the operation of the sound processing apparatus 100 will be described.
FIG. 12 is a flowchart showing the operation flow of the sound processing apparatus according to the first embodiment.
The digital stream of the AV clip 740 is divided into an interactive graphics stream or the like by the playback device and decoded by the interactive graphics decoder 150 or the like. The interactive graphics stream is stored in the CB 155 for the ICS 801 and in the decoding object buffer 156 for the ODS 901 or the like.

  The graphics controller 105 searches the CB 155 for the ICS 801 (S1305, where S is an abbreviation of step, the same applies hereinafter), and if stored, clears the display information buffer 102, reads video_width 802 and video_height 803, and displays them. The information is written in the information buffer 102 as a video width 502 and a video height 503, respectively. Similarly, number_of_butons 804 of ICS 801 is written as the number of buttons 504, and for each button, button_id 812 is set as button ID 512, button_horizontal_position 813 and button_vertical_position 814 are written into the display information buffer 102 as button horizontal position 513 and button vertical position 514, respectively. (S1310).

  Next, the graphics controller 105 searches the decoding object buffer 156 for an ODS 901 (the leading ODS if there is a plurality) that matches the object ID of the selected_start_object_id_ref 818 (S1315). The button width 515 and button height 516 are written into the display information buffer 102 (S1320).

The graphics controller 105 outputs the button object and the like in the decoding object buffer 156 to the interactive graphics plane 151.
The graphics controller 105 determines whether or not the interactive process for accepting the button operation is finished (S1325), and when finished, the operation of the sound processing apparatus 100 is finished (S1330).

  The user waits for a button operation, that is, UO 170 (S1335). If there is UO 170, the graphics controller 105 obtains the button ID of the operated button from the ICS 801 in the CB 155 and the ODS 901 in the decoding object buffer 156, and at the time of the button selection operation, the selected_state_sound_id_ref 817 of the button in the ICS 801. Similarly, in the activation operation, activated_state_sound_id_ref 820 is obtained as the sound ID corresponding to the button (S1340).

The graphics controller 105 notifies the processing unit 110 of the button ID and the sound ID. The processing unit 110 that has received the notification performs the following processing (S1345).
The processing unit 110 performs the processing. The processing unit 110 reads the video width 502, the horizontal position 513 of the button corresponding to the notified button ID, and the button width 128 from the display information buffer 102 (S1360).

The sound data of the notified sound ID is read from the sound buffer 101 (S1360). The horizontal position of the center of the button is calculated from the horizontal position 513 of the button and the button width 515 (S1370).
A ratio a: b by which the button divides the width of the display 1111 in the horizontal direction is calculated from the video width 502 and the horizontal position of the center of the button (S1375).

The processing unit 110 obtains l and r from equations (1) and (2) based on the parameters shown in FIG. Here, w, d, h, and v are, for example, numerical values of 0.6, 2, 0.2, and 340 (meter units), respectively.
The processing unit 110 outputs the monaural sound data indicated by the sound ID designated by the graphics controller 105 as it is as the R (channel) sound of the stereo sound, and the sound of L is processed with the intensity according to Equation (4). The sound subjected to is delayed with respect to R by the time indicated by the expression (3) (S1380) and output to the mixer 161 (S1385).

(Embodiment 2)
In the second embodiment, the localization of the horizontal position of the button performed in the first embodiment is corrected.
The configuration of the second embodiment is the same as that of the first embodiment, and only the processing of the processing unit 110 is different.

FIG. 13A is an external view of the display and the speaker when the left and right speakers are separated from the display, and FIG. 13B is a diagram illustrating the relationship between the position of the button and the position where it is localized. It is.
FIG. 13A shows a display 1401, a left speaker 1402, a right speaker 1403, and a button 1404 displayed on the right side of the display 1401. The width of the display 1401 is w, the distance between the speakers is s, and each speaker is shown. Indicates equidistant from the display 1401. Here, the origin of the X axis 1411 is the center of the display 1401.

In the first embodiment, as shown in FIG. 11, it is assumed that the center line of the speaker is installed at the positions of both ends of the screen, and the horizontal localization is realized. At this time, for example, when there is a button near the right end of the display 1401, the localization is closer to the right speaker 1403 than the position where the actual button is displayed, and is shifted to the right from the position of the button to be actually localized.
In FIG. 13 (b), the X axis indicates the horizontal position of the button display in pixels, and the Y axis is the position where the processed stereo sound is localized on the X axis in FIG. 13 (a). Is shown in meters.

A graph 1422 indicates that the buttons are correctly positioned, and a graph 1421 is a diagram illustrating the localization according to the first embodiment. When s is equal to w, the graph 1421 is the same as the graph 1422.
The difference 1412 between the button position and the click sound localization position is caused by the difference between the speaker position (both ends of the display) assumed by the processing unit 110 and the actual speaker position. Near the center of the display 1401, the difference between the display position of the button and the position where the click sound is localized is relatively small. However, the closer the display position of the button is to both ends of the screen, the larger the shift. Therefore, the processing unit 110 may be localized by assuming that the position of the speaker is previously set apart from the screen. This is realized by giving the localization a click sound as shown in the graph 1423. That is, the closer the button is to the both ends of the screen, the more the click sound localization is corrected to be inside the graph 1421.

By applying the graph 1423 for this correction, it is possible to output a click sound having a natural localization with little deviation.
(Embodiment 3)
In the third embodiment, a monaural sound is processed so as to be localized at a position in the vertical direction of a button to create a stereo sound and output it.

The configuration of the sound processing apparatus in the present embodiment is the same as that of the first embodiment, and only the processing of the processing unit 110 is different. Since processing unit 110 in the present embodiment performs filter processing, the filter will be described first.
FIG. 14 is a diagram illustrating examples of various filters used in the second embodiment.
FIG. 14A shows an example of the high-pass filter 1501. The high-pass filter 1501 keeps the intensity of the sound in the frequency band having a frequency of approximately 2 kHz or higher as it is, and decreases the intensity of the sound closer to 0 Hz with the intensity of the sound less than approximately 2 kHz. Therefore, as a result of this filtering process, a high tone is mainly output.

FIG. 14B is an example of the low-pass filter 1511. Contrary to the high-pass filter 1501, the low-pass filter 1511 decreases the intensity of sound in a frequency band having a frequency of approximately 2 kHz or higher as the frequency approaches 20 kHz, and leaves the intensity of sound less than approximately 2 kHz as it is. Therefore, bass is mainly output as a result of this filtering process.
When the processing unit 110 according to the present embodiment receives the button ID of the button operated by the UO 170 and the sound ID associated with the button from the graphics controller 105, the processing unit 110 stores the button ID in the display information buffer 102. The video height 502, the vertical position 514 of the button with the button ID, and the button height 516 are read out. Then, the vertical position of the center of the button is calculated by adding a half value of the button height 516 to the vertical position 514 of the button.

When the vertical position of the center of the button is less than half the button height 516, the button is on the upper half of the screen, and when the vertical position of the center of the button is more than half the button height 516, the button Is on the lower half of the screen.
In the present embodiment, the processing unit 110 processes the monaural sound corresponding to the button with the high-pass filter 1501 when the button is on the upper half of the screen, and processes the monaural sound corresponding to the button when the button is on the lower half. By processing with the low-pass filter 1502, a click sound that gives an impression corresponding to the vertical position of the button, that is, the upper half or the lower half of the display screen is output.

  Further, as a modification of the above, when the button is on the upper half surface, the high-pass filter 1521 of FIG. 14C is used instead of the high-pass filter 1501 to further increase the intensity of high-band sound of approximately 2 kHz or higher. You may make it make it. Similarly, when the button is on the lower half, the low-pass filter 1521 of FIG. 14C is used instead of the low-pass filter 1511 to further increase the intensity of the low-band sound of less than about 2 kHz. Good.

In all of the four types of filters described above, approximately 2 kHz is set as a boundary value between the high band and the low band, but this value is variable in a range of approximately 2 kHz to r approximately 3 kHz, and may be approximately 3 kHz, for example.
In addition, although the sound to be filtered is a monaural sound, the left and right stereo sound processed in the first embodiment may also be subjected to horizontal localization.

(Embodiment 4)
In the fourth embodiment, the display width (m unit) and the distance between the speakers are added as configuration information to the configuration of the first embodiment, and the sound source localization accuracy in the horizontal direction of the button is referred to during sound processing. I try to raise.
FIG. 15 is a diagram illustrating the configuration of the fourth embodiment.

The configuration of the sound processing device 1600 in the present embodiment is different from that of the first embodiment in the configuration information buffer 1611 and the processing unit 1630.
Only differences from the first embodiment will be described below.
FIG. 16A shows an example of the configuration information buffer 1611.
In FIG. 16A, the configuration information buffer 1611 stores configuration information including a display width 1612 and a distance 1613 between the left and right speakers. The configuration information is written into the configuration information buffer 1701 with a dedicated tool in advance.

FIG. 16B is a diagram showing the display 1111, the left speaker 1112, and the right speaker 1113 similarly to FIG. 11 based on the numerical values shown in the example of the configuration information buffer 1611. The difference between these figures is that in FIG. 11, the inter-speaker distance 1713 is the same as the width of the display 1111, but in FIG.
It is assumed that the button 1114 in FIG. 16B is at a position where the display width w is divided into a: b. Assume that the display width is w, the distance between speakers is s, and each speaker is equidistant from the center of the display.

If the button 1114 divides the speakers into m: n, m: n is as shown in FIG.
m: n = ((s−w) / 2 + aw / (a + b)): ((s−w) / 2 + bw / (a + b))
M and n can be calculated from known values a, b, w, and s.
The division ratio a: b shown in FIG. 11, which is a premise for calculating “L delay time” shown in Expression (3), is actually a division ratio between speakers.

Therefore, the processing unit 1630 reads the configuration information in the configuration information buffer 1611 and calculates m: n. Then, the value of m obtained above is substituted for a in the expressions (1) and (2), and the value of n is substituted for b. The remaining variables d, l, and r are the same as in the first embodiment. Substituting the values, L delay time and L sound intensity are obtained from equations (3) and (4).
When the left and right speakers are separated from both ends of the display, the localization shift 1412 according to the first embodiment as shown in FIG. 13 or the correction shown in the second embodiment is not necessary, and the horizontal direction of the button is eliminated. The position to the position can be realized with high accuracy.

As mentioned above, although this invention has been demonstrated based on four embodiment, of course, this invention is not limited to them. The following cases are also included in the present invention.
In the first embodiment, the display information buffer 102 also stores information in the vertical direction, for example, the video height, the vertical position of the button, and the button height, but these are not limited to the first embodiment but the third and fourth embodiments. It does not have to be. Conversely, in the second embodiment, only the information in the vertical direction may be stored.

In the embodiment, the position of the button to be localized is the center position of the button. For example, when the button width is less than a predetermined ratio with respect to the video width, for example, 5% or less, the rectangular area of the button It may be the position of the upper left corner, for example, the upper left corner.
The stereo sound L delay and the sound intensity were set based on the equations (3) and (4). However, the present invention is not limited to these equations, and was obtained from experiments and the like. An expression may be applied.

The control method of the present invention may be a computer program for realizing using a computer system, or may be a digital signal representing the program.
The present invention may be a computer-readable recording medium that records the program or the digital signal, such as a semiconductor memory.
The present invention may be the computer program or the digital signal transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, or the like.

  The sound processing apparatus according to the present invention has high applicability in the movie industry and the reproduction equipment industry as an apparatus attached to the reproduction equipment.

1 is a diagram illustrating a configuration of a sound processing device according to a first embodiment. It is a figure which shows the reproducing | regenerating system containing the reproducing | regenerating apparatus of the said embodiment. It is a figure which shows the structure of BD-ROM. It is a figure which shows the content of sound data. It is a figure which shows the content of the localization information buffer of the said embodiment. It is a figure which shows the relationship between the screen of the said embodiment, and a button. It is a figure which shows the content of the AV clip of the said embodiment. It is a figure which shows the content of ICS of the said embodiment. It is a figure which shows the content of ODS of the said embodiment. It is a figure which shows an example of the display of the button of the said embodiment, and a remote control. It is a figure which shows the parameter of the process of the sound of the said embodiment. It is a flowchart which shows operation | movement of the sound processing apparatus of the said embodiment. It is a figure which shows the relationship between the display position of the screen of Embodiment 2, and a localization position. 10 is a diagram illustrating an example of a filter according to Embodiment 3. FIG. It is a figure which shows the structure of the sound processing apparatus of Embodiment 4. FIG. It is a figure which shows the content of the additional localization information buffer of the said embodiment.

Explanation of symbols

100 Sound processing equipment 101 Sound buffer
102 Display information buffer 105 Graphics controller (Graphics controller)
110 Processing unit 120 Sound data
121 MPEG-2TS packet (MPEG-2 Transport Stream Packet)
122 PID Filter
123-126 Transport Buffer (TB)
130 Presentation Graphics Decoder
131 Presentation Graphics Plane
132 CLUT (Color LookUp Table)
140 Video Decoder
141 Video Plane
150 Interactive Graphics Decoder
151 Interactive Graphics Plane (Interactive Graphics Plane)
152 CLUT
153 Coded Data Buffer (EB)
154 Stream Graphics Processor
155 Composition Buffer (CB)
156 Decoded Object Buffer
160 Audio Decoder
161 Mixer (audio Mixer)
210 BD-ROM
220 Playback unit
230 TV
240 Remote Controller
301 Composition of BD-ROM 801 ICS (Interactive Composition Segment)
901 ODS (Object Definition Segment)

Claims (11)

A sound processing device that processes and outputs a sound associated with an object designated by a user among a plurality of objects included in a graphics stream in accordance with reproduction of a digital stream including a graphics stream,
A sound buffer that records sound data associated with the object;
Extraction means for extracting display information comprising the size of the video screen and the display position of each object on the video screen from the graphics stream;
Based on the extracted display information, the sound data associated with the object is read from the sound buffer and processed so that the sound image is localized at the display position of the specified object, and the processed sound data is output. A sound processing apparatus comprising a processing output means. The digital stream to be played back is recorded on a BD-ROM,
The graphics stream includes an interactive graphics stream,
The display information is recorded in units of pixels,
The sound processing apparatus according to claim 1, wherein the extraction unit extracts the display information from an interactive graphics stream. The extraction means extracts display information consisting of the width of the video screen and the horizontal display position of each object,
The processing output means calculates a horizontal division ratio with respect to the width of the video screen by the object from the display information, and processes monaural sound data associated with the object into a stereo sound based on the calculated division ratio. 3. The sound processing apparatus according to claim 2, wherein the sound image is localized at a display position in a horizontal direction of the designated object by adjusting at least one of a difference in intensity between left and right sounds and a delay. The extraction means extracts display information composed of the height of the video screen and the vertical display position of each object,
The processing output means includes, in the display information extracted by the extraction means, the sound data associated with the object so that the sound image is localized at the vertical position of the object, and the vertical position of the object. 2. The sound processing apparatus according to claim 1, wherein different filters are applied according to the ratio of the sound. 5. The sound processing apparatus according to claim 4, wherein the processing means applies a high-pass filter when the vertical position of the object is on the upper half surface of the screen, and applies a low-pass filter when the object is on the lower half surface. The sound processing apparatus according to claim 1, wherein each of the plurality of objects is a button. Storage means for storing information on the size of the display that performs playback display and the positions of the left and right speakers as additional information;
The processing output means processes sound data associated with the object based on the display information and the additional information so as to be localized at a specified object position. Sound processing equipment. It has a sound buffer that records sound data associated with the object, and corresponds to the object specified by the user among multiple objects included in the graphics stream when playing a digital stream including the graphics stream A method of controlling a sound processing device that processes and outputs a sound attached thereto,
An extraction step of extracting display information comprising the size of the video screen and the display position of each object on the video screen from the graphics stream;
Based on the extracted display information, the sound data associated with the object is read from the sound buffer and processed so that the sound image is localized at the display position of the specified object, and the processed sound data is output. A method for controlling a sound processing apparatus, comprising: a processing output step. It has a sound buffer that records sound data associated with the object, and corresponds to the object specified by the user among multiple objects included in the graphics stream when playing a digital stream including the graphics stream To the sound processing device that processes and outputs the attached sound,
An extraction step of extracting display information comprising the size of the video screen and the display position of each object on the video screen from the graphics stream;
Based on the extracted display information, the sound data associated with the object is read from the sound buffer and processed so that the sound image is localized at the display position of the specified object, and the processed sound data is output. Program that executes machining output step. An integrated circuit that processes and outputs a sound associated with an object specified by a user among a plurality of objects included in the graphics stream in accordance with reproduction of the digital stream including the graphics stream,
An integrated circuit comprising each means of the sound processing apparatus according to claim 1. A playback device for a digital stream including a graphics stream,
A playback apparatus comprising the sound processing apparatus according to claim 1.

Images