JP2006148542A - Three-dimensional sound field information reproducing device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a format by which three-dimensional sound field information is reproduced by maintaining compatibility with normal audio information when recording and reproducing the three-dimensional sound field information. <P>SOLUTION: The three-dimensional sound field information reproducing device has: a means for reproducing the audio object of a normal audio; a means for controlling the reproduction of the audio object by using management information after reproducing the management information for managing the audio object; and a means for controlling the reproduction of three-dimensional sound field data by using information regarding the three-dimensional sound field information recorded in a user data region of the audio object for every prescribed unit of each frame of the normal audio after reproducing the information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an apparatus and program for reproducing three-dimensional sound field information, and more particularly to a method for suitably realizing three-dimensional sound field information reproduction while maintaining compatibility with normal audio information. The present invention relates to a highly realistic theater system technology that allows viewers to enjoy stereophonic sound with a high sense of reality using sound localization technology without impairing the compatibility of the media on which the video is recorded.

  Conventionally, a technique related to a 3D sound field, that is, a technique for localizing and reproducing sound as if there is a speaker from a place without a speaker, has been proposed in several systems. In addition, techniques related to standards such as DVD video and DVD audio have been proposed.

  Conventionally, for example, in a three-dimensional virtual reality system, a sound image localization apparatus has been used as a means for improving a sense of reality by a virtual experience. In this type of system, for example, a three-dimensional sound field is generated from a monaural sound source by generating a multi-channel signal having a time difference, an amplitude difference, and a frequency characteristic difference based on a binaural method, thereby giving a sense of direction and a sense of distance. Is generated. That is, the audio input signal has a specific frequency component attenuated by, for example, a notch filter to give a sense of vertical direction, is converted into a left and right channel signal having a time difference by a delay circuit, and is virtually converted by a FIR (finite impulse response) filter. A sound transfer characteristic from the sound source position is given. The filter coefficient of the FIR filter is given from an HRTF database storing a head related transfer function (HRTF) measured in advance by a dummy head.

  In Patent Document 1, since such a conventional sound image localization device cannot store HRTFs from all virtual sound source positions, normally, transfer characteristics from a position away from a listener by a predetermined distance are usually used. In order to solve the problem that the sound image felt by each ear at a distance outside the predetermined distance does not match and is not localized well, which is caused by measuring and storing only, the distance from the listener is different from the predetermined distance. When a position is designated as a virtual sound source position, the virtual sound source position is transferred from the virtual sound source position to each ear of the listener based on the transmission distance and transmission direction specified by the designated virtual sound source position and the distance between both ears of the listener. The transmission direction of the right channel and the transmission direction of the left channel are calculated respectively, and the acoustic transmission characteristics of the filter for the left and right channels are determined by the transmission directions of the left and right channels. Respectively determining techniques are disclosed.

Patent Document 2 discloses an example of a method for describing unique data with compatibility with a DVD video or DVD audio format.
Japanese Patent Laid-Open No. 10-17420 Japanese Patent Laid-Open No. 11-178090 NHK Broadcasting Technology Laboratory, “Basics of 3D Video”, Ohm, 1995 By Jens Brauert, “Spatial Acoustics”, Kashima Press, 1985

  The problem to be solved is that sound reproduction using spatial localization technology of spatial sound is compatible with existing reproduction methods including stereo reproduction like the existing DVD video standard and DVD audio, and recording. There is no format to play.

  The present invention has been made in view of such conventional circumstances, and is a format that enables reproduction of 3D sound field information while maintaining compatibility with normal normal audio information when recording and reproducing 3D sound field information. The purpose is to record and reproduce sound reproduction using spatial localization technology of spatial sound with compatibility with conventional reproduction methods including stereo reproduction like the existing DVD video standard and DVD audio. And

  In order to achieve the above object, a three-dimensional sound field information reproducing apparatus according to the present invention reproduces a normal audio audio object, management information for managing the audio object, and uses the management information to reproduce the audio. Means for controlling the reproduction of the object, and for each predetermined unit of each frame of the normal audio, information relating to the three-dimensional sound field information recorded in the user data area of the audio object is reproduced and three-dimensional using the information. And means for controlling reproduction of sound field data.

  In another aspect, the three-dimensional sound field information reproducing apparatus according to the present invention reproduces normal audio audio objects, management information for managing the audio objects, and uses the management information to reproduce the audio objects. Means for controlling reproduction, and for each predetermined unit of each frame of the normal audio, information relating to the three-dimensional sound field information is separated from the stream that is multiplexed and recorded as stream data different from the normal audio of the audio object. And means for controlling the reproduction of the three-dimensional sound field data using the information.

  In still another aspect, the three-dimensional sound field information reproducing device according to the present invention reproduces the audio object of normal audio, management information for managing the audio object, and uses the management information to reproduce the audio object. And a three-dimensional sound recorded in a management information area for a three-dimensional sound field different from the area in which the management information is recorded for each predetermined unit of each frame of the normal audio. And means for reproducing information relating to the field information and controlling reproduction of the three-dimensional sound field data using the information.

  In order to achieve the above object, a program according to the present invention includes a step of reproducing a normal audio audio object, a step of reproducing management information for managing the audio object, and controlling reproduction of the audio object using the information. For each predetermined unit of each frame of the normal audio, information related to the three-dimensional sound field information recorded in the user data area of the audio object is reproduced, and reproduction of the three-dimensional sound field data is controlled using the information. And causing the computer to execute the step of performing.

  In another aspect, a program according to the present invention reproduces a normal audio audio object, reproduces management information for managing the audio object, and controls reproduction of the audio object using the information, For each predetermined unit of each frame of the normal audio, information relating to the three-dimensional sound field information is separated and reproduced from the stream that is multiplexed and recorded as stream data different from the normal audio of the audio object, and the information is used. And a step of controlling the reproduction of the three-dimensional sound field data by a computer.

  In yet another aspect, the program according to the present invention includes a step of reproducing a normal audio audio object, a step of reproducing management information for managing the audio object, and controlling the reproduction of the audio object using the information, For each predetermined unit of each frame of normal audio, information related to 3D sound field information recorded in a management information area for 3D sound field different from the management information area is reproduced and 3 And a step of controlling the reproduction of the three-dimensional sound field data by a computer.

  According to the present invention, it is possible to provide a format that enables reproduction of 3D sound field information while maintaining compatibility with normal normal audio information when recording and reproducing 3D sound field information. Sound reproduction using the technology can be recorded and reproduced with compatibility with conventional reproduction methods including stereo reproduction like the existing DVD video standard and DVD audio. For example, audio data is highly correlated with normal data, such as biphonic recording, and in particular, sound compatibility can be maintained without compromising playback compatibility from existing normal audio recording media such as DVDs. By using this localization technology, it is possible to realize a highly realistic theater system capable of reproducing a three-dimensional sound field that enables viewers to enjoy stereoscopic sound with a high sense of reality.

  The best mode for carrying out a three-dimensional sound field information reproducing apparatus and program according to the present invention will be described below with reference to the accompanying drawings.

  This embodiment is a system capable of recording and reproducing 3D sound field information necessary for reproducing a three-dimensional sound field on a medium while maintaining compatibility with the DVD standard in a state compliant with, for example, a DVD video or DVD audio standard. It is to provide. This form is applicable not only to DVD but also to applications and media having a mechanism for reproducing normal audio.

  The present embodiment is composed of a recording device for three-dimensional sound field information and its reproducing device.

  FIG. 1 shows the overall configuration of the recording apparatus of this embodiment. The recording apparatus shown in FIG. 1 includes a normal audio microphone 1, a binaural audio microphone 2, a normal audio compressor 3, a buffer 4, a normal audio decoder 5, a subtractor 5a, a differential binaural audio compressor 6, an information multiplexer 7, A camera 8, a video compressor 9, a buffer 10, a DVD formatter 11, a recorder 12, a buffer 13, a time stamp generator 14, and a control unit 15 are provided. The control unit 15 outputs a command signal to each unit in the apparatus to perform individual operations.

  The normal audio microphone 1 records normal audio information and supplies it to the normal audio compressor 3. Normal audio here is defined as something other than three-dimensional sound field audio. For example, normal stereo audio.

  The binaural audio microphone 2 performs binaural recording as three-dimensional sound field data simultaneously with normal audio recording by the normal audio microphone 1, and supplies the binaural audio data to the subtractor 5a. This binaural recording is performed by using a dummy head or the like.

  The normal audio compressor 3 compresses normal audio data recorded by the normal audio microphone 1 using a predetermined compression method, and supplies the compressed data to the buffer 4 and the normal audio decoder 5. The compression method used here may be either MPEG or DOLBY-DIGITAL (AC3).

  The buffer 4 temporarily buffers the data compressed by the normal audio compressor 3 in order to synchronize with video and differential binaural audio described later.

  The normal audio decoder 5 decodes the data compressed by the normal audio compressor 3 and supplies it to the subtracter 5a.

  The subtractor 5a subtracts the binaural audio data recorded by the binaural audio microphone 2 from the normal audio data decoded by the normal audio decoder 5 to create differential binaural audio data, which is subtracted from the binaural audio data. Supply to the compressor 6.

  The differential binaural audio compressor 6 compresses the differential binaural audio data from the subtractor 5a using a predetermined compression method. The compression method may be MPEG or DOLBY-DIGITAL (AC3). The AAC method and the like have very good compression efficiency using variable length coding, and can be said to be a suitable algorithm in this method.

  Here, a compressor (encoding device) when the AAC method is used as the compression method will be described with reference to FIGS.

  The coding apparatus using the AAC method shown in FIG. 2 includes functional units shown in the drawing, that is, an auditory psychological analyzer 101, an MDCT (Modified Discrete Cosine Transform) device 102, a quantizer 103, and a group processor 104. , A variable length encoder 105, a bit number determination unit 106, a bit stream generator 107, and a processing control unit 108.

  The psychoacoustic analyzer 101 takes an audio signal in a frame unit of a predetermined number of samples, performs a fast Fourier transform (FFT) on the input audio signal, obtains a frequency spectrum, and based on the frequency spectrum Then, the auditory masking is calculated to calculate the allowable quantization noise power and the auditory psychological parameter for each preset frequency band, and the conversion block length for MDCT is determined based on the auditory psychological parameter.

  Similar to the auditory psychological analyzer 101, the MDCT unit 102 takes an audio signal in units of a frame consisting of a predetermined number of samples, performs MDCT on the input audio signal to convert it into a frequency spectrum, and calculates MDCT coefficients related to each frequency spectrum. Ask. In this case, the MDCT unit 102 overlaps the conversion block length by 50% at the time of conversion into the frequency spectrum, and converts, for example, 2048 samples into 1024 MDCT coefficients.

  The MDCT unit 102 is a block switching unit for switching the block length to be subjected to MDCT to a long conversion block (long block) or a short conversion block (short block) based on the conversion block length information obtained from the psychoacoustic analyzer 101. The function is adopted. This is because, generally, the longer the transform block length, the more concentrated the spectrum, so that efficient bit allocation can be performed. However, the quantization noise in the frequency domain spreads over the entire transform block length when it is returned to the time domain. When a waveform that has a steep rise (attack part) after the quiet part is converted with a long block length and quantized, the quantization noise spreads to the quiet part, which is extremely harsh to hearing. Because it becomes.

  That is, the MDCT unit 102 selects a conversion block length based on the conversion block length information obtained from the psychoacoustic analyzer 101, and in particular, switches from a long conversion block to a plurality of short conversion blocks before and after the attack part. ing. For example, in the case of a stationary signal, the conversion block length of MDCT is converted into 1024 MDCT coefficients as a long block of 2048 samples, while in the case of a transient signal, it is 128 blocks as a short block of 256 samples. Convert to MDCT coefficient of book. For the short block, eight consecutive short conversion lengths are selected, and the number of output MDCT coefficients is set to 1024 to match the long block.

  The quantizer 103 divides 1024 MDCT coefficients into a plurality of scale factor bands for each frequency band based on human auditory characteristics, normalizes the MDCT coefficients for each scale factor band, and performs quantization. At that time, in the case of a short block, 128 MDCT coefficients are divided into a plurality of scale factor bands. Also, the number of quantization steps for each scale factor band is controlled so that the quantization noise calculated for each scale factor band does not become larger than the allowable quantization noise power calculated by the psychoacoustic analyzer 101, and Quantization is performed by controlling the total number of quantization steps so that the number of bits required for quantization falls within a predetermined number of bits per frame. The scale factor band quantization step number is obtained by separating the sample data in each frequency band into a waveform and a magnification, normalizing the waveform so that the maximum amplitude of the waveform is 1.0, and encoding the magnification. It is also called a scale factor. The quantized data is supplied to the group processor 104.

  The group processor 104 groups the short blocks so as to obtain higher encoding efficiency for the quantized data from the quantizer 103. In the grouped blocks, auxiliary information is shared, and coding efficiency is improved.

  FIG. 3 shows an example of the grouping. Eight short blocks are divided into four groups (Group 0 to 4), and each group includes three (Group 0) and one (Group). 1) Two (Group 2) and two (Group 3) short blocks are included.

  The variable-length encoder 105 performs variable-length encoding processing on the encoding parameters such as the quantized value of the MDCT coefficient and the scale factor after being processed by the quantizer 103 and the group processor 104, thereby providing redundancy. Is output to the bit number determination unit 106.

  The bit number determination unit 106 determines whether or not the number of encoded bits for one frame is within a predetermined range set in advance. If the condition is satisfied, the encoded data is directly converted into a bit stream. Although it outputs to the generator 107, when not satisfy | filling, the determination result is output to the process control part 108.

  The processing control unit 108 causes the quantizer 103, the group processor 104, and the variable length encoder 105 to execute the series of processes again based on the determination result, and the bit number determiner 106 satisfies the condition. The process is repeated until it is determined that there is. The encoded data that satisfies the above condition in the bit number determination unit 106 is output to the bit stream generator 107 and transmitted as a multiplexed bit stream together with encoding parameters such as block information.

  The above is the description when the AAC method is used as the compression method.

  Subsequently, the description returns to the configuration of the recording apparatus shown in FIG.

  The video compressor 9 compresses the video signal input from the camera 8 using a predetermined compression method. As the compression method here, the MPEG method or the like is used.

  The buffer 10 compresses the video compressed data compressed by the video compressor 9 by data compressed by the normal audio compressor 3 and buffered in the buffer 4, or compressed by the differential binaural audio compressor 6 and buffered. In order to synchronize with the differential binaural audio buffered in FIG.

  The time stamp generator 14 uses the 27 MHz or 90 KHz counter information as the time stamp and supplies it to the information multiplexer 7.

  The information multiplexer 7 synchronizes the video compressed data buffered in the buffer 10, the normal audio compressed data buffered in the buffer 4, and the differential binaural audio compressed data buffered in the buffer 13. Multiplex while. In this case, multiplexing is performed by packing each elementary element using the program stream method in the MPEG system layer synchronization method, and by multiplexing the presentation time stamps so as to obtain synchronization during reproduction. As the time stamp, counter information of 27 MHz or 90 KHz from the time stamp generator 14 is used. Since this mechanism is possible using the MPEG multiplexing standard, a detailed description is omitted.

  The DVD formatter 11 formats the multiplexed stream into a format conforming to the DVD standard described later.

  The recorder 12 records the multiplexed stream formatted in the format conforming to the DVD standard by the DVD formatter 11 on the recording medium RM. Here, in order to create a DVD ROM type medium as the recording medium RM, it is once recorded on the HDD as DVD master data and then recorded on the DVD medium through a manufacturing process.

  Next, the reproducing apparatus of the present embodiment will be described with reference to FIG.

  4 includes a playback device 21, a DVD format decoder 22, an information separator 23, a differential binaural audio decoder 24, an adder 24a, a normal audio decoder 25, a sound source selector 26, a GUI (Graphical User). Interface) 27, speaker 28, buffer 29, video decoder 30, buffer 31, image display 32, buffer 33, STC time stamp comparator 34, and control unit 35. The control unit 35 outputs a command signal to each unit in the apparatus to perform individual operations.

  The reproducer 21 reproduces data from the recording medium 15 and supplies it to the DVD format decoder 22.

  The DVD format decoder 22 extracts an MPEG stream from the playback data DVD format. Although not shown in this figure, information for reproducing a DVD (for example, playlist information and special reproduction information) is separately extracted and interactively reproduced via a user interface or CPU (not shown). It can be carried out.

  The information separator 23 demultiplexes the extracted MPEG stream into MPEG, normal audio, and differential binaural audio.

  The video decoder 30 decodes the separated video and supplies it to the buffer 31. The buffer 31 temporarily buffers the decoded video.

  The normal audio decoder 25 decodes the separated normal audio and supplies it to the buffer 29 and the adder 24a. The buffer 29 temporarily buffers the decoded normal audio.

  The differential binaural audio decoder 24 decodes the differential binaural audio data and supplies it to the adder 24a. The adder 24 a adds the decoded differential binaural audio data and the normal audio from the normal audio decoder 25 and supplies them to the buffer 33. The buffer 33 temporarily buffers the added data.

  The STC time stamp comparator 34 detects the SCR (system clock reference) and time stamp recorded in the header of the packed data of each elementary by the information separator 23, and sets it by the MPEG multiplexing method. The STC (system time clock) time synchronized by the SCR and the presentation time stamp are compared, and when the presentation time stamp time matches the STC time, the decoded data of each of the buffers 33, 29, and 31 Outputs elementary information.

  The image display 32 displays the video from the buffer 31 as an image.

  The GUI 27 receives a selection signal for selecting a sound source designated by the user, that is, normal audio or binaural audio, and outputs it to the sound source selector 26. The sound source selector 26 selects an audio sound source from the buffers 33 and 29 according to a selection signal from the GUI 27, and outputs and reproduces the audio from the speaker 28.

  In this way, the normal audio data and the binaural audio data that is the three-dimensional sound field data are subjected to the difference to delete the strongly correlated portion, and the phase difference and the sounding portion information representing the sound field are used as the difference. By encoding, encoding efficiency and recording efficiency can be further increased.

  Next, using the DVD video standard shown in FIG. 5, information relating to the three-dimensional sound field information is multiplexed and recorded as stream data different from the normal audio of the audio object for each predetermined unit of each frame of normal audio. An embodiment for reproduction will be described.

  Since the above-described differential binaural audio data is not originally a DVD standard, it is conceivable to record it as a separate stream in a pack that is MPEG-multiplexed. In the lowest hierarchy shown in FIG. 5, the differential binaural audio data forms a layer as a single audio frame with a predetermined number of samples. A predetermined number of these audio frames are collected to form a pack with about 2 kB. This pack has a header, which is composed of a 14-byte pack header and an audio packet. The audio packet is a 9-29 byte packet header, 1 byte substream ID, 3 bytes of audio frame information, 3 bytes. Audio data information is recorded, followed by 2013-byte differential audio data. These are MPEG-multiplexed as a differential pack (D_PACK) as a binaural audio data pack together with other normal audio packs (A_PACK) and video packs (V_PACK).

  Here, looking at the DVD video format shown in FIG. 5 from the top, the DVD video is divided into a volume and file structure, a DVD-video zone, and a DVD-others zone as a volume space. Among these, the DVD-video zone has a structure of a video manager (VMG) and a video title set (VTS).

  The video manager describes information such as identification information of a subsequent video title set such as video manager information, start addresses and end addresses of various information itself, and from which video stream playback is started. Control data such as address information and identification information of audio and video data to be reproduced is described in the video title set.

  Control data in these video managers and video title sets is a management information area and is essential information for reproduction. Data in this area is recorded by the DVD formatter 11 or the DVD formatting step described above. Then, it is reproduced by the DVD format decoder 22 and the DVD format decoding step described above.

  On the rear side of the control data, there is a set of video and audio multiplexed MPEG streams called a video object set (VOBS), and the video object set includes a small unit MPEG stream called a video object (VOB). Below the video object is a unit called a subdivided cell (CELL), and below the cell is a video object unit (VOBU), which has a structure almost equivalent to a group of pictures (GOP) of an MPEG stream. And about 0.4 to 1.0 seconds.

  In the video object unit, stream search information such as a navigation pack (NV_PACK) is described at the top. In addition, there are data in which video compression data called a video pack (V_PACK) is packed, and data in which audio compression data called an audio pack (A_PACK) is packed, which are respectively MPEG-multiplexed. In this way, in a format compliant with the DVD video standard, the above-described binaural audio data pack called a differential pack (D_PACK) is packed and MPEG-multiplexed.

  Therefore, in this embodiment, if a differential pack is used, binaural three-dimensional sound field audio can be reproduced, and if a differential pack is not used, a standard normal audio can be output as a DVD video standard.

  Next, using the DVD video standard shown in FIG. 6, for each predetermined unit of each frame of normal audio, information relating to the three-dimensional sound field information is managed separately for the three-dimensional sound field from the management information area. An embodiment in the case of recording / reproducing in an area will be described.

  This method takes a format that conforms to the DVD video standard, but does not record it in the above-described differential pack, but in a DVD standard compliant format called DVD-others zone. This is a method for recording and reproducing audio field information.

  In the DVD-others zone, a structure called a video manager (DVMG) and a video title set (DVTS) is described. The video manager describes information such as identification information of a subsequent video title set such as video manager information, start addresses and end addresses of various information itself, and from which video stream playback is started. The video title set describes video title set information (DVTSI) such as address information and identification information of audio and video data to be reproduced.

  These video manager and video title set information are management information areas, which are indispensable information for reproduction. Data in this area is recorded by the DVD formatter 11 or the DVD formatting step described above, and is described above. It is reproduced by the DVD format decoder 22 or the DVD format decoding step.

  The video title set has a video object set (DVOBS) video and audio multiplexed MPEG stream set behind the video title set information, and a video object (DVOB) small unit MPEG stream. . Below the video object, there is a unit of further subdivided cells (DCELL), and further a video object unit (DVOBU). In this video object unit, several frames of the frame layer of binaural 3D sound field audio information are collected. It has a structure.

  In this way, the structure is the same as that of 2D video data in the DVD-video zone, and each video object unit, cell, video object, etc. has the same number of frames (same playback time length), so that search etc. Can improve accessibility.

  In this way, if binaural 3D sound field audio information data is described in a format compliant with the DVD video standard and linked to the DVD-video zone and DVD-others zone, binaural 3D sound field audio and Normal audio can be recorded and reproduced with DVD video standard compatibility.

  Next, by using the DVD audio standard shown in FIG. 7, information relating to the three-dimensional sound field information is recorded by multiplexing as stream data different from the normal audio of the audio object for each predetermined unit of each frame of normal audio. An embodiment for reproduction will be described.

  As shown in FIG. 7, the DVD audio format is composed of an audio manager (AMG) and areas of a plurality of audio title sets (ATS) following the audio manager. Corresponding to this, each of the audio title sets is composed of a leading ATS information (ATSI) followed by one or more audio object sets (AOBS).

  These audio manager and ATS information are management information areas, which are indispensable information for reproduction. Data in this area is recorded by the DVD formatter 11 and the DVD formatting step described above, and is decoded by the DVD format. It is reproduced by the device 22 and the DVD format decoding step.

  Each audio object set is composed of a plurality of audio objects (AOB). Each audio object is composed of a plurality of cells (CELL), and the cell is further composed of a plurality of audio object units (AOBU). Each audio object unit is composed of a plurality of packs, and one pack is composed of 2048 bytes. The audio object unit is composed of an arbitrary number of packs for 0.4 to 1.0 seconds. Adjacent audio packs (A_PACK) are arranged so that audio signals are related to each other. For example, in the case of stereo, an L channel pack and an R channel pack are arranged adjacent to each other. Adjacent to each other. Each of these is MPEG multiplexed. In the audio object unit, the above binaural audio data pack called a differential pack (D_PACK) is packed and MPEG-multiplexed.

  Therefore, if a differential pack is used, binaural three-dimensional sound field audio can be reproduced, and if a differential pack is not used, a standard normal audio can be output as a DVD video standard.

  Next, referring to FIG. 8A to FIG. 8-7 and FIG. 9, using the DVD video standard, information on the three-dimensional sound field information is obtained for each predetermined unit of each frame of normal audio. An embodiment for recording / reproducing data in the data area will be described.

  FIGS. 8-1 to 8-7 are explanatory tables of MPEG video stream video layers (video layers of the MPEG1 video standard (MPEG1 VIDEO SYNTAX (extracted from ISO-IEC11172-2))), and FIG. An explanatory table of multiplexed transport stream system layers is shown respectively.

  The MPEG standard provides a mechanism for transmitting data in a user data area or a private stream so that compatibility can be obtained.

  For example, in the MPEG video standard, user data areas are set in the picture layer, the GOP layer, and the like. These are recorded in a data packet such as user_data or private_data_byte which is set as a predetermined area in which data unrelated to video and audio can be embedded in MPEG syntax, or private_stream which can be arbitrarily set by the user.

  For example, the picture layer in MPEG1 video is as shown in FIGS. 8-1 to 8-7, and user_data can be recorded in 8-bit units after sending user_data_start_code before the slice layer. Mechanism is defined.

  Further, in the system layer of a multiplexed transport stream such as MPEG2, if the transport_private_data_flag is set to 1 as shown in FIG. 9, it is possible to clearly indicate that private_data exists and the data length does not protrude from the transport packet. Originally, private_data having the data length set in transport_private_data_length can be transmitted.

  In addition to this, there are several mechanisms for recording user-specific data in the MPEG system, such as transmitting by declaring a dedicated packet with private_stream set in stream_id, and binaural in the present invention. Three-dimensional sound field audio information can be recorded in these areas.

  An example using user_data of MPEG1 video will be described in a little more detail.

  user_data_start_code is defined in MPEG as 0x000001B2 before the slice layer. After sending the code, a code of, for example, 0x0f0f0f0f2428fdaa, which is a uniquely identifiable code indicating the presence of the function value used for authentication of the present invention in the user data area, is transmitted. This code is recorded for identification purposes when user_data is used in other applications, and the code value has no particular meaning. The audio frame layer structure of FIG. 6 is recorded after the code, and an audio frame layer corresponding to a picture display section is recorded for each MPEG picture. If the time width between the picture display section and the audio frame playback section is different, it is multiplexed in a format that allows an error of about one packet on average, and information on the difference between the presentation time at the beginning of the video and the beginning of the audio is obtained. The head count of 90 KHz or 27 MHz may be recorded at about 32 bits at the beginning of user_data, or playback may be performed in the order of data arrival using the playback side clock, and implicit synchronization may be established.

  In this example, recording in the user data area of an MPEG video with a high transmission rate has been described. However, in the audio compression method DOLBY-DIGITAL (AC3), a function of repeating compressed data called SyncFrame in a predetermined unit is used. Finally, there are functions called auxdata and errorcheck, and user data can be sent to auxdata if the first bit is set to 1. Therefore, such a mechanism may be used. DOLBY-DIGITAL (AC3) is an American ATSC standard, and this syntax is described in detail in the ATSC standard (20 Dec.1995) Digital audio Compression (AC-3) (Doc.A / 52). Has been.

  Next, a processing flowchart of a program used in the recording apparatus of the present embodiment will be described with reference to FIG. The detailed processing contents of the steps here are substantially the same as those described in the block diagram of the recording apparatus shown in FIG. 1 described above, so only the order of the steps will be briefly described here.

  First, in step S110, acoustic data is input from the normal audio microphone 1 and the binaural microphone 2. Further, image data from the camera 8 is input for a predetermined time and stored in the memory.

  Next, in step S120, the video data and the normal audio data are compressed, in step S130, the video compressed data and the normal audio compressed data are temporarily buffered, and in step S140, the normal audio data is decoded.

  Next, in step S150, subtraction calculation of binaural audio data and normal audio decoded data is performed, and in step S160, differential binaural audio data is compressed.

  Next, in step S170, the video compression data, normal audio compression data, and differential binaural audio compression data are multiplexed while being synchronized, and in step S180, the DVD is formatted. In step S190, in predetermined units. Record on media. If the recording medium is a DVD, 2 KB is the unit. When outputting to a communication path or the like, packetization peculiar to the communication path is performed at this step.

  Next, in step S200, it is determined whether there is still input image data. If there is (YES), the process returns to step S110, and if not (NO), the program ends.

  Next, a processing flowchart of a program used in the playback apparatus of this embodiment will be described with reference to FIG. Since the detailed processing contents of the steps here are substantially the same as those described in the block diagram of the playback apparatus shown in FIG. 4 described above, only the order of the steps will be briefly described here.

  First, in step S210, multiplexed data is read in a predetermined unit from a recording medium or a transmission path.

  Next, in step S220, the DVD format is decoded. For decoding of the DVD format, an MPEG stream is extracted from the DVD format. Although not shown in this step, information for reproducing the DVD (for example, playlist information and special playback information) is separately extracted, Including interactive playback via a user interface or CPU.

  Next, in step S230, information extraction of the extracted MPEG stream is performed. In step S240, the compressed video data and the compressed normal audio data are decoded. In step S250, the compressed video data and the compressed normal audio data are decoded. In step S260, the compressed differential binaural audio data is decoded.

  Next, in step S270, the difference binaural audio data and the normal audio decoded data are added and calculated, and in step S280, a sound source designated by the user from the GUI, that is, a signal for selecting normal audio or binaural audio is used. Select whether to play binaural audio data or normal audio.

  Next, in step S290, the video and the selected audio are synchronized and displayed and reproduced by a speaker.

  Next, in step S300, it is determined whether there is still display image acoustic data. If there is (YES), the process returns to step S210, and if not (NO), the program is terminated.

  In the present embodiment, the final information is recorded on the recording medium in each process by the apparatus shown in FIGS. 1 and 4 and the program shown in FIGS. 10 and 11, but otherwise, it is peculiar to communication and broadcasting. Packetization may be performed, and transmission or reception may be performed via a packetizer to a broadcast or communication network.

  Further, even if data is not recorded on the recording medium, it is possible to transmit the data via any transmission medium such as communication and broadcasting. In that case, the recording apparatus can also be used as a transmission apparatus. The playback device can also be used as a receiving device.

  In addition, the recording medium on which the signal data of this embodiment is recorded records a format that enables the reproduction of 3D sound field information while maintaining compatibility with normal normal audio information when recording and reproducing 3D sound field information. There is an effect peculiar to the medium that it is.

  In addition, the definition of the medium includes not only a narrowly-defined medium that can record data, but also electromagnetic waves, light, and the like for transmitting signal data. In addition, the information recorded on the recording medium includes data itself such as an electronic file when not recorded.

  The binaural audio data information according to the present embodiment has been described as being recorded for each picture when MPEG video user data is used. However, the binaural audio data information may be about every 0.5 seconds or every 1 second. Absent. In that case, it can be realized by using user data of the GOP layer of MPEG.

  Although this embodiment mainly describes audio, the present invention is effective even when audio data is present together with video and audio and video are multiplexed by MPEG multiplexing. The same applies to other sub-pictures and data such as control information.

  In addition to biphonically recorded audio data, 3D sound field data includes data that has special surround effects, and rare data that is recorded in an anechoic room that is necessary to create a 3D sound field. From sound source data, it is possible to create a sound field virtually similar to biphonic recording by simulation using head field transfer functions and sound field environment data such as a hall.

  In the present embodiment, the information about the three-dimensional sound field information has been described with the audio data recorded by biphonic recording. However, it is necessary to create the above-mentioned special surround effect data and the three-dimensional sound field. Rare sound source data (rare audio) as recorded in an anechoic room may be used. Rare audio has the advantage that it is easy to create a three-dimensional sound field because no special effects are applied.

  In the present embodiment, the compression method of the three-dimensional sound field data has been described by MPEG or DOLBY-DIGITAL (AC3), but may be a method of quantization by other orthogonal transformation such as DPCM or DCT. As the type of the audio object, one that is not compressed by linear PCM or one that is reversibly compressed, for example, Packed PCM (lossless compression method) adopted for DVD audio may be used. Linear PCM multi-channel audio can also be applied to normal audio. In other words, the difference between the multi-channel LR 2CH and the LR corresponding to the three-dimensional sound field data may be taken as in this embodiment. Other CHs are recorded as they are. Further, when there is a margin in the processing amount, the CH having the strongest correlation may be selected every predetermined time, and the difference from the three-dimensional sound field data may be adaptively taken. In that case, the information indicating which channel the difference is from may be indicated by a few bits in a format instructing the header or user area. The rare audio is not limited to data recorded in an anechoic room, but may be a hall sound recording using a multi-microphone or a master sound source mixed down to 6 channels.

  In this embodiment, binaural audio is used as a method for reproducing a three-dimensional sound field. However, for example, a local sound field may be created using an array speaker by having a plurality of CHs. . In other words, reproduction with a delay amount corresponding to the difference between the path from the center of the speaker array to the focal point and the path from each speaker array to the focal point so as to locally increase the sound pressure near a focal point in space. This is a method realized by a signal.

  The principle of this method will be described with reference to FIG. First, as the speaker 28 connected to the sound source selector of the playback apparatus described above, a plurality of speaker arrays 28a to 28a are assembled in an array as shown in FIG. 12, and a delay circuit is provided for each of the speaker arrays 28a to 28a. 28b to 28b are provided individually. In this configuration, when the delay time is set in each of the delay circuits 28b to 28b so as to focus on the vicinity of the listening position, the sound pressure component generated at the focal point at the listening position is greater than the direct sound from the speaker arrays 28 to 28. It is possible to reproduce it so as to be extremely high. By using this principle and continuously controlling in real time, the localization of the sound image can be controlled by linking to the position of the object of the stereoscopic moving image.

  In this case, as shown in FIG. 13, from the listening position P1 facing the speaker arrays 28a to 28a constituting the speakers of the reproducing apparatus and the LCD (liquid crystal display) 32a constituting the image display, several localized sound fields P2, P3 are obtained. To create a three-dimensional sound field space. In this case, a plurality of CHs are recorded in a differential pack (D_PACK) as shown in FIGS. 5 and 7, multiplexed in a plurality of streams. Alternatively, as shown in FIG. 6, an audio frame is created for each CH in the DVD others zone, and a plurality of CHs are multiplexed and recorded in a predetermined order, so that a three-dimensional sound field is maintained while maintaining compatibility with existing DVDs. It is possible to record (transmit) data.

  Note that the functions of the above-described apparatus may be realized by a computer by a program. This program may be read from a recording medium and loaded into a computer, or may be transmitted via a communication network and loaded into a computer.

  As mentioned above, although the Example of this invention was described, the suitable aspect of this invention is enumerated in the following (1)-(40).

  (1): In an information recording medium including a normal audio object recorded in a three-dimensional sound field information recording medium and a management information area recording information for managing the audio object, Information on the three-dimensional sound field information is recorded in the user data area of the object for each predetermined unit of the frame.

  (2): In an information recording medium including a normal audio object recording area and a management information area recording information for managing the audio object in the three-dimensional sound field information recording medium, Information on the three-dimensional sound field information is recorded as stream data different from the normal audio of the audio object by multiplexing for each predetermined unit of the frame.

  (3): In a three-dimensional sound field information recording medium, an information recording medium comprising an area where normal audio objects are recorded and a management information area where information for managing the audio objects is recorded. Information on 3D sound field information is recorded in a management information area for a 3D sound field separate from the management information area for each predetermined unit of the frame.

  (4): In the three-dimensional sound field information recording media of the above (1) to (3), the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio includes normal audio information and an anechoic chamber. The difference information with the rare audio information equivalent to that recorded in step 1 is encoded using at least one of differential encoding, predictive encoding, or encoding using orthogonal transform, and then recorded.

  (5): In the three-dimensional sound field information recording media of the above (1) to (3), the information regarding the three-dimensional sound field information for each predetermined unit of each frame of normal audio is the normal audio information, the hall or the head The parameter information for the purpose of hall acousticization related to the transfer function was added.

  (6): In the three-dimensional sound field information reproducing method, reproducing a normal audio audio object, reproducing information managing the audio object, and controlling reproduction of the audio object using the information, For each predetermined unit of each frame of normal audio, reproducing information related to the three-dimensional sound field information recorded in the user data area of the audio object, and controlling the reproduction of the three-dimensional sound field data using the information. It was characterized by having.

  (7): In the three-dimensional sound field information reproduction method, reproducing a normal audio audio object, reproducing information managing the audio object, and controlling reproduction of the audio object using the information, For each predetermined unit of each frame of normal audio, information related to the three-dimensional sound field information is separated and reproduced from the stream that is multiplexed and recorded as stream data different from the normal audio of the audio object, and the information is used for 3 And a step of controlling reproduction of the dimensional sound field data.

  (8): In the three-dimensional sound field information reproduction method, a step of reproducing an audio object of normal audio, a step of reproducing information managing the audio object, and controlling reproduction of the audio object using the information, For each predetermined unit of each frame of normal audio, information related to 3D sound field information recorded in a management information area for 3D sound field different from the management information area is reproduced, and the information is used to 3D And a step of controlling the reproduction of the sound field data.

  (9): In the three-dimensional sound field information reproducing method of (6) to (8) above, the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio includes normal audio information and an anechoic chamber. The difference information with the rare audio information equivalent to that recorded in the above is differentially encoded, predictively encoded, or encoded using orthogonal transform, and the information relating to the three-dimensional sound field information is differentially encoded and predicted code. The reproduction is performed after decoding using at least one of conversion or orthogonal transformation.

  (10): In the three-dimensional sound field information reproducing method of (6) to (8) above, the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio is normal audio information, hole or head It is characterized by parameter information for the purpose of hall acoustics related to the transfer function.

  (11): In the three-dimensional sound field information reproducing device, means for reproducing an audio object of normal audio, means for reproducing information for managing the audio object, and controlling reproduction of the audio object using the information; Means for reproducing information on the three-dimensional sound field information recorded in the user data area of the audio object for each predetermined unit of each frame of normal audio and controlling the reproduction of the three-dimensional sound field data using the information; It was characterized by having.

  (12): In the three-dimensional sound field information reproducing apparatus, means for reproducing an audio object of normal audio, means for reproducing information for managing the audio object, and controlling reproduction of the audio object using the information, For each predetermined unit of each frame of normal audio, information related to the three-dimensional sound field information is separated and reproduced from the stream that is multiplexed and recorded as stream data different from the normal audio of the audio object, and the information is used for 3 And means for controlling reproduction of the two-dimensional sound field data.

  (13): In the three-dimensional sound field information reproducing device, means for reproducing an audio object of normal audio, means for reproducing information for managing the audio object, and the management for each predetermined unit of each frame of normal audio And means for reproducing information relating to the three-dimensional sound field information recorded in the management information area for the three-dimensional sound field different from the information area and controlling the reproduction of the three-dimensional sound field data using the information. It was characterized.

  (14): In the three-dimensional sound field information reproducing device according to (11) to (13), the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio is the normal audio information and the anechoic chamber. The difference information from the recorded rare audio information equivalent to the recorded audio is differentially encoded, predictively encoded, or encoded using orthogonal transform, and the information relating to the three-dimensional sound field information is differentially encoded and predictively encoded. Alternatively, the reproduction is performed after decoding using at least one of orthogonal transforms.

  (15): In the three-dimensional sound field information reproducing device according to (11) to (13), the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio is transmitted from the normal audio information to the hall or head. It is characterized by parameter information for the purpose of hall acoustics related to functions.

  (16): In a three-dimensional sound field information reproduction program, reproducing a normal audio audio object, reproducing information managing the audio object, and controlling reproduction of the audio object using the information, For each predetermined unit of each frame of normal audio, reproducing information related to the three-dimensional sound field information recorded in the user data area of the audio object, and controlling the reproduction of the three-dimensional sound field data using the information. It is characterized in that it is executed by a computer.

  (17): In a three-dimensional sound field information reproduction program, reproducing a normal audio audio object, reproducing information managing the audio object, and controlling reproduction of the audio object using the information, For each predetermined unit of each frame of normal audio, information related to the three-dimensional sound field information is separated and reproduced from the stream that is multiplexed and recorded as stream data different from the normal audio of the audio object, and the information is used for 3 And a step of controlling the reproduction of the two-dimensional sound field data by a computer.

  (18): In a three-dimensional sound field information reproduction program, reproducing a normal audio audio object, reproducing information managing the audio object, and controlling reproduction of the audio object using the information, For each predetermined unit of each frame of normal audio, information related to 3D sound field information recorded in a management information area for 3D sound field different from the management information area is reproduced, and the information is used to 3D And causing the computer to execute the step of controlling the reproduction of the sound field data.

  (19): In the three-dimensional sound field information reproduction program of (16) to (18) above, the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio includes normal audio information and an anechoic chamber. The difference information with the rare audio information equivalent to that recorded in the above is differentially encoded, predictively encoded, or encoded using orthogonal transform, and the information relating to the three-dimensional sound field information is differentially encoded and predicted code. The reproduction is performed after decoding using at least one of conversion or orthogonal transformation.

  (20): In the three-dimensional sound field information reproduction program of (16) to (18) above, the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio is the normal audio information, the hall or the head It is characterized by parameter information for the purpose of hall acoustics related to the transfer function.

  According to the above (1) to (20), it is possible to provide a format that enables reproduction of 3D sound field information while maintaining compatibility with normal normal audio information when recording and reproducing 3D sound field information. In addition, sound reproduction using spatial localization technology of spatial sound can be recorded and reproduced with compatibility with conventional reproduction methods including stereo reproduction like the existing DVD video standard and DVD audio. For example, audio data is highly correlated with normal data, such as biphonic recording, and in particular, sound compatibility can be maintained without compromising playback compatibility from existing normal audio recording media such as DVDs. By using this localization technology, it is possible to realize a highly realistic theater system capable of reproducing a three-dimensional sound field that enables viewers to enjoy stereoscopic sound with a high sense of reality.

  (21) In a three-dimensional sound field information recording medium, an information recording medium comprising an area in which normal audio signals and video signals are recorded as audio objects, and a management information area in which information for managing the audio objects is recorded. Information on three-dimensional sound field information is recorded in the user data area of the object for each predetermined unit of each frame of normal audio.

  (22): In a three-dimensional sound field information recording medium, an information recording medium comprising an area in which normal audio signals and video signals are recorded as audio objects, and a management information area in which information for managing the audio objects is recorded. For each predetermined unit of each frame of normal audio, information on the three-dimensional sound field information is recorded by multiplexing as stream data different from the normal audio of the audio object.

  (23): In an information recording medium comprising a three-dimensional sound field information recording medium, an area in which normal audio signals and video signals are recorded as audio objects, and a management information area in which information for managing the audio objects is recorded. Information regarding 3D sound field information is recorded in a management information area for a 3D sound field separate from the management information area for each predetermined unit of each frame of normal audio.

  (24): In the three-dimensional sound field information recording media of the above (21) to (23), the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio includes normal audio information and an anechoic chamber. The difference information with the equivalent rare audio information recorded in step 1 is encoded using at least one of differential encoding, predictive encoding, or encoding using orthogonal transform, and then recorded.

  (25): In the three-dimensional sound field information recording medium of (21) to (23) above, the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio is the normal audio information, the hall or the head The parameter information for the purpose of hall acousticization related to the transfer function was added.

  (26): In the three-dimensional sound field information reproducing method, a normal audio signal and a video signal are used as audio objects, the audio object is reproduced, information for managing the audio object is reproduced, and the audio object is reproduced using the information. And reproducing information related to 3D sound field information recorded in the user data area of the audio object for each predetermined unit of each frame of normal audio, and using the information to reproduce 3D sound field data And the step of controlling the reproduction of the video.

  (27): In the three-dimensional sound field information reproducing method, a normal audio signal and a video signal are used as audio objects, the audio object is reproduced, information for managing the audio object is reproduced, and the audio object is reproduced using the information. The information relating to the three-dimensional sound field information is separated from the stream that is multiplexed and recorded as stream data different from the normal audio of the audio object, for each predetermined unit of each frame of normal audio. And a step of controlling reproduction of the three-dimensional sound field data using the information.

  (28): In the three-dimensional sound field information reproducing method, a normal audio signal and a video signal are used as audio objects, the audio object is reproduced, information for managing the audio object is reproduced, and the audio object is reproduced using the information. Information relating to 3D sound field information recorded in a 3D sound field management information area different from the management information area is reproduced for each predetermined unit of each frame of normal audio. And the step of controlling the reproduction of the three-dimensional sound field data using the information.

  (29): In the three-dimensional sound field information reproducing method of (26) to (28), the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio includes normal audio information and an anechoic chamber. The difference information with the rare audio information equivalent to that recorded in the above is differentially encoded, predictively encoded, or encoded using orthogonal transform, and the information related to the three-dimensional sound field information is differentially encoded and predicted code. The reproduction is performed after decoding using at least one of conversion or orthogonal transformation.

  (30): In the three-dimensional sound field information reproducing method of (26) to (28) above, the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio is the normal audio information, the hole or the head It is characterized by parameter information for the purpose of hall acoustics related to the transfer function.

  (31): In a three-dimensional sound field information reproducing apparatus, a normal audio signal and a video signal are used as audio objects, means for reproducing the audio object, information for managing the audio object is reproduced, and the audio object is reproduced using the information. And information on 3D sound field information recorded in the user data area of the audio object for each predetermined unit of each frame of normal audio, and 3D sound field data using the information. And means for controlling the reproduction of the video.

  (32): In the three-dimensional sound field information reproducing apparatus, a normal audio signal and a video signal are used as audio objects, means for reproducing the audio object, information for managing the audio object is reproduced, and the audio object is reproduced using the information. Information on the three-dimensional sound field information is separated from the stream that is multiplexed and recorded as stream data different from the normal audio of the audio object for each predetermined unit of each frame of the normal audio. And a means for controlling the reproduction of the three-dimensional sound field data using the reproduced information.

  (33): In a three-dimensional sound field information reproducing apparatus, normal audio signals and video signals are used as audio objects, means for reproducing the audio objects, information for managing the audio objects is reproduced, and the audio objects are used using the information. Information relating to 3D sound field information recorded in a management information area for 3D sound fields different from the management information area for each predetermined unit of each frame of normal audio. And means for controlling reproduction of the three-dimensional sound field data using the information.

  (34): In the three-dimensional sound field information reproducing device according to (31) to (33), the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio includes normal audio information and an anechoic chamber. The difference information with the rare audio information equivalent to that recorded in the above is differentially encoded, predictively encoded, or encoded using orthogonal transform, and the information related to the three-dimensional sound field information is differentially encoded and predicted code. The reproduction is performed after decoding using at least one of conversion or orthogonal transformation.

  (35): In the three-dimensional sound field information reproducing device of (31) to (33) above, the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio is the normal audio information, the hall or the head It is characterized by parameter information for the purpose of hall acoustics related to the transfer function.

  (36): In a three-dimensional sound field information reproduction program, a normal audio signal and a video signal are used as audio objects, the audio object is reproduced, information for managing the audio object is reproduced, and the audio object is reproduced using the information. And reproducing information related to 3D sound field information recorded in the user data area of the audio object for each predetermined unit of each frame of normal audio, and using the information to reproduce 3D sound field data And a step of controlling the reproduction of the computer.

  (37): In a three-dimensional sound field information reproduction program, a normal audio signal and a video signal are used as audio objects, the audio object is reproduced, information for managing the audio object is reproduced, and the audio object is reproduced using the information. The information relating to the three-dimensional sound field information is separated from the stream that is multiplexed and recorded as stream data different from the normal audio of the audio object, for each predetermined unit of each frame of normal audio. And reproducing the information and using the information to control the reproduction of the three-dimensional sound field data.

  (38): In a three-dimensional sound field information reproducing program, a normal audio signal and a video signal are used as audio objects, the audio object is reproduced, information for managing the audio object is reproduced, and the audio object is reproduced using the information. Information relating to 3D sound field information recorded in a 3D sound field management information area different from the management information area is reproduced for each predetermined unit of each frame of normal audio. And the step of controlling the reproduction of the three-dimensional sound field data using the information is executed by a computer.

  (39): In the three-dimensional sound field information reproduction program of (36) to (38) above, the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio includes normal audio information and an anechoic chamber. The difference information with the rare audio information equivalent to that recorded in the above is differentially encoded, predictively encoded, or encoded using orthogonal transform, and the information related to the three-dimensional sound field information is differentially encoded and predicted code. The reproduction is performed after decoding using at least one of conversion or orthogonal transformation.

  (40): In the three-dimensional sound field information reproduction program of (36) to (38) above, the information related to the three-dimensional sound field information for each predetermined unit of each frame of normal audio is the normal audio information, the hall or the head It is characterized by parameter information for the purpose of hall acoustics related to the transfer function.

  According to the above (20) to (40), it is possible to provide a format that enables reproduction of 3D sound field information while maintaining compatibility with normal normal audio information when recording and reproducing 3D sound field information. It is possible to record and reproduce sound reproduction using spatial localization technology of spatial sound, together with video images, with compatibility with conventional reproduction methods including stereo reproduction like the existing DVD video standard and DVD audio. Will be able to. For example, audio data is highly correlated with normal data, such as biphonic recording, and in particular, without compromising playback compatibility from existing normal audio recording media such as DVD. By using the localization technology, it is possible to realize a highly realistic theater system that can enjoy three-dimensional sound with high sense of presence for viewers and can reproduce a three-dimensional sound field.

  As described above, the present invention can be applied to an apparatus, a method, and a program for suitably realizing 3D sound field information reproduction while maintaining compatibility with normal audio information. In particular, a highly realistic theater system that allows viewers to enjoy stereophonic sound with a high sense of reality using sound localization technology without compromising the compatibility of existing normal audio recording media such as DVDs. Applicable to technical use.

It is a schematic block diagram which shows the whole structure of the three-dimensional sound field information recording device based on the Example of this invention. It is a schematic block diagram which shows the whole structure of the compressor which used the AAC system as a compression system. It is a figure explaining an example of grouping by the group processor in the compressor shown in FIG. It is a schematic block diagram which shows the whole structure of the three-dimensional sound field information reproducing | regenerating apparatus based on the Example of this invention. It is a figure explaining the example of a format of the DVD video standard used in the Example of this invention. It is a figure explaining the other format example of the DVD video specification used in the Example of this invention. It is explanatory drawing which shows the example of a format of the DVD audio standard used in the Example of this invention. 4 is an explanatory table (No. 1) of an MPEG video stream video layer. 4 is an explanatory table (2) of an MPEG video stream video layer. 4 is an explanatory table (No. 3) of an MPEG video stream video layer. 4 is an explanatory table (part 4) of an MPEG video stream video layer. FIG. 10 is an explanatory table (No. 5) of an MPEG video stream video layer. FIG. 7 is an explanatory table (No. 6) of an MPEG video stream video layer. FIG. 10 is an explanatory table (No. 7) of an MPEG video stream video layer. FIG. It is an explanatory table of the multiplexed transport stream system layer of MPEG. It is a flowchart figure which shows the recording program of the Example of this invention. It is a flowchart figure which shows the reproduction | regeneration program of the Example of this invention. It is explanatory drawing at the time of using an array speaker. It is a system diagram at the time of using an array speaker.

Explanation of symbols

1 Normal Audio Microphone 2 Binaural Audio Microphone 3 Normal Audio Compressor 4 Buffer 5 Normal Audio Decoder 5a Subtractor 6 Differential Binaural Audio Compressor 7 Information Multiplexer 8 Camera 9 Video Compressor 10 Buffer 11 DVD Formatter 12 Recorder 13 buffer 14 time stamp generator 15 control unit 21 player 22 DVD format decoder 23 information separator 24 differential binaural audio decoder 25 normal audio decoder 26 sound source selector 27 GUI
28 Speaker 28a Array speaker 28b Delay circuit 29 Buffer 30 Video decoder 31 Buffer 32 Image display 32a LCD
33 Buffer 34 STC Timestamp Comparator 35 Control Unit 101 Auditory Psychological Analyzer 102 MDCT Unit 103 Quantizer 104 Group Processor 105 Variable Length Encoder 106 Bit Number Determinator 107 Bit Stream Generator 108 Processing Control Unit

Claims (6)

Means for playing normal audio object,
Means for reproducing management information for managing the audio object and controlling reproduction of the audio object using the management information;
Means for reproducing information relating to the three-dimensional sound field information recorded in the user data area of the audio object for each predetermined unit of each frame of the normal audio and controlling the reproduction of the three-dimensional sound field data using the information. And a three-dimensional sound field information reproducing apparatus. Means for playing normal audio object,
Means for reproducing management information for managing the audio object and controlling reproduction of the audio object using the management information;
For each predetermined unit of each frame of the normal audio, information on the three-dimensional sound field information is separated and reproduced from the stream that is multiplexed and recorded as stream data different from the normal audio of the audio object, and the information is used. And a means for controlling the reproduction of the three-dimensional sound field data. Means for playing normal audio object,
Means for reproducing management information for managing the audio object and controlling reproduction of the audio object using the management information;
For each predetermined unit of each frame of the normal audio, information related to the 3D sound field information recorded in the 3D sound field management information area different from the area where the management information is recorded is reproduced. And a means for controlling the reproduction of the three-dimensional sound field data using the information. Playing normal audio object,
Replaying management information for managing the audio object and controlling the reproduction of the audio object using the information;
For each predetermined unit of each frame of the normal audio, reproducing information related to the three-dimensional sound field information recorded in the user data area of the audio object and controlling the reproduction of the three-dimensional sound field data using the information. 3D sound field information reproduction program characterized by causing a computer to execute. Playing normal audio object,
Replaying management information for managing the audio object and controlling the reproduction of the audio object using the information;
For each predetermined unit of each frame of the normal audio, information relating to the three-dimensional sound field information is separated and reproduced from the stream that is multiplexed and recorded as stream data different from the normal audio of the audio object, and the information is used. And a step of controlling a reproduction of the three-dimensional sound field data by a computer. Playing normal audio object,
Replaying management information for managing the audio object and controlling the reproduction of the audio object using the information;
For each predetermined unit of each frame of normal audio, information related to 3D sound field information recorded in a management information area for 3D sound field different from the management information area is reproduced, and the information is used to 3D 3. A three-dimensional sound field information reproduction program that causes a computer to execute a step of controlling reproduction of sound field data.

Images