JP7235590B2 - Audio distribution system, distribution server, playback device, and program - Google Patents

Description

The present invention relates to an audio distribution system, a distribution server, a playback device, and a program.

In the channel-based audio system, which is widely used at present, the complete program sound is transmitted on the distribution side, and the viewer has little freedom in selecting the sound. An example of a viewer's ability to select and listen to their favorite sounds is the utilization of primary and secondary sounds. For example, if the primary audio is in your native language and the secondary audio is in a foreign language, you can view the video in the foreign language by selecting the secondary audio. Also, there is a case where the sound of the sports commentary is provided as the main sound, and the background sound of the stadium without the sports commentary is provided as the sub-audio. As with the use of the main and sub-audio described above, a variety of audio services can be provided by generating multiple types of audio on the distribution side and selecting the audio to be played on the playback side. In the channel-based audio system, as a distribution technology that enables selective viewing of audio, for example, a technique of switching audio using MPEG-DASH (Dynamic Adaptive Streaming over HTTP) described in Non-Patent Document 1 is known. there is

Also, in recent years, as a next-generation audio service, an object-based audio system that can customize program audio according to the preferences of viewers and the viewing environment is attracting attention. In the object-based audio method, audio material and audio metadata are transmitted, and an audio signal to be played is generated using a rendering function on the playback side. As a result, the viewer can customize the sound according to his or her own preferences, such as adjusting the volume balance between the background sound and the commentary sound, switching to the commentary sound in a foreign language, and the like.

The following two points can be cited as examples of reproduction technology of the object-based audio system. The first is a hardware implementation of a rendering function that generates an audio signal to be reproduced from transmitted audio material and acoustic metadata. In this method, it is necessary to prepare a rendering function for object-based audio separately from a playback device for channel-based audio, or to prepare a playback device with a built-in rendering function for object-based audio. The second is a method of implementing a rendering function, which generates an audio signal to be reproduced from transmitted audio material and acoustic metadata, by software on a web browser. For example, as described in Non-Patent Document 2, a technique of rendering audio using Web Audio API, which is a web standard HTML5 audio signal control function, is known.

The reproduction technology of the object-based audio system enables the viewer to customize the program sound on the reproduction side according to his/her preference and viewing environment. Examples of voice services include: It is possible to provide a multi-audio service that can switch from Japanese audio commentary to English or other foreign language audio commentary, or switch to home-side commentary or visitor-side commentary for sports programs. In addition, it is possible to increase the volume of only the commentary voice, making it possible to adjust the voice so that it is easy to hear even for the elderly and non-native speakers. Furthermore, services such as adding sound effects and virtually switching the listening position are also conceivable. These audio services can support multi-channel audio playback environments such as 5.1-channel surround and 7.1-channel surround, in addition to widely used 2-channel stereo such as stereo speakers and headphones.

General Incorporated Foundation NHK Engineering Systems, “Hybridcast related technology”, [Searched May 8, 2019], Internet <URL: http://www.nes.or.jp/transfer/catalog/2018/01/71a/ > Chris Pike, Peter Taylour, Frank Melchior, “Proceedings of the 1st Web Audio Conference”, 2015

In order to enjoy the audio service of the object-based audio system, a separately prepared rendering device dedicated to object-based audio, a playback device with a built-in rendering function, or a viewing terminal that can implement the rendering function on the browser is required. Therefore, it is not necessarily easy to construct a reproduction environment for the object-based audio system.

The method of implementing a rendering function on a browser, described in Non-Patent Document 2, makes it possible to reproduce an object-based audio system without having a dedicated rendering function. However, since it is assumed that the viewing terminal is compatible with Web Audio API, TV browsers and some browsers of PCs and mobile terminals do not support Web Audio API, and the above technology cannot be used. can't

Also, in order to perform rendering on the playback side, it is necessary to distribute all the audio materials used for rendering. increases significantly. An increase in the transmission volume leads to an increase in distribution load and network congestion, so it is undesirable to increase the transmission volume unnecessarily.

An object of the present invention, which has been made in view of such circumstances, is to realize an object-based audio system voice service even if the playback side does not have a rendering function, and to reduce the amount of transmission compared to the conventional object-based audio system. To provide an audio distribution system, a distribution server, a reproduction device, and a program capable of

In order to solve the above problems, an audio distribution system according to the present invention is an audio distribution system that distributes an audio stream to a first playback device that does not have a rendering function and a second playback device that has a rendering function. N an audio mixer for generating ' rendered first audio material; and encoding each of said N' rendered first audio materials to generate N' first audio streams. an encoding device for encoding each of M second audio materials to generate M second audio streams; a distribution server that distributes one of the first audio streams among the M audio streams, and distributes the requested number of the second audio streams among the M audio streams in response to a request from only the second playback device; It is characterized by having

Further, in the audio distribution system according to the present invention, the audio mixing unit groups the first audio materials by category, selects and combines one volume-controlled first audio material for each category, and performs the rendering. The method is characterized by generating a finished first audio material.

Further, in the audio distribution system according to the present invention, the audio mixing unit excludes combinations that can reconstruct an equivalent volume balance by increasing or decreasing the volume of the channel-based audio system on the receiving side, and removes the rendered first It is characterized by generating voice material.

In order to solve the above problems, a distribution server according to the present invention is a distribution server that distributes an audio stream to a first playback device that does not have a rendering function and a second playback device that has a rendering function. and storing N′ first audio streams and M second audio streams respectively encoding N′ rendered first audio materials and M second audio materials in different combinations; One of the N′ first audio streams is delivered in response to requests from the first playback device and the second playback device, and M audio streams are delivered in response to requests from the second playback device only. It is characterized by distributing the second audio stream as many as requested.

Further, in order to solve the above problems, a playback device according to the present invention is a playback device that receives a first audio stream, which is an audio stream of a first audio material, from a distribution server, and is a playback device that receives a channel selected by a viewer. A parameter acquisition unit for acquiring a first volume balance parameter representing a volume balance capable of reconstructing a first audio stream as a base acoustic scheme, and a first audio material on the distribution side corresponding to the first volume balance parameter a request stream determination unit that determines a first volume balance reference parameter that represents a volume balance when mixing audio streams, and determines one first audio stream that follows the first volume balance reference parameter; a stream receiving unit configured to receive the determined first audio stream from the distribution server; and a volume controller for controlling the volume balance of the audio stream.

Further, in the reproducing apparatus according to the present invention, the parameter acquisition unit further acquires a material combination parameter selected by the viewer, and the requested stream determination unit complies with the material combination parameter and the first volume balance parameter. It is characterized by determining the number of first audio streams.

Further, in the reproducing apparatus according to the present invention, the parameter acquisition unit further acquires a parameter indicating volume balance of one or more second audio streams selected by the viewer as a second volume balance parameter, and determines the requested stream. The unit further determines one or more second audio streams according to a second volume balance parameter, and the stream receiving unit further transmits the one or more second audio streams determined by the requested stream determination unit from the distribution server. Further, the volume control unit controls the volume balance of the second audio stream received by the stream receiving unit based on the second volume balance parameter.

Further, in order to solve the above problems, a program according to the present invention causes a computer to function as the playback device.

According to the present invention, it is possible to easily realize an object-based audio system audio service that allows a viewer to customize audio according to his or her preferences and playback environment, even if the playback side does not have a rendering function. can. Also, if the playback side has a rendering function dedicated to object-based audio, the advantages of the conventional playback technology are not lost. Furthermore, compared to the conventional object-based acoustic system, it is possible to reduce the audio distribution load for realizing the audio service.

1 is a diagram showing a configuration example of an audio distribution system according to one embodiment of the present invention; FIG. 1 is a diagram showing a configuration example of a rendering device in an audio distribution system according to an embodiment of the present invention; FIG. FIG. 4 is a diagram showing a first method for reducing the number of sounds generated by a rendering device in the sound distribution system according to one embodiment of the present invention; FIG. 5 is a diagram illustrating a second method for reducing the number of sounds generated by a rendering device in the audio delivery system according to one embodiment of the present invention; FIG. 5 is a diagram illustrating a second method for reducing the number of sounds generated by a rendering device in the audio delivery system according to one embodiment of the present invention; 4 is a diagram showing input and output signals of an encoding device in the audio distribution system according to one embodiment of the present invention; FIG. 1 is a diagram showing a configuration example of a first playback device according to an embodiment of the present invention; FIG. 4 is a flow chart showing an example of volume control operation of the first playback device according to one embodiment of the present invention. FIG. 4 is a diagram showing a configuration example of a second playback device according to one embodiment of the present invention; FIG. 8 is a diagram showing an example of a selection screen for volume balance of a second audio material according to one embodiment of the present invention; FIG. 4 is a block diagram showing a configuration example of a rendering processing unit in the second playback device according to one embodiment of the present invention; 4 is a flow chart showing an example of an audio reproduction method of the first reproduction device and the second reproduction device according to one embodiment of the present invention;

An embodiment of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a diagram showing a configuration example of an audio distribution system according to one embodiment of the present invention. As shown in FIG. 1, an audio distribution system 1 includes a rendering device 10, an encoding device 20, and a distribution server 30, and distributes audio. Note that although audio distribution will be described in the following embodiments, video may also be distributed together. Therefore, the present invention can also be applied to audio contained in moving images.

The audio distribution system 1 distributes the audio stream of the first audio material via the network 60 to both the one or more first playback devices 40 that do not have the rendering function and the one or more second playback devices 50 that have the rendering function. deliver to The audio distribution system 1 also distributes the audio stream of the second audio material to one or more second playback devices 50 via the network 60 . For convenience, FIG. 1 shows one first playback device 40 and one second playback device 50 .

Here, the first audio material is an audio material that is frequently used in constructing the program audio, and the second audio material is an audio material that is not frequently used in constructing the program audio. As an example, in the case of a sports program, it is conceivable to assign Japanese commentary, English commentary, home side commentary, visitor side commentary, venue background sound, home side background sound, visitor side background sound, etc. to the first audio material. be done. In addition, the second audio material includes the voices of specific players and sound effects, foreign languages not included in the first audio material, and multi-channel audio such as 5.1-channel surround other than 2-channel stereo and 7.1-channel surround. It is conceivable to allocate

The rendering device 10 inputs N first audio materials from the outside of the audio distribution system 1 and generates N' audios (rendered first audio materials) with different volume balances and combinations.

FIG. 2 is a diagram showing a configuration example of the rendering device 10. As shown in FIG. As shown in FIG. 2 , the rendering device 10 includes N volume control units 11 and an audio mixing unit 12 .

Each volume control unit 11 increases or decreases the volume of the first audio material to generate G volume-controlled first audio materials and outputs them to the audio mixing unit 12 .

The audio mixing unit 12 mixes the volume-controlled first audio materials input from the N volume control units 11 in various different combinations to produce N′ rendered first audio materials with different volume balances and combinations. is generated and output to the encoding device 20 . Assuming that there are G gains to be adjusted by the volume control unit 11, the maximum number of voices N' of the rendered first audio material generated by the rendering device 10 is mathematically ^GN . However, if all G ^N sounds are generated, the processing load on the rendering device 10 and the encoding device 20 will increase, and the amount of data to be stored in the distribution server 30 will become enormous. Therefore, a method for reducing the number of voices N' will be described below.

FIG. 3 is a diagram showing a first method for reducing the number of sounds N' generated by the rendering device 10. As shown in FIG. In this method, the audio mixing unit 12 groups the volume-controlled first audio materials by category, selects one volume-controlled first audio material for each category, and combines them into N′ rendered first audio materials. 1 Generate audio material. As a result, unnecessary combinations can be excluded when constructing the program sound, and the number of sounds to be generated can be reduced. The grouping may be performed based on an instruction from the user of the rendering device 10, or may be automated based on a feature extracted from the audio signal of the volume-controlled first audio material.

In the example shown in FIG. 3, there are seven first audio materials of a sports program: Japanese commentary, English commentary, home-side commentary, visitor-side commentary, venue background, home-side background, and visitor-side background. The first audio material is grouped into a commentary audio category and a background sound category. There are four audio commentary categories: Japanese audio commentary, English audio commentary, home side commentary, and visitor side commentary. There are three background sound categories: venue background sound, home side background sound, and visitor side background sound. is. In the viewing operation, desired first audio materials are selected one by one from each category, so unnecessary combinations of commentary voices, background sounds, and the like can be reduced. Therefore, in the case of this sports program, 4×3=12 sounds with different combinations should be generated for one set of volume balance. Although an example is shown, the number of categories may be arbitrary, and generalized descriptions such as category A and category B may be used. A parameter indicating which audio materials are to be mixed in this way is called a material combination parameter.

FIG. 4 is a diagram showing a second method for reducing the number of sounds N' generated by the rendering device 10. As shown in FIG. In this method, the audio mixing unit 12 generates N' rendered first audio materials by excluding combinations that can reconstruct an equivalent volume balance by increasing or decreasing the volume of the channel-based audio system on the receiving side. do. Here, as in FIG. 3, consider an example in which the number of categories is two. Although the volume to be adjusted by the volume control unit 11 is arbitrary, as an example, it is assumed that volume control is performed in five ways (G=5) of −6 dB, −3 dB, 0 dB, +3 dB, and +6 dB for the original voice material. There are a total of 25 volume balances for a set of first audio materials selected from category A and category B, as shown in the figure.

Here, for example, if we focus on the combination of +6 dB in category A and -3 dB in category B, the combination of +3 dB in category A and -6 dB in category B, which both lower the volume by 3 dB, has the same volume as the channel-based acoustic system. is equivalent to lowering by 3 dB. Therefore, even in the first playback device 40 that does not have the rendering function of the object-based audio method, if a combination of +6 dB for category A and -3 dB for category B is distributed, category A is +3 dB and category B is -6 dB. can be reconstructed. Similarly, the volume balance combinations corresponding to the shaded areas in FIG. 4A can be reconstructed at the receiving end, and thus do not require audio generation by the rendering device 10 . Therefore, the audio mixing unit 12 performs (+6 dB, -6 dB), (+6 dB, -3 dB), (+6 dB, 0 dB), (+6 dB, +3 dB), (+6 dB, +6 dB), (+3 dB, +6 dB), (0 dB, +6 dB), (-3 dB, +6 dB), (-6 dB, +6 dB).

FIG. 4B is a diagram showing an example of a correspondence table between first volume balance reference parameters and first volume balance parameters. Here, the first sound volume balance reference parameter is a parameter representing the sound volume balance when mixing the first audio materials on the distribution side (rendering device 10). The first volume balance parameter is a volume balance that can be selected by a viewing operation and that can reconstruct the first audio stream as a channel-based audio system on the playback side (the first playback device 40 and the second playback device 50). is a parameter that represents

The correspondence table shown in FIG. 4B means that volume control according to the first volume balance parameter in the right column is performed on the basis of the sound volume-controlled according to the first volume balance reference parameter in the left column. If the audio material rendered according to the first volume balance reference parameter is distributed, the audio of the first volume balance parameter corresponding to the first volume balance reference parameter can be obtained by increasing or decreasing the volume in the channel-based acoustic method on the receiving side. can be converted. From the above, in the example shown in FIGS. 3 and 4, the audio mixing unit 12 should generate a total of 12×9=108 rendered first audio materials with different combinations and volume balances.

Furthermore, the audio mixer 12 may generate N′ rendered first audio materials by excluding combinations that are statistically less frequently selected by viewers. This makes it possible to further reduce the number of voices N' to be generated. For example, the access rates of combinations of audio materials used in programs with similar configurations that were broadcast in the past are obtained, and combinations with access rates below a threshold (for example, 1%) are not generated. Since the threshold varies depending on a plurality of factors such as the number of combinations and viewer demographics, it may be changed as appropriate. Alternatively, the access rate may be input by the user of the rendering device 10 .

FIG. 5 is a diagram showing input and output signals of the encoding device 20. As shown in FIG. As shown in FIG. 5, the encoding device 20 encodes each of the N′ rendered first audio materials input from the rendering device 10, converts them into a stream format, and converts N′ first audio streams. Generate and transmit to distribution server 30 . Further, the encoding device 20 encodes each of M second audio materials input from outside the audio distribution system 1 and converts them into a stream format, generates M second audio streams, and distributes them to the distribution server 30 . Send to A streaming method such as MPEG-DASH or HTTP Live Streaming may be used as the distribution protocol.

Refer to FIG. 1 again. The distribution server 30 stores the first audio stream and the second audio stream transmitted from the encoding device 20 in the storage unit 31 and distributes the streams requested by the first playback devices 40 and 50 . As for the first audio stream, the distribution server 30 receives requests from both the first reproduction device 40 and the second reproduction device 50, and distributes one of the N' first audio streams. As for the second audio streams, the distribution server 30 accepts requests only from the second playback devices 50, and distributes the requested number of the second audio streams among the M pieces.

<First playback device>
Next, the first playback device 40 that does not have a rendering function will be described. FIG. 6 is a diagram showing a configuration example of the first playback device 40. As shown in FIG. The first playback device 40 includes a communication interface 41, a parameter acquisition unit 42, a parameter storage unit 43, a request stream determination unit 44, a stream delivery request unit 45, a stream reception unit 46, and a volume control instruction unit 47. , a volume control unit 48 and a reproduction processing unit 49 .

The communication interface 41 is an Ethernet (registered trademark) interface, a wireless LAN interface, or the like, and connects to the network 60 by wire or wirelessly.

The parameter acquisition unit 42 causes the display unit (not shown) to display material combination parameters as shown in FIG. Then, it acquires the material combination parameter selected by the viewer and outputs it to the parameter holding unit 43 . Further, the parameter acquisition unit 42 causes the display unit to display the first volume balance parameters as shown in the right column of FIG. 4B for the combination of the first audio materials. Then, it acquires the first volume balance parameter selected by the viewer and outputs it to the parameter holding unit 43 . Note that the material combination parameter and the first sound volume balance parameter to be displayed on the display unit are received in advance from the distribution server 30 .

The parameter holding unit 43 holds the material combination parameter and the first volume balance parameter acquired by the parameter acquiring unit 42 until the parameter acquiring unit 42 acquires new parameters. The parameters held by the parameter holding unit 43 are presented in response to reference requests from the stream distribution requesting unit 45 and volume control instruction unit 47 .

The requested stream determination unit 44 determines the first volume balance reference parameter corresponding to the first volume balance parameter acquired by the parameter acquisition unit 42 . Then, the requested stream determination unit 44 determines one first audio stream according to the material combination parameter and the first volume balance reference parameter, and outputs it to the stream distribution request unit 45 .

The stream distribution requesting unit 45 requests the distribution server 30 for one first audio stream determined by the requested stream determining unit 44 .

The stream receiving unit 46 receives one first audio stream determined by the requested stream determining unit 44 from the distribution server 30 and buffers it.

Based on the difference between the first volume balance reference parameter and the first volume balance parameter, the volume control instruction unit 47 converts the volume balance of the first audio stream received by the stream receiving unit 46 into the volume balance based on the first volume balance parameter. determines a volume control instruction value for the first audio stream for reconstruction into a volume control unit 48;

The volume control unit 48 controls the volume balance of the first audio stream received by the stream reception unit 46 according to the instruction value determined by the volume control instruction unit 47 and outputs the result to the reproduction processing unit 49 .

The reproduction processing unit 49 reproduces the audio signal input from the volume control unit 48 . A speaker, headphones, or the like may be used to reproduce the audio signal. Note that the reproduction processing unit 49 may be separated from the first reproduction device 40 and the audio signal may be reproduced by another device via the communication interface 41 .

FIG. 7 is a flow chart showing an operation example of the volume control instructing section 47 and the volume control section 48. As shown in FIG. In step S<b>101 , the volume control instructing unit 47 acquires the first volume balance parameter selected by the viewing operation from the parameter holding unit 43 .

In step S102, the volume control instructing unit 47 selects the first volume balance parameter and the first volume balance parameter based on the correspondence table of the first volume balance reference parameter and the first volume balance parameter as illustrated in FIG. 4B. with a first volume balance reference parameter corresponding to . If both are equal, the process proceeds to step S103, and if they are not equal, the process proceeds to step S104.

In step S103, since the first volume balance reference parameter and the first volume balance parameter are equal, there is no need to reconstruct the volume balance. Therefore, the volume control instruction unit 47 sets the audio control value to 0 [dB]. In this case, the volume control unit 48 does not perform volume control.

In step S104, since the first volume balance reference parameter and the first volume balance parameter are different, it is necessary to reconstruct the volume balance. Therefore, the volume control instruction unit 47 calculates the difference obtained by subtracting the first volume balance reference parameter corresponding to the first volume balance parameter from the first volume balance parameter, and converts the difference x [dB] to the volume control instruction value. and For example, when the first volume balance parameter is (0 dB, -3 dB) and referring to the correspondence table shown in FIG. 4B, the first volume balance reference parameter corresponding to the first volume balance parameter is (+6 dB, +3 dB). Therefore, the volume control instruction value x=-6 [dB].

In step S105, the volume control unit 48 increases the volume by x [dB] when the volume control instruction value x is positive, and decreases the volume by |x| [dB] when x is a negative value. control.

<Second playback device>
Next, the second playback device 50 having a rendering function will be described. FIG. 8 is a diagram showing a configuration example of the second playback device 50. As shown in FIG. The second playback device 50 includes a communication interface 51, a parameter acquisition unit 52, a parameter storage unit 53, a request stream determination unit 54, a stream delivery request unit 55, a stream reception unit 56, and a volume control instruction unit 57. , a rendering processing unit 58 and a reproduction processing unit 59 .

The communication interface 51 connects to the network 60 by wire or wirelessly, like the communication interface 41 of the first playback device 40 .

Like the parameter acquisition unit 42 of the first playback device 40, the parameter acquisition unit 52 causes the display unit (not shown) to display material combination parameters as shown in FIG. Then, the material combination parameter selected by the viewer is acquired and output to the parameter holding unit 53 . Further, similarly to the parameter acquisition unit 42 of the first playback device 40, the parameter acquisition unit 52 displays the first volume balance parameters as shown in the right column of FIG. 4B for the combination of the first audio materials. to display. Then, it acquires the first volume balance parameter selected by the viewer and outputs it to the parameter holding unit 53 .

Furthermore, the parameter acquisition unit 52 causes the display unit to display a volume balance selection screen for the second audio material as shown in FIG. Then, a parameter representing the volume balance of the one or more second audio materials selected by the viewer is obtained as a second volume balance parameter. In the example shown in FIG. 9, the volume balance of the second audio material 1 is set to +2, and the volume balance of the second audio material 2 is set to -4.

The parameter holding unit 53 holds the material combination parameter, the first volume balance parameter, and the second volume balance parameter acquired by the parameter acquisition unit 52 until the parameter acquisition unit 52 acquires new parameters.

The requested stream determination unit 54 determines the first volume balance reference parameter corresponding to the first volume balance parameter acquired by the parameter acquisition unit 52, similarly to the requested stream determination unit 44 of the first playback device 40. FIG. Requested stream determining section 54 then determines one first audio stream according to the material combination parameter and the first volume balance reference parameter, and outputs it to stream delivery requesting section 55 .

Further, the requested stream determination unit 54 determines one or more (the number selected by the viewer) second audio streams according to the second volume balance parameter acquired by the parameter acquisition unit 52 , and outputs them to the stream delivery request unit 55 . do.

The stream distribution requesting unit 55 requests the distribution server 30 for one first audio stream and one or more second audio streams determined by the requested stream determining unit 54 .

The stream receiving unit 56 receives the one first audio stream and the one or more second audio streams determined by the requested stream determining unit 54 from the distribution server 30 and buffers them.

Similarly to the volume control instruction unit 47 of the first playback device 40, the volume control instruction unit 57 reproduces the volume balance of the first audio stream distributed with the first volume balance reference parameter to the volume balance of the first volume balance parameter. The audio control directive values for the first audio stream to be constructed are determined and output to the rendering processor 58 .

Also, the volume control instruction unit 57 determines a volume control instruction value for the second audio stream based on the second volume balance parameter, and outputs the volume control instruction value to the rendering processing unit 58 .

The rendering processing unit 58 performs rendering processing according to the volume control instruction value determined by the volume control instruction unit 57 and outputs the result to the reproduction processing unit 59 .

FIG. 10 is a block diagram showing a configuration example of the rendering processing unit 58. As shown in FIG. The rendering processing unit 58 includes (j+1) volume control units 581 and audio mixing units 582 . The volume control unit 581 issues volume control instructions determined by the volume control instruction unit 57 to each of the one first audio stream and the selected number (j) of second audio streams distributed from the distribution server 30. The volume balance is controlled according to the value and output to the audio mixer 582 . The audio mixing section 582 mixes the first audio stream and the second audio stream input from the volume control section 581 to generate a rendered audio signal, and outputs the rendered audio signal to the reproduction processing section 59 .

The reproduction processing unit 59 reproduces the rendered audio signal input from the rendering processing unit 58 . A speaker, headphones, or the like may be used to reproduce the rendered audio signal. Note that the reproduction processing unit 59 may be separated from the second reproduction device 50 and the audio signal may be reproduced by another device via the communication interface 51 .

<Audio playback method>
Next, the audio reproduction method of the first reproduction device 40 and the second reproduction device 50 will be described with reference to FIG. FIG. 11 is a flow chart showing an example of an audio reproduction method of the first reproduction device 40 and the second reproduction device 50. As shown in FIG.

In step S<b>201 , the first playback device 40 and the second playback device 50 receive the material combination parameter and the first volume balance parameter from the distribution server 30 via the communication interfaces 41 and 51 .

In step S202, the first playback device 40 and the second playback device 50 accept viewing operations until the audio distribution ends. If there is a viewing operation, the process moves to the process of detecting the viewing operation in step S203. While there is no viewing operation, the reproduction of the audio signal is continued.

In step S203, a viewing operation is detected. The viewer selects, by touching the screen or the like, the desired audio material and volume from among the material combination parameters and the first volume balance parameters presented on the display unit. The method of the viewing operation may be remote operation such as remote control button operation, gesture operation, laser pointer, or the like, in addition to screen touch.

In step S204, the parameter acquisition units 42 and 52 acquire the material combination parameter and the first volume balance parameter selected by the viewing operation. It should be noted that parameters may be added to the audio material by specifying them in the file name, or by describing them as metadata of the audio material using an array, list, or the like.

In step S205, subsequent processing is determined depending on whether or not the rendering function is provided. The first reproduction device 40 without the rendering function reproduces the audio signal after performing the processing of steps S206 to S209, and the second reproduction device 50 with the rendering function reproduces the audio signal after performing the processing of steps S210 to S213. Play an audio signal.

(without rendering function)
In step S206, the requested stream determining unit 44 determines a first volume balance reference parameter corresponding to the first volume balance parameter from the first volume balance parameter.

In step S207, the requested stream determination unit 44 requests the distribution server 30 for one first audio stream that conforms to the material combination parameter and the first volume balance reference parameter.

In step S<b>208 , the first audio stream distributed from the distribution server 30 is received by the stream reception unit 46 .

In step S209, the volume control instructing unit 47 determines an audio control instruction value from the first volume balance parameter and the first volume balance reference parameter, and the volume control unit 48 performs volume control according to the audio control instruction value. Specifically, the first playback device 40 compares the first volume balance parameter selected by the viewing operation and the first volume balance reference parameter corresponding to the first volume balance parameter, and if both are equal, does not perform volume control, and if both are different, volume control is performed based on the volume control instruction value.

(with rendering function)
In step S210, the requested stream determining unit 54 determines a first volume balance reference parameter corresponding to the first volume balance parameter from the first volume balance parameter. Also, the parameter acquisition unit 52 acquires the second volume balance parameter selected by the viewing operation.

In step S211, the requested stream determining unit 54 distributes one first audio stream according to the material combination parameter and the first volume balance reference parameter and a selected number of second audio streams according to the second volume balance parameter to the distribution server 30. request to.

In step S<b>212 , the stream receiving unit 56 receives the first audio stream and the second audio stream distributed from the distribution server 30 .

In step S213, the volume control instruction unit 57 determines an audio control instruction value from the first volume balance parameter, the first volume balance reference parameter, and the second volume balance parameter, and the rendering processing unit 58 renders according to the audio control instruction value. process. Specifically, for the first audio stream, the second playback device 50 sets the first volume balance parameter selected by the viewing operation and the first volume balance reference parameter corresponding to the first volume balance parameter. are compared, and if both are equal, volume control is not performed, and if both are different, volume control is performed based on the volume control instruction value. Also, the second playback device 50 performs volume control on the second audio stream so as to match the second volume balance parameter. The second playback device 50 then mixes the volume-controlled first and second audio streams to generate a rendered audio signal.

In step S214, the audio signal is reproduced by the reproduction processors 49 and 59 and output.

In step S215, the first reproduction device 40 and the second reproduction device 50 determine whether or not the audio distribution has ended, and if the audio distribution has not ended, the process returns to step S202.

It is also possible to use a computer to function as all or part of the rendering device 10, the encoding device 20, the distribution server 30, the first playback device 40, and the second playback device 50 described above. Such a computer stores a program describing processing details for realizing each function of the rendering device 10, the encoding device 20, the distribution server 30, the first playback device 40, and the second playback device 50 in the storage unit of the computer. It can be realized by storing the program and reading and executing the program by a CPU (Central Processing Unit) or DSP (Digital Signal Processor) of the computer.

Also, this program may be recorded in a computer-readable recording medium. By using such a recording medium, it is possible to install the program in the computer. Here, the recording medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be, for example, a recording medium such as a CD-ROM or a DVD-ROM. This program can also be provided by download over a network.

As described above, the rendering device 10 increases or decreases the volume of each of the N first audio materials by the volume control unit 11 to generate the volume-controlled first audio material, and the audio mixing unit 12 generates the volume-controlled first audio material. 1 audio material is mixed in different combinations to generate N′ rendered first audio materials with different volume balances and combinations. Encoding device 20 encodes the N' rendered first audio material and M second phonemes to generate N' first audio streams and M second audio streams. The distribution server 30 receives requests for the first audio streams from the first reproduction device 40 and the second reproduction device 50, distributes one of the N' first audio streams, and receives the first audio stream from the second reproduction device 50 only. A request for two audio streams is accepted, and the requested number of second audio streams out of M is delivered.

With such a configuration, according to the present invention, an object-based audio system audio service that allows a viewer to customize audio according to his/her preferences and playback environment can be easily provided even in the first playback device 40 that does not have a rendering function. can be practically realized. The browser rendering method described in Non-Patent Document 2 enables object-based audio reproduction without a dedicated rendering device, but it is premised on a viewing terminal that supports Web Audio API. According to the present invention, as long as the viewing terminal supports HTML5 and is capable of streaming playback, a simple voice service of the object-based audio system can be enjoyed in a playback environment of the channel-based audio system. Also, if it is compatible with Web Audio API, it does not impair the advantage of being able to construct a reproduction environment for the object-based audio system as well.

Further, according to the present invention, a plurality of types of audio rendered with parameters set in advance on the distribution side are prepared, and the selected audio is distributed, thereby reducing the audio distribution load for realizing the audio service. be able to. However, by limiting the audio to be distributed to preset parameters, the transmission amount of audio materials is reduced compared to the conventional object-based audio method, which requires transmission of all audio materials used for rendering. It is unrealistic to prepare in advance every pattern selected by every viewer's viewing operation. Therefore, it is preferable that the volume control unit 11 is devised to reduce the number of voices to be rendered and generated. In addition, audio that is not expected to be frequently selected by viewing operations can be distributed using the conventional object-based audio method, thereby reducing the number of sounds generated on the distribution side and reducing the distribution load. .

Although the above embodiments have been described as representative examples, it will be apparent to those skilled in the art that many modifications and substitutions may be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as limited by the embodiments described above, and various modifications and changes are possible without departing from the scope of the appended claims. For example, it is possible to combine a plurality of configuration blocks described in the configuration diagrams of the embodiments into one, or divide one configuration block.

1 audio distribution system 10 rendering device 11 volume control unit 12 audio mixing unit 20 encoding device 30 distribution server 31 storage unit 40 first playback device 41, 51 communication interface 42, 52 parameter acquisition unit 43, 53 parameter storage unit 44, 54 Request stream determining unit 45, 55 Stream distribution requesting unit 46, 56 Stream receiving unit 47, 57 Volume control instructing unit 48 Volume control unit 49, 59 Playback processing unit 50 Second playback device 58 Rendering processing unit 60 Network 581 Volume control unit 582 audio mixer

Claims (8)

An audio distribution system that distributes an audio stream to a first playback device that does not have a rendering function and a second playback device that has a rendering function,
a volume control unit for increasing or decreasing the volume of each of the N first audio materials to generate volume-controlled first audio materials; and mixing the volume-controlled first audio materials in different combinations to achieve volume balance and combination an audio mixer for generating N′ different rendered first audio material;
encoding each of the N′ rendered first audio materials to generate N′ first audio streams and encoding each of the M second audio materials to generate M second audio streams; an encoding device that
One of the N′ first audio streams is delivered in response to requests from the first playback device and the second playback device, and M audio streams are delivered in response to requests from the second playback device only. a distribution server for distributing the requested number of second audio streams among
An audio distribution system, comprising: The audio mixing unit groups the first audio materials by category, selects and combines one volume-controlled first audio material for each category, and generates the rendered first audio material. 2. The audio distribution system according to claim 1, wherein: The audio mixing unit generates the rendered first audio material by excluding combinations that can reconstruct an equivalent volume balance by increasing or decreasing the volume of the channel-based acoustic method on the receiving side, 3. The audio distribution system according to claim 1 or 2. A distribution server that distributes an audio stream to a first playback device that does not have a rendering function and a second playback device that has a rendering function,
storing N′ first audio streams and M second audio streams respectively encoding N′ rendered first audio materials and M second audio materials with different volume balances and combinations; ,
One of the N′ first audio streams is delivered in response to requests from the first playback device and the second playback device, and M audio streams are delivered in response to requests from the second playback device only. A distribution server that distributes the requested number of the second audio streams out of the above. A playback device for receiving a first audio stream, which is an audio stream of a first audio material, from a distribution server,
a parameter obtaining unit for obtaining a first volume balance parameter representing a volume balance that can reconstruct the first audio stream as a channel-based audio scheme selected by the viewer;
determining a first volume balance reference parameter corresponding to the first volume balance parameter and representing a volume balance when the first audio materials are mixed on the distribution side; a request stream determination unit that determines one audio stream;
a stream receiving unit that receives the first audio stream determined by the requested stream determining unit from the distribution server;
a volume control unit that controls the volume balance of the first audio stream received by the stream receiving unit based on the difference between the first volume balance parameter and the first volume balance reference parameter;
A playback device comprising: The parameter acquisition unit further acquires material combination parameters selected by the viewer,
6. The reproducing apparatus according to claim 5, wherein the requested stream determining unit determines one first audio stream according to the material combination parameter and the first volume balance parameter. The parameter acquisition unit further acquires a parameter representing volume balance of one or more second audio streams selected by the viewer as a second volume balance parameter,
The requested stream determining unit further determines one or more second audio streams according to a second volume balance parameter;
The stream receiving unit further receives the one or more second audio streams determined by the requested stream determining unit from the distribution server,
7. The reproduction according to claim 5, wherein the volume control unit further controls volume balance of the second audio stream received by the stream receiving unit based on the second volume balance parameter. Device. A program for causing a computer to function as the playback device according to any one of claims 5 to 7.

Images