JP5333043B2 - Mixing apparatus and mixing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform stable mixing processing without uselessly enlarging a buffer upper-limit accumulation amount while the fluctuation of packets becomes obvious between a plurality of channels. <P>SOLUTION: A mixing device 130 includes a packet acquisition unit for acquiring the sound signals of a plurality of channels in units of packets which are transmitted through a communication network 120, a packet loss decision unit 218 for defining the packets as the missing packets when a delay of arrival time of the acquired packets in the channels exceeds a prescribed time, a packet interpolating part 220 for interpolating the packets defined as the missing packets, a mixing buffer 216 for temporarily holding the packets of the channels, and a mixing processing unit 222 for adding the packets of the channels and generating one packet. The prescribed time is made different for each channel in response to the priority degree of the channels. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a mixing device and a mixing method for adding sound signals of a plurality of channels and generating a new sound signal.

  In recent years, techniques for transmitting sound signals such as music and voices through a communication network (IP network) using IP (Internet Protocol) have been adopted for various services. In the case of a transmission technique through such a communication network, for example, a sound signal is once divided into packets of a predetermined data amount, and each divided packet is individually decoded to the original sound signal at the transmission destination via the communication network.

  In such a communication network, since the transmission path is not specified, the arrival time of each packet is not uniform, and jitter (sound signal fluctuation) is found in the arrival time particularly when passing through a network device such as a router. Can occur. When decoding a sound signal from a packet, if the packet has not arrived at the timing at which the sound signal should be output, it is considered that the packet has been dropped, causing the sound to be interrupted and for the user listening to the sound signal. It made me feel uncomfortable. Therefore, a technique is known in which a packet receiving device is provided with a buffer for temporarily holding a packet related to a single sound signal, so as to absorb temporal fluctuation of the packet passing through the communication network and prevent sound interruption. (For example, Patent Document 1).

  Also, it is determined whether or not the fluctuation of the arrived packet related to a single sound signal can be absorbed by the buffer, and when it is determined that it cannot be absorbed, the importance of each arrived packet is determined and the importance is low. A technique for discarding a voice packet and reproducing a voice packet with high importance is disclosed (Patent Document 2).

JP 2007-259471 A JP 2007-258928 A

  On the other hand, regarding the way the sound signal is used, not only the above-mentioned single sound signal (one-channel sound signal) but also the sound signals of a plurality of channels with different sound sources and sound qualities are added, and a new sound signal is obtained. There are also so-called mixing applications that produce. At this time, if the sound signals of a plurality of channels can be acquired in a synchronized state, a desired sound signal can be obtained by mixing the acquired sound signals as they are. Conventionally, dedicated cables have been used for transmission of sound signals for such a plurality of channels. However, when dedicated cables were used, the number of dedicated cables had to be increased as the number of channels and transmission / reception devices increased.

  Therefore, it is considered to distribute a sound signal using the communication network described above. In the communication network, it is not necessary to physically change the wiring even when the number of channels is increased or the transmitter / receiver is reconfigured, and it is possible to flexibly cope with the addition of video signals and control information other than sound signals. . However, when acquiring packets based on sound signals of multiple channels through a communication network, in order to mix the packets of multiple channels that do not arrive at a uniform time with the same time axis, you must wait for unreachable packets. Therefore, in order to suppress the delay of the final sound signal, it was not possible to spend time on the mixing process.

  In addition, if a next packet arrives in another channel before a packet of an arbitrary channel arrives, a situation occurs in which the previous packet of the other channel is lost. Although the above-described prior art can absorb the fluctuation of the front and rear packets regarding the sound signal of one channel, it does not mention any means for absorbing the fluctuation of the arrival time of the packets between the plurality of channels.

  In view of such a problem, the present invention can execute stable mixing processing without unnecessarily increasing the upper limit accumulation amount of a buffer in a state where packet fluctuations are manifested between a plurality of channels. An object of the present invention is to provide a mixing device and a mixing method.

  In order to solve the above problem, a mixing device that adds sound signals of a plurality of channels transmitted through a communication network and newly generates a sound signal is provided for each sound signal of a plurality of channels transmitted through the communication network in units of packets. The packet acquisition unit acquired in step 1 compares the arrival time delay of a packet in the acquired plurality of channels with respect to the packet of the other channel acquired first and the predetermined time, and the packet arrival time delay is predetermined. A packet loss determination unit that considers that the packet is missing when the time is exceeded, a packet complementation unit that complements the packet deemed to be missing, and packets of multiple channels temporarily to absorb delays in the arrival time of the packet The mixing buffer that is held in the It includes a mixing processor for generating one packet by adding the door, and the predetermined time, varying per channel according to the priority of the plurality of channels.

  By providing a mixing buffer for each channel, fluctuations in packet arrival time between channels can be absorbed, and packet loss can be avoided. On the other hand, by providing the mixing buffer, a delay corresponding to the upper limit accumulation amount of the mixing buffer may occur until the mixing process is executed. Even in such a case, in order to suppress the delay of the final sound signal after mixing processing, it is necessary to execute the mixing processing time in a short time, but this may cause audible noise such as sound interruption. Increase sex. However, if the packet loss determination is uniformly advanced for all channels in order to provide a margin for the mixing process, it is determined that the packet has been dropped early to the channel with a higher priority, which causes a sound cut. Here, according to the priority, the time for which the packet is considered to be dropped is different, that is, the channel with high priority is not dropped as much as possible even if the mixing processing time is shortened, and the channel with low priority is advanced. It is possible to perform stable mixing processing by performing packet loss determination and executing mixing processing with a margin.

  The predetermined time described above may be set shorter for a low priority channel than for a high priority channel. Here, for a low-priority channel, a predetermined time for determining a delay is set shorter than that for a high-priority channel, and packet loss determination is advanced, thereby reducing the packet delay of a low-priority channel. Regardless, it is possible to execute the mixing process of the high priority channel at an early stage. By such stable mixing processing, it is possible to reliably and stably output a sound signal of at least a high priority channel.

  In addition, the packet complementing unit complements a packet that has been considered missing by any one of the replacement process of a silent packet or a dummy packet, the process of repeating a past packet, or the process of generating and replacing an interpolated packet. May be. With this configuration, it is possible to supplement other packets that can be played back to the packets that are regarded as missing, and it is possible to continue normal playback processing.

  In order to solve the above-mentioned problem, in the mixing method of adding sound signals of a plurality of channels transmitted through a communication network and newly generating a sound signal, arrival times of packets in the plurality of channels with respect to packets of other channels The predetermined time indicating the allowable delay time is set differently for each channel according to the priority of the plurality of channels, and the sound signals of the plurality of channels transmitted through the communication network are acquired in units of packets. The delay in the arrival time of the packet in the other channel with respect to the packet of the other channel acquired first is compared with the predetermined time set for each of the plurality of channels, and the delay in the arrival time of the packet exceeds the predetermined time And the packet is considered missing, complements the packet deemed missing, and the packet arrival time Held temporarily in the mixing buffer packets of a plurality of channels for absorbing a delay adds a packet of a plurality of channels aligned time axis by being held in the mixing buffer for generating one packet.

  The above-described components based on the technical idea of the mixing apparatus and the description thereof can also be applied to the mixing method.

  As described above, according to the present invention, it is possible to perform stable mixing processing without unnecessarily increasing the upper limit accumulation amount of a buffer in a state where packet fluctuations are manifested between a plurality of channels. It becomes.

It is explanatory drawing which showed the rough connection relation of the sound signal output system. It is the functional block diagram which showed the schematic structure of the mixing apparatus. It is explanatory drawing for demonstrating operation | movement of a mixing buffer. It is explanatory drawing for demonstrating operation | movement of a packet loss determination part. It is explanatory drawing for demonstrating operation | movement of the packet loss determination part at the time of changing predetermined time for every channel. It is the flowchart which showed the flow of the process of the mixing method.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(Sound signal output system 100)
FIG. 1 is an explanatory diagram showing a schematic connection relationship of the sound signal output system 100 according to the present embodiment. The sound signal output system 100 includes a distribution server 110, a communication network 120, a mixing device 130, and a sound output device 140.

  The distribution server 110 uses a so-called VoIP (Voice over Internet Protocol) that compresses sound signals such as music and voices using various existing encoding methods, packetizes them, and transmits them to the communication network 120, so that the user of the mixing device 130 can transmit them. A desired sound signal is output to the communication network 120. In this embodiment, in order to facilitate understanding, the reference source of the sound signal is the distribution server 110, but is not limited to the server, and various electronic devices that can be connected to the communication network 120 can be employed. In addition, VoIP is used as a transmission protocol through the communication network 120. However, the present invention is not limited to this, and various protocols capable of transmitting sound signals (voice data) in units of packets can be employed. In this embodiment, such a distribution server 110 outputs sound signals of a plurality of channels independently or in cooperation with a plurality of channels.

  The communication network 120 performs communication connection between arbitrary devices, and when using IP, for example, plays a role of transmitting information in units of packets composed of an IP header and a payload. However, since the transmission route of the packet is not specified in the communication network 120, the arrival time of each packet output from the distribution server 110 in the mixing device 130 is not uniform, especially when it passes through a network device such as a router. Jitter (sound signal fluctuation) may occur in the arrival time. Further, since even the arrival of the packet is not guaranteed in the communication network 120, it is necessary to consider the loss of the packet.

  The mixing device 130 acquires sound signals of a plurality of channels in units of packets through the communication network 120, adds the sound signals of the plurality of channels (mixing process), and generates a sound signal for each of one or a plurality of output lines. To the sound output device 140. The mixing device 130 can also be configured as a single device that performs mixing processing, but is not limited to a BD (Blu-ray Disc) recorder / player, DVD recorder / player, CD player, HDD (Hard disk drive) recorder / Players, personal computers, notebook personal computers, PDAs (Personal Digital Assistants), mobile phones, PHS (Personal Handy phone System) terminals, digital cameras, digital video cameras, televisions, game machines, etc. can be connected to the communication network 120 It can also be composed of various electric devices.

  The sound output device 140 is composed of a dynamic speaker, a condenser speaker, a bone conduction speaker, headphones, and the like, and converts an electrical sound signal output from the mixing device 130 into physical vibrations and outputs sound (sound).

  In the sound signal output system 100, sound signals of a plurality of channels transmitted from one or more distribution servers 110 are mixed by the mixing device 130, and sound signals for one or a plurality of output lines are output from the sound output device 140. ing. Here, in mixing apparatus 130, a buffer is provided for each of a plurality of channels to absorb fluctuations in packet arrival times between channels based on passing through communication network 120. However, if the total amount of the buffer is divided and allocated to each channel uniformly, the data amount of the buffer of each channel is limited, and a packet that does not arrive after a predetermined time must be regarded as missing. When the timing for determining that a packet is missing is set uniformly for each channel regardless of the priority of the channel, the mixing process is delayed due to the delay of the channel with the lower priority, and the mixing process is performed together. There is a risk that the sound of a channel with a high priority should be cut off. Therefore, in this embodiment, not only a buffer is provided for each channel, but packet processing according to the priority of the channel is performed.

  In this way, the mixing apparatus 130 according to the present embodiment can perform stable mixing processing without unnecessarily increasing the upper limit accumulation amount of the buffer in a state where packet fluctuations are manifested between a plurality of channels. It becomes. Hereinafter, a specific configuration of the mixing apparatus 130 for achieving such an object will be described, and the processing flow will be described in detail later.

(Mixing device 130)
FIG. 2 is a functional block diagram illustrating a schematic configuration of the mixing apparatus 130. The mixing apparatus 130 includes a communication unit 210, a control unit 212, an operation unit 214, a mixing buffer 216, a packet loss determination unit 218, a packet complementing unit 220, a mixing processing unit 222, and an output buffer 224. Consists of.

  The communication unit 210 is connected to the communication network 120 by wire, or wireless such as Bluetooth (registered trademark), IEEE802.11a / b / g / n, and establishes communication with the distribution server 110 by VoIP. The communication unit 210 functions as a packet acquisition unit, and acquires sound signals of a plurality of channels transmitted through the communication network 120 in units of packets.

  The control unit 212 controls the mixing device 130 as a whole by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing programs, a RAM as a work area, and the like. In particular, in the present embodiment, the control unit 212 sets a predetermined time for each mixing buffer described later according to the priority of each channel acquired by the communication unit 210. A detailed description of this configuration will be given later.

  The operation unit 214 includes a switch such as a keyboard, a cross key, a joystick, and a display, and a touch panel installed on a display surface of the display, and receives a user operation input to the mixing device 130.

  The mixing buffer 216 temporarily holds packets of a plurality of channels acquired by the mixing device 130 at a stage before mixing processing.

  FIG. 3 is an explanatory diagram for explaining the operation of the mixing buffer 216. In FIG. 3, the horizontal axis is the time axis, and the arrival on the left side is earlier. In addition, here, only two channels A and B will be described for ease of understanding, but it goes without saying that three or more channels can be handled.

For example, as shown in FIG. 3A, sound signal packets A ₁ , A ₂ , A ₃ ,..., A _n , B ₁ , in a plurality of channels A and B without the mixing buffer 216 being provided. Assume that B ₂ , B ₃ ,..., B _n (a packet with a fast arrival order is given a young number) is acquired. In order to mix the sound signals of a plurality of channels A and B, it is necessary to match the time axis of the packet of each channel before executing the mixing process. Here, the mixing process is performed when the packet A ₁ of the channel A and the packet B _{1 of the} channel B are aligned (1).

However, as shown in FIG. 3A, the packet A _{2 of the} channel A has arrived (2), but the packet B _{2 of the} channel B has not yet arrived, and the packet B ₂ is ahead of the packet B _2. If packet A _{3 of} channel A is acquired, either packet A ₂ or packet A ₃ must be discarded (4).

In this embodiment, since the mixing buffer 216 is provided, as shown in FIG. 3B, both the packet A ₂ and the packet A ₃ can be held in the mixing buffer 216 even in the above state (5 ) (6) When packet B ₂ arrives, it is possible to mix the packet A ₂ and packet B ₂ in the mixing buffer 216 without dropping the packet (7).

  Such a configuration of the mixing buffer 216 can absorb not only the fluctuation of the arrival time of each of the plurality of channels but also the delay of the arrival time of the sound signal packet between the plurality of channels based on the fact that it has passed through the communication network 120. This makes it possible to avoid missing packets and prevent sound interruptions.

  The larger the capacity of the mixing buffer 216 is, the better. However, the resource is limited, and when the time axis delay due to the provision of the mixing buffer 216 increases, a sound signal after mixing processing is output accordingly. Therefore, the upper limit accumulation amount of the mixing buffer 216 is set in consideration of the delay time required by the sound signal output system 100.

  Therefore, even if the above-described mixing buffer 216 is provided, it is not possible to completely absorb packet fluctuations caused by passing through the communication network 120. The amount of delay allowed before executing the mixing process corresponds to the upper limit accumulation amount prepared for each channel, so if the arrival of a packet is delayed beyond the upper limit accumulation amount prepared for each channel, that packet is The mixing process must be performed without waiting. Therefore, a means for determining whether or not the arrival of the packet exceeds the upper limit accumulation amount is required.

  The packet loss determination unit 218 compares, for each channel, the delay in arrival time of the acquired packets in the plurality of channels with respect to the packet acquired in the other channel first and the predetermined time set by the control unit 212. If the delay of the arrival time of the packet exceeds a predetermined time, the packet is regarded as missing. The delay in arrival time between packets may be calculated from the difference in time stamps indicated in the packets, or may be calculated by counting the time from the arrival of the packet of the channel acquired first. .

FIG. 4 is an explanatory diagram for explaining the operation of the packet loss determination unit 218. In FIG. 4, the horizontal axis is the time axis. For example, sound signal packets A ₁ , A ₂ ,..., A _n , B ₁ , B ₂ , A, B from a plurality of channels A, B are distributed in time as shown in FIG. ..., _Bn is acquired. Here, the time difference between the packets to be output at the same timing, that is, the time difference between the packets A ₁ and B ₁ and the packets A ₂ and B ₂ (indicated by white double arrows in FIG. 4A) is the arrival time. It will be late.

Here, the upper limit accumulation amount of mixing the buffer 216 allocated to each channel, the packet A _1, A 2, _..., if an amount can only accumulate to A _n, as shown in FIG. 4 (b), The channel B packet is significantly delayed with respect to the channel A packet. For example, the channel A packets A ₁ , A ₂ ,..., _An reach, but the channel B packet B ₁ has not yet arrived. And Then, when a new packet A _{n + 1} of channel A will reach while waiting for the arrival of packets B _1, data area overflows about the channel A of mixing the buffer 216. Thus, the packet loss determining unit 218 regards the packet delay in the arrival time of the packet A ₁ is missing a packet B ₁ exceeds a predetermined time, advances the mixing processing of packets A _1, B _1. At this time, the packet B ₁ does not exist, but some data in place of the packet B ₁ is required to perform the mixing process.

  The packet complementing unit 220 supplements a packet regarded as missing with some data that can be mixed and reproduced. Therefore, the true packet that will arrive after that is discarded without being used in the mixing process. Here, as a complementary process, a replacement process with a silent packet or a predetermined dummy packet, a process of repeating one or a plurality of past packets, or a process of generating an interpolation packet from previous and subsequent packets and replacing it with the interpolation packet Various processes can be applied. With this configuration, it is possible to continue normal reproduction processing.

Accordingly, as shown in FIG. 4 (b), any packet, wherein the delay is significantly reaches the packet B ₁ is other channel A, when exceeds a predetermined time, for example, the dummy packet of the packet B ₁ Is generated, and the generated dummy packet is held in the mixing buffer 216 as the packet B ₁ . In this way, when the packets A ₁ and B ₁ are ready, it is possible to perform the mixing process for the two packets A ₁ and B ₁ .

  Meanwhile, in the state shown in FIG. 4B, a delay corresponding to the upper limit accumulation amount of the mixing buffer 216 occurs until the mixing process is executed. Therefore, the final sound signal after the mixing process is output with such a delay as tolerable. Here, in order to suppress the delay of the final sound signal after the mixing process as much as possible, it is necessary to execute the mixing process time in a short time. However, it increases the possibility of audible noise such as sound interruption.

  For example, in the mixing apparatus 130 of the present embodiment, a plurality of processes such as packet transmission / reception, packet loss determination, packet complementation, and mixing in each of a plurality of channels are executed by the control unit 212 having limited processing capability. . Therefore, if a waiting time is caused by an arbitrary process, it affects other processes, and the mixing process must be executed in a short time, but the mixing process is not in time. Then, there is a possibility that the mixing processing unit 222 described later cannot transmit the mixed packet to the output buffer 224 at an appropriate timing. This causes a sound interruption.

  In order to provide a margin for such mixing processing, if the timing for determining packet loss is uniformly advanced for all channels, it will be determined that the packet has been dropped early to the channel with the higher priority, As a result, the sound signal is interrupted and the user who is listening to the sound signal feels uncomfortable.

  Here, according to the priority, the time for which the packet of each channel is regarded as missing is made different, that is, the channel with high priority is not dropped as much as possible even if the mixing processing time is shortened, and the priority is low. The channel makes it possible to perform stable mixing processing by performing packet loss determination early and executing mixing processing with a margin.

  First, in accordance with the priority of each channel, the control unit 212 sets in advance a predetermined time that indicates an allowable time of arrival delay of a packet in a channel with respect to a packet in another channel.

  Specifically, in the mixing buffer 216, different predetermined times are set in advance for each data area corresponding to a plurality of channels, and the channel is mixed with reference to the priority indicated in the IP header or the like of the packet. 216. For example, a sound signal packet of a high priority channel such as an announcement is associated with a data area of an arbitrary predetermined time, and a sound signal packet of a low priority channel such as BGM (Back-Ground Music) is Corresponding to a data area of a predetermined time shorter than the predetermined time.

  As another example, first, a channel can be assigned to a plurality of data areas of the mixing buffer 216, and then a predetermined time corresponding to the priority can be set for each data area with reference to a table or the like.

  More specifically, the predetermined time is set shorter for the low priority channel than for the high priority channel. In this way, for a low-priority channel, the packet delay of a low-priority channel is set by shortening the predetermined time for determining the delay compared to a high-priority channel and accelerating packet loss determination. Regardless of this, it is possible to execute the mixing process of the high priority channel at an early stage. By such stable mixing processing, it is possible to reliably and stably output a sound signal of at least a high priority channel. Here, a configuration has been described in which the predetermined time for a channel with a low priority is shortened. Naturally, the object can also be achieved by extending the predetermined time for a channel with a high priority.

FIG. 5 is an explanatory diagram for explaining the operation of the packet loss determination unit 218 when the predetermined time is varied for each channel. When the priority of channel A is higher than that of channel B in a plurality of channels A and B, until packets B ₁ , B ₂ ,..., B _{n of} channel B become equal to the upper limit accumulation amount of mixing buffer 216 (the longest predetermined time) 5), the packet loss determination unit 218 determines that the arrival of the channel A packet is a long predetermined amount corresponding to the upper limit accumulation amount of the mixing buffer 216, as shown in FIG. It is determined that a packet is missing for the first time after the time has elapsed. Then, the packet complementing unit 220 supplements the packet determined to be missing with a reproducible packet.

On the other hand, in channel B, which has a lower priority than channel A, the predetermined time is set shorter than that of channel A, and the packet of channel A reaches the upper limit accumulation amount of mixing buffer 216 as shown in FIG. For example, even when only the packets A ₁ and A ₂ are held in the mixing buffer 216, the delay in the arrival time of the channel B packet with respect to the channel A packet exceeds the short predetermined time of the channel B. The packet loss determination unit 218 determines that the channel B packet is missing. Again, the packet complementing unit 220 supplements a packet determined to be missing with a reproducible packet.

Thus, for a channel with a low priority, for example, channel B, as shown in FIG. 5B, packet B _{1 that} can be reproduced in a shorter time without waiting for the time corresponding to the upper limit accumulation amount of mixing buffer 216. Is generated, and the packet A ₁ of the channel with a high priority can be promptly transferred to the mixing processing unit 222 together with the packet B ₁ . Therefore, since the packets A ₁ and B ₁ can be mixed at an early stage, there is a margin in the mixing process, and it is possible to reduce the frequency of sound interruption and improve the sound quality on hearing.

  Examples of channels with different priorities include, for example, a channel formed by voice such as an announcement and a channel such as BGM. When the configuration described with reference to FIG. 5 is used, announcements that are high-priority channels can wait for arrival of packets for a predetermined amount of time corresponding to the upper limit accumulation amount of the mixing buffer 216, thereby avoiding packet loss. can do. On the other hand, a BGM or the like having a low priority channel can execute a mixing process with a margin by performing packet loss determination early and passing it to the mixing processing unit 222 at an early stage.

  In this case, the BGM, which is a channel with a low priority, has a higher possibility that the packet will be complemented by the packet complementer 220, that is, the possibility that the true packet will be discarded, but compared with the announcement with a higher priority. In many cases, the playback volume itself is low and the necessity of continuous listening is low, so that the result of mixing the announcement and BGM is not affected.

  Other combinations of channels with different priorities, such as earthquake early warning, announcements and hourly reports with high urgency, BGM, voice and performance in karaoke, individual advertising music and BGM dedicated to retail stores, etc. can do. As will be described later, when there are a plurality of output lines from the output buffer 224, the result of mixing both the announcement and BGM channels is output on any output line, and only the BGM is mixed on the other output lines. The processed result may be output.

  When the time axis of the packets of each channel (including the packets complemented by the packet complementing unit 220) is aligned by the mixing buffer 216, the mixing processing unit 222 adds the packets of each channel to generate one packet. Whether or not the packets for each channel are ready may be confirmed by the time stamp of the packet, or may be confirmed by whether or not the packet exists at a predetermined position in the mixing buffer 216.

  The output buffer 224 is provided corresponding to the mixing processing unit 222, temporarily holds the packet processed by the mixing processing unit 222, and absorbs fluctuations between the output lines. The fluctuations between the output lines are due to a difference in the start timing of the distribution server 110, a difference in the issue timing of various commands in the mixing device 130 or a packet acquisition timing, a slight difference in the reference clock, a difference in the consumption time of the mixing process for each channel, and the like. Can occur.

  Then, when the output timing of the packet for each output line arrives, the output buffer 224 decodes the packet and outputs it to the sound output device 140. In this way, even when there are a plurality of output lines for the sound signal, it is possible to absorb fluctuation for each output line and maintain the real-time property of the sound signal.

  As described above, the mixing device 130 described above makes it possible to perform stable mixing processing without unnecessarily increasing the upper limit accumulation amount of the buffer in a state where packet fluctuations are manifested between a plurality of channels. .

(Mixing method)
Next, a mixing method for newly generating a sound signal by adding the sound signals of a plurality of channels transmitted through the communication network 120 using the above-described mixing device 130 will be described.

  FIG. 6 is a flowchart showing a processing flow of the mixing method. Before the processing shown in FIG. 6 is started, the control unit 212 sets a predetermined time indicating a delay time of arrival time of a packet in a plurality of channels with respect to a packet in another channel as a priority for the plurality of channels. It is set differently for each channel according to the degree.

  When the user desires to start the reproduction of the sound signal by the mixing device 130 (YES in S302), the distribution server 110 transmits the sound signal packets of a plurality of channels through the communication network 120, and the packet acquisition unit (communication unit 210). Acquires such a packet (S304).

  Subsequently, the packet loss determination unit 218 compares the arrival time delay of the packet in the plurality of channels with respect to the packet of the other channel acquired first and a predetermined time preset for each of the plurality of channels. When the delay of the arrival time of the packet exceeds the predetermined time (YES in S308), the packet is regarded as missing, and the packet complementing unit 220 complements the packet regarded as missing (S310). .

  The mixing buffer 216 temporarily holds the packet acquired by the packet acquisition unit or the packet complemented by the packet complementing unit 220 (S312), and the mixing processing unit 222 holds the time axis by being held in the mixing buffer 216. Are added (mixing process) to generate one packet (S314).

  The output buffer 224 temporarily holds the packet mixed by the mixing processor 222 (S316), and after decoding, outputs the packet in time synchronization with other output lines (S318). Such mixing processing is repeated until the user desires to stop the reproduction of the sound signal (YES in S320).

  Even with such a mixing method, it is possible to perform stable mixing processing without unnecessarily increasing the upper limit accumulation amount of the buffer in a state where packet fluctuations are manifested between a plurality of channels.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  Note that each step in the mixing method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or by a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for a mixing apparatus and a mixing method that add sound signals of a plurality of channels and newly generate a sound signal.

DESCRIPTION OF SYMBOLS 100 ... Sound signal output system 110 ... Distribution server 120 ... Communication network 130 ... Mixing apparatus 140 ... Sound output apparatus 210 ... Communication part (packet acquisition part)
212 ... Control unit 214 ... Operation unit 216 ... Mixing buffer 218 ... Packet loss determination unit 220 ... Packet complementation unit 222 ... Mixing processing unit 224 ... Output buffer

Claims (4)

A mixing device that adds sound signals of a plurality of channels transmitted through a communication network and generates a new sound signal,
A packet acquisition unit for acquiring sound signals of a plurality of channels transmitted through the communication network in units of packets;
Compare the arrival time delay of the acquired packets in the plurality of channels with respect to the packet of the other channel acquired first and a predetermined time, and if the arrival time delay of the packet exceeds the predetermined time, A packet loss determination unit that considers that a packet is missing,
A packet complementer that complements the packet deemed to be missing;
A mixing buffer that temporarily holds packets of the plurality of channels to absorb delays in arrival times of the packets;
A mixing processor that adds the packets of the plurality of channels whose time axes are aligned by the mixing buffer to generate one packet;
With
The mixing apparatus according to claim 1, wherein the predetermined time is varied for each of the channels according to the priority of the plurality of channels.   The mixing apparatus according to claim 1, wherein the predetermined time is set shorter for a low priority channel than for a high priority channel.   The packet complementing unit complements the packet deemed to be missing by any one of a replacement process with a silent packet or a dummy packet, a process of repeating a past packet, or a process of generating and replacing an interpolation packet. The mixing apparatus according to claim 1, wherein the mixing apparatus is characterized in that A mixing method for generating a new sound signal by adding sound signals of a plurality of channels transmitted through a communication network,
A predetermined time indicating an allowable delay time of arrival time of a packet in the plurality of channels with respect to a packet in another channel is set differently for each channel according to the priority of the plurality of channels,
Obtaining sound signals of a plurality of channels transmitted through the communication network in units of packets;
Comparing the delay in arrival time of the acquired packets in the plurality of channels with respect to the packet of the other channel acquired first and the predetermined time set for each of the plurality of channels;
When the delay of the arrival time of the packet exceeds the predetermined time, the packet is regarded as missing,
Complement the packet considered to be missing,
In order to absorb delays in arrival times of the packets, the packets of the plurality of channels are temporarily held in a mixing buffer,
A mixing method characterized in that the packets of the plurality of channels whose time axes are aligned by being held in the mixing buffer are added to generate one packet.

Images