JP4498011B2 - Distribution apparatus, processing method thereof, program, and storage medium - Google Patents

Description

  The present invention relates to a technique for transmitting continuous data such as voice and video to a large number of people via a network such as the Internet.

  In recent years, data distribution systems that require real-time performance, such as live image distribution systems and video conference systems using an IP (Internet Protocol) network, have been proposed.

  In general, when the above-mentioned live distribution system or video conference system is used by a large number of people, the video may be seen as dropped frames or the audio may be interrupted. In addition, there are few products that synchronize video and audio, and there are systems where video and audio are out of sync when used for a long time. These are caused by the difference in the properties of video and audio data.

  In video distribution represented by MotionJPEG, a large amount of data must be transmitted in a short time. For this reason, when live distribution is performed with emphasis on real-time performance for a large number of people, it is not always possible to transmit all the data to everyone, and the playback side plays back with a few dropped frames. The best-effort transmission method is used because it is difficult for human eyes to understand even if the video is dropped a little.

  In voice distribution represented by VoIP (Voice over IP), a small amount of data must be transmitted at regular intervals over time. Even when distributed to a large number of people, if all data cannot be transmitted to everybody at regular intervals, the reproduction side will reproduce the sound as an unpleasant sound such as sound interruption or echo. On the playback side, the audio data is buffered to some extent in consideration of the delay on the network and the audio is played back. However, live distribution with an emphasis on real-time performance may only buffer several hundred milliseconds to several seconds, and it is necessary for the distribution server to perform transmission processing at a fixed interval.

  Here, an outline of the operation of the prior art when video and audio are distributed to a large number of people will be described below.

  First, the video / audio distribution server is generally configured in a system as shown in FIG. 1, and the distribution server 102 has a plurality of functional blocks.

  The plurality of clients 109 transmit video and audio distribution requests to the distribution server 102 via the network 108. In the distribution server 102, the session management unit 105 receives a transmission request from the client through the network unit 107, and creates and stores client information 106. The audio data processing unit 103 and the video data processing unit 104 respectively create data to be transmitted, refer to the previous client information 106, and request data transmission to the network unit 107 through the session management unit 105. Then, the network unit 107 sends the data to the destination client according to the transmission request.

In the case of distribution to a large number of people, the audio data processing unit 103 and the video data processing unit 104 perform processing for transmitting the input data to all clients according to the client information 106.
JP 2002-223425 A

  FIG. 2 shows the time relationship between the generation timing of the audio data and the transmission processing when only audio distribution is performed for a large number of people in the prior art. In this example, it is assumed that the number N of data transmitted at a time is N = 4.

  When the audio data is generated in the order of 200, 201,... And the fourth audio data 203 is generated, the data of 200 to 203 is started to be transmitted to the clients collectively, and the transmission completion timing to all the clients is the audio. The timing when the data 205 is generated is assumed. In this case, the transmission cycle 210 and the transmission cycle 211, which are the cycles for distribution processing to the final client, are substantially equal.

  FIG. 3 shows the entire transmission processing timing when the video distribution processing is simultaneously performed in addition to the audio distribution processing as shown in FIG.

  The video distribution process and the audio distribution process can be transmitted in parallel because of the multitasking operation. However, since only one network unit 107 exists as shown in FIG. 1, the transmission process timing overlaps. And sending time will take longer than ever. For this reason, the transmission cycle 310 takes longer than the transmission cycle 210 in the case of only the voice distribution processing of FIG. Further, in the transmission cycle 311, the transmission is completed in the same time as before without much influence because it does not overlap with the video distribution process.

  As described above, in the related art, when video distribution and audio distribution are performed for a large number of people, it is difficult to keep the audio transmission cycle constant. In particular, it becomes more difficult when the processing capability of the transmission apparatus is low.

  Accordingly, the present invention has been made in view of the above-described circumstances, and an object of the present invention is to maintain a constant transmission cycle of audio data even when audio data and video data are distributed to a large number of people. That is.

To solve the above problems and achieve the object, the distribution device according to the present invention, there is provided a delivery apparatus for transmitting video data and audio data to a plurality of clients, said plurality of clients, a plurality of groups Allocating means for allocating; a first buffer for storing audio data to be transmitted to a first group of clients; a second buffer for storing audio data to be transmitted to a second group of clients; After transmitting voice data of the first period from the first buffer to the clients of the first group, starting from the generation timing of the voice data after a predetermined time from the start of the first period to the clients of the second group and transmitting means for transmitting the voice data of the second period is from the second buffer, said allocation means, glue In response to the reduction in the number, the first group of first clients is changed to the second group of clients, and the transmission means converts the first group of audio data to the first group. The first client changes from the first group to the second group between the transmission of the first period and the second period of audio data to the second group of clients. When the voice data of the second period is transmitted to the second group of clients, the voice data already transmitted to the first client is not transmitted to the first client. It is characterized by.

  In the distribution apparatus according to the present invention, the allocating unit changes the number of groups according to an increase or decrease in the number of clients.

The processing method of distributing device according to the present invention, the allocation process to allocate a processing method of a delivery apparatus for transmitting video data and audio data to a plurality of clients, said plurality of clients, a plurality of groups, the A management step of storing audio data to be transmitted to a group of clients in a first buffer and storing audio data to be transmitted to a second group of clients in a second buffer; After the audio data of the first period is transmitted from the first buffer to the client, the second group of clients is started from the generation timing of the audio data after a predetermined time from the start of the first period. the second period of audio data and a transmission step of transmitting from said second buffer, the allocation process, the number of groups In response, the first group of first clients is changed to the second group of clients, and the transmitting step converts the first period of audio data into the first group of clients. The first client is changed from the first group to the second group between the transmission of the voice data of the second period and the transmission of the voice data of the second period to the second group of clients. In this case, when the audio data of the second period is transmitted to the second group of clients, the audio data that has already been transmitted to the first client is not transmitted to the first client. And

  Further, in the processing method of the distribution apparatus according to the present invention, the allocating step changes the number of groups according to increase / decrease of the number of clients.

A program according to the present invention causes a computer to execute the above processing method.

A storage medium according to the present invention is characterized in that the above-described program is stored in a computer-readable manner.

  According to the present invention, even when audio data and video data are distributed to a large number of people, the transmission cycle of audio data can be kept constant.

  Preferred embodiments of the present invention will be described below with reference to the drawings.

  FIG. 4 is a block diagram illustrating a system configuration common to some embodiments of the present invention.

  In FIG. 4, 400 is an imaging device, 401 is an audio input device, 402 is a distribution server, 403 is an audio data processing unit, 404 is a video data processing unit, 405 is a session management unit, and 407 is a network unit. Functions similar to those of the imaging device 100, the audio input device 101, the distribution server 102, the audio data processing unit 103, the video data processing unit 104, the session management unit 105, and the network unit 107 shown in FIG.

  In the embodiment of the present invention, in addition to these blocks, an area for storing a group number is prepared in a storage unit that stores client information 406 referred to by the audio data processing unit 403 and the video data processing unit 404. In addition, a group number management unit 408 is provided to manage how much buffer accumulation is completed for each group and which group transmission processing is executed when the next audio data is input.

(First embodiment)
In FIG. 4, the session management unit 405 assigns a group number to the client in response to a transmission request from the client, creates client information 406, and stores it.

  In this embodiment, the group number range is divided into four groups of 1 to 4. The group number is assigned so that the number of clients belonging to each group number is the same at the time of connection.

  The voice data processing unit 403 divides the voice data input from the voice input device 401 into an appropriate size, assigns a serial number (identifier), and sends it to the group number management unit 408. The group number management unit 408 refers to the identifier from the buffer managed for each group, and collects a predetermined number of audio data from the audio data next to the divided audio data that has already been transmitted via the network unit 407. Send to the target group.

  The flow of processing performed by the group number management unit 408 will be described with reference to the flowchart of FIG.

  When the session management unit 405 creates and stores the client information, the session management unit 405 starts the group number management unit 408.

  The group number management unit 408 first sets the working group number to 0 (step S501).

  Next, referring to the client information, it is determined whether or not there is a connected client (step S502). If there is no connected client, the process ends here.

  If there is a connected client in step S502, the working group number is incremented, and the working group number is set so as to increase sequentially in the range of 1 to N (N = 4 in this embodiment). (Step S503).

  Then, the input voice data is divided and read for each fixed size (step S504), and stored in each buffer managed for each group number (step S505).

  Next, the client information is searched (step S506), and it is determined whether there is next client information (step S507). When the search for all connected clients is completed, the process returns to the process of determining whether there is a connected client (step S502). On the other hand, if it is determined in step S507 that all connected clients have not been searched, it is determined whether the group number in the currently searched client information matches the group number currently being worked on (step S508). Then, the audio data stored in the buffer corresponding to the currently working group number is transmitted (step S509). If they do not match, nothing is done. Then, the process returns to the client information search (step S506).

  FIG. 6 shows the time relationship between the generation timing of audio data and the transmission processing when only audio distribution is performed for a large number of people in the present embodiment.

  When attention is paid to the transmission cycle of the last client of group 1, since it is a case of only audio distribution, the transmission cycle 610 and the transmission cycle 611 are naturally equal (the transmission cycle is the same as in FIG. 2).

  FIG. 7 shows the timing of all transmission processing when the video distribution processing is simultaneously performed in addition to the audio distribution processing of FIG.

  When the video distribution process and the audio distribution process are executed simultaneously, the audio transmission cycle for the group having the same processing timing is slightly delayed, but the influence on other groups is suppressed, and the transmission cycle 710 and 711 as a whole are greatly different. It turns out that does not occur.

  Thus, in the audio distribution process, the transmission timing of the same divided audio data is shifted for each group, so that the load of the transmission process can be distributed to the number of groups. For this reason, even when overlapped with other processing timings such as video distribution, it is possible to minimize the error in the transmission cycle of the audio data and the influence on the client.

  Further, even when viewed from the client, since the buffer corresponding to the group to which the client belongs is transmitted from the server, the real-time property is not impaired.

(Second Embodiment)
When a large number of people use the video distribution / audio distribution function, even if allocation is performed so that the number of people is equal for each group number, the number of people belonging to each group number varies depending on the cutting timing. In order to further reduce the error in the transmission interval, it is better to reduce the number of people belonging to each group number and to distribute the transmission process.

  For this reason, in this embodiment, rearrangement of the belonging group is performed not only at the time of connection but also at the timing when the client is disconnected. This will be described with reference to FIG.

  For example, there are currently three clients belonging to group 1 and two clients belonging to groups 2 to 4, and if the clients of group 4 are disconnected, the number of members belonging to each group becomes 3, 2, 2, 1. From the viewpoint of distribution of transmission processing, it is preferable that the number of members in each group is two. For this reason, consider moving a client of group 1 to group 4.

  As the operation of group 1, transmission processing is performed at the time when the audio data 803 is accumulated. Thereafter, the affiliation of one client in group 1 is changed to group 4, and the fact that transmission to audio data 803 has been completed is stored in the client information.

  In the operation of the group 4, as shown in FIG. 8, when the audio data 806 is generated, the transmission process is performed again, and the audio data 803 to 806 are transmitted. In this case, since the audio data 803 has already been transmitted to the clients whose affiliation has been changed from the group 1 to the group 4, the fact that the transmission to the audio data 803 has been completed is stored in the client information. A total of three pieces of audio data 804 to 806 are transmitted.

  In this way, when distributing to a large number of people, the number of clients belonging to each group number can be kept even by rearranging the group at the disconnection timing of the client, and the transmission cycle is always shifted. Can be kept to a minimum.

(Third embodiment)
In the first embodiment, the number of groups = 4 and the number of buffer accumulations = 4. However, when the number of connected clients is small, it is not necessary to prepare as many as four groups. Is likely to be able to keep. Also, reducing the number of groups is very effective for improving the efficiency of resources such as memory of the apparatus.

  Specifically, until the number of connected clients reaches a certain number A, the number of groups is one, the number of clients is A to 2xA, the number of groups is 2, ..., and the number of clients is (M-1) xA. Up to MxA, the number of groups is divided into M. The maximum value of M is the buffer accumulation number.

  Even if an attempt is made to increase the number of groups M beyond the buffer accumulation number, the transmission timing overlaps with any other group, so it is meaningless to divide the buffer accumulation number M or more.

  Although the group number to which the client belongs changes due to increase / decrease in the number of groups, the method of moving the client between the group numbers can be solved by using the method of the second embodiment.

  As described above, by changing the number of groups in accordance with the increase or decrease of the number of connected clients, it is possible to efficiently use the resources of the device itself.

  As described above, according to the above embodiment, when an audio distribution request is received from a client, a group number is assigned to each client, and a buffer for each group having a different start position each time data is generated. When a certain number of data is stored in the client, the transmission processing to the client is performed, so that the load of the transmission processing can be distributed over the number of groups, and the deviation of the transmission cycle of the audio data can be minimized. it can. As a result of actually experimenting with the number of groups of 4 and the number of storage buffers of 4 × 10 ms by the video and audio distribution server, the deviation of the transmission cycle was changed from ± 5 ms to ± 1 ms, and a high effect could be obtained.

  Also, since buffers with different start positions are used for each group, the timing for completing the specified number of accumulations is different, and the data can always be sent to the client at the timing when the latest data is generated, and real-time performance is not impaired. .

(Other embodiments)
In addition, an object of each embodiment is to supply a storage medium (or recording medium) on which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and a computer (or CPU) of the system or apparatus Needless to say, this can also be achieved by reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Furthermore, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts described above.

It is a figure which shows the structure of the conventional video / audio delivery system, and the functional block structure of a delivery server. It is the conventional operation | movement timing diagram at the time of performing audio | voice delivery to many people. It is the conventional operation | movement timing diagram at the time of performing audio | voice delivery and video delivery to many people. It is a figure which shows the functional block structure of the video / audio delivery server concerning embodiment of this invention. It is a flowchart which shows the process implemented in a group number management part. It is an operation | movement timing diagram at the time of performing audio | voice delivery to many people in the 1st Embodiment of this invention. It is an operation | movement timing diagram at the time of performing audio | voice delivery and video delivery to many people in the 1st Embodiment of this invention. It is a transmission data transition diagram by movement of a client between groups.

Explanation of symbols

400 Imaging Device 401 Audio Input Device 402 Distribution Server 403 Audio Data Processing Unit 404 Video Data Processing Unit 405 Session Management Unit 406 Client Information 407 Network Unit 408 Group Number Management Unit

Claims (6)

A distribution device that transmits video data and audio data to a plurality of clients,
Allocating means for allocating the plurality of clients to a plurality of groups;
A first buffer for storing audio data to be transmitted to a first group of clients;
A second buffer for storing audio data to be transmitted to a second group of clients;
After transmitting audio data of a first period from the first buffer to the first group of clients, generation of audio data after a predetermined time from the start of the first period to the second group of clients Transmission means for transmitting audio data of the second period starting from the timing from the second buffer ,
The allocating means changes the first client of the first group to the client of the second group in response to reducing the number of groups,
The transmission means transmits the audio data of the first period to the first group of clients until the audio data of the second period is transmitted to the second group of clients. When the first client is changed from the first group to the second group, when the second period of voice data is transmitted to the second group of clients, the first client A delivery apparatus , wherein audio data that has already been sent to a client is not sent to the first client .   The distribution apparatus according to claim 1, wherein the allocating unit changes the number of groups according to an increase or decrease in the number of the clients. A processing method of a distribution apparatus for transmitting video data and audio data to a plurality of clients,
An allocating step of allocating the plurality of clients to a plurality of groups;
A management step of storing audio data to be transmitted to a first group of clients in a first buffer and storing audio data to be transmitted to a second group of clients in a second buffer;
After transmitting audio data of a first period from the first buffer to the first group of clients, generation of audio data after a predetermined time from the start of the first period to the second group of clients A transmission step of transmitting audio data of the second period starting from the timing from the second buffer ,
The allocating step changes the first client of the first group to the client of the second group in response to reducing the number of groups;
In the transmission step, the audio data of the first period is transmitted to the first group of clients until the audio data of the second period is transmitted to the second group of clients. When the first client is changed from the first group to the second group, when the second period of voice data is transmitted to the second group of clients, the first client A processing method of a distribution apparatus, wherein audio data that has already been transmitted to a client is not transmitted to the first client . 4. The processing method of the distribution apparatus according to claim 3 , wherein in the allocation step, the number of groups is changed in accordance with an increase or decrease in the number of clients. A program for causing a computer to execute the processing method according to claim 3 or 4 . A storage medium storing the program according to claim 5 in a computer-readable manner.

Images