JP4917497B2 - Video distribution device, distribution video switching method, distribution video switching program, and distribution video switching program recording medium - Google Patents

Description

  The present invention relates to a technique for switching a video to be distributed without switching a session when distributing the video to a terminal at each point in a video conference between multiple points.

  In order to communicate various data, audio, video, and other media over an IP network, such as video conferencing and IP telephony, it is necessary to start and disconnect communication between two or more communication applications. . Such association between applications on the Internet in communication is called a session. As a procedure for establishing a session, for example, there is a protocol such as SIP (Session Initiation Protocol).

  As technologies for performing multipoint connection of terminals for video conferences and the like, there are the following prior art 1 and prior art 2.

  The prior art 1 is a method of establishing a video session corresponding to each point on a one-to-one basis. When paying attention to a certain point α, a one-to-one session is prepared between the point α and all other points, and images from all other points are distributed to the user terminal at the point α. The video to be displayed on the user terminal at the point α is selected on the user terminal side at the point α.

  However, the method of the prior art 1 has a problem that the processing load on the distribution server side and the network increases as the number of connection points increases. With the spread of video conferencing systems in recent years, the number of connection bases tends to increase. In some cases, tens to hundreds of points are connected. There is a possibility that such a case cannot be handled by the method of the prior art 1.

  In video conferencing systems that connect a large number of points, there are many use cases in which only video at some points among all participants need to be distributed. Therefore, in the prior art 2, for example, at a certain point β, as many sessions as the number of images to be displayed on the user terminal at that point β are prepared, and the user terminal side of the point β is designated among all other points. Only video from the selected point is distributed from the distribution server. In the method of Prior Art 2, when changing the point of the video displayed on the user terminal, it is necessary to switch the session.

Note that Non-Patent Document 1, for example, is a prior art document describing a technique related to multipoint connection such as a video conference. Non-Patent Document 1 describes a technique related to a multipoint video conference using an IP network.
Takeda, Moriuchi, Imae, Honda, Anno, TV conference technology that realizes high-quality PC multipoint connection, NTT Technical Journal, Vol.17, No.7, pp.18-22, July 2005

  In order to display the video of other points as in the prior art 1, a plurality of video sessions corresponding to each point are performed, and the video to be displayed on the user terminal side by distributing all the points is displayed. The method to be selected has a problem that the processing load on the distribution server side and the network becomes high.

On the other hand, in the method of the prior art 2, for example, when video for an arbitrary P (P <N) point among the N points of the TV conference participant is distributed, P sessions are appropriately connected / disconnected. There is a need to. At this time,
・ As the number of sessions increases, the processing load for session switching increases, which increases the processing cost of session management.
-When switching sessions, there is an increase in the probability of distribution delays and distribution abnormalities, resulting in disturbances in the displayed video.
There is.

  An object of the present invention is to solve these problems and to perform switching of a large amount of video information at a higher speed without causing disturbance in the displayed video.

  The present invention is characterized by using the following means in order to solve the above problems.

  (1) In a system in which as many sessions as the number of videos to be displayed on the user terminal are prepared and only videos from the point designated on the user terminal side among all other points are distributed from the video distribution device. In the video distribution apparatus, a switch for switching the video signal data distributed to the user terminal is provided.

  (2) As a method of switching the video to be distributed without switching the session, the association between the point and the session is performed by means different from the session establishment, and when the point to be distributed is switched after the session is established, A switch in the distribution apparatus changes only video signal data distributed to a data channel in which a session is established and data communication is performed.

  (3) At the time of video switching, there is a time interval between detecting the frame end point of the video before video switching and the frame start point of the video after video switching, and between the video before video switching and the video after video switching. By inserting a pre-stored video when it is available, it is possible to switch the distribution video with little display video disturbance.

  That is, the present invention is intended to be used in a multipoint video conference system, etc., and obtains a video signal from a video signal stream transmitted from a plurality of points or a video signal file stored in advance, and serves as original information. In a video distribution system for generating a distribution stream to be distributed to multiple points, comprising a buffer for receiving a signal, a buffer for transmitting a video signal as transmission information, and a switch for switching between the received video and the transmitted video, Provides a function to switch the input information to be distributed by changing only the video signal to be distributed without changing the session information such as the sender information and reception information of any distribution stream, the port used for communication, etc. It is characterized by that.

Further, in the above invention, in order to suppress disturbance of the video on the display terminal side at the time of switching, the video stream before switching is detected after detecting the break point of the DCI bit string indicating the break of the video signal data and waiting for the break point. When the video stream after switching is started, a meaningful start point is found in the compression format, and transmission from the start point suppresses video disturbance on the video display terminal side. And

  Also, the present invention inserts an arbitrary video signal prepared in advance into an idle time during video switching, which is during switching from the end of the video before switching to the start of video after switching, thereby enabling the video signal in the video stream to be inserted. When the video stops, the display method due to the difference in the processing method on the playback side will not cause a difference, and the process that waits for a preset time interval will not cause the subsequent video signal to come. Further, the switching process is surely performed.

  Furthermore, the display terminal side is provided with a means for determining that the received video stream is transmission information of the own terminal so that the video acquired in the own terminal is displayed so that the transmission delay is not conscious. It is also possible to reduce the processing load using an input board or the like.

According to the present invention, the following effects can be obtained.
(1) Since session switching is not required, the cost of session management can be reduced, and video disturbance due to session switching can be eliminated.
(2) By detecting the video frame start point and determining the appropriate switching timing as video signal data, the video start point after video switching is correctly displayed.
(3) When there is a time interval at the time of video switching, unified display is performed at all points by inserting video signal data prepared in advance. For this reason, it is possible to reduce a sense of incongruity and display disturbance when switching images.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. This embodiment is an example when the distribution video switching method according to the present invention is applied to a multipoint video conference system.

  FIG. 1 is a diagram for explaining the outline of a multipoint video conference system according to this embodiment. The multipoint video conference system according to the present embodiment includes a video distribution device 10, a terminal 20, and a user operation processing device 30.

  The video distribution device 10 relays the video between the terminals 20 and switches the video signal to be distributed as necessary. The video distribution device 10 includes an input buffer 11, a switch 12, an output buffer 13, and a video distribution information management table 14. The input buffer 11 receives video from each terminal 20. The switcher 12 switches the video distributed to each terminal 20. The output buffer 13 delivers video to each terminal 20. The video distribution information management table 14 is a storage means for storing and managing information about which terminal's video is distributed to which terminal. In particular, the video distribution information management table 14 is prepared for at least the number of points at which video is displayed on the terminal 20 at one point. The session information and information indicating the video switching state by the switch 12 are stored.

  The terminal 20 is used by an end user, and inputs and transmits a local video and receives and plays a video of another site.

  The user operation processing device 30 issues a video switching request to the video distribution device 10 and accepts a video switching request from an end user as an opportunity. Although the user operation processing device 30 can be realized on the terminal 20, this embodiment shows an example in which the user operation processing device 30 is explicitly distinguished and implemented for the sake of explanation.

  In the multipoint video conference system, as shown in FIG. 1A, the video input at the terminal 20 </ b> A is transmitted to the video distribution device 10. Similarly, the video input at the terminals 20B and 20C is also transmitted to the video distribution device 10. In the video distribution device 10, the input buffer 11 receives video signals from the terminals 20 at all points. In FIG. 1, the input buffer 11 is schematically shown as one square. However, as an implementation, the input buffer 11 may be appropriately divided depending on processes and threads.

  As shown in FIG. 1A, in the initial stage, it is assumed that the video of the terminal 20A is the distribution target video. The video received by the input buffer 11 is transferred to the switch 12. The switch 12 stores that the video of the terminal 20A is a video to be distributed, selects the video signal of the terminal 20A to be distributed from the video signal delivered from the input buffer 11, and stores it in the output buffer 13. Deliver. The output buffer 13 distributes the video signal of the terminal 20A delivered from the switcher 12 to the terminals 20 at all points.

  Here, it is assumed that the distribution target video is switched from the video of the terminal 20A to the video of the terminal 20B. The change from the state shown in FIG. 1A to the state shown in FIG. 1B means switching of the distribution target video.

  In the multipoint video conference system, when the video to be distributed is to be switched, the end user issues a video switching request to the video distribution device 10 via the user operation processing device 30. This means, for example, that the video to be distributed is changed from the conference status change page configured by the WWW page after accessing the server using the WWW browser.

  Here, it is assumed that the distribution target video is changed from the video of the terminal 20A to the video of the terminal 20B. As a result of the operation, the user operation processing device 30 issues a video switching request from the video of the terminal 20A to the video of the terminal 20B to the video distribution device 10.

  As shown in FIG. 1B, the video distribution device 10 returns a video switching response (OK) to the user operation processing device 30 and stores the video distribution information management table 14 when the video switching is possible. The information to be updated is updated, and a switching control signal is sent to the switch 12. As a result, the video signal to be distributed is switched from the video signal of terminal A to the video signal of terminal B.

  After the video switching, the switcher 12 selects the video signal of the terminal 20B to be distributed from the video signal delivered from the input buffer 11, and delivers it to the output buffer 13. The output buffer 13 distributes the video signal of the terminal 20B delivered from the switcher 12 to the terminals 20 at all points.

  The above is the outline of the multipoint video conference system according to the present embodiment. Next, the internal processing of the switcher 12 at the time of video switching will be described.

  FIG. 2 is a diagram for explaining an example of internal processing of the switch according to the present embodiment. The switch 12 includes a DCI detector 120. The DCI detector 120 is means for detecting a bit string of DCI (Decoder Configuration Information) from the bit string of the video signal. DCI is data configuration information necessary for terminal side decoding processing.

  When the video switching occurs, the information of the distribution target video stored in the switch 12 is updated. In this embodiment, however, the video signal to be transferred to the output buffer 13 is immediately switched after the update. Not performed.

  When video switching occurs, the switcher 12 first detects information indicating the head of a new frame when the encoded data is decoded from the video signal of the current distribution target video. The detection of the head information of the frame includes, for example, a method in which a DCI bit string is stored and a DCI bit string is detected from a bit string of a video signal by a matching process by a simple bit operation. In this embodiment, the video signal is not decoded and the arithmetic processing is performed as a bit string with the content of the payload in the packet. When the head of a new frame is detected, the packet before the packet in which the head of the frame is detected is distributed as a video before video switching.

  Next, the head information of the new frame is detected from the video signal to be distributed after the video switching. When the head of a new frame is detected, the video after the video switching is distributed from the packet in which the head of the frame is detected.

  As a result of the above processing, a consistent video signal in units of frames is distributed to each terminal 20 during video playback, so that it is possible to suppress video disturbance in the decoding / playback processing at the terminal 20. .

  However, when detecting a new frame before and after video switching, there may be a time lag due to processing delay, packet loss, or packet data abnormality. In such a case, in order to prevent the playback on the terminal 20 of the end user from being disturbed, and in order to prevent a display difference due to a difference in the implementation of the decoding device on the terminal 20 side, As shown in FIG. 2, a stored video signal (Stored Data) stored in advance is inserted between the video signal before video switching and the video signal after video switching in the switch 12. As a result, it is possible to avoid changing the display video during video switching depending on the decoding device on the terminal 20 side.

  In addition, the switcher 12 manages two timers in order to reliably switch the video in case the end point of the video signal before video switching or the start point of the video signal after video switching cannot be detected. If the end point of the video signal before the video switching cannot be detected within a predetermined time, the switching to the stored video signal is forcibly performed. In addition, when the video signal start point after video switching cannot be detected within a predetermined time and there is a video signal after video switching, the video signal after video switching is forcibly switched to. Do.

  Here, an example of specific video switching processing by the switch 12 will be described. As shown in FIG. 2, the video signal of the terminal 20 </ b> A and the video signal of the terminal 20 </ b> B are passed from the input buffer 11 to the switch 12. Here, the video signal of the terminal 20A is a video signal before video switching, and the video signal of the terminal 20B is a video signal after video switching. In FIG. 2, A-1, A-2, and A-3 indicate packets in the video signal of the terminal 20A, and the smaller number indicates the earlier time. Similarly, B-1, B-2, and B-3 indicate packets in the video signal of the terminal 20B, and the smaller number indicates the earlier time.

  When receiving a video switching request from the video of the terminal 20A to the video of the terminal 20B from the user operation processing device 30, in the switcher 12, the DCI detector 120 detects a DCI bit string from the video signal of the terminal 20A. Here, as shown in FIG. 2, a bit string of DCI is detected from packet A-2. The switch 12 determines that the packet before the packet A-2 in which the DCI bit string is detected, that is, the packet A-1 is output as the video signal before the video switching.

  Next, the DCI detector 120 detects a DCI bit string from the video signal of the terminal 20B. Here, as shown in FIG. 2, it is assumed that a bit string of DCI is detected from packet B-3. The switch 12 determines that the packet B-3 from which the DCI bit string is detected is output as a video signal after video switching.

  Here, a stored video signal (Stored Data) is inserted between the video signal before video switching and the video signal after video switching for adjustment.

  As shown in FIG. 2, the video signal delivered to the output buffer 13 at the time of video switching is, in order of time, the video signal of the packet A-1 before video switching, the stored video signal (Stored Data), and the video signal after video switching. It becomes the video signal of packet B-3.

  FIG. 3 is a distribution video switching process flowchart by the switching unit of the video distribution apparatus according to the present embodiment. In the example of FIG. 3, a case where the distribution target video is switched from the video of the terminal 20A to the video of the terminal 20B will be described.

  The switch 12 receives the video packets of all the terminals 20 from the input buffer 11 (step S10). If no video switching request is received from the user operation processing device 30 (step S11), a packet of the distribution target video (here, a video packet of the terminal 20A) is output to the output buffer 13 (step S12).

  When a video switching request (here, a video switching request from the video of the terminal 20A to the video of the terminal 20B) is received from the user operation processing device 30 (step S11), the DCI from the video packet of the terminal 20A by the DCI detector 120 is received. The detection of the bit string is started.

  When the DCI detector 120 detects a DCI bit string from the video packet of the terminal 20A (step S13), a packet subsequent to the video packet of the terminal 20A from which the DCI bit string is detected is not transmitted to the output buffer 13. To do. The DCI detector 120 starts detecting the DCI bit string from the video packet of the terminal 20B.

  While the DCI detector 120 cannot detect the DCI bit string from the video packet of the terminal 20A (step 13), timer processing for detecting the end point of the terminal 20A is performed. Until the end point detection timer of the terminal 20A times out (step S14), the video packet of the terminal 20A is transmitted to the output buffer 13 (step S15), and the video packets of all the terminals 20 are received from the input buffer 11. (Step S16).

  If the end point detection timer of the terminal 20A times out (step S14), the subsequent video packet of the terminal 20A is not transmitted to the output buffer 13, and the DCI from the video packet of the terminal 20B by the DCI detector 120 is prevented. The detection of the bit string is started.

  When the DCI detector 120 detects the DCI bit string from the video packet of the terminal 20B (step S17), the video packet of the terminal 20B is transmitted from the video packet of the terminal 20B from which the DCI bit string is detected. To the output buffer 13 (step S18).

  While the DCI detector 120 cannot detect the DCI bit string from the video packet of the terminal 20B (step 17), timer processing for detecting the start point of the terminal 20B is performed. Until the start point detection timer of the terminal 20B times out (step S19), the stored video signal is transmitted to the output buffer 13 (step S20), and video packets of all the terminals 20 are received from the input buffer 11 (step S20). S21).

  If the end point detection timer of the terminal 20B times out (step S19), the terminal 20B video packet received from the input buffer 11 is transmitted to the output buffer 13 as a distribution target video packet (step S18).

  Assume that, for example, a request for switching to the video of the terminal 20C is sent from the user operation processing device 30 when the video of the terminal 20A is switched to the video of the terminal 20B. At this time, the video of the terminal 20B displayed so far by the switcher 12 is switched to the video of the terminal 20C by the same processing as the processing described in FIG. 2 and FIG.

  The major differences between this method and the prior art 1 described in the section “Background Art” are as follows. In the prior art 1, in order to sequentially display the images of the terminals 20B and 20C on the terminal 20A, a session is established in advance using a protocol such as SIP between the terminals 20A and 20B and between the terminals 20A and 20C. It is necessary to keep it. In addition, when a video of another terminal 20D (not shown) is displayed on the terminal 20A, it is necessary to establish a session in advance between the terminal 20A and the terminal 20D. In other words, a session must be established between terminals at all points.

  On the other hand, in the present invention, a session is established in advance between the terminal 20A and the video distribution apparatus 10, or between the terminal 20A and the terminal 20B or the terminal 20C. If it is set that it may be used for video distribution of other terminals, in response to a video switching request from the user operation processing device 30, the sender information and reception information of the distribution stream, and the port used for communication Switching the input information to be distributed is realized by changing only the video signal to be distributed without changing the session information. Therefore, it is not necessary to establish a session between terminals at all points in advance.

  The major differences between the present invention and the prior art 2 are as follows. In the prior art 2, it is not necessary to establish a session between terminals at all points in advance, as in the present invention. However, in the case of the prior art 2, for example, a session is first established between the terminal 20A and the terminal 20B. In this state, when it is desired to switch the video of the terminal 20B displayed on the terminal 20A to the video of the terminal 20C, it is necessary to newly disconnect and connect the session. As a result, the display of the image may be disturbed when the display image is switched.

  On the other hand, in the present invention, since an already connected session is reused for a plurality of terminals, a session is established and only a video distributed to a data channel on which data communication is performed is changed. It is not necessary to repeat session disconnection and connection at the time of video switching, and video switching with less disturbance of the displayed video can be realized by detecting the frame start point of the video and inserting previously stored video at the time of video switching.

  In the example of FIGS. 1 to 3, an example in which the video displayed on each terminal is a video of one terminal has been described in order to make the description easy to understand. For example, when displaying images of four terminals on the terminal 20A, the four switches 12 shown in FIGS. 1 and 2 can be used to display the images of the four terminals simultaneously on the terminal 20A. , It is possible to switch each video to a video of another terminal as required. In this case, it is sufficient that at least four sessions are connected. Note that the session may be divided into an uplink to the video distribution device 10 and a downlink from the video distribution device 10.

  Information on the established session, the data channel in which data communication is performed by the establishment of the session, and information on which terminal's video signal data is currently flowing through the data channel are provided by the video distribution device 10. The distribution information management table 14 is stored in a table in the internal memory and managed. This information is updated when a session is connected or disconnected and when a distribution video is switched. This makes it possible to associate the correspondence between the point and the session by a method different from the session establishment.

  Further, for example, the terminal 20A (same for other terminals) is provided with means for determining whether or not the received video stream is the transmission information of the own terminal. It is also preferable to provide a means for directly displaying the video acquired in the own terminal instead of the video sent from the video distribution device 10. By doing so, it is possible to display without making the user aware of the transmission delay. Also, when acquiring the video of the terminal itself, the processing load can be reduced by using a video input board or the like.

  The distribution video switching process of the video distribution apparatus 10 described above can be realized by a computer and a software program. The program can be recorded on a computer-readable recording medium or provided through a network. It is.

It is an outline explanatory view of a multipoint video conference system by this embodiment. It is a figure explaining the example of the internal process of the switch by this Embodiment. It is a delivery video switching process flowchart by the switch of the video delivery apparatus of this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Video delivery apparatus 11 Input buffer 12 Switcher 120 DCI detector 13 Output buffer 14 Video delivery information management table 20 Terminal 30 User operation processing apparatus

Claims (6)

In a video distribution device that distributes video signal data transmitted from terminals at multiple points to terminals at multiple points,
An input buffer for receiving video signal data transmitted from each terminal;
An output buffer for storing video signal data to be transmitted to each terminal;
A switch for switching between the video signal data stored in the input buffer and the video signal data stored in the output buffer, and a session prepared for at least the number of points at which video is displayed at the terminal at one point Video distribution information storage means for storing information and information indicating a video switching state by the switch;
When switching a video distribution point, the session information is distributed to a data channel in which data communication is performed without changing the session information by referring to the video distribution information storage unit and operating the switch. Control means for changing only the video signal data,
And said switch, said comprising a detection means for detecting the bit sequence of the DCI indicating a break of the video signal data odor Te Film image signal data, the video signal data before switching is detected by said detection means separated and switched after A video distribution apparatus characterized by switching video signal data output to the output buffer at video signal data delimiters. The video distribution apparatus according to claim 1,
The switching unit stops the output of the video signal data before switching to the output buffer at the time when the detection unit detects a break of the video signal data before switching at the time of video switching, and then the detection unit To output the video signal data after switching after detecting the delimiter of the video signal data after switching to the output buffer, and from the output stop of the video signal data before switching to the output buffer of the video signal data after switching A video distribution apparatus comprising means for inserting other video signal data prepared in advance during the time until the start of output to the video. A distribution video switching method executed by a video distribution device that distributes video signal data transmitted from a plurality of terminals to a plurality of terminals,
Receiving video signal data transmitted from each terminal into an input buffer;
Transferring video signal data stored in the input buffer to the output buffer via a switch;
Storing the session information prepared for at least the number of points for displaying video in the terminal at one point and the information indicating the video switching state by the switch in the video distribution information storage means;
When switching a video distribution point, the session information is distributed to a data channel in which data communication is performed without changing the session information by referring to the video distribution information storage unit and operating the switch. And changing only the video signal data,
And, in the switch, at the detected bit sequence of DCI indicating a break of movies image signal data Te video signal data smell, separator and after switching of the video signal data of the detected before switching of the video signal data separator, wherein A delivery video switching method comprising: switching the output of video signal data to an output buffer. The delivery video switching method according to claim 3,
At the time of switching the video by the switch, the output of the video signal data before switching to the output buffer is stopped at the time when the separation of the video signal data before switching is detected, and then the switching by the detection means Outputs the video signal data after switching after detecting the delimiter of the video signal data to the output buffer, and outputs the video signal data after the switching to the output buffer from the stop of the output of the video signal data before the switching. A distribution video switching method characterized by inserting other video signal data prepared in advance during the time until the start of the video.   A distribution video switching program for causing a computer to execute the distribution video switching method according to claim 3 or 4.   A computer-readable distribution video switching program recording medium in which the distribution video switching program according to claim 5 is recorded.

Images