WO2023021547A1

WO2023021547A1 - Transfer system, transfer method, and transfer program

Info

Publication number: WO2023021547A1
Application number: PCT/JP2021/029891
Authority: WO
Inventors: 拓郎山口; 康弘持田; 大介白井
Original assignee: 日本電信電話株式会社
Priority date: 2021-08-16
Filing date: 2021-08-16
Publication date: 2023-02-23
Also published as: JPWO2023021547A1

Abstract

This transfer system comprises: a copy unit 22 that copies a plurality of packets formed by packetizing a stream of a non-compressed video; a transmission unit 24 that transmits the copied packets through a narrow band path 3; and a generation unit 122 that generates a low-resolution image from packets sampled through packet-loss occurring in the narrow band path 3.

Description

TRANSMISSION SYSTEM, TRANSMISSION METHOD, AND TRANSMISSION PROGRAM

The present invention relates to transmission systems, transmission methods, and transmission programs.

SMPTE ST2110 is a method for transmitting uncompressed video and audio over networks. SMPTE ST2110-20 describes a method for storing and transmitting a video essence in the payload of an RTP (Real-time Transport Protocol) packet (Non-Patent Document 1). In uncompressed video transmission, the number of packets per frame is constant as long as the video parameters and the method of mapping pixels to RTP packets are unchanged.

The payload part of the RTP packet contains the pixel values of the pixels that make up the video and the RTP payload header required for reconstruction on the receiving side. In particular, since the RTP payload header contains the coordinates of the leading pixel in the packet, the receiving side reconstructs the pixel values by arranging the pixel values accordingly.

Uncompressed video can now be transmitted over the network, and it is expected that more streams will be transmitted within the network than ever before. Furthermore, in order to use technologies such as redundant transmission, it became necessary to change the construction of network paths in real time. When making such a configuration change, it is essential not to adversely affect the current stream transmission, and for that, it is necessary to know which stream is flowing on which path. In addition, if the system does not operate correctly after the route construction is changed, it is desirable to know how far the stream is flowing correctly on the route after the change.

However, in next-generation transmission, where high-resolution video is expected to be sent without compression, extremely high processing performance is required to acquire all stream information in the operating network as it is and process it in real time. can only be realized in a limited environment or device.

The present invention has been made in view of the above circumstances, and an object of the present invention is to reduce the load on the network while making it possible to grasp the stream flowing through the network.

In order to achieve the above object, one aspect of the present invention is a transmission system comprising: a duplicating unit for duplicating a plurality of packets obtained by packetizing an uncompressed video stream; and a generator for generating a low-resolution image from packets sampled due to packet loss occurring in the narrowband path.

One aspect of the present invention is a transmission method performed by a transmission system, comprising the steps of: duplicating a plurality of packets obtained by packetizing an uncompressed video stream; and transmitting the duplicated packets via a narrowband path. and generating a low-resolution image from packets sampled due to packet loss occurring in the narrowband path.

One aspect of the present invention is a transmission program that causes a computer to function as the transmission system.

According to the present invention, it is possible to understand the stream flowing through the network while reducing the load on the network.

1 is a diagram illustrating a configuration example of a transmission system according to a first embodiment; FIG. 4 is a flowchart showing transmission processing; FIG. 10 is a diagram illustrating a configuration example of a transmission system according to a second embodiment; FIG. FIG. 11 is a diagram illustrating a configuration example of a transmission system according to a third embodiment; FIG. It is a hardware configuration example.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First embodiment>
FIG. 1 is a system configuration diagram showing a configuration example of a transmission system according to the first embodiment.

The transmission system of this embodiment includes at least one relay device 2 arranged on the network and a management device 1 that aggregates information. The management device 1 and relay device 2 are connected by a wireless or wired narrowband path 3 (communication path).

The relay device 2 is connected to the

network devices

4 and 5 by wire or wirelessly. The

network devices

4 and 5 include, for example, transmitting/receiving devices. Relay device 2 relays a stream transmitted from at least one network device 4 to at least one network device 5 . In the illustrated example, a relay device 2 relays a plurality of streams transmitted from a plurality of network devices 4 to corresponding network devices 5, respectively.

It is assumed that the stream of this embodiment is an uncompressed image and holds pixel value data and coordinate information, like RTP packets packetized according to SMPTE ST 2110.

In the transmission system of this embodiment, when changing the route of an uncompressed video stream or adding or deleting equipment, it is possible to grasp the route of each stream in the current network, check whether the system after the change is operating as expected, etc. It facilitates management and operation.

Also, the stream of this embodiment will be described below as an example of an uncompressed video stream (hereinafter referred to as "video stream"), but it may be an uncompressed video/audio stream.

The illustrated interconnecting device 2 has at least one reception port 21 , a duplicator 22 , at least one first transmission port 23 and a second transmission port 24 .

Each reception port 21 receives uncompressed video packets (hereinafter referred to as "video packets"), which are packetized video streams, from the network device 4.

The duplication unit 22 duplicates (mirrors) the video packets received at each reception port 21 and sends them to the second transmission port 24 . In the illustrated example, the duplicator 22 duplicates video packets of multiple video streams. Also, the duplicator 22 outputs the received video packet to the corresponding first transmission port 23 . The first transmission port 23 transmits the video packets to the next network device 5 .

Although RTP packets are used as video packets in this embodiment, they are not limited to RTP packets, and packets other than RTP packets may be used.

The second transmission port 24 transmits the video packets duplicated by the duplicator 22 to the management device 1 via the narrowband path 3 .

The video packets output from the second transmission port 24 are transmitted to the management device 1 while causing packet loss by passing through the narrowband path 3 . In the narrowband path 3, video packets are discarded when the amount of video packets exceeds the amount of video packets that can be transmitted. Specifically, the second transmission port 24 discards video packets that do not fit within the bandwidth (transmission capacity) of the narrowband path 3 and outputs video packets that fit within the bandwidth to the narrowband path 3 . In this embodiment, by using the narrowband path 3 as the communication path between the relay device 2 and the management device 1 , packet loss occurs in the narrowband path 3 .

The narrowband path 3 may be a path with a transmission capacity smaller than the transmission capacity of the video stream input to the relay device 2 . In the example shown in FIG. 1, the path may have a transmission capacity smaller than the total transmission capacity of video packets of n (n=3) video streams input to the relay device 2 .

For example, when an uncompressed video stream with an image size of HD (1280×720) and a frame rate of 60 fps flows from one network device 4 to the relay device 2, the HD size video stream conforming to SMPTE2110 is about 1.3 Gbps. be. In this case, 1 GbE (Gigabit Ethernet) with a transmission capacity smaller than 1.3 Gbps may be used as narrowband path 3 connecting relay device 2 and management device 1 . In this case, even one video stream lacks a transmission capacity of about 0.3 Gbps, and packet loss occurs. Also, as shown in the figure, when video streams flow from n network devices 4 to relay device 2, narrowband path 3 may be a path with a transmission capacity smaller than n×1.3 Gbps.

The transmission capacity of the narrowband path 3 may be determined according to the processing performance of the management device 1. For example, the amount of processing for generating a low-resolution image is determined by the number of known points (the number of remaining packets after packet loss), the resolution (reduction ratio) of the output low-resolution image, and the like. Therefore, if the processing performance of the management device 1 is low, it is better to reduce the transmission capacity of the narrowband path 3 .

On the other hand, if the number of remaining packets decreases too much, the quality of low-resolution images tends to deteriorate. descend. Therefore, the transmission capacity of the narrowband path 3 may be determined not only by the processing performance of the management device 1 but also by considering the resolution of target video streams, the number of streams, and the resolution of low-resolution images to be output.

Since the number n of video streams flowing through the network (network device 4, relay device 2, network device 5) changes, the narrow band path 3 with a transmission capacity that causes packet loss when n increases to some extent may be For example, for an HD size video stream group, if the narrowband path 3 is set to 2 Gbps, packet loss occurs from n=2.

Alternatively, a path with a relatively large transmission capacity such as 10 GbE is used as the narrowband path 3, and packet loss occurs when the number of video streams or bit rate input to the relay device 2 exceeds a predetermined threshold. Depending on the presence or absence of loss, the low-resolution image generation processing performed by the management device 1 may be stopped.

Specifically, when packet loss does not occur, the management device 1 can acquire an image without any loss, so it does not generate a low-resolution image and displays a high-quality video stream as it is, or displays a low-load image. Any method may be used to generate and display an image of sufficient quality. On the other hand, when a packet loss occurs, an image cannot be displayed unless a low-resolution image is generated, so the management device 1 generates and displays a low-resolution image. In this manner, the management device 1 may switch whether to generate a low-resolution image depending on the presence or absence of packet loss.

Alternatively, the relay device 2 connects to the management device 1 through a plurality of narrowband paths 3 having different transmission capacities, and switches the narrowband path 3 to be used according to the number of video streams and bit rate to change the packet loss rate. good too. In this case, the relay device 2 is provided with a second transmission port 24 for each narrowband path 3 .

Specifically, the interconnecting device 2 has at least one reception port 21 and at least one first transmission port 23 . The route is set depending on whether to output from. Therefore, the route selection unit (not shown) of the interconnecting device 2 acquires the number of packets flowing into each reception port 21, bit rate information, etc., and selects the first transmission port 23 as a route. In addition, the route selection unit uses the number of packets flowing into each receiving port 21, bit rate information, etc. to determine to which of the plurality of second transmitting ports 24 the duplicated packet group is to be allocated. Narrowband path 3 is switched by determining. The route selector is connected to the reception port 21, the first transmission port 23 and the second transmission port 24. FIG.

Remaining packets sampled due to packet loss on narrowband path 3 are sent to management device 1 .

Note that the illustrated relay device 2 may be a device that relays a video stream transmitted from one network device 4 to one network device 5 . In this case, the interconnecting device 2 has one reception port 21 , a duplicator 22 , one first transmission port 23 and a second transmission port 24 .

Video packets transmitted from one or more relay devices 2 via the narrowband path 3 are input to the management device 1 .

The management device 1 has a reception port 11, a processing section 12, and a display section 13. The processing unit 12 includes a packet storage unit 121 and a generation unit 122 . Note that the management device 1 may have a display device (display) outside the management device 1 without the display unit 13 .

The reception port 11 receives video packets from the relay device 2 and sends them to the packet storage section 121 of the processing section 12 .

The video packets are accumulated in the packet accumulation unit 121 . The packet accumulating unit 121 classifies and accumulates video packets for each stream using information such as the transmission source address of the video packets, and stores the accumulated video at regular time intervals or when a certain amount of packets is collected for each stream. The packet is sent to the generator 12 . That is, the packet storage unit 121 sends the video packets to the generation unit 122 at the timing when the amount of packets necessary for generating low-resolution images such as thumbnail images is accumulated for each stream.

The generation unit 122 generates a low-resolution image from the video packets sampled due to the packet loss that occurs on the narrowband path 3 and outputs it to the display unit 13 . The generator 122 generates a low-resolution image for each video stream. The generation unit 122 acquires the pixel value data of the uncompressed video and the coordinates (position information) of the pixel value data from the video packet, and uses these to generate a low-resolution image. The low-resolution image provides an overview of the video stream and serves, for example, as a thumbnail image.

Specifically, the generation unit 122 acquires the coordinates of the pixel value data held in the media payload from the payload header of the RTP packet (video packet) defined by SMPTE2110. An RTP packet has an RTP header, a payload header containing information (such as coordinates) about pixel value data, and a media payload containing pixel value data.

As a method for creating a low-resolution image such as a thumbnail image, the generation unit 122 may perform image reduction using a classical resampling method (resolution conversion method) such as Bilinear or Bicubic on the acquired coordinates.

When performing processing from pixel value data in multiple frames, the generation unit 122 considers object movement between frames, and does not treat frames equally, but processes a certain frame as main information. Techniques such as selecting frames or increasing the reduction ratio to reduce the effect of a small amount of movement may be adopted.

As another reduction method, the generation unit 122 may utilize machine learning (learning model). In this case, the generation unit 122 may generate a low-resolution image using a learning model that has undergone machine learning using the pixel value data and coordinates of the RTP packet, or the output image obtained by the above-described resampling method as input. .

In addition, the generation unit 122 may reconstruct an incomplete image in which only the known pixels of the acquired pixel value data are arranged based on the coordinates, and solve the regression problem of generating a low-resolution image therefrom.

In addition, the generation unit 122 may be notified in advance of important part coordinates representing the outline of the video as sub information, and the generation unit 122 may adopt trimming or seam carving including them to reduce the image size.

In addition, the generation unit 122 may switch the generation method based on the required resolution of the display unit 13, the amount of parallel-processed streams, etc., instead of using a single method. For example, the generation unit 122 may use a resampling method when displaying multiple streams side by side, and may utilize machine learning when a request to confirm an overview of one of the streams is input.

The display unit 13 displays low-resolution images such as thumbnail images. As a result, the low-resolution image generated by the generation unit 122 is displayed on the display unit 13 of the management device 1 or on a display device installed outside via the network. Thus, the video stream on the network is visualized and displayed in a user-visible manner.

The display unit 13 may use low-resolution images to display information indicating what kind of video stream is flowing between which devices on the network. That is, the display unit 13 may display the low-resolution image as a video stream flowing between the

network devices

4 and 5. FIG. For example, the display unit 13 may display a line representing the connection between devices and a low-resolution image superimposed on the line.

Since the video packet contains the source address and the destination address assigned to each port of the

network devices

4 and 5, it is possible to analyze the video packet to determine which network device 4 to which network device 5 the video packet is sent from. It is possible to determine whether the video stream is a

In the display unit 13, for example, a simple display such as a block diagram using a GUI is assumed. A line representing a connection between devices may be displayed by obtaining a route in advance by confirming that the management device 1 or the relay device 2 can actually communicate by ping or the like. In addition, the line representing the connection between devices indicates that when transmission is actually performed, the management device 1 acquires the route by acquiring information from the NMOS (which manages information on terminals existing on the network), which will be described later. may be displayed as Further, the generation unit 122 may acquire the transmission source address and the destination address included in the video packet, and acquire the line representing the connection between the devices.

Alternatively, when a user views a fiber (transmission path) through which a video stream flows, a display device outside the management device 1, such as a head-mounted display, may display a thumbnail image or the like superimposed on the fiber. Specifically, when the user visually recognizes or gazes at the fiber actually connected in the real world through a transparent display such as AR glasses, the thumbnail image of the video stream flowing through the fiber can be displayed on the fiber. is displayed overlaid on the

FIG. 2 is a flowchart showing the transmission processing of this embodiment.

The relay device 2 receives video packets (RTP packets) in which an uncompressed video stream is packetized from each network device 4, and duplicates the received video packets (S11). Also, the relay device 2 transmits the received video packet to the corresponding network device 5 .

The relay device 2 transmits the duplicated video packet to the management device 1 via the narrowband path 3 (S12). The management device 1 receives the video packets sampled through the narrowband path 3, and uses the video packets to generate low-resolution images such as thumbnail images (S13). The management device 1 displays the low resolution image (S14).

<Second embodiment>
FIG. 3 is a system configuration diagram showing a configuration example of a transmission system according to the second embodiment.

The transmission system of this embodiment includes management equipment 1A. The management device 1A acquires network device information and stream information from an NMOS (Networked Media Open Specifications) RDS (Registration & Discovery System) server 8, and acquires the multicast address of a desired multicast terminal 9 from the device information. Then, the management device 1A uses the multicast address to send join to the multicast terminal 9 and participates in the multicast. As a result, the management device 1A can acquire the same video stream (video packet) as the video stream (video packet) flowing from the multicast terminal 9 to the network device 5. FIG.

The illustrated multicast terminal 9 includes a transmission source terminal that transmits a video stream and a multicast network. The multicast terminal 9 and the reception port 11 of the management device 1A are connected by the narrowband path 3. FIG.

The illustrated management device 1A has a reception port 11, a processing unit 12, a display unit 13, another reception port 14, and a transmission port 15. The managed device 1A differs from the managed device 1 of the first embodiment shown in FIG. However, the reception port 11 and the transmission port 15 may be a single port having both transmission and reception functions.

The other receiving port 14 acquires network device information (NMOS information) and stream information from the RDS server 8 . The RDS server 8 is a server that detects and registers devices within the network. The other receiving port 14 acquires the multicast address of the multicast terminal 9 that distributes the desired stream from the device information and the stream information, and sends the multicast address to the transmitting port 15 .

The sending port 15 sends a join to the multicast terminal 9 using the multicast address. Thereby, the multicast terminal 9 transmits the same video stream (video packet) as the video stream (video packet) transmitted to the network device 5 or the like to the management device 1A via the narrowband path 3 . The narrowband path 3 of the embodiment is, like the narrowband path 3 of the first embodiment, a path with a transmission capacity smaller than the transmission capacity of the video packets of the video stream distributed by the multicast terminal 9 . As a result, the video packets output from the multicast terminal 9 pass through the narrowband path 3 and are transmitted to the management device 1A while packet loss occurs.

The receiving port 11 of the management device 1A receives video packets sampled due to packet loss on the narrowband path 3, and outputs the received packets to the processing unit 12. The processing unit 12 generates a low-resolution image such as a thumbnail image from the sampled video packets, as in the first embodiment. The display unit 13 displays the generated low-resolution image.

<Third Embodiment>
FIG. 4 is a system configuration diagram showing a configuration example of a transmission system according to the third embodiment.

The transmission system of this embodiment includes a management device 1B. The management device 1B is different from the management device 1A of the second embodiment in that it acquires video packets of a plurality of video streams from the multicast network 7, and otherwise is the same as the management device 1A.

A plurality of source terminals 6 are connected to the multicast network 7 . The source terminal 6 multicasts (transmits) video packets of a video stream (multicast source) via the multicast network 7 . The multicast network 7 and the reception port 11 of the management device 1B are connected by the narrowband path 3. FIG.

As in the second embodiment, other receiving ports 14 acquire network device information (NMOS information) and stream information from the RDS server 8, and acquire multiple multicast addresses of desired video streams from these information. and sends the multicast address to the transmission port 15 .

The sending port 15 sends a join to the multicast network 7 using multiple multicast addresses. As a result, the multicast network 7 transmits a plurality of video streams identical to the video stream (video packet) transmitted to the network device 5 and the like to the management device 1B via the narrowband path 3. FIG.

The narrowband path 3 of the embodiment is a path with a transmission capacity smaller than the total transmission capacity of video packets of a plurality of video streams distributed by the multicast network 7 to the management device 1 . As a result, the video packets output from the multicast network 7 are transmitted to the management device 1B while passing through the narrowband path 3, causing packet loss.

The reception port 11 of the management device 1B receives video packets sampled due to packet loss on the narrowband path 3, and outputs the received packets to the processing unit 12. As in the first embodiment, the processing unit 12 generates a low-resolution image for each video stream from the sampled video packets. The display unit 13 displays the generated low-resolution image.

<Effects of Embodiment>
The transmission system according to the first to third embodiments described above includes a duplicating unit 22 that duplicates a plurality of packets obtained by packetizing an uncompressed video stream, and transmits the duplicated packets via the narrowband path 3. and a generation unit 122 for generating a low-resolution image from packets sampled due to packet loss occurring in the narrowband path 3 . The replicator may replicate packets using multicast joins.

As described above, in this embodiment, a low-resolution image showing an outline of a video stream is generated from packet sampling by simply preparing the environment by preparing the narrowband path 3 and remaining packets after sampling.

As a result, in this embodiment, it is possible to understand the stream flowing through the network while reducing the load on the network. Specifically, the video packet passing through the narrowband path 3 intentionally causes packet loss. As a result, in this embodiment, the number of video packets to be transmitted is reduced without providing a special device or function, and a low-resolution image is generated from the reduced video packets and transmitted to the management device 1 (maintenance base). Therefore, in a wide range of usage environments, visualization of video streams can be realized in parallel and in real time.

In addition, processing costs can be reduced by reducing the number of video packets due to packet loss on the narrowband path 3 and generating low-resolution images from limited information, and multiple video streams can be visualized in parallel in real time. .

In this way, the present embodiment suppresses an increase in the processing load due to an increase in network size and an increase in the amount of video streams flowing through the network without preparing a special device, and enables video streams that can be widely used in networks in the future. Visualization can be achieved.

<Hardware configuration>
For the management devices 1, 1A, 1B and relay device 2 described above, for example, a general-purpose computer system as shown in FIG. 5 can be used. The illustrated computer system includes a CPU (Central Processing Unit, processor) 901, a memory 902, a storage 903 (HDD: Hard Disk Drive, SSD: Solid State Drive), a communication device 904, an input device 905, and an output device. 906. Memory 902 and storage 903 are storage devices. In this computer system, CPU 901 executes a predetermined program loaded on memory 902 to realize each function of each device. For example, each function of the management devices 1, 1A, 1B and relay device 2 is executed by the CPU of the management device in the case of the program for the management device, and by the CPU of the relay device in the case of the program for the relay device. It is realized by

Also, the management device and the relay device may be implemented by one computer or may be implemented by multiple computers. Also, the management device and the relay device may be virtual machines implemented in computers. Programs for management devices and programs for relay devices can also be stored on computer-readable recording media such as HDDs, SSDs, USB (Universal Serial Bus) memories, CDs (Compact Discs), and DVDs (Digital Versatile Discs). , can also be distributed over a network.

It should be noted that the present invention is not limited to the above embodiments, and many modifications are possible within the scope of the gist. For example, a plurality of management devices 1 shown in FIG. 1 may be provided, and the plurality of management devices 1 may be connected to a central management server via a network. This allows the central management server to grasp video streams flowing through a large-scale network. The same applies to the management devices 1A and 1B in FIGS. 3 and 4. FIG.

1, 1A, 1B: management device 11, 14: reception port 12: processing unit 121: packet storage unit 122: generation unit 13: display unit 15: transmission port 2: relay device 21: reception port 22: duplication unit 23, 24 : Transmission port 4, 5: Network device 6: Source terminal 7: Multicast network 8: RDS server 9: Multicast terminal

Claims

a duplicating unit that duplicates a plurality of packets obtained by packetizing an uncompressed video stream;
a transmitter that transmits the duplicated packet through a narrowband path;
a generator that generates a low-resolution image from packets sampled due to packet loss that occurs on the narrowband path.
2. The transmission system according to claim 1, wherein the generation unit acquires pixel value data of the uncompressed video and position information of the pixel value data from the packet, and generates the low-resolution image.
3. The transmission system according to claim 1, further comprising a display unit that displays the low-resolution image as the stream flowing between network devices.
4. The transmission system according to any one of claims 1 to 3, wherein the narrowband path has a transmission capacity smaller than that of the uncompressed video stream.
The duplicating unit duplicates packets of a plurality of streams of uncompressed video,
The transmission system according to any one of claims 1 to 3, wherein the generator generates the low-resolution image for each stream.
A transmission method performed by a transmission system,
replicating a plurality of packetized packets of a stream of uncompressed video;
transmitting the duplicated packet over a narrowband path;
generating a low-resolution image from packets sampled due to packet loss occurring on the narrowband path.
A transmission program that causes a computer to function as the transmission system according to any one of claims 1 to 5.