JP4836871B2 - Video data transmitting apparatus, program, and method for assigning priority to encoded image - Google Patents

Description

  The present invention relates to a video data transmission apparatus, program, and method for assigning priorities to encoded images.

Conventionally, as a representative multimedia coding standard, MPEG (Moving Picture Experts)
Group) method. MPEG is an inter-frame coding technique for image compression by reducing time domain redundancy. This technique uses the fact that the correlation between adjacent frames in video data is very high, and divides it into three types (at least two types) of pictures.

The following pictures are used:
・ I picture (Intra-Picture, intra-frame coded image)
The amount of data is the largest, but it can be decoded by itself.
P picture (Predictive-Picture, inter-frame encoded image, forward predictive encoded image)
A forward interframe prediction is performed from a past frame (I picture), and the difference is encoded. The data amount of the P picture is smaller than that of the I picture, and the P picture cannot be decoded by itself.
・ B picture (Bi-directional Predictive-Picture, inter-frame encoded image, bidirectional predictive encoded image)
Inter-frame prediction in both directions of the past frame and the future frame is performed, and the difference is encoded. The data amount of the B picture is smaller than that of the P picture, and the B picture cannot be decoded by itself.

  In order to use the future frame for prediction, the B picture rearranges the frames at the time of encoding, first encodes the future frame, and then performs bidirectional prediction encoding. For this reason, only two types of pictures, I and P, are used for bidirectional real-time applications such as videophones where delay is a problem (IP prediction).

  Here, there is a technique for assigning a high priority to packets constituting an I picture (see, for example, Patent Document 1). When packet discarding occurs, it is discarded in the order of B picture, P picture, and I picture. This minimizes the degradation of video quality due to packet discard.

  In addition, there is a technique for transmitting a total of seven types of streams in which audio is added to six types of high-frequency components and low-frequency components of I, P, and B pictures, respectively, using different multicast addresses (see, for example, Non-Patent Document 1). According to this technique, the receiving device side can select a reception hierarchy according to an available bandwidth or a resource such as a CPU.

  Further, regarding a wireless network system that can use a plurality of communication media, there is a technique in which an I picture that is a basic layer communicates with a communication medium (for example, a mobile phone network) that covers a wide area (see, for example, Non-Patent Document 2). . According to this technique, a P picture is communicated by a communication medium (for example, PHS) that covers a medium area, and a B picture is communicated by a communication medium (for example, a wireless LAN) that covers a narrow area. As a result, in the case of frequently occurring wireless LAN or PHS handovers, the I picture is not discarded, and the video quality for the mobile terminal is improved.

Sazawa et al., “Client Configuration and Operation Analysis in Hierarchical Video Multicast”, IEICE, Society Conference, D-11-38, September 2000 Honma et al., “Study on Hierarchical Multicast Suitable for Wireless Environment”, IEICE General Conference, B-7-127, March 2001 JP 2000-078573 A

  According to the above-described prior art, three levels of priority are basically provided for each image type of I, P, and B pictures. However, for example, IP prediction is usually used for streaming packets of video data. In this case, only two levels of priority of I and P pictures can be provided.

  The P picture is used for inter-frame prediction in the forward direction from the past frame. Therefore, when the next P picture after the I picture is lost, the P picture received until the next I picture is received cannot be decoded normally. Since the interval between I pictures is usually about 15 to 30 frames, the loss of the first half P picture has a large effect on the quality.

  Incidentally, by including the priority based on the data in the IP (Internet Protocol) header, the router arranged on the relay route can recognize the priority of the packet. The router can perform relay control (priority transfer, waiting transfer, or discard) of the packet, for example, according to the priority of the packet.

  Therefore, the present invention assigns multiple levels of priority to encoded images, so that even in a communication environment in which packet loss occurs, video data transmission that can prevent degradation of the reception quality of video data as much as possible. An object is to provide an apparatus, a program, and a method.

According to the present invention, in a video data transmitting apparatus for transmitting video data,
Encoding means for generating at least an intra-frame encoded image and an inter-frame encoded image from video data;
A priority assignment unit that assigns a higher priority to the intra-frame encoded image than the inter-frame encoded image, and assigns different priorities according to the transmission order of the frames in the inter-frame encoded image;
Constitute a packet including information constituting the image, have a packet structure means for including a priority in the header of the packet,
The priority allocation means sets the number of inter-frame encoded images sandwiched between two intra-frame encoded images as I_int, sets the number of inter-frame encoded image priorities as N_prio, and the priority is an absolute value thereof. , The priority becomes lower, the highest priority for the inter-frame coded image is PRI, the variable parameter i = 1,
If I_int ≦ N_prio,
Until i exceeds N_prio, the process of assigning (PRI + I_int-1 + i-1) from the intra-frame encoded image to the i-th inter-frame encoded image is repeated while incrementing i by 1.
It is characterized by that.

  According to another embodiment of the video data transmitting apparatus of the present invention, the encoding means performs encoding based on MPEG, the intra-frame encoded image is an I picture, and the inter-frame encoded image is P. A picture is also preferred.

  According to another embodiment of the video data transmitting apparatus of the present invention, it is also preferable that the packet composing means includes the priority for the IP packet in the Diff-Serv field of the IP header.

According to the present invention, in a program for causing a computer mounted on an apparatus for transmitting video data to function,
Encoding means for generating at least an intra-frame encoded image and an inter-frame encoded image from video data;
A priority assignment unit that assigns a higher priority to the intra-frame encoded image than the inter-frame encoded image, and assigns different priorities according to the transmission order of the frames in the inter-frame encoded image;
Configure a packet including information constituting the image, and cause the computer to function as a packet configuration unit that includes priority in the header of the packet ;
The priority allocation means sets the number of inter-frame encoded images sandwiched between two intra-frame encoded images as I_int, sets the number of inter-frame encoded image priorities as N_prio, and the priority is an absolute value thereof. , The priority becomes lower, the highest priority for the inter-frame coded image is PRI, the variable parameter i = 1,
If I_int ≦ N_prio,
Until i exceeds N_prio, the process of assigning (PRI + I_int-1 + i-1) from the intra-frame encoded image to the i-th inter-frame encoded image is repeated while incrementing i by 1.
The computer is made to function as described above .

According to the present invention, in the video data transmitting method in the video data transmitting apparatus,
A first step of generating at least an intra-frame encoded image and an inter-frame encoded image from video data;
A second step of assigning a higher priority to the intra-frame encoded image than the inter-frame encoded image, and assigning a different priority according to the transmission order of the frames in the inter-frame encoded image;
Constitute a packet including information constituting the image, we have a third step of including a priority in the header of the packet,
In the second step, the number of inter-frame encoded images sandwiched between two intra-frame encoded images is I_int, the number of inter-frame encoded image priorities is N_prio, and the priority is an absolute value thereof. , The priority becomes lower, the highest priority for the inter-frame coded image is PRI, the variable parameter i = 1,
If I_int ≦ N_prio,
The process of assigning (PRI + I_int-1 + i-1) from the intra-frame encoded image to the i-th inter-frame encoded image is repeated while incrementing i by 1 until i exceeds N_prio. .

  According to the video data transmitting apparatus, program and method of the present invention, even in a communication environment in which packet loss occurs by assigning different priorities according to the transmission order of frames among inter-frame encoded images, It is possible to prevent deterioration of data reception quality as much as possible. This is particularly effective for receiving streaming packets of video data.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a system configuration diagram according to the present invention.

  According to FIG. 1, a video data transmission device (video content distribution server) 1 transmits a packet including an encoded image based on MPEG to a terminal 2 via a network. A plurality of nodes intervene in the route through which the packet is transferred. For example, there are a plurality of routers included in a wired network and a relay terminal that is interposed by a wireless link. These nodes buffer received packets and transmit the buffered packets. At this time, packet priority transfer, waiting transfer, or discard control is performed according to the priority of the packet.

In the case of an Ethernet (registered trademark) frame, the priority is included in the CoS field, for example. In the case of the IP header, the priority is included in the Diff-Serv field, for example. The Diff-Serv field is defined as follows.

  The Diff-Serv field is included in the TOS (Type Of Service) field in the case of IPv4, and is included in the Flow-ID in the case of IPv6. According to Table 1 described above, the high priority transfer and low discard type has the highest priority, and the low priority transfer and high discard type has the lowest priority.

  According to FIG. 1, the video data transmitting apparatus 1 according to the present invention includes an encoding unit 101, a priority assignment unit 102, and a packet configuration unit 103. These functional units can also be realized by executing a program that causes a computer mounted on the video data transmitting apparatus to function.

  The encoding unit 101 encodes video data based on MPEG. At this time, at least an I picture of the intra-frame encoded image and a P picture of the inter-frame encoded image are generated.

  The priority assignment unit 102 assigns a higher priority to the intra-frame coded image than the inter-frame coded image, and assigns different priorities in the inter-frame coded image according to the frame transmission order. Here, the priority assigning unit 102 assigns the highest priority to the first image among the inter-frame encoded images, and assigns the lower priority to the rear image.

  The packet configuration unit 103 configures an IP packet including an image. The priority is included in the Diff-Serv field of the header of the IP packet. Of course, in the case of an Ethernet (registered trademark) frame, the priority may be included in the CoS field, for example.

  FIG. 2 is a flowchart of the priority assignment unit in the present invention.

The priority assignment unit 102 defines the following parameters.
I_int: Number of P pictures sandwiched between two I pictures N_prio: Number of P picture priorities i: Variable parameter with initial value “1” PRI: Maximum priority for P pictures Here, priority Is assumed to have a lower priority as its absolute value increases. That is, it is assumed that the priorities become lower in the order of priorities 1, 2, 3, and so on.

(S201) First, it is determined whether I_int> N_prio. That is, it is determined whether it is necessary to assign the same priority to a plurality of P pictures.

(S202) When I_int> N_prio, that is, when it is necessary to assign the same priority to a plurality of P pictures, it is determined whether i <N_prio. Initially, i = 1 and S203 is repeated for the number of N_prio-1.

(S203) When i <N_prio, it is determined from what number to what number P picture the priority (PRI + i-1) is assigned. For example, when priority 1 (PRI = 2) is assigned to an I picture and i = 1, the first priority assigned to a P picture is 2 (= 2 + 1−1). The priority is assigned to the P pictures of the order number calculated by the following equation. This order is the order from the I picture.
((i-1) × [I_int / N_prio] +1) th
~ (I × [I_int / N_prio]) th (Formula 1)

(S204) Increment i by 1 and return to S202.

(S205) When i ≧ N_prio, it is determined whether to assign the priority (PRI + i−1) to the P picture from the order to the end. For example, when i = 5, the priority assigned to the last P picture is 6 (= 2 + 5-1). The priority is assigned to the P pictures of the order number calculated by the following equation.
((i-1) × [I_int / N_prio] +1) th
~ (I × [I_int / N_prio] + (I_int mod N_prio)) th (Expression 2)
Then, the process ends.

(S206) When I_int ≦ N_prio, that is, when a different priority can be assigned to each of the plurality of P pictures, it is determined whether i ≦ N_prio. In other words, the process ends when priority assignment for all P pictures is completed (i> N_prio).

(S207) When i ≦ N_prio, the priority of (PRI + i−1) is assigned to the data of the i-th P picture. Note that when the I picture interval is short (that is, when the number of P pictures sandwiched between I pictures is small), the speed of recovery from video degradation due to packet loss or the like is faster than when the interval is long. . In that case, a priority of (PRI + I_int-1 + i-1) may be assigned to data of the i-th P picture.

(S208) Increment i by 1 and return to S206.

  FIG. 3 is an explanatory diagram showing priorities assigned to images according to the present invention.

For example, the following first example will be described.
I_int = 14: Number of P frames sandwiched between two I pictures N_prio = 6: Number of P picture priorities (7 priority levels, I picture priority 1, P picture priority) Degrees 2-7)
PRI = 2: highest priority for P picture data

[Processing of S203 when i = 1]
Priority (PRI + i−1) = 2 + 1−1 = 2
((i−1) × [I_int / N_prio] +1) to (i × [I_int / N_prio])
= ((1-1) × [14/6] +1) to (1 × [14/6])
= (0 × [2.33] +1) to (1 × [2.33])
= 0 to 2
Priority 2 is assigned to the first to second P pictures.

[Processing of S203 when i = 2]
Priority (PRI + i-1) = 2 + 2-1 = 3
((i−1) × [I_int / N_prio] +1) to (i × [I_int / N_prio])
= ((2-1) × [14/6] +1) to (2 × [14/6])
= (1 × [2.33] +1) to (2 × [2.33])
= 3 to 4
A priority of 3 is assigned to the third to fourth of the P pictures.

[Processing of S203 when i = 3]
Priority (PRI + i-1) = 3 + 2-1 = 4
((i−1) × [I_int / N_prio] +1) to (i × [I_int / N_prio])
= ((3-1) × [14/6] +1) to (3 × [14/6])
= (2 × [2.33] +1) to (3 × [2.33])
= 5-6
Priority 4 is assigned to the fifth to sixth P pictures.

[Processing of S203 when i = 4]
Priority (PRI + i-1) = 4 + 2-1 = 5
((i−1) × [I_int / N_prio] +1) to (i × [I_int / N_prio])
= ((4-1) × [14/6] +1) to (4 × [14/6])
= (3 × [2.33] +1) to (4 × [2.33])
= 7-9
A priority of 5 is assigned to the seventh to ninth P pictures.

[Processing of S203 when i = 5]
Priority (PRI + i-1) = 5 + 2-1 = 6
((i−1) × [I_int / N_prio] +1) to (i × [I_int / N_prio])
= ((5-1) × [14/6] +1) to (5 × [14/6])
= (4 × [2.33] +1) to (5 × [2.33])
= 10 to 11
A priority of 6 is assigned to the 10th to 11th P pictures.

[Processing of S205 when i = 6]
Priority (PRI + i-1) = 6 + 2-1 = 7
((i-1) × [I_int / N_prio] +1) ~
(i × [I_int / N_prio] + (I_int mod N_prio)
= ((6-1) × [14/6] +1) to (6 × [14/6] + (14 mod 6))
= (5 × [2.33] +1) to (6 × [2.33] +2)
= 12-15
A priority of 7 is assigned to the 12th to 15th P pictures.

Next, for example, the following second example will be described.
I_int = 14: Number of P frames sandwiched between two I pictures N_prio = 3: Number of P picture priorities (4 priority levels, I picture priority 1, P picture priority) Degrees 2-4)
PRI = 2: highest priority for P picture data

[Processing of S203 when i = 1]
Priority (PRI + i−1) = 2 + 1−1 = 2
((i−1) × [I_int / N_prio] +1) to (i × [I_int / N_prio])
= ((1-1) × [14/3] +1) to (1 × [14/3])
= (0 × [4.66] +1) to (1 × [4.66])
= 0 to 4
Priority 2 is assigned to the first to fourth P pictures.

[Processing of S203 when i = 2]
Priority (PRI + i-1) = 2 + 2-1 = 3
((i−1) × [I_int / N_prio] +1) to (i × [I_int / N_prio])
= ((2-1) × [14/3] +1) to (2 × [14/3])
= (1 × [4.66] +1) to (2 × [4.66])
= 5 to 9
Priority 3 is assigned to the fifth to ninth P pictures.

[Processing of S205 when i = 3]
Priority (PRI + i-1) = 3 + 2-1 = 4
((i-1) × [I_int / N_prio] +1) ~
(i × [I_int / N_prio] + (I_int mod N_prio)
= ((3-1) × [14/3] +1) to (3 × [14/3] + (14 mod 3))
= (2 × [4.66] +1) to (3 × [4.66] +2)
= 10-15
Priority 4 is assigned to the 10th to 15th P pictures.

Next, for example, the following third example will be described.
I_int = 3: Number of P frames sandwiched between two I pictures N_prio = 6: Number of P picture priorities (7 priority levels, I picture priority 1, P picture priority Degrees 2-7)
PRI = 2: highest priority for P picture data

[Processing of S207 when i = 1]
Priority (PRI + i−1) = 2 + 1−1 = 2
Priority 2 is assigned to the first P picture.
[Processing of S207 when i = 2]
Priority (PRI + i-1) = 2 + 2-1 = 3
The priority 3 is assigned to the second P picture.
[Processing of S207 when i = 3]
Priority (PRI + i-1) = 2 + 3-1 = 4
Priority 4 is assigned to the third P picture.

Next, when the number of P pictures sandwiched between I pictures is small, for example, assignment may be performed as in the following fourth example.
I_int = 3: Number of P frames sandwiched between two I pictures N_prio = 6: Number of P picture priorities (7 priority levels, I picture priority 1, P picture priority Degrees 2-7)
PRI = 2: highest priority for P picture data

[Processing of S207 when i = 1]
Priority (PRI + I_int-1 + i-1) = 2 + 3-1 + 1-1 = 4
Priority 4 is assigned to the first P picture.
[Processing of S207 when i = 2]
Priority (PRI + I_int-1 + i-1) = 2 + 3-1 + 2-1 = 5
Priority 5 is assigned to the second P picture.
[Processing of S207 when i = 3]
Priority (PRI + I_int-1 + i-1) = 2 + 3-1 + 3-1 = 6
A priority of 6 is assigned to the third P picture.

  As described above, according to the video data transmission apparatus, program, and method of the present invention, in a communication environment in which packet loss occurs by assigning different priorities according to the transmission order of frames among inter-frame encoded images. Even so, it is possible to prevent the reception quality of the video data from being lowered as much as possible. This is particularly effective for receiving streaming packets of video data.

  According to the various embodiments of the present invention described above, those skilled in the art can easily make various changes, modifications and omissions within the scope of the technical idea and the viewpoint of the present invention. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

It is a system configuration diagram in the present invention. It is a flowchart of the priority allocation part in this invention. It is explanatory drawing showing the priority allocated to the image by this invention.

Explanation of symbols

1 Video Data Transmission Device 101 Encoding Unit 102 Priority Allocation Unit 103 Packet Configuration Unit

Claims (5)

In a video data transmission device that transmits video data,
Encoding means for generating at least an intra-frame encoded image and an inter-frame encoded image from the video data;
A priority assigning unit that assigns a higher priority to the intra-frame encoded image than the inter-frame encoded image, and assigns different priorities according to the transmission order of frames in the inter-frame encoded image;
Constitute a packet including information constituting the image, have a packet structure unit including the priority in the header of the packet,
The priority allocating means sets the number of inter-frame encoded images sandwiched between two intra-frame encoded images as I_int, the number of inter-frame encoded image priorities as N_prio, and the priority is absolute The higher the value, the lower the priority, the highest priority for the inter-frame coded image is PRI, the variable parameter i = 1,
If I_int ≦ N_prio,
The process of assigning (PRI + I_int-1 + i-1) from the intra-frame encoded image to the i-th inter-frame encoded image is repeated while incrementing i by 1 until i exceeds N_prio. Video data transmission device. The encoding means performs encoding based on MPEG (Moving Picture Experts Group),
The video data transmitting apparatus according to claim 1, wherein the intra-frame encoded image is an I picture, and the inter-frame encoded image is a P picture. The packet configuration unit, IP for (Internet Protocol) packet, the video data transmitting apparatus according to claim 1 or 2, characterized in that inclusion of the priority Diff-Serv field of the IP header. In a program for causing a computer mounted on a device for transmitting video data to function,
Encoding means for generating at least an intra-frame encoded image and an inter-frame encoded image from the video data;
A priority assigning unit that assigns a higher priority to the intra-frame encoded image than the inter-frame encoded image, and assigns different priorities according to the transmission order of frames in the inter-frame encoded image;
Configuring a packet including information constituting an image, and causing the computer to function as a packet configuration unit including the priority in a header of the packet ;
The priority allocating means sets the number of inter-frame encoded images sandwiched between two intra-frame encoded images as I_int, the number of inter-frame encoded image priorities as N_prio, and the priority is absolute The higher the value, the lower the priority, the highest priority for the inter-frame coded image is PRI, the variable parameter i = 1,
If I_int ≦ N_prio,
Until i exceeds N_prio, the process of assigning (PRI + I_int-1 + i-1) from the intra-frame encoded image to the i-th inter-frame encoded image is repeated while incrementing i by 1.
A program characterized by causing a computer to function . In the video data transmission method in the video data transmission device,
A first step of generating at least an intra-frame encoded image and an inter-frame encoded image from the video data;
A second step of assigning a higher priority to the intra-frame encoded image than the inter-frame encoded image, and assigning a different priority according to the transmission order of frames in the inter-frame encoded image;
Forming a packet including information constituting the image, and including the priority in the header of the packet,
In the second step, the number of inter-frame encoded images sandwiched between two intra-frame encoded images is I_int, the number of inter-frame encoded image priorities is N_prio, and the priority is an absolute value thereof. , The priority becomes lower, the highest priority for the inter-frame coded image is PRI, the variable parameter i = 1,
If I_int ≦ N_prio,
The process of assigning (PRI + I_int-1 + i-1) from the intra-frame encoded image to the i-th inter-frame encoded image is repeated while incrementing i by 1 until i exceeds N_prio. Video data transmission method.

Images