JP2001078181A - Method for transmitting packet and device for receiving and reproducing packet - Google Patents

Abstract

PROBLEM TO BE SOLVED: To discriminate contents included in picture data and to judge the efficient decoding or abandoning of a packet by adding the division rule of picture data to the header part of an RTP packet to transmit. SOLUTION: A data transmitting device 100 inputs picture data which are recorded in a recording device 101 or generated by a picture encoding device 102. A division control part 104 obtains the number of video packets(VP) included in segmented succeeding picture data, the encoding the type of each VP and a position from the first byte of segmented picture data. A data dividing part 105 segments picture data from an input buffer by size indicated by the part 104 and outputs it to a header multiplexing part 106. The part 106 adds a data start type indicated by the part 104, the number of VP, the encoding type of each VP and its start position to a header, besides, multiplexes picture data which are inputted from the part 105 and outputs them as packet.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a packet transmission method, such as MPEG-4 or H.264. Image data encoded by an image compression encoding method such as R.263 (real-time t
This is related to a method of transmitting a packet by using a transport protocol.

[0002]

2. Description of the Related Art At present, a method of transmitting image data encoded by MPEG-4 is described in IETF (Internet / Internet).
Engineering Task Force), and MPE
G (Moving Picture Experts Group) is in the process of being standardized, and is described in Document 1: "RTP Payload Format for
MPEG-4 Streams "(INTERNET-
DRAFT, draft-ietf-avt-rtp-
mpeg4-01. txt, issued February 25, 1999).

FIG. 7 shows a configuration of an RTP packet shown in Reference 1. The RTP packet 701 includes a header 702 and a payload 703. The header 702 includes an RTP version number 702a, a payload type 702c for identifying the type of data, a time stamp 702e indicating a data display time, and other areas. The payload 703 includes an SL packet defined by MPEG-4. Payload 703 is
It is specified that one SL packet is always included.

[0004] The configuration of the SL packet is described in Reference 2: "ISO
/ IEC JTC 1 / SC29 / WG11 N250
1, ISO / IEC14496-1, Informat
ion Technology-Coding of
audio-visualobjects-Part-
1 Systems "(Issued December 18, 1998)
It is described in. FIG. 8 shows the structure of a typical SL packet. The SL packet 801 has a header 8
02, and a payload 803. Header 802
Are present when the access unit start flag 802a, the access unit presence / absence flag 802c present when the access unit start flag is 1, the display time presence / absence flag 802d, and the decode time presence / absence flag indicate 1. Decoding time 802f, the display time 802g existing when the display time presence / absence flag indicates 1, and other areas. Here, an access unit is one frame of image data.

The structure of MPEG-4 image data is described in Document 3: "ISO / IEC JTC 1 / SC29 / WG1".
1 N2502, ISO / IEC14496-2, In
formation Technology-Codi
ng of audio-visual object
s-Part-2 Visual "(issued December 18, 1998). FIG. 9 shows a typical configuration of image data. As shown at 901,
The image data includes a VOL (video object layer) header 902 and a VOP (video object plane) 903. VOL header 902
Is setting information including information necessary for decoding and displaying image data, such as information on the width and height of image data or a table for determining the number of quantization steps for decoding. V
OP903 is the same as that generally called a frame, and corresponds to one frame in a moving image. Also, V
OP includes a plurality of video packets 9 as shown in FIG.
03a to 903c. FIG. 10 is a diagram for explaining the relationship between the video packet and the decoded display of the frame. As shown in FIG. 10A, the video packet is 1
Although it forms part of a frame, as shown in the example shown in (b), even if the video packet 1001b is lost due to a transmission error or the like, other video packets are decoded and displayed, and Decoding display is possible.

FIG. 11 shows a process of RTP packetization of image data as described in FIG. The image data (a) is divided into access units (b). Each access unit is added with an SL packet header, and becomes an SL packet (c). When the size of the access unit is large, such as several kilobytes, one frame data can be divided into a plurality of SL packets. At this time, the access unit start flag is set to 1 only for SL packets including the head data of the access unit, and the flag is set to zero for other SL packets. Also, the SL packet can be configured in video packet units. Again,
The access unit start flag is set to 1 only for the head data of each access unit, that is, the access unit including the first video packet of each image frame, and the flag is set to zero for SL packets including the second and subsequent video packets. set. In addition, the RTP
An RTP packet (d) is generated by adding a packet header.

[0007]

SUMMARY OF THE INVENTION As described above, the conventional M
The following problems occur in the packet transmission method using RTP for PEG-4 image data.

First, when one VOP is divided into a plurality of SL packets by an SL packet, whether the SL packet is divided in video packet units or not is determined.
Cannot be determined. When divided into video packets and converted into SL packets, as shown in FIG. 10B, even if some SL packets are lost, display decoding of a part of an image frame becomes possible. However, since it is not known whether the SL packet has a start point of the video packet, a process of searching for the SL packet payload is necessary. Also, information for identifying the presence or absence of a video packet does not exist in the RTP packet header. Therefore, the receiving terminal that receives and decodes and displays the packet data cannot determine the presence or absence of the video packet unless the data terminal of the SL packet is searched, and there is a concern that the processing load increases.

Second, there is no information indicating the presence of the VOL header in either the RTP packet header or the SL packet header. The receiving terminal cannot decode the image data unless the received RTP packet contains a VOL header. Unless the data part of the SL packet is searched,
Since it is not possible to determine the presence or absence of a VOL header, this also increases the processing load on the terminal.

Third, one SL packet, that is, RTP
It is possible to include a plurality of video packets in the packet. In this case, information such as the number of video packets included, the start position of each video packet, and the encoding type of the video packet includes the SL packet header and the RTP packet header. None specified. For example, when the load on the receiving terminal is heavy, it is conceivable to selectively decode and display only some of the received packets. However, at this time, an index is not given as to which packet is to be decoded and displayed. The selection may not be efficient.

[0011]

SUMMARY OF THE INVENTION In view of the above problems, a packet transmission method according to the present invention (claim 1) has
The TP packet is added with an N-bit area for identifying the division rule of the image data, that is, what type of data the payload data starts with, in the header of the TP packet, and is transmitted.

[0012] In the packet transmission method according to the present invention (claim 2), the image data division rule may be started from setting information necessary for decoding and displaying image data, such as a VOL header, or a frame of image data. N bits of N bits capable of identifying at least one of a start, a start of a slice of image data, a start of a macroblock of image data, or a start of a video packet of image data defined in MPEG-4. The transmission is performed by adding an area.

[0013] In the packet transmission method according to the present invention (claim 3), an N-bit area for identifying whether setting information necessary for decoding and displaying image data such as a VOL header is included in a packet. It is transmitted in addition.

[0014] In the packet transmission method according to the present invention (claim 4), an N-bit area designating the number of image frames included in the packet is transmitted.

[0015] A packet transmission method according to the present invention (claim 5) adds an N-bit area specifying the number of video packets defined in MPEG-4 included in the packet and transmits the packet.

[0016] In the packet transmission method according to the present invention (claim 6), when the packet includes the head data of the video packet, the packet is transmitted by adding an N-bit area designating the position of the head data of the video packet. Things.

[0017] In the packet transmission method according to the present invention (claim 7), when the packet includes the head data of the image frame, N specifies the position of the head data of the image frame.
A bit area is added and transmitted.

[0018] In the packet transmission method according to the present invention (claim 8), the video packet included in the packet is specified whether it is predictively coded or intra-screen coded. For transmission with an additional N-bit area.

A packet transmission method according to the present invention (claim 9) is characterized in that the packet transmission method according to claims 1 to 8 is an RTP packet particularly defined by IETF.

A packet receiving / reproducing apparatus according to the present invention (claim 10) receives a packet sequence generated by the packet transmission method according to claims 1 to 9, and includes a header attached to each packet. The decoding of the image data in the packet or the discarding of the packet while referring to at least one of the areas from (1) to (8).

The packet receiving / reproducing apparatus according to the present invention (claims 11 and 13) digitizes the load of the decoding process of the image data, and when the numerical value exceeds a predetermined threshold value,
If the image data is prediction coded image data, it is discarded without decoding.

The packet receiving / reproducing apparatus according to the present invention (claims 12 and 14), when the packet containing the reference image data necessary for decoding the image data contained in the packet cannot be correctly received, This is to discard the packet without decoding it.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a packet transmission method according to the present invention will be described below.

(Embodiment 1) A data transmission apparatus according to an embodiment of the packet transmission method of the present invention will be described.
This will be described with reference to FIGS.

FIG. 1 shows a schematic configuration of a data transmission apparatus. The recording device 101 records image data. Further, the image encoding device 102 outputs image data.
A data transmission device 100 that receives image data from these devices and generates and outputs a packet includes an input buffer 103 that receives and holds image data, a division control unit 104 that outputs a division instruction according to a predetermined image division rule. ,
It is composed of a data division unit 105 that cuts out and outputs image data in accordance with a control signal from the division control unit 104, and a header multiplexing unit 106 that multiplexes a header on the image data and outputs it as a packet. FIG. 2 shows a configuration of a packet generated in the header multiplexing unit. FIG. 3 is a table used in the division control unit 104. FIG. 4 is a flowchart illustrating the operation of data transmission apparatus 101. FIG. 5 shows an example of a packet sequence output by the packet output unit.

Hereinafter, the operation of the data transmission apparatus according to the present embodiment will be described with reference to the flowchart of FIG.
The flowchart in FIG. 4 includes six steps S401 to S406, and shows a procedure for creating one RTP packet including a part of image data. Image data is sequentially converted into RTP packets by repeating steps S401 to S406.

S401 (Step 401): The data transmission apparatus 100 inputs image data recorded in the recording apparatus 101 or generated by the image encoding apparatus 102. The recording device is composed of, for example, an optical / magneto / magnetic disk, tape, semiconductor memory, etc.
-4 for recording the image data compressed and encoded, and the recorded image data can be read out as needed. On the other hand, the image coding apparatus inputs uncompressed image data or non-MPEG-4 format image data,
This is a device that compresses and encodes with PEG-4 and outputs it. The image data input to the data transmission device is input buffer 1
03 is temporarily held.

S402 (step 402): The division control unit 104 converts the image data stored in the input buffer
The start code and resync marker defined in MPEG-4 are searched. By detecting these codes and counting the number of bytes included between the codes, the sizes of the VOL header, VOP, video packet, and the like can be obtained. The transmission device uses RTP in advance.
Limit packet size to a certain range. For example, 1
The range is from 000 bytes to 5000 bytes (hereinafter referred to as condition A). Due to this restriction, it becomes necessary to divide one VOP or video packet into a plurality of RTP packets, or to group a plurality of VOPs or video packets into one RTP packet. The division control unit divides the image data so that the VOL header, the VOP, and the video packet fall within the above restrictions. Now, it is assumed that the image data shown in FIG. The division control unit includes a VO
Count the size of the L header (500 bytes),
Since the condition A is not satisfied, the search of the input buffer is continued. Next, the size (3500 bytes) of the video packet forming a part of the VOP is counted. The total size of the VOL header and video packet is 4000 bytes,
In order to satisfy the condition A, the size (4
000 bytes) and the data start type (VOL header). In the present embodiment, as shown in the table of FIG. 3A, four types of data start types are defined in advance, and when the identification value is 0, the start of the VOL header, when 1, the start of the VOP, and In the case of, the video packet starts, and in the case of 3, it is other than the above. Therefore, 0 (VOL header) is currently selected as the value of the data start type.

S403 (Step 403): The division control unit determines the number of video packets (VP) included in the next clipped image data, the encoding type of each VP, and the number of video packets (VP) from the first byte of the clipped image data. Find the position. Each V
As a method for obtaining the encoding type of P, an encoding type of VOP or the like can be used. In this embodiment, as shown in the table of FIG. 3B, three types of coding types are defined in advance, and when the identification value is 0, intra-frame coding is performed, and when the identification value is 1, unidirectional predictive coding is performed. 2, the bidirectional prediction coding is performed. Now, when the image data of H501 is targeted, the number of video packets is 1, and the start position is 500. The encoding type here is 0
(Intra-screen coding).

S404 (step 404): The data division unit 105 cuts out image data from the input buffer by the size specified by the division control unit, and
Output to Now, when the image data of H501 is targeted, 4000 bytes are cut out and output.

S405 (Step 405): The header multiplexing unit adds the data start type, the number of video packets, the same encoding type, and the same start position specified by the division control unit to the header, and further adds the image input from the image division unit. It is multiplexed with data and output as a packet. Now, when the image data of H501 is targeted, these values are 0, 1,
0, 500. H502 in FIG. 5B shows a part of the header at this time. As shown in the figure, information such as the data start type, the number of video packets, the same encoding type, and the same start position are specified in the header. Have been.

In the data transmission apparatus 100,
This is a procedure for generating one RTP packet. FIG.
When the image data of (a) is sequentially processed according to the above procedure, the second RTP packet is composed of the image data indicated by H503 and the header indicated by H504, and the third RTP packet is
The TP packet is composed of image data indicated by H505 and a header indicated by H506.

The data transmission apparatus according to the embodiment of the packet transmission method of the present invention has been described above.

In the present embodiment, the data start type, the number of video packets, the same encoding type,
Although the start position is designated, a part of the information may be omitted. For example, the information added to the header can be a pattern such as only the data start type or only the data start type and the number of video packets. As information to be added to the header, in addition to the above, the number of VOPs, the encoding type of the VOP, and each VOP when one RTP packet includes a plurality of VOPs.
May be information such as the start position or size. Further, a flag set when including setting information necessary for decoding or display, such as a VOL header, may be included.

Further, in the present embodiment, only the start of a VOL header, the start of a VOP, and the start of a video packet are used as predetermined data division rules, but the present invention is not limited to this. It may be the start of a macro block.

Further, in the present embodiment, the image data is included as the RTP packet payload. However, the present invention is not limited to this, and the SLTP packet image data defined in MPEG-4 may be included.

(Embodiment 2) An embodiment of a packet receiving and reproducing apparatus according to the present invention will be described with reference to FIG. FIG. 6 is a schematic diagram of a packet reception / reproduction device.
It comprises an input buffer 601, a header analysis unit 602, a data decoding unit 603, and a display unit 604. Hereinafter, the operation will be described.

In FIG. 6, reference numeral 100 denotes the data transmission device described in the first embodiment, which outputs the RTP packet sequence including the image data shown in FIG. The packet reception / reproduction device 600 receives this packet sequence and temporarily stores it in the input buffer 601. The header analysis part is RT
The header of P is analyzed, and whether to decode the packet to be analyzed is determined. At the time of the decision, whether or not the packet immediately before the packet to be analyzed is correctly received is used as an index. If the immediately preceding packet is not correctly received, the header of the received packet shown in FIG. 2 is checked, and if the value of the video packet encoding type (VP encoding type) shown in FIG. Discards the received packet. Further, decoding or discarding is determined depending on the processing load of the data decoding unit. The processing load of the data decoding unit is quantified according to a predetermined rule. If the numerical value exceeds a predetermined threshold, the header of the received packet shown in FIG. 2 is checked, and the VP coding type shown in FIG. When the value of is 1 or 2, the received packet is discarded. Note that as a predetermined rule for quantifying the processing load of the data decoding unit, a difference between a time stamp indicated by the header and a time at which decoding is completed can be used. If this difference is large, that is, if the time until the actual display is available at the time when the data decryption ends,
This indicates that the data decoding unit has room for processing. If the difference is small, conversely, there is no margin, and if the difference value indicates a negative value, the decoding process is not in time. An appropriate threshold value is about 0.1 second to 1 second.

When the header analysis unit determines to decode the packet, it instructs the data decoding unit 603 to perform a data decoding process. The data decoding unit decodes the data input from the input buffer and outputs the data to the display unit 604. The display unit displays the decoded image frame.

The data transmission apparatus according to the embodiment of the packet transmission method of the present invention has been described above.

In the present embodiment, data decoding or discarding is determined. However, the present invention is not limited to this. A request for retransmission of a packet is made to the data transmission apparatus with reference to header information. It is also possible to send out.

[0042]

As described above, according to the data transmission method according to the present invention, image data is input, the image data is divided according to a predetermined rule, a header is added to create a packet, and the packet is transmitted. In addition, by specifying the rule of division in the header and specifying the number of image frames in the packet or the number of video packets in the header and specifying By specifying the start position of the frame or the start position of the video packet by adding it to the header, the video packet in the packet is either predictively coded or intra-screen coded. By explicitly identifying the header by referring to the header, the packet receiving / reproducing apparatus refers to the header without searching for image data in the packet. Allows determination of the content image data comprises by, decoding of efficient packet, or allows the determination of such discarded.

[Brief description of the drawings]

FIG. 1 is a schematic diagram of a configuration of a data transmission device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration of a packet generated in a header multiplexing unit 106 for describing the data transmission apparatus according to the first embodiment of the present invention.

FIG. 3 is a diagram illustrating a table referred to in a division control unit 104 for describing the data transmission apparatus according to the first embodiment of the present invention.

FIG. 4 is a flowchart for explaining the operation of the data transmission device according to the first embodiment of the present invention;

FIG. 5 is a diagram showing an example of input image data and an RTP packet sequence used to describe the data transmission device according to the first embodiment of the present invention.

FIG. 6 is a schematic diagram of a configuration of a packet reception / reproduction device according to a second embodiment of the present invention.

FIG. 7 is a diagram showing a configuration of an RTP packet;

FIG. 8 is a diagram showing a configuration of an SL packet.

FIG. 9 is a diagram illustrating a configuration of MPEG-4 image data.

FIG. 10 is a schematic diagram for explaining the concept of a video packet defined by MPEG-4.

FIG. 11 is a diagram used to explain an example in which image data is converted into RTP packets.

[Explanation of symbols]

REFERENCE SIGNS LIST 100 data transmission device 101 recording device 201 RTP header (basic region) 202 RTP header (extended region) 203 RTP payload 204 pair region repeatedly appearing by the number of VPs 600 packet receiving / reproducing device 701 RTP packet 702 RTP packet header 702a to 702f Elements of RTP packet header 703 RTP packet payload 801 SL packet 802 SL packet header 802a to 802h Elements of SL packet header 803 SL packet payload 901 Image data 1001a Display image for video packet 903a 1001b Display image for video packet 903b 1001c Video packet Display image for 903c VOL Video object layer (header) VOP1, VOP2 Video object plane 1, 2 VP Video packet H501, H503, H505 RTP packet H502, H504, H506 RTP packet header

Continued on front page F term (reference) 5C056 BA01 DA06 DA11 DA20 5C059 KK25 MA00 MA05 RB02 RB09 SS06 TA00 TC00 TD12 UA05 UA38 5K028 EE03 KK32 MM05 MM12 SS14 5K030 HB02 JA01 JA05 JT04 KA19 LC18

Claims (14)

[Claims] 1. A packet transmission method for inputting image data, dividing the image data according to a predetermined rule, creating a packet by adding a header to each of the divided data, and transmitting the packet. The header includes N indicating the content of the predetermined rule.
A packet transmission method comprising a bit area (N is an integer of 1 or more). 2. The method according to claim 1, wherein the predetermined rule includes a start of setting information necessary for decoding or displaying data included in the image data, a start of a frame of the image data, a start of a slice of the image data, and a start of a slice of the image data. The packet transmission method according to claim 1, further comprising one or more of a start of a macro block and a start of a video packet defined by MPEG-4. 3. A packet transmission method for inputting image data, dividing the image data, adding a header to each of the divided data to create a packet, and transmitting the packet, wherein the header is Packet transmission characterized by including an N-bit area (N is an integer of 1 or more) for determining that data in a packet includes setting information necessary for decoding or displaying image data. Method. 4. A packet transmission method for inputting image data, dividing the image data, adding a header to each of the divided data to create a packet, and transmitting the packet, wherein the header is And a N-bit area (N is an integer of 1 or more) for designating the number of image frames included in the packet. 5. A packet transmission method for inputting image data, dividing the image data, adding a header to each of the divided data, creating a packet, and transmitting the packet, wherein the header is And a N-bit area (N is an integer of 1 or more) for designating the number of video packets defined by MPEG-4 included in the packet. 6. A packet transmission method for inputting image data, dividing the image data, adding a header to each of the divided data to create a packet, and transmitting the packet, wherein the header is When the packet includes the head data of a video packet defined by MPEG-4, the packet includes an N-bit area (N is an integer of 1 or more) that specifies the position of the head data of the video packet. Packet transmission method. 7. A packet transmission method for inputting image data, dividing the image data, adding a header to each of the divided data to create a packet, and transmitting the packet, wherein the header is A packet transmission method comprising: when a packet includes head data of an image frame, the packet includes an N-bit area (N is an integer of 1 or more) that specifies the position of the head data of the image frame. 8. A packet transmission method for inputting image data, dividing the image data, adding a header to each of the divided data, creating a packet, and transmitting the packet, wherein the header is , An N-bit area (where N is 1) for designating whether a video packet defined by MPEG-4 contained in the packet is a prediction-encoded video or an intra-picture-encoded video packet A packet transmission method comprising: 9. The method according to claim 1, wherein the packet is an RTP packet defined by IETF.
The packet transmission method according to any one of the above. 10. A packet sequence generated by the packet transmission method according to claim 1 is received,
In the header added to each of the received packets, an N-bit area according to any one of claims 1 to 8,
A packet receiving and reproducing apparatus for decoding and reproducing image data in a packet or discarding a packet while referring to at least one area. 11. A packet receiving and reproducing apparatus for receiving, decoding, and displaying a packet sequence including image data, wherein the load of image data decoding processing is quantified, and when the numerical value exceeds a predetermined threshold value, A packet receiving and reproducing apparatus for discarding predicted encoded image data without decoding it. 12. A packet receiving / reproducing apparatus for receiving, decoding, and displaying a packet sequence including image data, wherein the packet receiving / reproducing apparatus correctly decodes a packet including reference image data necessary for decoding image data included in the packet. A packet reception / playback apparatus, wherein when the packet cannot be received, the packet is discarded without decoding. 13. The packet receiving and reproducing apparatus according to claim 11, wherein the packet receiving and reproducing apparatus receives a packet sequence generated by the packet transmitting method according to claim 1. 14. A packet receiving / reproducing apparatus according to claim 12, which receives a packet sequence generated by the packet transmission method according to any one of claims 1 to 9.