JP2009153198A - Moving picture data reception apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving picture data reception apparatus capable of more surely receiving a macroblock including a significant portion. <P>SOLUTION: The moving picture data reception apparatus includes: a packet receiving unit for sequentially receiving a packet including a macroblock obtained by dividing an image frame; a significance receiving unit for receiving macroblock significance indicative of a weight imparted to the macroblock; a significance determining unit for determining specific macroblock significance of a macroblock included in a specific packet that cannot be received by the packet receiving unit but is lost, on the basis of the macroblock significance of a plurality of macroblocks received by the significance receiving unit; and a retransmission requesting unit for requesting retransmission of the specific packet when the specific macroblock significance determined by the significance determining unit exceeds a prescribed threshold. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a moving image data receiving apparatus that receives moving image data, and more particularly to a moving image data receiving apparatus that can more reliably receive an important part of a moving image on a screen.

  It is known that a moving image communication using a moving image encoding method such as MPEG generally has a higher processing load than normal data communication and voice communication.

  Therefore, various methods have been proposed and implemented in order to reduce the transmission (processing) delay of moving image data due to such a high processing load.

For example, by generating header information of moving image data in units of macroblocks encoded based on MPEG-4 / FGS (Fine Granularity Scalability) earlier than the moving image data, the transmission delay of moving image data is reduced. A reduction method is disclosed (see Patent Document 1).
JP 2004-343511 A (pages 11-13, FIG. 3)

  By the way, the moving image data generally includes an “important portion” that has a motion on the screen (image frame) and is preferably transmitted to the transmission destination, and other portions other than the important portion. It is.

  However, in the conventional method described above, a macroblock including an “important part” and a macroblock including other parts other than the important part are handled in the same manner. For this reason, there is a problem that a macroblock including an important part cannot be transmitted differently from other macroblocks.

  That is, in the above-described conventional method, a macroblock including an important part is treated differently from other macroblocks (for example, a macroblock including an important part is transmitted a plurality of times). There was a problem that it could not be received more reliably.

  Therefore, the present invention has been made in view of such a situation, and an object thereof is to provide a moving image data receiving apparatus that can more reliably receive a macroblock including an important part.

  A feature of the present invention is that a packet receiving unit that sequentially receives a packet including a macroblock obtained by dividing an image frame, an importance receiving unit that receives a macroblock importance indicating a weight assigned to the macroblock, Importance determination for determining a specific macroblock importance of a macroblock included in a specific packet that cannot be received by the packet receiver based on the macroblock importance of a plurality of macroblocks received by the importance receiver And a retransmission request unit that requests retransmission of the specific packet when the importance level of the specific macroblock determined by the importance level determination unit exceeds a predetermined threshold.

  According to such a feature, retransmission of a specific packet is requested only when the degree of importance of a specific macroblock exceeds a predetermined threshold. That is, since retransmission of a packet including a macroblock that is not very important for reproduction of moving image data is not required, the macroblock decoding process can be executed at a higher speed.

  Furthermore, since retransmission of the packet is not requested, the amount of packets transmitted / received via the communication network can be reduced. In particular, the loss of packets is likely to occur when the communication network is congested. Therefore, reducing the number of retransmitted packets contributes to improving the congestion of the communication network.

  According to the characteristics of the present invention, it is possible to provide a moving image data receiving apparatus that can more reliably receive a macroblock including an important part.

[First Embodiment]
Next, a first embodiment of the present invention will be described. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, it should be noted that the drawings are schematic and ratios of dimensions are different from actual ones.

  Accordingly, specific dimensions and the like should be determined in consideration of the following description. Moreover, it is a matter of course that portions having different dimensional relationships and ratios are included between the drawings.

(Logical block configuration of moving image data transmission device)
FIG. 1 shows a logical block configuration of a moving image data transmission apparatus 100 according to the present embodiment. Hereinafter, portions related to the present invention will be mainly described. Therefore, it should be noted that the moving image data transmission apparatus 100 may include a logical block (such as a power supply unit) that is not shown in the drawing or that is omitted in order to realize the functions of the moving image data transmission apparatus 100. .

  As shown in the figure, the moving image data transmission apparatus 100 is connected to the communication network 10 by wire or wireless.

  The communication network 10 is a communication network for transmitting IP packets including moving image data transmitted by the moving image data transmission apparatus 100.

  The moving image data transmission apparatus 100 includes a moving image data acquisition unit 101, a motion vector calculation unit 103, a reference image block determination unit 105, a weighting unit 107, and a communication unit 109.

  The moving image data acquisition unit 101 acquires moving image data. Specifically, the moving image data acquisition unit 101 can acquire moving image data from a server (not shown) that can be accessed via the communication network 10.

  The moving image data acquisition unit 101 may include an access interface with a recording medium (for example, a DVD or a tape) and acquire moving image data recorded on the recording medium.

  The motion vector calculation unit 103 calculates a motion vector of a macroblock (16 images × 16 lines) included in an image frame.

Specifically, as shown in FIG. 3, the motion vector calculation unit 103 is reproduced after the position of the reference image element R on the image frame F _n (first image frame) and after the image frame F _n. Based on the position of the macro block MB (n + 1, k) (backward macro block) including the reference image element R on the image frame F _{n + 1} (second image frame), the motion vector of the macro block MB (n + 1, k) V (n + 1, k), that is, the moving direction and moving amount of the macroblock MB (n + 1, k) on the image frame are calculated.

  Note that the macroblock MB (n + 1, k) may include a part of the reference image element R instead of all.

  More specifically, the motion vector calculation unit 103 calculates a motion vector V (n + 1, k) using the luminance signal and the color difference signal constituting the macro block MB (n + 1, k). Note that the motion vector calculation unit 103 may calculate the motion vector V (n + 1, k) using either a luminance signal or a color difference signal.

The reference image block determination unit 105 illustrated in FIG. 1 is based on the motion vector V (n + 1, k) calculated by the motion vector calculation unit 103, and is a reference image that is an area on the image frame F _n including the reference image element R. The block SB (n + 1, k) (see FIG. 4) is determined.

  Specifically, the reference image block determination unit 105 determines a reference image block SB (n + 1, k) having the same size (16 pixels × 16 lines) as the macro block based on the motion vector V (n + 1, k). .

Weight assigning unit 107, the macro block MB in the image frame _{F n (n,} m) and the reference picture block SB (n + 1, k) is in accordance with the ratio of the overlapping area overlapping on the image frame F, the macro block MB ( n, m) is given a weight.

    Specifically, as illustrated in FIG. 5, the weight assigning unit 107 includes an overlapping region OL and a macroblock in which the macroblock MB (n, m) and the reference image block SB (n + 1, k) overlap on the image frame F. The ratio of the area occupied by MB (n, m) is given as a weight.

More specifically, the weight assigning unit 107 calculates the weight (W _s ) using (Equation 1).

  Here, SB (n + 1, k) ∩MB (n, m) is the area of the overlapping region OL where the reference image block SB (n + 1, k) and the macroblock MB (n, m) overlap.

For example, when the macro block MB (n, k) and the reference image block SB (n + 1, k) overlap, assuming that the motion vector V (n + 1, k) = (a, b), the macro block MB (n, k) The weight (W _s ) is calculated as follows.

  Further, the weight assigning unit 107 can determine the weight to be given to the macroblock MB (n, m) using the plurality of reference image blocks determined by the reference image block determining unit 105.

For example, as illustrated in FIG. 6, when there are two reference image blocks, that is, a reference image block SB (n + 1, i) and a reference image block SB (n + 1, j), the weight assigning unit 107 selects the macro block MB (n, Based on the overlap areas OL1 and OL2 with _m ), the weight (W _m ) based on the plurality of reference image blocks is calculated.

Specifically, the weight assigning unit 107 calculates a weight (W _m ) based on a plurality of reference image blocks using (Expression 2).

  The communication unit 109 illustrated in FIG. 1 provides a communication interface for connecting to the communication network 10. The communication unit 109 can appropriately include a communication interface depending on the configuration of the communication network 10. For example, the communication unit 109 can include a LAN interface such as 100BASE-TX.

Further, the communication unit 109 controls the macroblock transmission method based on the weight (for example, the weight (W _s )) given by the weighting unit 107. In the present embodiment, the communication unit 109 constitutes a transmission method control unit.

  Specifically, as illustrated in FIGS. 7A and 7B, the communication unit 109 connects via the communication network 10 when the weight value exceeds a predetermined threshold (for example, 1.0). The macro block is transmitted a plurality of times toward a transmission destination (not shown).

  More specifically, the communication unit 109, when the sum of weights assigned to a plurality of macroblocks included in the transmitted IP packet (packet) exceeds a predetermined threshold (for example, 1.0), The IP packet is transmitted a plurality of times toward the destination.

  Further, when the weight assigned to the macroblock is lower than a predetermined threshold (for example, 0.5), the communication unit 109 discards the macroblock without transmitting it.

  Further, as shown in FIGS. 8A and 8B, the communication unit 109 determines that the sum of the weights assigned to the plurality of macroblocks included in the transmitted IP packet is within a predetermined range (for example, 0). .8 to 1.2), it is also possible to generate an IP packet. A more specific operation of the communication unit 109 will be described later.

(Operation of moving image data transmission device)
Next, the operation of the moving image data transmission apparatus 100 according to this embodiment described above will be described. FIG. 2 is an operation flowchart of the moving image data transmission apparatus 100.

  As shown in the figure, in step S10, the moving image data transmission apparatus 100 calculates a motion vector of a macroblock (16 images × 16 lines) included in the image frame F.

Specifically, the moving picture data transmission apparatus 100, as described above, the image frame F _n and the position of the reference picture element R on the image frame F _{n +} reference image on ₁ to be reproduced later than the image frame F _n Based on the position of the macro block MB (n + 1, k) including the element R, the motion vector V (n + 1, k) of the macro block MB (n + 1, k) is calculated (see FIG. 3).

In step S20, the moving image data transmission apparatus 100, based on the calculated motion vector V (n + 1, k), the reference image block SB (n + 1, k) which is an area on the image frame F _n including the reference image element R. Is determined (see FIG. 4). Note that the moving image data transmission apparatus 100 can also determine a plurality of reference image blocks in step S20.

  In step S30, the moving image data transmission apparatus 100 determines whether the macro block MB (n, m) and the reference image block SB (n + 1, k) overlap on the image frame F according to the ratio of overlapping areas. The weight of (n, m) is calculated (see FIGS. 5 and 6).

Specifically, as described above, the moving image data transmission apparatus 100 calculates the weight (W _s ) (or weight (W _m )) using Expression 1 (or Expression 2).

In step S40, the moving image data transmission device 100 assigns the calculated weight (W _s ) (or weight (W _m )) to the macroblock MB (n, m).

  For example, as shown in FIG. 7A, the moving image data transmission apparatus 100 assigns calculated weights (such as “0.2” and “0.3” in the figure) to the macroblocks MB1 to MB4, respectively. To do.

  Note that the information indicating the weight may be expressed using appropriate bits of the data field constituting the macroblock MB (n, m).

    Next, as illustrated in FIG. 2, in step S50, the moving image data transmission apparatus 100 performs the processing on the image frame F in addition to the reference image block SB (n + 1, k) targeted in the processing of steps S30 and S40 described above. It is determined whether or not there is a reference image block overlapping with the macro block MB (n, m).

  If there is another reference image block that overlaps the macro block MB (n, m) (YES in step S50), the moving image data transmission apparatus 100 repeats the processing from step S30.

  When there is no other reference image block overlapping the macro block MB (n, m) (NO in step S50), in step S60, the moving image data transmission apparatus 100 determines the macro block MB (n, n, m) according to the assigned weight. m) is transmitted. The moving image data transmission apparatus 100 determines how to transmit the macroblock MB (n, m) according to the set content.

  For example, as shown in FIGS. 7A and 7B, the total weight assigned to the macroblocks MB1 to MB4 included in the IP packet P1 is “1.0”. Further, the total weight given to the macroblocks MB5 to MB8 included in the IP packet P2 is “0.7”.

  In this case, the moving image data transmission apparatus 100 transmits only the IP packet P1, and discards it without transmitting the IP packet P2.

  Also, as shown in FIGS. 8A and 8B, the moving image data transmission apparatus 100 allows the total weight assigned to a plurality of macroblocks included in the transmitted IP packet to be within a predetermined range ( For example, an IP packet can be generated so as to be 0.8 to 1.2).

  The IP packet P3 includes “four” macroblocks MB11 to MB14, and the total weight of the macroblocks is “1.0”.

  On the other hand, the IP packet P4 includes “five” macroblocks MB21 to MB25, and the total weight of the macroblock is “1.0”, which is the same as the IP packet P3.

  That is, the moving image data transmission apparatus 100 includes the number of macroblocks included in the IP packet so that the sum of the weights assigned to the plurality of macroblocks included in the transmitted IP packet is within a predetermined range. That is, the packet length is adjusted.

(Action / Effect)
According to the moving image data transmission apparatus 100 according to the present embodiment described above, the macroblock MB (n, m) and the reference image block SB (n + 1, k) are overlapped on the image frame F at the ratio of the overlapping area OL. Accordingly, a weight (W _s ) is assigned to the macroblock MB (n, m).

Furthermore, controlling a transmission method for a communication unit 109, based on the assigned weights _{(W s)} macroblock MB (n, m), specifically, if the weight _{(W s)} exceeds a predetermined threshold value, Since the macro block MB (n, m) is transmitted a plurality of times, the macro block including the important part can be transmitted more reliably.

According to the moving image data transmission apparatus 100, a plurality of reference image blocks (reference image blocks SB (n + 1, i) and SB (n + 1, j)) are used to give weights ( W _m ) is determined. For this reason, it is possible to more accurately determine a macroblock including an “important part” that is desirably transmitted to the transmission destination, that is, a macroblock having a motion on the image frame F.

  According to the moving image data transmission apparatus 100, the IP packet is transmitted so that the sum of the weights assigned to the plurality of macroblocks included in the transmitted IP packet (IP packet P3, P4) is within a predetermined range. Generated.

  For this reason, even if the IP packet transmitted by the moving image data transmission apparatus 100 is lost on the communication network 10 (so-called packet loss) or includes an error, the moving image data to be reproduced at the transmission destination The influence can be made almost constant.

  That is, even when the IP packet is lost without reaching the transmission destination, the degree of degradation of the moving image data can be suppressed to a certain level.

  According to the moving image data transmission apparatus 100, when the weight given to a macroblock is lower than a predetermined threshold, the macroblock can be discarded without being transmitted.

  For this reason, when the communication band (throughput) of the communication network 10 is limited, by deleting a macroblock having a relatively low weight, the image quality is not greatly deteriorated, and the communication band of the communication network 10 is matched. Transmission of moving image data can be realized.

(Example of change)
In the above-described embodiment, an example in which the macroblock transmission method is controlled based on the weight assigned to the macroblock has been described. However, the transmission method is not limited to the above-described embodiment.

  For example, QoS control such as priority control may be executed for IP packet transmission based on the weight assigned to the macroblock.

  In the above-described embodiment of the present invention, the ratio between the overlap area OL obtained using (Equation 1) and the area occupied by the macroblock MB (n, m) is used as a weight. An expression different from (Expression 1) may be used as long as the ratio of the area occupied by the block MB (n, m) is obtained.

  Furthermore, the calculation method of the weight given to a macroblock can be changed as follows.

FIG. 9 (a) and (b) is an image frame F _n, respectively show a macro block included in the image frame F _{n + 1} to be reproduced later than the image frame F _n. Here, the macro blocks included in an image frame _{F n} and A to I, the macroblocks contained in the picture frame _{F n + 1} and a to i.

  FIG. 10 illustrates reference image blocks SBa to SBi determined based on the motion vectors of the macro blocks a to i illustrated in FIG.

Moving picture data transmission apparatus 100, based on the proportion of the overlap region where the macroblock A~I the image on the frame _{F n} shown in FIG. 9 (a), and the reference image block SBa~SBi shown in FIG. 10 overlap The weights given to the macroblocks A to I are calculated.

For example, moving picture data transmission apparatus 100 calculates a weight _{W E} of (Equation 3) using a macro block E to be located in the center of the image frame _{F n} (MB (E)).

Here, for example, MB (E) ∩SB (a) is the area of the overlapping region where the macroblock A and the reference image block SBa overlap. Further, w _{a to} w _i can be set to arbitrary values depending on the content of moving image data handled by the moving image data transmission apparatus 100.

According to the calculation of the weight of the macroblock using (Expression 3), for example, w _a and w _b used for the macro blocks A, B, C, G, H, and I positioned in the vertical direction of the macro block E. _{, w c, w g, w} h, by increasing the value of w _i, can be transmitted the macroblock preferentially than the other macro blocks.

Furthermore, a weight may be calculated for an image frame (for example, up to two to three image frames ahead) reproduced after the image frame F _{n + 1} , and the weight may be reflected on w _a to w _i .

  According to such a method, it is possible to assign a weight to a macroblock reflecting the state of an image frame several frames ahead of the current image frame.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the present embodiment, the macroblock weight including the “important portion” is more reliably transmitted by using the macroblock weight more effectively.

(Logical block configuration of moving image data transmission device)
FIG. 11 shows a logical block configuration of the moving image data transmission apparatus 100A and the moving image data transmission apparatus 200 according to the present embodiment. Hereinafter, parts different from those of the first embodiment will be mainly described, and description of similar parts will be appropriately omitted.

(1) Moving picture data transmission device 100A
Compared to the moving image data transmission apparatus 100 (see FIG. 1) according to the first embodiment, the moving image data transmission apparatus 100A includes a packet importance degree transmission unit 111 and an error correction processing unit 113.

  The packet importance transmission unit 111 transmits the macroblock importance indicating the weight assigned to the macroblock to the transmission destination, specifically, the moving image data transmission apparatus 200.

  FIG. 17 illustrates an example of an IP packet transmitted from the moving image data transmission apparatus 100A to the moving image data transmission apparatus 200. As shown in FIG. 17, the IP packet P11 includes macroblocks MB31 to MB34.

  The packet importance transmitter 111 includes the weight values (0.2 / 0.3 / 0.1 / 0.4) of the macroblocks MB31 to MB34 in the macroblock importance W0. The packet importance transmitter 111 transmits the macroblock importance W0 added to the IP packet P11.

  The packet importance transmission unit 111 can also transmit the IP packet different from the IP packet including the macroblock including the macroblock importance of the macroblock.

  FIG. 18 shows another example of an IP packet transmitted from the moving image data transmission apparatus 100A to the moving image data transmission apparatus 200. As shown in FIG. 18, the IP packet P12 includes a macroblock importance W1 that is the sum of the weights of the macroblocks MB1 to MB4 included in the IP packet P1 (see FIG. 7A), and an IP packet P2 (FIG. 7). Macroblock importance W2 that is the total weight of the macroblocks MB5 to MB8 included in (b).

  The packet importance transmitter 111 can transmit the macroblock importance W1 and W2 to the IP packet P12 different from the IP packets P1 and P2 including the macroblocks (MB1 to MB8).

  The packet importance transmission unit 111 can also transmit the macroblock importance to the moving image data transmission apparatus 200 in a form different from the forms shown in FIGS. 17 and 18.

  19A to 19C show other examples of IP packets transmitted from the moving image data transmission apparatus 100A to the moving image data transmission apparatus 200. FIG.

  As shown in FIG. 19A, IP packets P21, P22, and P23 are sequentially transmitted from the moving image data transmission apparatus 100A to the moving image data transmission apparatus 200.

  IP packet P21 includes macroblocks MB31 to MB34. Similarly, IP packet P22 includes macroblocks MB35 to MB38, and IP packet P23 includes macroblocks MB39 to MB42. Further, macroblock importance levels W21 to W23 are added to the IP packets P21 to P23, respectively.

  FIG. 19B shows the contents of the macroblock importance W22 added to the IP packet P22. The macroblock importance W22 is assigned to the macroblock importance W2 that is the total weight (0.7) of the macroblocks MB35 to MB38 included in the IP packet P22, and the IP packet P21 transmitted before the IP packet P22. And the macroblock importance W1 that is the total weight (1.0) of the included macroblocks MB31 to MB34.

  In other words, the packet importance transmitter 111 transmits the IP packet P22 including the macroblock importance W1 of the macroblocks MB31 to MB34 included in the IP packet P21 (forward packet) transmitted before the IP packet P22. can do.

  Further, the packet importance transmission unit 111 can transmit a macroblock importance W22A as shown in FIG. 19C instead of the macroblock importance W22. The macroblock importance W22A is a macroblock importance W3 that is the sum (1.3) of the weights of the macroblocks MB39 to MB42 included in the IP packet P23 transmitted after the IP packet P22. Including.

  That is, the packet importance transmitter 111 transmits the IP packet P22 including the macroblock importance W3 of the macroblocks MB39 to MB42 included in the IP packet P23 (backward packet) transmitted after the IP packet P22. You can also.

  The error correction processing unit 113 adds an error correction code, specifically an FEC (forward error correction) code, to the IP packet transmitted to the moving image data transmission apparatus 200.

  FIG. 20 shows a specific configuration example of an IP packet transmitted by the moving image data transmission apparatus 100A. As shown in FIG. 20, the IP packet P10 includes an IP header, a UDP header, an RTP header, a weight W (macroblock importance), image data (macroblock), and an FEC code.

  In particular, in this embodiment, the error correction processing unit 113 adds an FEC code to the IP packet P10 based on the weight (W) value assigned to the macroblock included in the transmitted IP packet P10. Whether or not to do so can be determined. In the present embodiment, the error correction processing unit 113 constitutes a determination unit.

  Further, when the error correction processing unit 113 determines that the FEC code is not added to the IP packet P10, the communication unit 109 according to the present embodiment transmits the IP packet P10 without the FEC code added.

  As shown in FIG. 20, the weight W (macroblock importance) is not limited to being added as the header of the IP packet P10, but may be added to the header portion of the image data.

(2) Moving picture data transmission device 200
As illustrated in FIG. 11, the moving image data transmission apparatus 200 receives the IP packet transmitted by the moving image data transmission apparatus 100 </ b> A via the communication network 10. Furthermore, the moving image data transmission apparatus 200 performs a decoding process of the macroblock included in the received IP packet, and reproduces the moving image data.

  The moving image data transmission apparatus 200 includes a communication unit 201, a packet importance processing unit 203, a moving image data processing unit 205, and an error correction processing unit 207.

  The communication unit 201 provides a communication interface for connecting to the communication network 10. The communication unit 201 can include an appropriate communication interface depending on the configuration of the communication network 10.

In particular, in the present embodiment, the communication unit 201 sequentially receives IP packets including macroblocks obtained by dividing an image frame (for example, the image frame F _n illustrated in FIG. 3). In the present embodiment, the communication unit 201 constitutes a packet reception unit.

  In addition, when the macroblock importance determined by the packet importance processing unit 203 exceeds a predetermined threshold, the communication unit 201 retransmits an IP packet (specific packet) including the macroblock to which the macroblock importance is given. Request. In the present embodiment, the communication unit 201 constitutes a retransmission request unit.

  The packet importance processing unit 203 executes a process related to the macroblock importance indicating the weight given to the macroblock.

  The packet importance processing unit 203 receives the macroblock importance included in the IP packet (for example, the IP packet P11 shown in FIG. 17) transmitted by the moving image data transmission apparatus 100A. In the present embodiment, the packet importance processing unit 203 constitutes an importance receiving unit.

  The packet importance processing unit 203 also determines the macroblock importance of the macroblock included in the IP packet (specific packet) lost by the communication unit 201 based on the macroblock importance of the received plurality of macroblocks. Determining the degree (specific macroblock importance). In the present embodiment, the packet importance level processing unit 203 constitutes an importance level determination unit.

Specifically, the packet importance processing unit 203 is based on the IP packet received by the communication unit 201, and weights of macroblocks (macroblock importance) constituting one image frame (for example, the image frame F _n ). The total value (reception total value) is calculated. Here, the total weight value (reference total value) for each image frame is substantially constant.

  Therefore, the packet importance processing unit 203 determines the weight (macroblock importance) of the macroblock included in the lost IP packet by subtracting the received total value from the reference total value.

  Further, the packet importance processing unit 203 can also receive an IP packet including only the macroblock importance (for example, the IP packet P12 shown in FIG. 18).

  Further, as shown in FIGS. 19A and 19B, the packet importance processing unit 203, when the lost IP packet is the IP packet P21 (specific packet), receives the IP packet received after the IP packet P21. The macroblock importance W1 of the macroblock included in the lost IP packet P21 can be determined using the macroblock importance W1 (specific macroblock importance) included in P22 (backward packet).

  Alternatively, as shown in FIGS. 19A and 19C, the packet importance processing unit 203, when the lost IP packet is the IP packet P23 (specific packet), the IP received before the IP packet P23 The macroblock importance W3 of the macroblock included in the lost IP packet P23 can be determined using the macroblock importance W3 (specific macroblock importance) included in the packet P22 (forward packet).

  The moving image data processing unit 205 executes a decoding process of the macroblock included in the IP packet received by the communication unit 201. In the present embodiment, the moving image data processing unit 205 constitutes a decoding processing unit.

  In particular, in the present embodiment, the moving image data processing unit 205 can change the content of the decoding process based on the macroblock importance received by the packet importance processing unit 203.

  For example, the moving image data processing unit 205 can omit at least a part of the decoding process when the macroblock importance is equal to or less than a predetermined threshold. Specifically, the moving image data processing unit 205 selects a macro having a weight greater than or equal to a predetermined threshold (for example, 0.3) among macroblocks (for example, macroblocks MB31 to MB34) corresponding to the macroblock importance. Decoding processing can be executed only for the block, or all decoding processing of the macroblock can be omitted.

  The error correction processing unit 207 performs error correction processing using the FEC code (see FIG. 20) included in the IP packet received by the communication unit 201. Note that the error correction processing unit 207 does not perform any error correction processing when the FEC code is not added to the IP packet.

(Operation of moving image data transmission device)
Next, operations of the moving image data transmission apparatus 100A and the moving image data transmission apparatus 200 according to the above-described embodiment will be described.

(1) Transmission of Macroblock Importance FIGS. 12A to 12C are flow charts of macroblock importance transmission operation in the moving image data transmission apparatus 100A.

(1.1) Transmission of Macro Block Importance Total Value As shown in FIG. 12A, in step S110, the moving image data transmission apparatus 100A uses the macro block weight total value included in the IP packet as a macro value. Calculated as block importance W1. For example, the moving image data transmission apparatus 100A calculates the macroblock importance W1 and W2 shown in FIG.

  In step S120, the moving image data transmission apparatus 100A transmits the calculated total value to the receiving side moving image data transmission apparatus 200 as the macroblock importance W1 and W2.

  Specifically, the moving image data transmission device 100A transmits the IP packet P12 including the macroblock importance W1 and W2 to the moving image data transmission device 200.

(1.2) Transmission of Part of Macroblock Importance The moving image data transmission apparatus 100A can execute the operation shown in FIG. 12B instead of the operation shown in FIG. As shown in FIG. 12B, in step S110A, the moving image data transmission apparatus 100A acquires a predetermined number of weights in descending order of the weights of the macroblocks included in the IP packet.

  For example, in the case of the IP packet P11 shown in FIG. 17, the moving image data transmission apparatus 100A determines the weight (0.4 and 0...) Of the macroblock MB34 having the largest weight and the macroblock MB32 having the second largest weight after the macroblock MB34. 3) is acquired.

  In step S120A, the moving image data transmission apparatus 100A transmits the predetermined number of weights (0.4 and 0.3) to the moving image data transmission apparatus 200 on the receiving side as the macroblock importance W0 (see FIG. 17). To do.

  Specifically, the moving image data transmission apparatus 100A transmits an IP packet P11 including the macroblock importance W0 to the moving image data transmission apparatus 200.

(1.3) Transmission of Specific Macroblock Importance The moving image data transmission apparatus 100A can further execute the operation shown in FIG. As illustrated in FIG. 12C, in step S110B, the moving image data transmission device 100A acquires the macroblock number having a larger (or smaller) weight among the weights of the macroblocks included in the IP packet.

  For example, in the case of the IP packet P11 shown in FIG. 17, the moving image data transmission apparatus 100A acquires the number of the macroblock MB34 having the largest weight.

    In step S120B, the moving image data transmission apparatus 100A transmits the macroblock number as the macroblock importance W0 (see FIG. 17) to the moving image data transmission apparatus 200 on the receiving side.

(2) Decoding Processing Using Macroblock Importance Next, operations related to decoding processing using macroblock importance will be described. Specifically, three operation examples in the moving image data transmission apparatus 200 on the receiving side will be described.

(2.1) Operation example 1
In this operation example, it is determined whether to request retransmission of a lost IP packet using the macroblock importance.

  As shown in FIG. 13, in step S210, the moving image data transmission apparatus 200 receives an IP packet (for example, lost in the communication network 10) based on the sequence number included in the IP packet received from the moving image data transmission apparatus 100A. , IP packet P11) is detected.

  In step S220, the moving image data transmission apparatus 200 acquires the macroblock importance of the macroblock included in the received IP packet.

  In step S230, the moving image data transmission apparatus 200 determines the macroblock importance of the macroblock included in the lost IP packet (IP packet P11) based on the acquired macroblock importance.

  Specifically, as described above, the moving image data transmission apparatus 200 determines the macroblock importance of the macroblock included in the received IP packet from the reference total value of the macroblock importance of the macroblock constituting the image frame. By subtracting the received total value, which is the total value of, the weight of the macroblock (macroblock importance) included in the lost IP packet is determined.

  In step S240, the moving image data transmission apparatus 200 determines whether the macroblock importance of the lost IP packet (macroblock) exceeds a predetermined threshold.

  If the macroblock importance exceeds a predetermined threshold (YES in step S240), in step S250, the moving image data transmission apparatus 200 requests retransmission of the lost IP packet.

  In step S260, the moving image data transmission apparatus 200 determines whether a lost IP packet has been received.

  When the lost IP packet is received (YES in step S260), in step S270, the moving image data transmission apparatus 200 performs a decoding process using the macroblock included in the retransmitted IP packet.

  When the lost IP packet cannot be received (NO in step S260), the moving image data transmission apparatus 200 repeats the processing from step S250.

  If the macroblock importance is equal to or lower than the predetermined threshold (NO in step S240), in step S280, the moving image data transmission apparatus 200 executes the decoding process without using the macroblock included in the lost IP packet. To do.

  As illustrated in FIG. 18, when the macroblock importance is transmitted by an IP packet P12 different from the IP packet including the macroblock, the moving image data transmission apparatus 200 uses the macroblock importance included in the IP packet P12. Based on the above, the processing from step S240 may be executed.

(2.2) Operation example 2
FIG. 14 is an operation flow chart executed in place of steps S210 to S230 in the flow according to the operation example 1 shown in FIG.

  Step S210A is the same as step S210 shown in FIG. In step S220A, the moving image data transmission apparatus 200 determines the macroblock importance (included in the IP packet (IP packet P22) received after the lost IP packet (for example, the IP packet P21 shown in FIG. 19A)) ( Macro block importance W22) is acquired.

  In step S230A, the moving image data transmission apparatus 200 determines the macroblock importance (macroblock importance W1) of the lost IP packet based on the acquired macroblock importance (macroblock importance W22).

  Specifically, as illustrated in FIG. 19B, the moving image data transmission apparatus 200 determines the macroblock importance of the lost IP packet P21 based on the macroblock importance W1 included in the macroblock importance W22. To decide.

  Note that, as shown in FIGS. 19A and 19C, the moving image data transmission apparatus 200 includes the IP packet P22 received before the IP packet P23 when the lost IP packet is the IP packet P23. The macroblock importance W3 can also be determined based on the macroblock importance W22A.

(2.3) Operation example 3
In this operation example, it is determined whether or not to perform a macroblock decoding process based on the received macroblock importance.

  As shown in FIG. 15, in step S310, the moving image data transmission apparatus 200 receives an IP packet from the moving image data transmission apparatus 100A.

  In step S320, the moving image data transmission apparatus 200 acquires the macroblock importance of the macroblock included in the received IP packet.

  In step S330, the moving image data transmission apparatus 200 determines whether the macroblock importance of the received IP packet (macroblock) exceeds a predetermined threshold.

  When the macroblock importance exceeds a predetermined threshold (YES in step S330), in step S340, the moving image data transmission apparatus 200 performs a decoding process using the macroblock included in the received IP packet.

  When the macroblock importance is equal to or less than the predetermined threshold (NO in step S330), in step S350, the moving image data transmission apparatus 200 executes the decoding process without using the macroblock included in the received IP packet.

(3) Omission of Error Correction Code FIG. 16 is a flowchart of an operation for adding an error correction code (FEC code) in the moving image data transmission apparatus 100A.

  As shown in FIG. 16, in step S410, the moving image data transmission apparatus 100A calculates the total value of the macroblock weights included in the IP packet as the macroblock importance.

  In step S420, the moving image data transmission apparatus 100A determines whether or not the calculated macroblock importance degree exceeds a predetermined threshold value.

  When the macroblock importance exceeds a predetermined threshold (YES in step S420), in step S430, the moving image data transmission apparatus 100A adds an FEC code to the IP packet.

  In step S440, the moving image data transmission device 100A transmits the IP packet to the moving image data transmission device 200 on the receiving side.

  When the macroblock importance is equal to or lower than the predetermined threshold (NO in step S420), the moving image data transmission apparatus 100A adds the IP packet to the moving image data on the receiving side without adding the FEC code to the IP packet. It transmits to the transmission apparatus 200.

(Action / Effect)
According to the moving image data transmission apparatus 100A and the moving image data transmission apparatus 200 according to the present embodiment described above, only when the macroblock importance (specific macroblock importance) of the lost IP packet exceeds a predetermined threshold, The retransmission of the IP packet is requested. That is, since retransmission of an IP packet including a macroblock that is not very important for reproduction of moving image data is not required, the moving image data transmission apparatus 200 can perform macroblock decoding processing at a higher speed.

  Furthermore, since retransmission of the IP packet is not requested, the amount of IP packets transmitted / received via the communication network 10 can be reduced. In particular, since loss of IP packets is likely to occur when the communication network 10 is congested, reducing the number of IP packets to be retransmitted contributes to improving the congestion of the communication network 10.

  According to this embodiment, macroblock importance can be transmitted by an IP packet (IP packet P12 shown in FIG. 18) different from the IP packet including the macroblock.

  For this reason, when the moving image data transmission apparatus on the receiving side supports processing using macroblock importance, the IP packet may be used. On the other hand, when the moving image data transmission apparatus on the receiving side does not support the processing, the IP packet including the macroblock is not different from the conventional one, and therefore the moving image data transmission apparatus can also operate normally.

  Further, as shown in FIGS. 19B and 19C, the macroblock importance of the macroblock included in the IP packet P23 (IP packet P21) transmitted after (before) the IP packet P22 is expressed as follows: It can be transmitted by being included in the IP packet P22.

  For this reason, even when a specific IP packet is lost, the macroblock importance of the lost IP packet can be quickly recognized.

  According to the present embodiment, when the macroblock importance is equal to or less than the predetermined threshold, at least a part of the decoding process of the macroblock is omitted, so that the decoding process can be executed at a higher speed. In particular, when the decoding process is executed in software, it is possible to reduce the load on the CPU and to more reliably execute a moving picture decoding process that is often delayed compared to voice. .

  According to the present embodiment, since the addition of the FEC code is omitted when the macroblock importance is equal to or less than a predetermined threshold, the amount of data transmitted to the communication network 10 while maintaining the error resistance of important macroblocks Can be reduced.

[Other Embodiments]
As described above, the contents of the present invention have been disclosed through the first and second embodiments of the present invention. However, it should not be understood that the description and drawings constituting a part of this disclosure limit the present invention. Absent. From this disclosure, various alternative embodiments will be apparent to those skilled in the art.

  For example, the above-described moving image data transmission apparatuses 100, 100A, and 200 (excluding the hardware part configuring the LAN interface) can be provided as a computer program that can be executed by a personal computer or the like.

  In the second embodiment described above, the moving image data transmission device 100A and the moving image data transmission device 200 are connected via the communication network 10, but the moving image data transmission device 100A and the moving image data transmission device 200 are different. However, the communication network 10 may be used instead of the communication network 10.

  The IP packet P12 (see FIG. 18) may be transmitted through a communication network that is separate from the communication network 10 to which the IP packet including the macroblock is transmitted.

  In the first and second embodiments described above, the macroblock is converted into an IP packet. However, the macroblock may be transmitted in another form (for example, an ATM cell) instead of the IP packet.

  Furthermore, in the second embodiment, the FEC code is used as the error correction code, but other error correction codes such as CRC may be used instead of the FEC code.

  As described above, the present invention naturally includes various embodiments that are not described herein. Therefore, the technical scope of the present invention is defined only by the invention specifying matters according to the scope of claims reasonable from the above description.

It is a logic block block diagram of the moving image data transmission apparatus which concerns on 1st Embodiment of this invention. It is an operation | movement flowchart of the moving image data transmission apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows the structural example of an image frame. It is a figure which shows the example of a reference image block. It is a figure which shows the example of a reference image block and a macroblock. It is a figure which shows the example of a reference image block and a macroblock. It is a figure which shows the structural example of a packet. It is a figure which shows the structural example of a packet. It is a figure which shows the structural example of the image frame which concerns on the example of a change of 1st Embodiment of this invention. It is a figure which shows the structural example of the reference image block which concerns on the example of a change of 1st Embodiment of this invention. It is a logic block block diagram of the moving image data transmission apparatus which concerns on 2nd Embodiment of this invention. It is an operation | movement flowchart of the moving image data transmission apparatus which concerns on 2nd Embodiment of this invention. It is an operation | movement flowchart of the moving image data transmission apparatus which concerns on 2nd Embodiment of this invention. It is an operation | movement flowchart of the moving image data transmission apparatus which concerns on 2nd Embodiment of this invention. It is an operation | movement flowchart of the moving image data transmission apparatus which concerns on 2nd Embodiment of this invention. It is an operation | movement flowchart of the moving image data transmission apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows the structural example of the packet which concerns on 2nd Embodiment of this invention. It is a figure which shows the structural example of the packet which concerns on 2nd Embodiment of this invention. It is a figure which shows the structural example of the packet which concerns on 2nd Embodiment of this invention. It is a figure which shows the specific structural example of the packet which concerns on 2nd Embodiment of this invention.

DESCRIPTION OF SYMBOLS 10 ... Communication network, 100, 100A ... Moving image data transmission apparatus, 101 ... Moving image data acquisition part, 103 ... Motion vector calculating part, 105 ... Reference image block determination part, 107 ... Weight assignment part, 109 ... Communication part, 111 ... packet importance level transmission unit, 113 ... error correction processing unit, 200 ... moving image data transmission apparatus, 201 ... communication unit, 203 ... packet importance level processing unit, 205 ... moving image data processing unit, 207 ... error correction processing unit, F: Image frame, MB: Macroblock, OL, OL1, OL2 ... Overlapping region, P1 to P4 ... IP packet, R ... Reference image element, SB ... Reference image block, V ... Motion vector, W ... Macroblock importance ( weight)

Claims (3)

A packet receiver that sequentially receives packets including macroblocks obtained by dividing an image frame;
An importance receiving unit for receiving the macroblock importance indicating the weight given to the macroblock;
The importance level for determining the specific macroblock importance level of the macroblock included in the specific packet lost by the packet reception unit based on the macroblock importance level of the plurality of macroblocks received by the importance level reception unit A decision unit;
When the specific macroblock importance determined by the importance determination unit exceeds a predetermined threshold, a retransmission request unit that requests retransmission of the specific packet;
A moving image data receiving apparatus.
The importance level determination unit determines a specific macroblock importance level based on a reception total value that is a total value of macroblock importance levels of macroblocks that can be received by the packet reception unit, and a reference total value. Item 4. The moving image data receiving device according to Item 1.
A decoding processing unit that performs decoding processing of the macroblock received by the packet receiving unit;
The moving image data receiving apparatus according to claim 1, wherein the decoding processing unit omits at least a part of the decoding processing when the macroblock importance received by the importance receiving unit is equal to or less than a predetermined threshold.