JP2012100187A - Image distribution system and image distribution method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image distribution system capable of maintaining high image quality even when a part of a frame constituting image is deficient.SOLUTION: In an image data distribution system for transmitting and receiving image data frame by frame, a receiver receives image data from a transmitter which includes an image composition classification means for classifying image compositions constituting a frame into brightness information and hue information; a transmission means for transmitting the brightness information of the classified image compositions to the receiver prior to the hue information; and a retransmission means for retransmitting image compositions which have already been transmitted in response to a request from the receiver. The receiver includes a retransmission request means for requesting the transmitter to retransmit only brightness information when the brightness information are deficient in the image compositions received from the transmitter; a frame destruction means for destructing a frame containing deficiency of brightness information; and a means by which when a deficiency of image compositions is limited to hue information, a frame containing the deficiency is regarded as a reproduction object.

Description

  The present invention relates to a video distribution system and method via a network.

  A video distribution system that distributes video acquired from a video source connected to a transmitter to an output device such as a monitor connected to a receiver through a network has been put into practical use. Generally distributed video is encoded in order to keep the transmission rate small, and the MPEG system, Motion-JPEG system, etc. are used as representative ones. When using Motion-JPEG, the transmitter compresses each frame constituting the video using the JPEG algorithm, and transmits this to the receiver. On the other hand, the receiver receives the compressed video frame, converts (decodes) it into video data, and outputs it to a video output device such as a monitor.

  By the way, since the video distribution system described above flows video data on the network, a packet including the video data may be lost on the transmission path for some reason. To deal with such problems, conventionally, when a receiving device detects a loss of a packet that constitutes received video data, the video frame in which the loss has occurred is discarded or a packet retransmission request is sent to the transmitter as necessary. To deal with it. However, with such means, it is not easy to satisfy the required video frame rate, and as a result, the output video may be disturbed. In particular, as disclosed in Patent Document 1, when a partial loss of a frame is detected, handling the entire frame as a loss is a major factor that causes a decrease in the frame rate.

JP 2001-111993 A

  In view of the above points, the present invention discloses a video distribution system capable of maintaining high video quality even when a part of a frame constituting a video is lost.

  According to a first aspect of the present invention, there is provided a transmitter in a video data distribution system for transmitting video data in units of frames from a transmitter to a receiver, and the video components constituting a frame are visually affected. According to the request from the receiver, the video component classification means for classifying into multiple types, the transmission means for sequentially transmitting the classified video components from the one with the strong visual influence to the receiver, Re-transmission means for re-transmitting the already transmitted video component.

  According to a second aspect of the present invention, there is provided a transmitter in a video data distribution system that transmits video data in units of frames from a transmitter to a receiver, wherein video components constituting a frame are converted into luminance information and hue information. Video component classification means for classifying, transmission means for transmitting the luminance information of the classified video components to the receiver prior to the hue information, and the video component already transmitted in response to a request from the receiver again. Retransmitting means for transmitting.

  According to a third aspect of the present invention, there is provided a video component classification means for classifying video components constituting a frame into a plurality of types according to the level of visual influence in a video data distribution system that transmits and receives video data in units of frames. And transmitting means for sequentially transmitting the classified video components from the one having the strong visual influence to the receiver, and resending means for retransmitting the already transmitted video components in response to a request from the receiver. If the video component received from the transmitter is a video component received from the transmitter and the video component stronger than the predetermined visual influence is missing, only the video component is retransmitted. Resending request means for requesting the transmitter, frame discarding means for discarding a frame including a missing video component stronger than a predetermined visual influence degree, and a missing visual component being a predetermined visual If only weaker than Hibikido, and it means for the frame containing the missing reproduced.

According to a fourth aspect of the present invention, in a video data distribution system for transmitting and receiving video data in units of frames, video component classification means for classifying video components constituting a frame into luminance information and hue information, and classified video components Video data from a transmitter comprising: transmission means for transmitting the luminance information to the receiver prior to hue information; and retransmission means for retransmitting the already transmitted video component in response to a request from the receiver. A retransmission request means for requesting the transmitter to retransmit only the luminance information when the luminance information is missing for the video component received from the transmitter;
A frame discarding unit for discarding a frame including a defect in luminance information; and a unit for reproducing a frame including the defect when the defect in the video component is only the hue information.

  According to a fifth aspect of the present invention, a transmitter in a video data distribution system that transmits video data in units of frames from a transmitter to a receiver classifies video components constituting a frame into luminance information and hue information. Video component classifying means, transmission means for transmitting luminance information of the classified video components to the receiver prior to the hue information, and retransmitting the already transmitted video components in response to a request from the receiver A program for functioning as a retransmission means.

  According to a sixth aspect of the present invention, in a video data distribution system for transmitting and receiving video data in units of frames, video component classification means for classifying video components constituting a frame into luminance information and hue information, and classified video components Video data from a transmitter comprising: transmission means for transmitting the luminance information to the receiver prior to hue information; and retransmission means for retransmitting the already transmitted video component in response to a request from the receiver. When the luminance information is missing for the video component received from the transmitter, the retransmission request means for requesting the transmitter to retransmit only the luminance information, and the frame including the luminance information loss A frame discarding means for discarding the image, and when the loss of the video component is only the hue information, the function for making the frame including the defect as a reproduction target means Program.

According to the present invention, a video component (for example, luminance information) having a strong visual influence is preferentially transmitted from the transmitter, so that even if a packet loss occurs on the network, the influence is limited. Can be suppressed. This is because the more preferentially transmitted packets can secure more time for retransmission at the time of loss.
Furthermore, the receiver does not discard the frame even if a part of the video component is missing, specifically, the video component (eg, hue information) that has little visual impact is lost. In addition, a decrease in frame rate can be suppressed.

Embodiments according to the present invention will be described below with reference to the drawings.
[Example 1]
[System overview]

  FIG. 1 is an overall view of a system according to the present invention. The transmitter 101 and the receiver 103 are connected to the network. Furthermore, the transmitter 101 is connected to the PC 105 as a video source via a cable such as VGA, and the receiver 103 is connected to the monitor 107 as a video output device via a cable such as VGA. In the video distribution system of the present invention, the transmitter 101 transmits the video data output from the PC 105 to each receiver 103, and the receiver 103 that receives the video data outputs the data to the monitor 107. It is.

[Block diagram of transmitter]
FIG. 2 is a block diagram of the transmitter 101. The video data received from the PC 105 is received by the video receiver 201 and then sent to the encoder 203. The encoding unit 203 encodes the received video data in a predetermined encoding format, and stores it in the video data storage unit 205 as video data in units of frames. In this embodiment, the encoding format is JPG2000. Video data encoded by JPEG2000 is in frame units, and each frame is composed of the following three types of video components.

  In JPEG2000, video data is called code stream data, and this code stream data is composed of three video components of luminance information (Y), hue information 1 (Cr), and hue information 2 (Cb). Present for each frame. The video data in one frame is in the format shown in FIG. Specifically, it is composed of components composed of three types of video components: luminance information (Y), hue information 1 (Cr), and hue information 2 (Cb). In general, there are a plurality of components in one frame, and there are four in this embodiment. In addition, the subscript of each information means that it becomes high resolution as it becomes large. For example, Y1 is luminance information with the lowest resolution, and Y4 is luminance information with the highest resolution.

  By the way, luminance information and hue information are equivalent in one code stream, that is, one video frame. In other words, there is no difference in the amount of information. However, for human vision, luminance information and hue information are not necessarily equivalent, and in general, luminance information that influences the overall image tends to be regarded as dominant. The same applies to the resolution, and the tendency of the luminance information and the hue information becomes stronger as the information becomes lower resolution. That is, the degree of influence of each information on human vision is luminance information> hue information and low resolution information> high resolution information. In the present invention, video distribution is performed in consideration of the visual impact.

  Returning to the block diagram of FIG. The video data detection unit 207 is a unit that detects whether video data is stored in the video data storage unit 205. This detection is performed in units of frames. Specifically, this is the time when all the video components (Y1 to Cb4 in FIG. 5A) constituting one frame have been prepared.

  The video component rearranging unit 209 rearranges the video components of the detected frames shown in FIG. 5A in a predetermined order. FIG. 5B is a diagram after rearrangement. As is apparent from the figure, the video components related to the luminance information are collectively arranged in the front, followed by the hue information. Further, in each piece of information, low resolution information is arranged in the front and high resolution information is arranged in the rear. The reason for this is that, as described above, the luminance information is stronger than the hue information, and the low resolution information is stronger in human visual influence than the high resolution information, and these video components are transmitted preferentially. It is to do.

  This rearrangement process is the point of the present invention. As will be described later, the video distribution system of the present invention performs retransmission when packet loss occurs in the network. By preferentially transmitting data that has a strong visual influence by rearranging video components, even if such data is lost, more time for data retransmission can be secured. As a result, this contributes to the improvement of the reproduced video quality.

The video data transmission unit 213 is a unit that transmits the rearranged video data to the receiver 103 via the network transmission / reception unit 215. The unit of the video data transmitted to the network is not the video component described above, but is transmitted in units of network packets as shown in FIG.
The retransmission control unit 211 is a unit that transmits a packet to the receiver 103 when a packet retransmission request is received from the receiver 103.

[Flowchart 1]
In the following, the order in which the functions described above operate will be described.
As shown in FIG. 3, the transmitter 101 includes two tasks, a video transmission task and a retransmission task.

First, the video transmission task will be described.
In step 301, the video data detection means 207 confirms whether video data is stored in the video data storage means 205. If video data is detected, the process proceeds to the next step, otherwise it is detected. Repeat the process.
In step 303, the video component rearranging means 209 forwards the luminance information and the low resolution information in the forward direction as described in FIG. 5B for the video components (luminance information and hue information) of the stored video data. Next, the hue information and the high resolution information are rearranged and rearranged.
In step 305, the video data transmission unit 213 transmits the rearranged video data packets to the receiver 103 at a predetermined timing based on a frame rate (for example, 30 fps) to be satisfied by the video distribution system.

Next, the retransmission task will be described.
In step 351, the retransmission control unit 211 determines whether or not a retransmission request has been received from the receiver 103. If received, the process proceeds to step 353, and if not, the reception determination process is repeated. In step 353, the retransmission control unit 211 transmits the packet requested by the receiver 103 to the receiver 103 via the video data transmission unit 213.

[Receiver block diagram]
FIG. 4 is a block diagram of the receiver 103. The video data receiving unit 403 receives the video data transmitted from the transmitter 101 through the network transmission / reception unit 401 and stores it in the video data storage unit 405. The video data stored in the video data storage unit 405 is rearranged by the video component rearranging unit 411 in the original order suitable for decoding. That is, the video data received in the state of FIG. 5B is rearranged to the state of FIG. The decoding unit 407 decodes the rearranged video data at a predetermined timing, and the decoded video data is output to the monitor by the video output unit 409.

  The frame discarding unit 413 is a unit that discards the video data stored in the video data storage unit 405 in units of frames. As shown in FIG. 6A, in the present invention, a predetermined amount of loss is allowed for each video component of the received video data. According to the figure, Y1 to Cb1 are assumed to be deficiency unacceptable data, and Cr2 to Cb4 are assumed to be deficiency allowable data. As described above, Y, which is luminance information, has a greater visual influence than hue information, and the tendency is stronger as the resolution is lower. That is, video components having a strong visual influence are essential, and hue information with relatively little influence and information with higher resolution are allowed to be lost.

  The reason for this is related to the fact that the present invention relates to a video distribution system. In the prior art, when a part of the video component, for example, Cr5 of the hue information is lost, the entire frame including the video component is lost. This is effective when video quality is given the highest priority. However, in an environment where video continuity is important, such as a video distribution system, even if a part of the video quality is lost, the specified video quality can be obtained. If the frame is maintained, it is considered that it is likely that it is more natural to reproduce it. Therefore, in the present invention, a defect in the frame is allowed. In other words, the loss of video components with a visually weak influence is allowed.

  On the other hand, the loss of video components that have a strong visual impact will cause a significant drop in video quality, and if it is forcibly played back, it will result in an unnatural video, so we decided to discard the frame itself. This is the role of the frame discarding means 413. That is, when the data that cannot be tolerated is missing, the video data storage unit 405 discards the frame in which the data is included.

  The missing data detection means 415 detects the loss of the video data transmitted from the transmitter 101. Specifically, as shown in FIG. 6B, a loss occurs in units of network packets. In the figure, an example in which network packets 2 and 4 are lost is shown. The retransmission request transmission means 417 determines whether the detected missing data is missing acceptable data or missing acceptable data, and if it is impossible data, requests retransmission to the transmitter 101. On the other hand, if the missing allowance data is missing, no retransmission request is made, and the information is replaced with image information with zero information amount.

  FIG. 7A shows a case where missing data has been successfully complemented by a network packet retransmitted from the retransmission control unit 211 of the transmitter 101, and FIG. 7B shows a case where complement has failed. Examples of cases in which complementation fails include a case where a retransmission request to the transmitter 101 has not arrived, a case where a retransmission packet has further lost, and a case where the retransmission packet has not arrived in time for the playback timing depending on the frame rate and so on. In the event that complementation has not succeeded finally, the frame discarding means 413 discards the frame. This is because, as described above, the retransmission request transmission means 417 requests retransmission only for data that cannot be tolerated.

  The decoding unit 407 reads and decodes the video data from the video data storage unit 405 based on a predetermined timing for satisfying the frame rate. The decoded video data is output to the monitor by the video output means 409.

[Summary]
As described above, according to the present invention, a visually strong video component (for example, luminance information) is preferentially transmitted from the transmitter, so even if a packet loss occurs on the network, the effect is not affected. Can be suppressed to a limited one. This is because the more preferentially transmitted packets can secure more time for retransmission at the time of loss.
Furthermore, the receiver does not discard the frame even if a part of the video component is missing, specifically, the video component (eg, hue information) that has little visual impact is lost. In addition, a decrease in frame rate can be suppressed.

Overall system diagram of the present invention Transmitter block diagram Transmitter flow Receiver block diagram Image components in the frame Missing and unacceptable data Complement by missing data retransmission

209 Video component rearranging means 211 Retransmission control means 413 Frame discarding means 415 Missing data detection means 417 Retransmission request transmission means

Claims (6)

A transmitter in a video data distribution system for transmitting video data in frame units from a transmitter to a receiver,
Video component classification means for classifying video components constituting a frame into a plurality of types according to the level of visual influence;
A transmission means for sequentially transmitting the classified video components from the one having the strong visual influence to the receiver;
In response to a request from the receiver, a retransmission means for retransmitting the already transmitted video component;
Transmitter with. A transmitter in a video data distribution system for transmitting video data in frame units from a transmitter to a receiver,
Video component classification means for classifying the video components constituting the frame into luminance information and hue information;
Transmitting means for transmitting the luminance information of the classified video components to the receiver prior to the hue information;
In response to a request from the receiver, a retransmission means for retransmitting the already transmitted video component;
Transmitter with. In a video data distribution system that transmits and receives video data in frame units,
Video component classification means for classifying video components constituting a frame into a plurality of types according to the level of visual influence;
A transmission means for sequentially transmitting the classified video components from the one having the strong visual influence to the receiver;
In response to a request from the receiver, a retransmission means for retransmitting the already transmitted video component;
A receiver for receiving the video data from a transmitter comprising:
A retransmission request means for requesting the transmitter to retransmit only the video component when the video component received from the transmitter is missing a video component stronger than a predetermined visual influence;
A frame discarding unit for discarding a frame including a missing video component stronger than the predetermined visual influence;
If the video component loss is only weaker than the predetermined visual influence, means for reproducing a frame including the loss;
Receiver with. In a video data distribution system that transmits and receives video data in frame units,
Video component classification means for classifying the video components constituting the frame into luminance information and hue information;
Transmitting means for transmitting the luminance information of the classified video components to the receiver prior to the hue information;
In response to a request from the receiver, a retransmission means for retransmitting the already transmitted video component;
A receiver for receiving the video data from a transmitter comprising:
Retransmission request means for requesting the transmitter to retransmit only the luminance information when luminance information is missing for the video component received from the transmitter;
A frame discarding means for discarding a frame including a defect in luminance information;
If the missing video component is only the hue information, means for reproducing the frame containing the missing image,
Receiver with. A transmitter in a video data distribution system that transmits video data frame by frame from a transmitter to a receiver,
Video component classification means for classifying the video components constituting the frame into luminance information and hue information;
Transmitting means for transmitting the luminance information of the classified video components to the receiver prior to the hue information;
In response to a request from the receiver, a retransmission means for retransmitting the already transmitted video component;
Program to make it function. In a video data distribution system that transmits and receives video data in frame units,
Video component classification means for classifying the video components constituting the frame into luminance information and hue information;
Transmitting means for transmitting the luminance information of the classified video components to the receiver prior to the hue information;
In response to a request from the receiver, a retransmission means for retransmitting the already transmitted video component;
A receiver for receiving the video data from a transmitter comprising:
Retransmission request means for requesting the transmitter to retransmit only the luminance information when luminance information is missing for the video component received from the transmitter;
A frame discarding means for discarding a frame including a defect in luminance information;
If the missing video component is only the hue information, means for reproducing the frame containing the missing image,
Program to make it function.