JP2010028573A - Video processing device, control method therefor, computer program, and video processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent disturbance of image caused by underflow of a packet in a video processing device for performing processing for the packet. <P>SOLUTION: The video processing device has a receiving means for receiving a packet dividing data from another device; a processing means for performing prescribed processing for improving quality of data for the received packet; a measurement means for measuring elapsed time after starting processing; a transmission means for transmitting a processed packet to a following device and a retaining means for retaining a processing time which is allowed for processing. When the elapsed time exceeds the allowed processing time, before processing by the processing means, is exceeded, the processing means stops processing and transmits a received packet or a packet at the time of the stoppage of processing to a subsequent device. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video processing apparatus, a control method thereof, a computer program, and a video processing system.

  The video processing system needs to synchronize data such as images and audio when outputting video data. In order to enable synchronization between the image and the sound, the international standard encoding scheme MPEG1 or MPEG2 adds a time stamp as output timing information in the header of the packet data.

  FIG. 9 is an example of a schematic block diagram of the video processing system. The video processing system includes a packet transmission device 920 and a packet reception device 940. When the video packet generation circuit 921 of the packet transmission device 920 receives the video data from the video input device 910, the video data is packetized and transmitted. At this time, a time stamp is generated based on the time indicated by the system time clock generation circuit 922 and added to the header of the packet.

  The packet transmitted by the packet transmission device 920 is received by the packet reception device 940 via the transmission path 930. The packet receiving device 940 restores the video data and transmits it to the video output device 950 when the time indicated by the system time clock recovery circuit 942 matches the time indicated by the time stamp in the packet header. The system time clock generation circuit 922 periodically transmits the system time clock as a packet in order to synchronize with the system time clock recovery circuit 942. The time from when the packet transmitting device 920 transmits a packet until the packet receiving device 940 receives the packet is constant depending on the system. Therefore, the time stamp can be generated based on the system time clock generation circuit 922.

  In the above video processing system, a packet processing device for performing image quality enhancement processing of video data may be arranged between the packet transmission device 920 and the packet reception device 940. In this case, since the processing time varies depending on the state of the packet processing device, the packet reception device 940 may not output the video data at the time indicated by the time stamp, and the video may be disturbed. This phenomenon is called underflow.

Patent Document 1 proposes a technique for preventing display corruption by re-executing a decoding process when the decoding process is interrupted in a display device that has received video data. However, with this method, it is not always possible to display at the time specified by the time stamp.
JP 09-233425 A

  It is an object of the present invention to prevent image disturbance due to packet underflow in a video processing apparatus that performs predetermined processing on a packet.

In view of the above problems, a video processing apparatus according to the present invention is
Receiving means for receiving a packet obtained by dividing the data from another device;
Processing means for performing predetermined processing for improving the quality of the data on the received packet;
Measuring means for measuring the elapsed time since the start of the process;
Transmitting means for transmitting the packet for which the processing has been completed to a subsequent device;
Holding means for holding a processing time allowed for the processing,
When the elapsed time exceeds the allowable processing time before the processing by the processing means is completed,
The processing means stops the processing,
The transmission means transmits the received packet or a packet at the time of stopping the processing to the subsequent apparatus.

  Even when the processing capability of the video processing device is reduced, it is possible to prevent image disturbance due to underflow.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

<First Embodiment>
The configuration of the video processing apparatus 100 in this embodiment will be described with reference to FIGS. 1 and 2. FIG. 1 is an example of a functional block diagram of a video processing apparatus 100 in the present embodiment.

  The video processing apparatus 100 includes a reception unit 101, a processing unit 102, a transmission unit 103, a processing stop determination unit 104, a measurement unit 105, and an allowable processing time holding unit 106. All or part of these elements may be realized by hardware such as a circuit, or may be realized by software.

  The receiving unit 101 receives a packet obtained by dividing video data from another device. The other device may be another video processing device 100 or a video transmission device 310 described later. The receiving unit 101 outputs the received packet to the processing unit 102 and simultaneously notifies the measuring unit 105 that the packet has been received.

The processing unit 102 performs predetermined processing on the input packet. The processing performed here includes, for example, high-resolution conversion, error correction, sharpness conversion, and progressive conversion for improving the quality by improving the image quality of video data. The processing to be performed is not limited to the processing for improving the image quality, and may be processing for improving the sound quality, for example. In order to apply the present invention, any process having a plurality of modes with different processing speeds may be used. When the processing is completed, the processed packet is output to the transmission unit 103.
The transmission unit 103 transmits the input processed packet to another device. The other device may be another video processing device 100 or a video receiving device 330 described later.

  The measuring unit 105 measures an elapsed time after receiving the notification from the receiving unit 101. This elapsed time is equal to the time elapsed since the processing unit 102 started processing. In the present embodiment, the time is measured based on the clock cycle included in the video processing apparatus 100, but any device having a timekeeping function such as real time may be used. The measurement unit 105 outputs the elapsed time to the process stop determination unit 104.

  The allowable processing time holding unit 106 holds an allowable processing time that is a processing time that the processing unit 102 can spend on processing. The processing stop determination unit 104 reads the allowable processing time from the allowable processing time holding unit 106.

  The process stop determination unit 104 compares the elapsed time input from the measurement unit 105 with the allowable processing time input from the allowable processing time holding unit 106. When the elapsed time exceeds the allowable processing time, the processing unit 102 is instructed to stop the processing. The processing unit 102 that has received this command before the processing ends stops processing the packet. Even when the processing is stopped, the processing unit 102 outputs the packet to the transmission unit 103. The packet output from the processing unit 102 to the transmission unit 103 may be a packet before the processing unit 102 starts processing, or may be a packet at the time when processing is stopped. Even when the packet processing is stopped, the transmission unit 103 transmits the packet input from the processing unit 102 to another device.

  FIG. 2 is an example of a block diagram illustrating a hardware configuration of the video processing apparatus 100. Note that FIG. 2 shows a minimum configuration for realizing the configuration of the video processing apparatus 100 corresponding to the embodiment of the present invention, and other mechanisms related to the video processing apparatus 100 are simply described. Is omitted.

  A CPU 201 that is a microprocessor controls the video processing apparatus 100 based on programs and data stored in a storage medium set in the ROM 203, the hard disk (HD) 212, and the external memory drive 211.

  The RAM 202 functions as a work area for the CPU 201 and holds programs stored in the ROM 203, the HD 212, and the like. It also holds the packet being processed.

  The storage medium set in the ROM 203 and the external memory drive 211 or the HD 212 stores a program executed by the CPU 201 as shown in a flowchart to be described later. The ROM 203 functions as the allowable processing time holding unit 106 shown in FIG.

  A keyboard controller (KBC) 205 controls input from a keyboard (KB) 209 or a pointing device such as a mouse (not shown). A display controller (DPC) 206 controls display on the display 210. A disk controller (DKC) 207 controls access to the HD 212 and the external memory drive 211, and reads and writes various programs and various data such as font data, user files, and edit files to and from these storage media. A network controller (NC) 208 performs communication with the network 220. The network controller 208 functions as the reception unit 101 and the transmission unit 103 illustrated in FIG.

  The CPU 201 executes, for example, a display information area allocated on the RAM 202 or an outline font development (rasterization) process to a dedicated video memory (VRAM), and enables display on the display 210. Further, the CPU 201 opens various registered windows based on commands instructed by a mouse cursor or the like on the display 210 and executes various data processing.

  A video processing system using the video processing apparatus 100 according to this embodiment will be described with reference to FIG. FIG. 3 is an example of a video processing system in the present embodiment. The video processing system includes a video transmission device 310, a video processing device A 321, a video processing device B 322, a video processing device C 323, and a video reception device 330. In the example shown in FIG. 3, there are three video processing devices 100, but any number of video processing devices 100 may be used as long as the number is one or more. Moreover, the video processing apparatus 100 may be arrange | positioned in series as shown in FIG. 3, or one part or all part may be arrange | positioned in parallel.

  The video transmission device 310 divides the video data into packets and transmits the packets to the video processing device A321. Each video processing apparatus 100 that has received the packet performs processing according to a flowchart that will be described later. The video receiving device 330 restores and displays the video data from the received packet.

  Here, the configuration of the packet 400 will be described with reference to FIG. FIG. 4 is a diagram illustrating an example of the configuration of the packet 400 in the present embodiment. The packet 400 includes a header 410 and a payload 420. The header 410 includes information necessary for packet communication and video data restoration. The header 410 also includes a time stamp 411. The time stamp 411 represents a time for displaying video data restored from a packet received by the video receiving device 330. Payload 420 includes divided video data.

  Returning to FIG. 3, the description of the video processing system will be continued. In the present embodiment, an example in which a time after 12 cycles of a time at which the video transmitting apparatus 310 transmits a packet is designated as the time stamp 411 is handled. Further, an example in which the allowable processing times of the video processing device A321, the video processing device B322, and the video processing device C323 are set to 3 cycles, 4 cycles, and 5 cycles, respectively. The allowable processing time may be set in advance in the ROM 203, or the video transmission apparatus 310 may specify the setting individually in accordance with the setting of the time stamp 411. Each video processing apparatus 100 does not hold a packet beyond the allowable processing time. Therefore, the video reception device 330 receives the packet 400 within 12 cycles after the video transmission device 310 transmits the packet 400. As described above, the total value of the allowable processing times of the respective video processing devices 100 is set so as not to exceed the time from when the video transmission device 310 transmits the packet 400 until the time specified by the time stamp 411. In the present embodiment, for simplicity, the network overhead is not considered, but the allowable processing time may be defined taking this into consideration.

  Next, the operation of the video processing apparatus 100 will be described with reference to FIG. FIG. 5 is a flowchart for explaining an example of the operation of the video processing apparatus 100 in the present embodiment. This flowchart is processed by the CPU 201 executing the computer program written in the ROM 203.

  In step S501, the reception unit 101 receives the packet 400. The reception unit 101 outputs the received packet 400 to the processing unit 102 and notifies the measurement unit 105 that the packet has been output. For example, the video processing device A 321 receives the packet 400 from the video transmission device 310.

  In step S502, the processing unit 102 starts predetermined processing on the input packet 400.

  In step S503, the measurement unit 105 that has received the notification from the reception unit 101 starts measuring elapsed time. Each time the elapsed time increases, the elapsed time is output to the process stop determination unit 104.

  In step S <b> 504, the process cancellation determination unit 104 compares the elapsed time input from the measurement unit 105 with the allowable processing time held by the allowable processing time holding unit 106. If the elapsed time is longer than the allowable processing time (“YES” in step S504), the process proceeds to step S506. If the elapsed time is not longer than the allowable processing time (“NO” in step S504), the process proceeds to step S505.

  In step S505, the processing unit 102 determines whether the processing for the packet 400 is completed. When the process is completed (“YES” in step S505), the processed packet 400 is output to the transmission unit 103, and the process proceeds to step S507. If the process has not ended ("NO" in step S505), the process returns to step S504.

  In step S <b> 506, the processing unit 102 stops the processing and outputs the packet 400 to the transmission unit 103 even when the processing has not ended. The packet 400 output here may be the packet 400 received by the reception unit 101 as it is, or may be the packet 400 at the time when the processing is stopped. The processing unit 102 determines at which point the packet 400 is to be output depending on the content of the processing.

  In step S507, the transmission unit 103 transmits the packet 400 output from the processing unit 102 to a subsequent apparatus. For example, the video processing device A321 transmits the packet 400 toward the video processing device B322.

  An example of the operation of the video processing system based on the above-described flowchart will be described. For example, it is assumed that the video processing apparatus A321 requires two cycles for processing. In this case, since it is less than 3 cycles which is the allowable processing time, all the processing is terminated and the packet 400 is transmitted to the video processing device B322. The time from when the video processing apparatus A 321 receives the packet 400 to when it is transmitted is two cycles.

  Next, it is assumed that the video processing device B322 requires 6 cycles for processing. In this case, since the allowable processing time is longer than 4 cycles, the processing is interrupted and the packet 400 is transmitted to the video processing device C323. The time from when the video processing apparatus B 322 receives the packet 400 to when it is transmitted is four cycles.

  Finally, it is assumed that the video processing device C323 requires 5 cycles for processing. In this case, since it is equal to 5 cycles, which is the allowable processing time, all the processing is finished and the packet 400 is transmitted to the video receiving device 330. The time from when the video processing device C 323 receives the packet 400 to when it is transmitted is 5 cycles.

  In the above example, when a packet is processed without applying the present invention, it takes 13 cycles from when the video transmitting apparatus 310 transmits the packet 400 until the video receiving apparatus 330 receives it. However, by applying the present invention, there are 11 cycles, and the packet 400 arrives at the video reception device 330 earlier than the time designated by the video transmission device 310 as the time stamp 411.

  As described above, according to the present embodiment, even when the processing performance of the video processing device 100 is deteriorated, the video receiving device 330 can display the video data at the time designated by the video transmitting device 310, and the video is disturbed. Can be reduced.

<Second Embodiment>
The configuration of the video processing apparatus 100 in the second embodiment will be described. FIG. 6 is an example of a functional block diagram of the video processing apparatus 100 in the present embodiment. The same elements as those in the first embodiment are denoted by the same reference numerals as those in FIG.

  The video processing apparatus 100 further includes a header analysis unit 601 and a switching unit 603. All or some of these elements may be realized by hardware such as a circuit, or may be realized by software.

  The header analysis unit 601 analyzes the header 410 of the packet 400 input from the reception unit 101, and determines whether or not the processing unit 602 should process the packet 400. When it is determined that the packet 400 is to be processed, the switching unit 603 is switched so that the packet 400 is input to the processing unit 602. When it is determined not to process the packet 400, the switching unit 603 is switched so that the packet 400 is not input to the processing unit 602.

  When the processing is interrupted, the processing unit 602 changes the processing flag 701 included in the header 410 of the packet 400 to “invalid”.

  Here, the configuration of the packet 400 in this embodiment will be described with reference to FIG. FIG. 7 is a diagram illustrating an example of the configuration of the packet 400 in the present embodiment. The same elements as those in the first embodiment are denoted by the same reference numerals as those in FIG.

  The header 410 further includes a processing flag 701. In the processing flag 701, one of “valid” and “invalid” is set. If “valid”, the video processing apparatus 100 processes the packet 400. In the case of “invalid”, the video processing apparatus 100 does not process the packet 400.

  The hardware block diagram of the video processing apparatus 100 in the present embodiment is the same as that described in the first embodiment with reference to FIG. The video processing system is also the same as that described in the first embodiment with reference to FIG.

  Next, the operation of the video processing apparatus 100 will be described with reference to FIG. FIG. 8 is a flowchart for explaining an example of the operation of the video processing apparatus 100 in the present embodiment. This flowchart is processed by the CPU 201 executing the computer program written in the ROM 203.

  In step S801, the video processing apparatus 100 receives the packet 400 from another apparatus. The received packet 400 is output to the header analysis unit 601 and the switching unit 603.

  In step S802, the header analysis unit 601 analyzes a processing flag included in the header 410 of the input packet 400. If the processing flag 701 is “valid” (“YES” in step S802), the process proceeds to step S803. When the processing flag 701 is “invalid” (“NO” in step S802), the process proceeds to step S809. In this case, the transmission unit 103 transmits the packet 400 to the subsequent device without performing any processing on the packet 400.

  Steps S802 to S807 are the same as steps S502 to S506 shown in FIG. 5 described in the first embodiment, and a description thereof will be omitted.

  In step S808, the processing unit 602 changes the processing flag 701 included in the header 410 to “invalid”. By doing so, the subsequent video processing apparatus 100 does not process the packet 400.

  Step S809 is the same as step S507 shown in FIG. 5 described in the first embodiment, and a description thereof will be omitted.

  An example of the operation of the video processing system will be described with reference to FIG. In the present embodiment, as in the first embodiment, an example in which the time stamp 411 designates the time after 12 cycles of the time for the video transmitting apparatus 310 to transmit a packet is handled. Further, an example in which the allowable processing times of the video processing device A321, the video processing device B322, and the video processing device C323 are set to 3 cycles, 4 cycles, and 5 cycles, respectively.

  First, the video transmission apparatus 310 sets the processing flag 701 to “valid” and transmits the packet toward the video processing apparatus A321.

  Assume that the video processing apparatus A321 requires two cycles for processing. In this case, since it is less than 3 cycles which is the allowable processing time, all the processing is terminated and the packet 400 is transmitted to the video processing device B322. The time from when the video processing apparatus A 321 receives the packet 400 to when it is transmitted is two cycles. Here, the processing flag 701 remains “valid”.

  Next, it is assumed that the video processing device B322 requires 6 cycles for processing. In this case, since it is less than 4 cycles, which is the allowable processing time, the processing is interrupted and the packet 400 is transmitted to the video processing device C323. At this time, the processing flag 701 is changed to “invalid”. The time from when the video processing apparatus B 322 receives the packet 400 to when it is transmitted is four cycles.

  Finally, it is assumed that the video processing device C323 requires 5 cycles for processing. However, since the processing flag 701 is “invalid”, the packet 400 is transmitted to the video reception device 330 without performing any processing. The time from when the video processing device C323 receives the packet 400 to when the packet 400 is transmitted is 0 cycle.

  In the above example, when a packet is processed without applying the present invention, it takes 13 cycles from when the video transmitting apparatus 310 transmits the packet 400 until the video receiving apparatus 330 receives it. However, by applying the present invention, the number of cycles becomes six, and the packet 400 arrives at the video reception device 330 earlier than the time specified by the video transmission device 310 in the time stamp 411.

  As described above, according to the present embodiment, even when the processing performance of the video processing device 100 is deteriorated, the video receiving device 330 can display the video data at the time designated by the video transmitting device 310, and the video is disturbed. Can be reduced.

  In the present embodiment, the processing in all subsequent video processing devices 100 is uniformly disabled, but the processing of some video processing devices 100 may be disabled. For example, the video processing device 100 whose allowable processing time is equal to or greater than a predetermined threshold may be invalidated. Alternatively, each video processing apparatus 100 may be defined with a priority, and the video processing apparatus 100 having a priority level equal to or lower than a predetermined threshold may be invalidated.

<Other embodiments>
Note that the present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, and a printer), and a device (for example, a copying machine and a facsimile device) including a single device. You may apply to.

  The object of the present invention can also be achieved by supplying, to a system, a storage medium that records the code of a computer program that realizes the functions described above, and the system reads and executes the code of the computer program. In this case, the computer program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the computer program code constitutes the present invention. In addition, the operating system (OS) running on the computer performs part or all of the actual processing based on the code instruction of the program, and the above-described functions are realized by the processing. .

  Furthermore, you may implement | achieve with the following forms. That is, the computer program code read from the storage medium is written in a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Then, based on the instruction of the code of the computer program, the above-described functions may be realized by performing a part or all of the actual processing by the CPU or the like provided in the function expansion card or function expansion unit.

  When the present invention is applied to the above-described storage medium, the computer program code corresponding to the flowchart described above is stored in the storage medium.

It is an example of the functional block diagram of the video processing apparatus 100 in the 1st Embodiment of this invention. It is an example of the hardware block diagram of the video processing apparatus 100 in the embodiment of the present invention. It is a figure explaining an example of the video processing system in the embodiment of the present invention. It is a figure explaining an example of the structure of the packet in the 1st Embodiment of this invention. It is a flowchart explaining an example of operation | movement of the video processing apparatus 100 in the 1st Embodiment of this invention. It is an example of the functional block diagram of the video processing apparatus 100 in the 2nd Embodiment of this invention. It is a figure explaining an example of the structure of the packet in the 2nd Embodiment of this invention. It is a flowchart explaining an example of operation | movement of the video processing apparatus 100 in the 2nd Embodiment of this invention. 1 is an example of a schematic block diagram of a video processing system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 101 Reception part 102 Processing part 103 Transmission part 104 Processing stop determination part 105 Measurement part 106 Permissible processing time holding | maintenance part

Claims (7)

Receiving means for receiving a packet obtained by dividing the data from another device;
Processing means for performing predetermined processing for improving the quality of the data on the received packet;
Measuring means for measuring the elapsed time since the start of the process;
Transmitting means for transmitting the packet for which the processing has been completed to a subsequent device;
Holding means for holding a processing time allowed for the processing,
When the elapsed time exceeds the allowable processing time before the processing by the processing means is completed,
The processing means stops the processing,
The video processing apparatus, wherein the transmission means transmits the received packet or a packet at the time of stopping processing to the subsequent apparatus. Analyzing means for analyzing the received packet and determining whether to perform the processing;
The transmission unit transmits the received packet to the subsequent device without the processing unit performing the processing when the analyzing unit determines that the processing should not be performed. The video processing apparatus according to 1. If the elapsed time exceeds the allowed processing time, the processing means is further configured to determine that the analysis means included in a subsequent video processing device should not perform the processing, The video processing apparatus according to claim 2, wherein the packet at the time when the processing is stopped is changed, and the transmission unit transmits the changed packet to the subsequent apparatus.   The predetermined processing includes any one of high-resolution conversion, error correction, sharpness conversion, and progressive conversion for improving the image quality of the data. Video processing equipment. One or more video processing devices;
A video transmission device that divides the video data into packets, and attaches a time stamp indicating the time to display the video data to the packet and transmits the packet to any of the video processing devices;
A video receiving device that receives a packet from any of the video processing devices and reproduces video data at a time specified by the time stamp;
Each of the one or more video processing devices is the video processing device according to any one of claims 1 to 4,
The total value of the allowable processing time included in each of the video processing devices does not exceed the time specified by the time stamp after the video transmission device transmits the packet. Processing system. A receiving step for receiving a packet obtained by dividing the data from another device;
A processing step in which a processing unit performs a predetermined process for improving the quality of the data on the received packet;
A measuring step in which the measuring means measures an elapsed time since the start of the processing;
A transmission means comprising a transmission step of transmitting the packet for which the processing has been completed to a subsequent device;
When the elapsed time exceeds the processing time allowed for the processing before the processing in the processing step ends,
Stopping the processing in the processing step,
The method of controlling a video processing apparatus, wherein, in the transmission step, the received packet or a packet at the time of stopping processing is transmitted to the subsequent apparatus.   A computer program for causing a computer to operate as the video processing apparatus according to any one of claims 1 to 4.