JP2005039739A - Moving picture cluster processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving picture cluster processing system capable of surely correcting omission of data and an image due to a processing delay of each node. <P>SOLUTION: The moving picture cluster processing system comprises: a plurality of information processing apparatuses; and a management apparatus for allowing each information processing apparatus to distributingly carry out moving picture processing. The moving picture cluster processing system is provided with: a moving picture processing instruction means for allowing each information processing apparatus to perform the moving picture processing in units of pictures; a processing result acquisition means for acquiring processing results of the moving picture processing by each picture returned from each information processing apparatus respectively; and a processing result replacement means and the management apparatus that compare a variation evaluation value denoting a degree of a change in patterns between a delayed picture including a delayed processing result and another reference picture with a prescribed threshold when the return of the processing result from the information processing apparatuses is delayed, replace only the delayed processing result with a processing result corresponding to the reference picture when the variation evaluation value is less than the threshold, and replace the processing result of the entire delayed pictures with the processing result of the reference picture when the variation evaluation value is the threshold or over. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a moving image cluster processing system, and is suitable for application to a case where moving image processing is performed using a plurality of personal computers, workstations, and other low-cost CPU boards (hereinafter referred to as personal computers).

  In recent years, along with high performance and low price of personal computers, expensive computers are conventionally required by interconnecting multiple personal computers via a network and sharing the processing (this is called cluster processing). Complicated processing can be executed at low cost. Application fields of such cluster processing include cryptanalysis, genome analysis, and computer graphics rendering processing (see, for example, Patent Document 1).

  FIG. 20 shows a configuration of a general cluster processing system using a personal computer. A host node personal computer 101 and a plurality of computing node personal computers 102 (102A to 102N) are connected via a network 103, and the cluster processing system 100 is connected. Is configured. The host node personal computer 101 executes the processing in a distributed manner with respect to the managed computation node personal computers 102A to 102N.

  FIG. 21 shows an application structure in the cluster processing system 100, and the cluster application program 110 of the host node personal computer 101 controls the distributed job 113 on each computation node personal computer 102 via the cluster management program 111 and the communication layer 112.

The cluster management program 111 receives instructions from the cluster application program 110 and executes data transmission / reception to / from another personal computer, activation and termination of the distributed job 113, etc. via the communication layer 112.
JP 2000-30047 JP

  Here, it is also conceivable to apply the above-described cluster processing to moving image compression encoding or decompression decoding processing (moving image compression / decompression processing). In such moving image compression / decompression processing, there is a case where real-time processing is required in which input data is processed and transmitted in real time, and non-real-time processing in which processed data is stored in a storage medium such as a hard disk. Divided into two.

  Since non-real-time processing has no limitation on processing time, simply increasing the number of personal computers (hereinafter referred to as nodes) that can perform cluster processing to execute high-speed compression / decompression processing at a higher speed Can do.

  On the other hand, in real-time processing, it is required that the processing is completed within a certain time, and when the processing is not completed within the certain time, problems such as missing of processing occur. For this reason, in order to guarantee the real-time property in the cluster processing, measures are taken in advance such that the processing load is evaluated for each node, and processing corresponding to each processing load is allocated to each node.

  On the other hand, in each node of the cluster, a program for managing cluster processing generally operates on a basic program (operating system) such as Linux, Unix (registered trademark), Windows (registered trademark), or the like. .

  Here, the above-described operating system does not have a mechanism (hereinafter referred to as a real-time OS) that guarantees the end of processing within a certain period of time for an application program that runs under its management. Further, in most cases, a plurality of programs (tasks) are executed at each node at the same time, and there is no guarantee that a predetermined computer resource is allocated to a specific task. Some operating systems such as Unix (registered trademark) have a function of specifying the priority of a task, but this also does not guarantee that a predetermined process will be completed within a predetermined time.

  Accordingly, when real-time moving image compression / decompression processing is performed using a cluster configured with a personal computer, real-time performance may be impaired due to variations in processing time of each node, which may cause data processing delay.

  In addition, when the real-time moving image compression / decompression process is performed using a cluster configured with a personal computer as described above, a delay in data processing in each node may occur when the number of nodes is increased.

  In this way, when performing real-time video compression / decompression using a cluster composed of personal computers, data processing delays may occur at each node, which may cause loss of encoded data and decoded images. There was a problem of having sex.

  The present invention has been made in view of the above points, and an object of the present invention is to propose a moving image cluster processing system capable of reliably correcting data and image loss due to processing delay of nodes.

  In order to solve such a problem, in the present invention, in a moving image cluster processing system including a plurality of information processing devices and a management device that distributes moving image processing to each of the plurality of information processing devices, the plurality of information processing devices Moving image processing instruction means for executing moving image processing for each picture, processing result acquisition means for acquiring processing results of moving picture processing for each picture returned from each of the plurality of information processing apparatuses, and information processing apparatus When the return of the processing result from is delayed, the fluctuation evaluation value indicating the degree of change in the pattern between the delayed picture including the delayed processing result and another reference picture is compared with a predetermined threshold, and the fluctuation evaluation value is If it is less than, only the delayed processing result is replaced with the corresponding processing result of the reference picture. It provided a processing result replacing means for replacing the processing result of the reference picture management result to the management apparatus.

  By determining whether to replace only the delayed processing result or the entire processing result of the delayed picture according to the fluctuation evaluation value between the delayed picture and the reference picture, An image defect can be reliably corrected according to the degree of change in the picture pattern.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

(1) First Embodiment (1-1) Overall Configuration of Moving Image Cluster Processing System In FIG. 1, reference numeral 1 denotes a moving image cluster processing system according to the first embodiment as a whole, and a host as a management apparatus The node personal computer 2 and four calculation node personal computers 3A to 3D as information processing apparatuses are connected via a network 4.

  The moving image cluster processing system 1 is configured to perform real-time compression encoding processing of an analog video signal supplied from the outside in a JPEG (Joint Photographic Cording Experts Group) 2000 system by cluster processing.

  In the host node personal computer 2, a memory 12, a display unit 13, a hard disk drive 14, a network interface 15, and a video capture circuit 16 are connected via a bus 17 to a CPU 11 that performs overall control of the host node personal computer 2. .

  The hard disk drive 14 stores an operating system such as Windows (registered trademark) and various application programs such as a moving image cluster processing program. The CPU 11 reads the operating system from the hard disk drive 14 in response to the activation of the host node personal computer 2, expands it in the memory 12, and executes it.

  Then, the CPU 11 as the moving image processing instruction means executes the moving image cluster processing program under the operating system execution environment, and distributes the compression encoding and expansion decoding processing of the moving image to the calculation node personal computers 3A to 3D. Let

  That is, at the time of compression encoding of a moving image, the video capture circuit 16 of the host node personal computer 2 digitally converts the analog video signal S1 supplied from the outside to generate a digital video signal D1, and supplies this to the CPU 11. The CPU 11 distributes the compression encoding of the digital video signal D1 to the calculation node personal computers 3A to 3D according to the moving image cluster processing program.

  Here, in the JPEG2000 system, compression encoding of a digital video signal is completed for each frame (or field) as a picture, and each frame is divided into a plurality of image blocks (hereinafter referred to as tiles). Thus, each tile can be individually compressed and encoded.

  That is, as shown in FIG. 2, the CPU 11 divides each frame of the digital video signal S1 having, for example, 720 × 480 pixels into four according to the number of the calculation node personal computers 3A to 3D, and tiles 1 to tile having 360 × 240 pixels. 4 is generated. Then, the CPU 11 sequentially supplies the data of the divided tiles 1 to 4 to the corresponding computation node personal computers 3A to 3D as the divided digital video data D2A to D2D (FIG. 3A).

  In the calculation node personal computers 3 </ b> A to 3 </ b> D, a memory 22, a display unit 23, a hard disk drive 24, and a network interface 25 are connected via a bus 27 to the CPU 21 that performs overall control of the calculation node personal computer 3.

  The hard disk drive 24 stores various application programs such as an operating system and a cluster application program. The CPU 21 reads the operating system from the hard disk drive 24 in response to the activation of the computing node personal computer 3, expands it in the memory 22, and executes it.

  The CPU 21 executes the moving image cluster processing program under the operating system execution environment, thereby compressing and encoding the divided digital video data D2 (D2A to D2D) supplied from the host node personal computer 2 using the JPEG2000 method. Generated data D3 (D3A to D3D) is returned to the host node personal computer 2 (FIG. 3B).

  The CPU 11 as the processing result acquisition unit sequentially synthesizes the divided encoded data D3A to D3D returned from the calculation node personal computers 3A to 3D in units of frames, and encodes the encoded video composed of a series of compression-encoded frame data. The signal D4 is generated and stored in the hard disk drive 14, or transmitted to an external transmission destination via the network 4 (FIG. 3C).

  Thus, the moving image cluster processing system 1 performs real-time compression encoding of the analog video signal S1 by cluster processing.

  At the time of decompression decoding of the moving image, the CPU 11 of the host node personal computer 2 follows the cluster application program, and the encoded video signal D4 read from the hard disk drive 14 or the encoded video signal supplied from the outside via the network 4 is used. The decompression decoding of D4 is distributed to the computation node personal computers 3A to 3D.

  That is, the CPU 11 generates divided encoded data D3A to D3D by dividing the encoded data for each frame of the encoded video signal D4 into four corresponding to the tiles 1 to 4 shown in FIG. The data is supplied to the computation node personal computers 3A to 3D (FIG. 3C).

  In accordance with the moving image cluster processing program, the CPU 21 of each of the calculation node personal computers 3A to 3D decompresses and decodes the divided encoded data D3A to D3D supplied from the host node personal computer 2, respectively, thereby dividing the divided digital data composed of continuous images of tiles 1 to 4. Video data D2A to D2D are generated and returned to the host node personal computer 2 (FIG. 3D).

  The CPU 11 as the processing result acquisition means sequentially synthesizes the divided digital video data D2A to D2D returned from the calculation node personal computers 3A to 3D for each frame to generate a digital video signal D1, and displays it on the display unit 13. Or transmitted to an external transmission destination via the network 4 (FIG. 3E).

  Thus, the moving image cluster processing system 1 decodes the encoded video signal D4 in real time by cluster processing.

(1-2) Error concealment at the time of compression encoding and decompression decoding In addition to such a configuration, the host node personal computer 2 determines the encoding processing time or decoding processing time for each tile in each of the calculation node personal computers 3A to 3D. Always monitoring.

  Then, when either the encoding processing time or the decoding processing time exceeds a predetermined reference processing time (that is, processing delay occurs in the calculation node), the host node personal computer 2 includes the frame including the tile in which the processing delay has occurred. Is interpolated using the previous or subsequent frame to prevent loss of the decoded image due to the processing delay of the computation node. Such interpolation processing is called error concealment.

  First, the cluster encoding processing procedure in the host node personal computer 2 for performing the above-described moving image compression encoding with error concealment will be described in detail with reference to the flowchart shown in FIG.

  The CPU 11 of the host node personal computer 2 enters from the start step of the routine RT1 and proceeds to the next step SP1. In step SP1, the CPU 11 as the moving image processing instruction means transmits various processing parameters such as an encoding target rate to each of the calculation node personal computers 3A to 3D, and proceeds to the next step SP2.

  In step SP2, the CPU 11 stores image data for one frame of the digital video signal D1 in the memory 12, and proceeds to the next step SP3.

  In step SP3, the CPU 11 divides the image data for one frame taken into the memory 12 corresponding to the four tiles to generate divided digital video data D2A to D2D, and proceeds to the next step SP4.

  In step SP4, the CPU 11 calculates the difference value of the pixel value of each RGB pixel between each tile of the current frame and the tile of the previous frame corresponding thereto (referred to as a comparison target frame). .

  Then, the CPU 11 averages the absolute values of the calculated difference values for each tile, and stores each average value in the memory 12 as the image fluctuation evaluation values Diff1 to Diff4 representing the degree of change in the pattern between the frames of the tiles 1 to 4. Then, the process proceeds to the next step SP5.

  FIG. 5 shows an example of the transition of the image fluctuation evaluation value accompanying the change of the frame. For example, in frame 3, since a triangular object appears over tile 1 and tile 3, the image fluctuation evaluation value Diff1 of tile 1 and the image fluctuation evaluation value Diff3 of tile 3 are increased. Further, since the frame 4 having a completely different pattern is sandwiched between the frame 3 and the frame 5, each of the image fluctuation evaluation values Diff1 to Diff4 of the frame 4 and the frame 5 is a large value.

  In step SP5, the CPU 11 as the moving image processing instruction means supplies the corresponding tiles 1 to 4 to the respective computation node personal computers 3A to 3D, and instructs the start of compression encoding for the tiles. Control goes to step SP6.

  In step SP6, the CPU 11 as the processing result acquisition unit starts waiting for the divided encoded data D3A to D3D returned from the calculation node personal computers 3A to 3D. Then, when the reference processing time has elapsed since the start of the standby, the CPU 11 proceeds to the next step SP7.

  In step SP7, the CPU 11 determines whether or not a delay has occurred in the compression encoding based on the return status of the divided encoded data D3A to D3D returned from the respective computation node personal computers 3A to 3D.

  In step SP7, when all the divided encoded data D3A to D3D are returned, this means that no processing delay occurs in all the calculation node personal computers 3A to 3D, and no error concealment is required. At this time, the CPU 11 moves to step SP8, generates an encoded video signal D4 by combining all the returned divided encoded data D3A to D3D (tile), and returns to step SP2.

  On the other hand, if any of the divided encoded data D3A to D3D has not yet been returned in step SP7, this causes a processing delay in any of the calculation node personal computers 3A to 3D, and error concealment is required. In this case, the CPU 11 proceeds to step SP9.

  In step SP9, the CPU 11 as the processing result replacing unit compares the image fluctuation evaluation value Diff of the tile in which the processing delay has occurred with a predetermined fluctuation evaluation threshold value Dlim, and based on the comparison result, an error concealment method (one Or replace all tiles or all tiles).

  That is, in step SP9, when the image fluctuation evaluation value Diff of the tile in which the processing delay has occurred is less than the fluctuation evaluation threshold value Dlim, this means that the tile between the tile in which the processing delay has occurred and the tile of the comparison target frame corresponding thereto. Even if the divided encoded data D3A to D3D are synthesized by using the tile of the comparison target frame instead of the tile in which the processing delay has occurred with little difference in the design, it represents that there is little uncomfortable feeling in the decoded video, At this time, the CPU 11 proceeds to step SP10.

  In step SP10, the CPU 11 as the processing result replacement unit replaces the divided encoded data D3n of the tile in which the processing delay has occurred with the divided encoded data D3n of the tile of the comparison target frame, and then the divided encoded data D3A to D3D. The encoded video signal D4 is generated by combining, and the process returns to step SP2.

  On the other hand, in step SP9, when the image fluctuation evaluation value Diff of the tile in which the processing delay has occurred is equal to or greater than the fluctuation evaluation threshold Dlim, this means that the pattern between the tile in which the processing delay has occurred and the tile of the comparison target frame. When the divided encoded data D3A to D3D are synthesized using the tile of the comparison target frame instead of the tile in which the processing delay has occurred, the sense of incongruity in the decoded video increases. The CPU 11 moves to step SP11.

  In step SP11, the CPU 11 as the processing result replacement unit replaces the divided encoded data D3A to D3D of all tiles including the tile having the processing delay with the divided encoded data D3A to D3D of the tile of the comparison target frame. Post-synthesis is performed to generate an encoded video signal D4, and the process returns to step SP2.

  6 and 7 show examples of error concealment during encoding. Here, it is assumed that the fluctuation evaluation threshold value Dlim = 10.

  For example, as shown in FIG. 6B, when the tile 1 of the frame 3 (image fluctuation evaluation value Diff1 = 4) is delayed, the image fluctuation evaluation value Diff1 of the tile 1 is less than the fluctuation evaluation threshold Dlim. The CPU 11 replaces only the tile 1 in which the processing delay has occurred with the tile 1 of the frame 2 that is the comparison target frame. In this case, the tip of the triangle shaped on the tile 1 is erased by replacement, but since the area is relatively small, there is little discomfort in the decoded video.

  As shown in FIG. 6C, when the tile 2 (image fluctuation evaluation value Diff1 = 0) of the frame 3 is delayed, the image fluctuation evaluation value Diff1 of the tile 2 is less than the fluctuation evaluation threshold value Dlim. The CPU 11 replaces only the tile 2 in which the processing delay has occurred with the tile 2 of the frame 2.

  On the other hand, as shown in FIG. 6D, when the tile 3 of the frame 3 (image fluctuation evaluation value Diff1 = 30) is delayed, the image fluctuation evaluation value Diff1 of the tile 3 is equal to or larger than the fluctuation evaluation threshold Dlim. For this reason, the CPU 11 replaces all tiles with the tiles of the frame 2 that is the comparison target frame. In this case, two frames with the same pattern are continuous, but the sense of incongruity in the decoded video is less than when the tile 3 is replaced.

  Further, as shown in FIG. 7, when any tile of frame 5 (FIG. 5E) is delayed in processing, all the image fluctuation evaluation values Diff <b> 1 to Diff <b> 4 of each tile 1 to tile 4 of the frame 5 fluctuate. Since it is equal to or greater than the evaluation threshold value Dlim, the CPU 11 replaces all tiles with the tiles of the frame 4 that is the comparison target frame.

  Next, a cluster decoding processing procedure in the host node personal computer 2 for performing moving image decompression decoding with error concealment will be described in detail with reference to the flowchart shown in FIG.

  The CPU 11 of the host node personal computer 2 enters from the start step of the routine RT2 and proceeds to the next step SP21. In step SP21, the CPU 11 as the moving image processing instruction means transmits various processing parameters such as a decoding level to the respective calculation node personal computers 3A to 3D, and proceeds to the next step SP22.

  In step SP22, the CPU 11 stores the encoded data for one frame of the encoded video signal D4 in the memory 12, and proceeds to the next step SP23.

  In step SP23, the CPU 11 divides the encoded data for one frame taken into the memory 12 in correspondence with the four tiles to generate divided encoded data D3A to D3D, and proceeds to the next step SP24.

  In step SP24, the CPU 11 as the moving image processing instruction means supplies the corresponding divided encoded data D3A to D3D to the respective calculation node personal computers 3A to 3D, and proceeds to the next step SP25.

  In step SP25, the CPU 11 as the processing result acquisition means starts waiting for the divided digital video data D2A to D2D returned from the calculation node personal computers 3A to 3D. Then, when the reference processing time has elapsed from the start of the standby, the CPU 11 proceeds to next step SP26.

  In step SP26, the CPU 11 as the processing result replacing means determines whether or not there has been a delay in the decompression decoding based on the return status of the divided digital video data D2A to D2D returned from the calculation node personal computers 3A to 3D. To do.

  If all the divided digital video data D2A to D2D are returned in step SP26, this indicates that no processing delay occurs in all the calculation node personal computers 3A to 3D, and no error concealment is required. At this time, the CPU 11 proceeds to step SP26, combines them using all the returned divided digital video data D2A to D2D, and returns to step SP22.

  On the other hand, if any of the divided digital video data D2A to D2D has not yet been returned in step SP26, this causes a processing delay in any of the calculation node personal computers 3A to 3D, and error concealment is required. In this case, the CPU 11 proceeds to step SP28.

  Here, in the error concealment at the time of compression encoding described above, the error concealment method is selected based on the image fluctuation evaluation value Diff of the tile in which the encoding processing delay has occurred. In the error concealment at the time of decoding, since the tile itself in which the decoding processing delay has occurred is not decoded, the image fluctuation evaluation value Diff of the tile in which the processing delay has occurred cannot be obtained.

  Therefore, in this case, the error concealment method is selected based on the average value of the image fluctuation evaluation values Diff of other tiles in which no processing delay has occurred (tiles that have been decoded within the reference processing time).

  That is, in step SP28, the CPU 11 as the processing result replacement means determines the image fluctuation evaluation value Diff between each tile of the current frame for which decoding has been completed and the tile of the previous frame (comparison target frame) corresponding thereto. Is calculated (this is called the average fluctuation evaluation value Dave), and the process proceeds to the next step SP29.

  In step SP29, the CPU 11 compares the calculated average fluctuation evaluation value Dave with the fluctuation evaluation threshold value Dlim.

  In step SP29, when the average fluctuation evaluation value Dave is less than the fluctuation evaluation threshold value Dlim, this means that there is little difference in the pattern between the tile in which the processing delay has not occurred and the tile of the comparison target frame corresponding thereto, That is, it shows that there is little difference in the pattern in the whole frame.

  For this reason, it is assumed that there is little discomfort in the decoded video even if only the tile in which the processing delay has occurred is replaced with the tile of the comparison target frame. At this time, the CPU 11 as the processing result replacing means moves to step SP30, After replacing the divided digital video data D2n of the tile in which the processing delay has occurred with the divided digital video data D2n of the tile of the comparison target frame, the divided digital video data D2A to D2D are synthesized to generate the digital video signal D1. Return to SP22.

  On the other hand, when the average fluctuation evaluation value Dave is equal to or larger than the fluctuation evaluation threshold value Dlim in step SP29, this means that the pattern difference between the tile in which the processing delay has not occurred and the tile of the comparison target frame corresponding to this tile. Is large, that is, the difference in the pattern in the entire frame is large.

  For this reason, it is assumed that when only the tile in which the processing delay has occurred is replaced with the tile of the comparison target frame, the sense of incongruity in the decoded video is increased. At this time, the CPU 11 as the processing result replacing unit moves to step SP30, The divided digital video data D2A to D2D of all the tiles including the tile in which the processing delay has occurred are replaced with the divided digital video data D2A to D2D of the comparison target frame and then combined to generate the digital video signal D1, and the process returns to step SP22. .

  9 and 10 show examples of error concealment at the time of decoding. Here, it is assumed that the fluctuation evaluation threshold value Dlim = 10.

  For example, as shown in FIG. 9B, when the tile 1 of the frame 3 is delayed, the average fluctuation evaluation value Dave of the tile 2, the tile 3, and the tile 4 in time is (0 + 30 + 0) / 3 = 10. Since the average fluctuation evaluation value Dave is equal to or larger than the fluctuation evaluation threshold value Dlim, the CPU 11 replaces all tiles with the tiles of the frame 2 that is the comparison target frame.

  As shown in FIG. 9C, when the tile 2 of the frame 3 is delayed in processing, the average variation evaluation value Dave of the tile 1, tile 3 and tile 4 in time is (4 + 30 + 0) /3=11.3. Since the average fluctuation evaluation value Dave is equal to or larger than the fluctuation evaluation threshold value Dlim, the CPU 11 replaces all tiles with the tiles of the frame 2.

  On the other hand, as shown in FIG. 9D, when the tile 3 of the frame 3 is delayed, the average fluctuation evaluation value Dave of the tile 1, the tile 2 and the tile 4 in time is (4 + 0 + 0) / 3 = Since the average fluctuation evaluation value Dave is less than the fluctuation evaluation threshold value Dlim, only the tile 3 in which the processing delay has occurred is replaced with the tile 3 of the frame 2 that is the comparison target frame.

  As shown in FIG. 10, when any tile in frame 5 (FIG. 5E) is delayed in processing, the average fluctuation evaluation value Dave is equal to or greater than the fluctuation evaluation threshold Dlim regardless of which tile is delayed in processing. Therefore, the CPU 11 replaces all tiles with the tiles of the frame 2 that is the comparison target frame.

  Next, a calculation processing procedure in the calculation node personal computer 3 (3A to 3D) for performing the above-described moving image compression encoding and decompression decoding processing will be described in detail with reference to the flowchart shown in FIG.

  The CPU 21 of the computing node personal computer 3 enters from the start step of the routine RT3 and proceeds to the next step SP41. In step SP41, the CPU 21 waits for various processing parameters such as an encoding target rate and a decoding level transmitted from the host node personal computer 2, and when receiving the processing parameters, moves to the next step SP42.

  In step SP42, the CPU 21 waits for the divided digital video data D2n or the divided encoded data D3n transmitted from the host node personal computer 2. When receiving the data, the CPU 21 proceeds to the next step SP43.

  In step SP43, the CPU 21 performs the encoding process of the divided digital video data D2n received from the host node personal computer 2 or the decoding process of the divided encoded data D3n. When the process is completed, the process proceeds to the next step SP44.

  In step SP44, the CPU 21 returns the divided encoded data D3n or the divided digital video data D2n as a processing result to the host node personal computer 2 and returns to step SP42.

(1-3) Operation and Effect In the above configuration, the host node personal computer 2 of the moving image cluster processing system 1 corresponds to four tiles of image data for one frame of the digital video signal D1 during the compression encoding process. The divided digital video data D2A to D2D divided in this way are supplied to the corresponding computation node personal computers 3A to 3D for compression coding, and for each tile of the current frame, a pattern between the tile of the comparison target frame and the tile of the comparison target frame An image fluctuation evaluation value Diff representing the degree of change is calculated and stored in the memory 12.

  Then, the host node personal computer 2 synthesizes the divided encoded data D3A to D3D that have been compression-encoded and returned by the calculation node personal computers 3A to 3D to generate an encoded video signal D4, thereby compressing and encoding the moving image. To cluster.

  At this time, when a delay occurs in the compression encoding process in any of the calculation node personal computers 3A to 3D, the host node personal computer 2 compares the image fluctuation evaluation value Diff of the tile in which the processing delay has occurred with the fluctuation evaluation threshold value Dlim. .

  When the image fluctuation evaluation value Diff is less than the fluctuation evaluation threshold Dlim, the host node personal computer 2 has a small change in the pattern between the tile having the processing delay and the tile of the comparison target frame, and the tile having the processing delay has occurred. Assuming that there is little sense of incongruity in the decoded video even if it is replaced with the tile of the comparison target frame, the encoded video signal D4 is generated by replacing the tile having the processing delay with the tile of the comparison target frame.

  On the other hand, when the image fluctuation evaluation value Diff is equal to or larger than the fluctuation evaluation threshold Dlim, the host node personal computer 2 has a large change in the pattern between the tile having the processing delay and the tile of the comparison target frame, and processing delay occurs. If only the tiles that have been replaced are replaced, the sense of incongruity in the decoded video increases, and the entire frame including the tile in which the processing delay has occurred is replaced with the comparison target frame to generate the encoded video signal D4.

  In addition, the host node personal computer 2 divides the divided encoded data D3A to D3D obtained by dividing the image data for one frame of the encoded video signal D4 corresponding to the four tiles at the time of the expansion decoding process, respectively. The data is supplied to the personal computers 3A to 3D and decompressed and decoded.

  The host node personal computer 2 generates the digital video signal D1 by combining the divided digital video data D2A to D2D that have been decompressed and decoded by the computation node personal computers 3A to 3D and returned to generate the digital video signal D1. Clustering.

  At this time, when a delay occurs in the tile decompression decoding process in any of the calculation node personal computers 3A to 3D, the host node personal computer 2 averages the image fluctuation evaluation values Diff of other tiles that did not cause the processing delay. The fluctuation evaluation value Dave is calculated, and the average fluctuation evaluation value Dave is compared with the fluctuation evaluation threshold value Dlim.

  When the average fluctuation evaluation value Dave is less than the fluctuation evaluation threshold value Dlim, the host node personal computer 2 has a small change in the pattern between the frame including the tile having the processing delay and the comparison target frame, and the processing delay occurs. The digital image signal D1 is generated by replacing the tile in which the processing delay has occurred with the tile of the comparison target frame, assuming that there is little discomfort in the decoded video even if the tile is replaced with the tile of the comparison target frame.

  On the other hand, when the average fluctuation evaluation value Dave is equal to or larger than the fluctuation evaluation threshold value Dlim, the host node personal computer 2 has a large change in pattern between the frame including the tile in which the processing delay has occurred and the comparison target frame. If the tile with the delay is replaced with the tile of the comparison target frame, the sense of incongruity in the decoded video increases, and the entire frame including the tile with the processing delay is replaced with the comparison target frame to generate the digital video signal D1.

  According to the above configuration, when a processing delay occurs in the moving image cluster processing, the image variation evaluation value Diff of the tile in which the processing delay has occurred, or the image variation evaluation value Diff of a tile other than the tile in which the processing delay has occurred. Based on the average value, it is determined whether to replace only the tile in which the processing delay has occurred with the tile of the comparison target frame or to replace the entire frame. Error concealment can be performed.

(1-4) Other Embodiments In the above-described embodiment, the case where the moving image cluster processing system 1 executes the moving image compression / decompression processing according to the JPEG 2000 method has been described. However, the present invention is not limited to this. For example, MPEG (Moving Picture Experts
Various encoding schemes such as (Group) 2 scheme can be applied to the present invention.

  The MPEG2 system differs from the JPEG2000 system in which encoding is completed for each frame, and motion-compensated interframe encoding is performed using a block matching method that improves encoding efficiency with reference to the previous and subsequent frames. Yes.

  That is, in the MPEG2 system, an I picture that performs intraframe coding, a P picture that performs forward predictive coding with reference to the I picture, and a B picture that performs bidirectional predictive coding with reference to the preceding and following I pictures and P pictures These three picture types are appropriately selected. In the MPEG2 system, as shown in FIG. 12, a frame is composed of a sequence of digital video signals before encoding (FIG. 12A) and a sequence of encoded video signals after encoding (FIG. 12B). Differences occur in the order.

  For this reason, when the present invention is applied to the MPEG2 system, it is necessary to change the comparison target frame in accordance with the picture type of the encoding target frame. For example, when the encoding target frame is a B picture or a P picture, the forward I picture or P picture is set as a comparison target frame. When the encoding target frame is an I picture, the closest forward I picture or P picture is set as a comparison target frame.

  Furthermore, in the MPEG2 system, as shown in FIG. 13, the encoding processing time and the decoding processing time vary depending on the picture type and the motion search level (the motion search target range of the block matching method). For this reason, when the present invention is applied to the MPEG2 system, it is necessary to change the reference processing time for determining the processing delay according to the picture type and motion search level of the encoding target frame.

  In the above embodiment, the CPU 11 of the host node personal computer 2 executes the moving image cluster processing program stored in the hard disk drive 14, but the present invention is not limited to this, and the moving image cluster processing program is not limited thereto. The above-described moving image cluster processing may be executed by installing a program storage medium in which is stored in the host node personal computer 2.

In this case, as a program storage medium for installing the moving image cluster processing program in the host node personal computer 2, for example, a CD-ROM (Compact Disk-Read Only Memory) or a DVD (Digital Versatile).
In addition to a package medium such as a disk), it may be realized by a semiconductor memory or a magnetic disk in which a program is temporarily or permanently stored. Further, as means for storing the programs in these program storage media, wired and wireless communication media such as a local area network, the Internet, and digital satellite broadcasting may be used.

(2) Second Embodiment In FIG. 14, in which parts corresponding to those in FIG. 1 are assigned the same reference numerals, 30 is a virtual reality cluster processing system (hereinafter referred to as VR hereinafter) according to a second embodiment of the present invention as a whole. A host node personal computer 2 and three calculation node personal computers 3A to 3C are connected via a network 4.

  In this VR cluster processing system 30, a virtual reality image generated by computer graphics processing is arranged so as to surround the front and right and left of a virtual reality (virtual reality) experience person (hereinafter referred to as a VR experience person). In order to detect three displays 32A to 32C (front display 32A, right side display 32B, and left side display 32C) for displaying (hereinafter referred to as a VR image), and the line-of-sight direction and movement of the VR experience person. The VR device 34 including the motion sensor 33 is connected to the video interface 31 of the host node personal computer 2. Note that the motion sensor 33 may be mounted on a human head, or the direction of the line of sight and motion may be detected by a video camera or the like.

  The host node personal computer 2 of the VR cluster processing system 30 executes the VR cluster processing program under the operating system execution environment, detects the visual direction and operation of the VR experience person via the motion sensor 33, and responds to the detection result. The VR image data D5A to D5C are distributed and generated in the calculation node personal computers 3A to 3C.

  That is, the computation node personal computer 3A generates VR image data D5A composed of VR image data displayed on the front display 32A. Similarly, the calculation node personal computer 3B generates VR image data D5B which is data of a VR image displayed on the right side display 32B, and the calculation node personal computer 3C is data of a VR image displayed on the left side display 32C. VR image data D5C is generated.

  Then, the CPU 11 of the host node personal computer 2 supplies the VR image data D5A to D5C generated by the respective computation node personal computers 3A to 3C to the corresponding displays 32A to 32C, respectively, and displays the VR image, so The virtual reality space where the display changes according to the movement of the VR experience person himself is made to experience.

  FIG. 16 shows an example of display changes of VR images on the front display 32A, the right side display 32B, and the left side display 32C. In this case, the VR experience person is moving toward the front display 32A while keeping the line-of-sight direction toward the front display 32A.

  In addition to such a configuration, the host node personal computer 2 constantly monitors the processing time required to generate the VR image data D5A to D5C in each of the calculation node personal computers 3A to 3D, and any one of the VR image data D5A to D5C is generated. In the case of delay (occurrence of processing delay), error concealment is performed using the VR image data of the previous frame, thereby preventing loss of the VR image due to processing delay of the calculation node.

  Next, the cluster VR processing procedure in the host node personal computer 2 for generating the VR image data with the error concealment described above will be described in detail with reference to the flowchart shown in FIG.

  The CPU 11 of the host node personal computer 2 enters from the start step of the routine RT4 and proceeds to the next step SP51. In step SP51, the CPU 11 as the moving image processing instruction means instructs the calculation node personal computers 3A to 3C to generate the VR image data D5A to D5C in the initial state, and proceeds to the next step SP52.

  In step SP52, the CPU 11 as the processing result acquisition means supplies the VR image data D5A to D5C returned from the respective computation node personal computers 3A to 3C to the corresponding displays 32A to 32C to display the VR image, and the next step Move on to SP53.

  In step SP53, the CPU 11 extracts the line-of-sight direction and motion of the VR experience person based on the change in the VR experience person image D6 supplied from the motion sensor 33, and proceeds to the next step SP54.

  In step SP54, the CPU 11 determines whether or not a change has occurred in the line-of-sight direction and operation of the extracted VR experience person. If it is determined in step SP53 that there has been no change in the VR experience person's line-of-sight direction and movement, the CPU 11 returns to step SP53 and extracts the VR experience person's line-of-sight direction and movement again.

  On the other hand, if it is determined in step SP53 that changes have occurred in the line of sight and operation of the VR experiencer, the CPU 11 proceeds to the next step SP55.

  In step SP55, the CPU 11 as the moving image processing instruction means instructs each of the calculation node personal computers 3A to 3C to generate a new VR image in accordance with the extracted VR experience person's line-of-sight direction and movement change. The process moves to the next step SP56.

  In step SP56, the CPU 11 as the processing result acquisition unit starts waiting for the VR image data D5A to D5C returned from the calculation node personal computers 3A to 3C. Then, when the reference processing time has elapsed from the start of the standby, the CPU 11 proceeds to the next step SP57.

  In step SP57, the CPU 11 determines whether or not a delay has occurred in the generation of the VR image data based on the return status of the VR image data D5A to D5C returned from the respective computation node personal computers 3A to 3C.

  If all the VR image data are returned in step SP57, this indicates that no processing delay occurs in all the computation node personal computers 3A to 3C, and no error concealment is required. At this time, the CPU 11 Moves to step SP58, supplies the returned VR image data D5A to D5C to the corresponding displays 32A to 32C, displays the VR image, and returns to step SP53.

  On the other hand, if any of the VR image data D5A to D5C has not yet been returned in step SP57, this causes a processing delay in any of the calculation node personal computers 3A to 3C, and error concealment is required. At this time, the CPU 11 proceeds to step SP59.

  In step SP59, the CPU 11 as the processing result replacing unit determines whether or not the VR image data D5 in which the processing delay has occurred is an image in the visual axis direction of the VR experience person. If it is determined in step SP59 that the VR image data D5 in which the processing delay has occurred is not an image in the line-of-sight direction, the CPU 11 proceeds to step SP60.

  In this case, the VR image data D5 in which the processing delay has occurred is not an image in the line-of-sight direction, so even if a discontinuity occurs in the image due to error concealment, it is difficult for the VR experience person to detect this. Therefore, in step SP60, the CPU 11 uniformly replaces and displays only the VR image data D5 in which the processing delay has occurred with the VR image data D5 one frame before, and returns to step SP53.

  The state of error concealment in this case is shown in FIG. It is assumed that a processing delay occurs in the left side image at the timing of T = 3. In this case, since the VR image in which the processing delay has occurred is not an image in the line-of-sight direction, the replacement is performed with an image of T = 2.

  On the other hand, if it is determined in step SP59 that the VR image data D5 in which the processing delay has occurred is an image in the line-of-sight direction, the CPU 11 proceeds to step SP61.

  In this case, the VR image data D5 in which processing delay has occurred is an image in the line-of-sight direction. Therefore, if a discontinuity occurs in the image due to error concealment, the VR experience person can easily detect this. For this reason, the CPU 11 as the processing result replacing means calculates the average fluctuation evaluation value Dave for the other two VR image data D5 that has not been delayed in step SP61, and proceeds to the next step SP62 to calculate the average fluctuation evaluation value. Dave is compared with the fluctuation evaluation threshold value Dlim.

  In step SP62, when the average fluctuation evaluation value Dave is less than the fluctuation evaluation threshold value Dlim, this indicates that there is little change in the VR image in which no processing delay has occurred. For this reason, it is assumed that there is little discomfort even if only the VR image in which the processing delay has occurred is replaced with the VR image of the previous frame. At this time, the CPU 11 as the processing result replacing means moves to step SP63 and the processing delay has occurred. Only the VR image data D5 is replaced with the VR image data D5 of the previous frame and displayed, and the process returns to step SP53.

  The state of error concealment in this case is shown in FIG. Assume that processing delay occurs in the front image at the timing of T = 2. In this case, since the VR image in which the processing delay has occurred is a front image in the line-of-sight direction, the average fluctuation evaluation value Dave of the right side image and the left side image in which no processing delay has occurred is calculated, and the average fluctuation evaluation value Dave is calculated. Is less than the fluctuation evaluation threshold value Dlim, replacement is performed with an image of T = 1.

  On the other hand, when the average fluctuation evaluation value Dave is greater than or equal to the fluctuation evaluation threshold value Dlim in step SP62, this indicates that the change in the VR image in which no processing delay has occurred is large. For this reason, it is assumed that the discomfort increases when only the VR image in which the processing delay has occurred is replaced with the VR image of the previous frame. At this time, the CPU 11 as the processing result replacing means moves to step SP64 and all the VR image data D5. Is replaced with the VR image data D5 of the previous frame, and the process returns to step SP53.

  The state of error concealment in this case is shown in FIG. Assume that processing delay occurs in the front image at the timing of T = 4. In this case, since the VR image in which the processing delay has occurred is a front image in the line-of-sight direction, the average fluctuation evaluation value Dave of the right side image and the left side image in which no processing delay has occurred is calculated, and the average fluctuation evaluation value Dave is calculated. Is equal to or greater than the fluctuation evaluation threshold value Dlim, all the images are replaced with images of T = 1.

  In the above configuration, the host node personal computer 2 of the VR cluster processing system 30 compares the VR image data D5A to the computation node personal computers 3A to 3C according to the line-of-sight direction and operation of the VR experience detected through the motion sensor 33. D5C is generated by cluster processing, and the VR image data D5A to D5C are supplied to the corresponding displays 32A to 32C, respectively, to display VR images.

  At this time, when VR image data other than the direction of the visual line of the VR experience person is delayed in generation, the host node personal computer 2 replaces only the VR image data in which the processing delay has occurred with the VR image data of one frame before and displays it.

  On the other hand, when the VR image data in the line-of-sight direction of the VR experience person is delayed in generation, the host node personal computer 2 determines the average fluctuation evaluation value Dave and the fluctuation evaluation threshold value Dlim of the other VR image data that did not cause the processing delay. In comparison, when the average fluctuation evaluation value Dave is less than the fluctuation evaluation threshold value Dlim, only the VR image data in which the processing delay has occurred is replaced with the VR image data of the previous frame and displayed, whereas the average fluctuation evaluation value Dave fluctuates. If it is equal to or greater than the evaluation threshold value Dlim, all the VR image data is replaced with the VR image data of the previous frame and displayed.

  According to the above configuration, when a processing delay occurs in the VR cluster processing, only the VR image data in which the processing delay has occurred is obtained based on the average fluctuation evaluation value Dave of the other VR image data that did not cause the processing delay. By determining whether to replace or to replace all the VR image data, smooth error concealment according to the change of the VR image can be performed.

  In the above-described embodiment, the case where the present invention is applied to the VR cluster processing system 30 that generates a VR image by computer graphics processing has been described. However, the present invention is not limited to this, for example, a game screen in a computer game It is possible to apply the present invention to various computer graphics processing such as generation of.

  Furthermore, in the above-described embodiment, the CPU 11 of the host node personal computer 2 executes the VR cluster processing program stored in the hard disk drive 14, but the present invention is not limited to this, and the VR cluster processing program is stored. The above-described moving image cluster processing may be executed by installing the program storage medium on the host node personal computer 2.

  In this case, the program storage medium for installing the VR cluster processing program in the host node personal computer 2 is not only a package medium such as a CD-ROM (Compact Disk-Read Only Memory) or a DVD (Digital Versatile Disk), The program may be realized by a semiconductor memory or a magnetic disk in which the program is stored temporarily or permanently. In addition, as means for storing the programs in these program storage media, wired and wireless communication media such as a local area network, the Internet, and digital satellite broadcasting may be used.

  The present invention can be applied when moving image processing is performed using a plurality of personal computers, workstations, personal computers, and the like.

It is a block diagram which shows the whole structure of a moving image cluster processing system. It is a basic diagram which shows the division | segmentation state of a flame | frame. It is a basic diagram which shows the flow of a moving image cluster process. It is a flowchart which shows a cluster encoding process procedure. It is a basic diagram which shows the relationship between an input image and an image fluctuation evaluation value. It is a basic diagram which shows the mode of the error concealment at the time of encoding. It is a basic diagram which shows the mode of the error concealment at the time of encoding. It is a flowchart which shows a cluster decoding process procedure. It is a basic diagram which shows the mode of the error concealment at the time of decoding. It is a basic diagram which shows the mode of the error concealment at the time of decoding. It is a flowchart which shows a calculation node personal computer process sequence. It is a basic diagram which shows the change of the sequence before and behind encoding. It is a table | surface which shows the standard encoding process time of an MPEG2 system. It is a block diagram which shows the whole structure of a VR cluster processing system. It is a flowchart which shows a cluster VR process sequence. It is a basic diagram which shows the example of a display of a VR image. It is a basic diagram which shows the mode of the error concealment of a VR image. It is a basic diagram which shows the mode of the error concealment of a VR image. It is a basic diagram which shows the mode of the error concealment of a VR image. It is a block diagram which shows the structure of the conventional cluster processing system. It is a block diagram which shows the application structure of a cluster processing system.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Moving image cluster processing system, 2 ... Host node personal computer, 3A-3D ... Computation node personal computer, 4 ... Network, 11, 21 ... CPU11, 12, 22 ... Memory, 13, 23 ... Display Parts, 14, 24... Hard disk drive, 15, 25... Network interface, 16... Video capture circuit, 17 and 27... Bus, 30... VR cluster processing system, 31. 32C …… Display, 33 …… Motion sensor.

Claims (7)

In a moving image cluster processing system comprising a plurality of information processing devices and a management device that distributes moving image processing to each of the plurality of information processing devices,
The management device
Moving image processing instruction means for executing the moving image processing in units of pictures for each of the plurality of information processing devices;
Processing result acquisition means for acquiring the processing result of the moving image processing for each picture returned from each of the plurality of information processing devices;
When the return of the processing result from the information processing apparatus is delayed, the fluctuation evaluation value indicating the degree of change in the pattern between the delayed picture including the delayed processing result and another reference picture is compared with a predetermined threshold. When the variation evaluation value is less than the threshold value, only the delayed processing result is replaced with the corresponding processing result of the reference picture. When the variation evaluation value is equal to or greater than the threshold value, the entire delay picture A moving image cluster processing system comprising: processing result replacement means for replacing a processing result with the processing result of the reference picture. In a management device that distributes moving image processing to a plurality of information processing devices,
Moving image processing instruction means for executing the moving image processing in units of pictures for each of the plurality of information processing devices;
Processing result acquisition means for acquiring the processing result of the moving image processing for each picture returned from each of the plurality of information processing devices;
When the return of the processing result from the information processing apparatus is delayed, the fluctuation evaluation value indicating the degree of change in the pattern between the delayed picture including the delayed processing result and another reference picture is compared with a predetermined threshold. When the variation evaluation value is less than the threshold value, only the delayed processing result is replaced with the corresponding processing result of the reference picture. When the variation evaluation value is equal to or greater than the threshold value, the entire delay picture A management apparatus comprising: processing result replacement means for replacing a processing result with the processing result of the reference picture. The moving image process is a moving image compression encoding process,
The moving image processing instruction means divides the moving picture into image blocks corresponding to the plurality of information processing devices, and supplies the image blocks to the corresponding information processing devices,
The information processing apparatus returns encoded data obtained by compression encoding the image block to the management apparatus as the processing result,
The management apparatus according to claim 2, wherein the processing result replacement unit calculates the variation evaluation value from a degree of change in the design of the image block. The moving image processing is moving image decompression decoding processing,
The moving image processing instruction means divides encoded data obtained by compressing and encoding the moving image into divided encoded data corresponding to the plurality of information processing devices, and each of the divided encoded data corresponds to the corresponding information processing. Supply to the device,
The information processing apparatus returns an image block obtained by decompressing and decoding the divided encoded data to the management apparatus as the processing result,
The management apparatus according to claim 2, wherein the processing result replacement unit calculates the variation evaluation value from a degree of change in the design of the image block. The moving image processing is computer graphic image generation processing,
The moving image processing instruction means causes the information processing apparatus to generate a computer graphic image,
The information processing apparatus returns the generated computer graphic image to the management apparatus as the processing result,
The management apparatus according to claim 2, wherein the processing result replacement unit calculates the variation evaluation value from a degree of change in a pattern of the computer graphic image. In a moving image cluster processing method in which moving image processing is distributed to each of a plurality of information processing apparatuses,
A moving image processing instruction step for causing each of the plurality of information processing apparatuses to execute the moving image processing in units of pictures;
A processing result acquisition step for acquiring a processing result of the moving image processing for each picture returned from each of the plurality of information processing devices;
When the return of the processing result from the information processing apparatus is delayed, the fluctuation evaluation value indicating the degree of change in the pattern between the delayed picture including the delayed processing result and another reference picture is compared with a predetermined threshold. When the variation evaluation value is less than the threshold value, only the delayed processing result is replaced with the corresponding processing result of the reference picture. When the variation evaluation value is equal to or greater than the threshold value, the entire delay picture A moving image cluster processing method comprising: a processing result replacement step of replacing a processing result with the processing result of the reference picture. In a management device that distributes moving image processing to a plurality of information processing devices,
A moving image processing instruction step for causing each of the plurality of information processing apparatuses to execute the moving image processing in units of pictures;
A processing result acquisition step for acquiring a processing result of the moving image processing for each picture returned from each of the plurality of information processing devices;
When the return of the processing result from the information processing apparatus is delayed, the fluctuation evaluation value indicating the degree of change in the pattern between the delayed picture including the delayed processing result and another reference picture is compared with a predetermined threshold. When the variation evaluation value is less than the threshold value, only the delayed processing result is replaced with the corresponding processing result of the reference picture. When the variation evaluation value is equal to or greater than the threshold value, the entire delay picture A moving image cluster processing program for executing a processing result replacement step of replacing a processing result with the processing result of the reference picture.

Images