JP2007074042A - Moving picture processing apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving picture processing apparatus and a moving picture processing method capable of improving motion amount estimate accuracy by adjusting a frame image on the basis of a motion amount in a three-dimensional display. <P>SOLUTION: The moving picture processing method for consecutively reproducing right eye and left eye image data for reproducing a three-dimensional image in the unit of a frame executes: a correlation calculation processing for calculating the correlation between two image data in the unit of the frame; a motion amount estimate processing (S11) that estimates a motion amount to be small when a degree of the depth of the three-dimensional image is smaller than a presetting value on the basis of the calculated correlation value; and an interleave reproduction processing (S12) that thins frame image data and reproduces the resulting image when the motion amount estimate processing estimates that the motion amount is small. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a moving image processing apparatus that continuously reproduces right-eye and left-eye image data for three-dimensional image reproduction in units of frames, and in particular, the depth of a reproduced image of three-dimensional image display data. The present invention relates to a moving image processing apparatus and a moving image processing method for estimating a motion amount in a direction and performing frame correction based on the motion amount.

Generally, in 3D stereoscopic image display, 3D display processing is performed by separately preparing and reproducing image data for the right eye and left eye having different fields of view. The difference (correlation) between the image data for the right eye and the left eye is reproduced as the depth, and the magnitude of the correlation between the two images represents the degree of the depth. There is one that improves the estimation accuracy of.
As methods for estimating the amount of motion, Patent Documents 1 to 4 and the like have been proposed.
In the moving image processing apparatus disclosed in Patent Document 1, a difference in image data of a subject between frames is obtained, and an arithmetic process for obtaining a moving speed is performed based on the obtained difference data. For this reason, the amount of data to be processed is large, time is required for calculation, and a large amount of memory is required.
Japanese Patent Application Laid-Open No. 2004-259542 discloses a technique for detecting a horizontal comb-shaped portion constituted by frequency components of an input image, outputting an amount representing the degree of detection, and performing encoding by performing vertical high-pass filtering on the input image. Is disclosed.
In Patent Document 3, it is determined whether or not an image is moving by using an interlace comb, and one of block non-interlace data (frame data) and block interlace data (field data) is selected and encoded. A technique for performing the conversion is disclosed.
Patent Document 4 discloses a technique for detecting image motion between frames based on coefficients converted by discrete wavelet transform and reducing the amount of image data encoded using the detected image motion. It is disclosed.
Patent Document 5 discloses a parallax image imaging device capable of optically detecting a parallax amount obtained when viewing a subject from different points and obtaining accurate depth information of the subject.
Japanese Patent Publication No. 04-77517 JP 2002-271789 A JP 2002-64830 A JP 2001-309281 A JP 2001-16612 A

However, in moving image reproduction, since the code amount is large, frame code data may be dropped during transfer. Therefore, in the process of encoding a moving image, editing is performed such as frame decimation or code amount reduction processing when there is no motion amount when encoding is performed independently between frames as in Motion-JPEG2000. However, if the amount of motion cannot be estimated with high accuracy, the image quality may be deteriorated due to editing.
Therefore, a moving image processing apparatus and a moving image that are made in view of the above points and that can improve the motion amount estimation accuracy by adjusting the frame image based on the motion amount in the three-dimensional moving image display. An object is to provide an image processing method.

In order to achieve the above object, the invention described in claim 1 is a moving image processing apparatus that continuously reproduces two image data for right eye and left eye for three-dimensional image reproduction on a frame basis. A correlation calculating means for calculating a correlation between the two pieces of image data in frame units, and a motion amount when a depth degree of the three-dimensional image is smaller than a preset set value by the correlation value calculated by the correlation calculating means. A motion amount estimating means for estimating that the amount of motion is small, and a thinning reproduction means for thinning and reproducing the frame-unit image data when the motion amount is estimated to be small by the motion amount estimating means. To do.
According to a second aspect of the present invention, in a moving image processing apparatus that continuously reproduces image data for the right eye and left eye for reproducing a three-dimensional image in units of frames, the motion in the vertical and horizontal directions of the image is estimated. A motion amount estimating means; and a thinning reproduction means for thinning and reproducing the frame-unit image data when the motion amount in the depth direction is small and the motion amount in the vertical and horizontal directions is smaller than a preset motion amount reference value. It is characterized by having.
The invention according to claim 3 encodes frame image data in a moving image processing apparatus that continuously reproduces two image data for right eye and left eye for 3D image reproduction in units of frames. An encoding means, a correlation calculating means for calculating a correlation between the two image data in units of frames, and a depth value of the three-dimensional image by a correlation value calculated by the correlation calculating means than a preset setting value A motion amount estimation means for estimating that the motion amount is small when the motion amount is small, and a frame code amount reduction means for reducing the code amount of the code data obtained by encoding the frame image data when the motion amount is estimated to be small; It is provided with.

According to a fourth aspect of the present invention, there is provided a moving image processing apparatus that continuously reproduces right-eye and left-eye image data for three-dimensional image reproduction in units of frames, and encodes the image data in units of frames. Encoding means, motion amount estimation means for estimating the vertical and horizontal motion of the image, and the frame when the motion amount in the depth direction is small and the vertical and horizontal motion amount is smaller than a preset motion amount reference value. Frame code amount reduction means for reducing the code amount of the code data obtained by encoding the unit image data.
According to a fifth aspect of the present invention, in the moving image processing apparatus according to the second or fourth aspect of the present invention, the moving image processing apparatus further includes processing speed request input means for inputting a processing speed request, wherein the motion amount estimating means In the case of a request, the amount of motion is estimated by the amount of motion in the depth direction, or the amount of motion is estimated by the amount of vertical and horizontal direction motion that estimates the vertical and horizontal direction motion of the image. The amount of motion is estimated from the amount of motion and the amount of motion in the vertical and horizontal directions for estimating the vertical and horizontal motion of the image.
The invention according to claim 6 is the moving image processing apparatus according to any one of claims 1 to 4, further comprising a frame correction accuracy request input means for inputting a frame correction accuracy request, wherein the motion amount estimation means. If the frame correction accuracy requirement is high, the amount of motion is estimated by the amount of motion in the depth direction, or the amount of motion is estimated by the amount of vertical and horizontal motion that estimates the vertical and horizontal motion of the image, otherwise In this case, the amount of motion is estimated from the amount of motion in the depth direction and the amount of motion in the vertical and horizontal directions for estimating the vertical and horizontal motion of the image.

According to a seventh aspect of the present invention, in the moving image processing apparatus according to any one of the first to sixth aspects, a motion amount estimation unit for each region that estimates a motion amount for each image region, and for each image region And a code amount reducing means for reducing the code amount.
The invention according to claim 8 is the moving image processing apparatus according to any one of claims 1 to 7, wherein the calculation of the correlation is a partial region of the image data.
According to a ninth aspect of the present invention, in the moving image processing apparatus according to any one of the first to eighth aspects, the frame code data is encoded based on the Motion-JPEG2000 standard.
According to a tenth aspect of the present invention, in the moving image processing apparatus according to the ninth aspect, the data storing means for storing the frame code data obtained by encoding the image data, and the code data encoded after the encoding of the frame image data Reconstructing means for reconstructing the frame code data.
According to the eleventh aspect of the present invention, in the moving image processing apparatus according to the ninth aspect, the sub-band for calculating the code amount of the sub-band unit code data obtained in the wavelet transform process for wavelet transforming the image data. A code amount calculating unit; and a motion amount estimating unit configured to estimate the vertical and horizontal direction motion amount using a code amount in the sub-band 1LH or a code amount in the sub-band 1HL.
According to a twelfth aspect of the present invention, in the moving image processing apparatus according to any one of the first to eleventh aspects, the frame image data is an interlaced image, and the interlaced image data is divided into odd-numbered field image data and even-numbered field data. Field dividing means for dividing into image data, and an amount of motion for estimating an amount of motion using field data of odd field image data of an interlaced image and image data of even field of an interlaced image immediately before the interlaced image Means.
According to a thirteenth aspect of the present invention, in the moving image processing apparatus according to the twelfth aspect, there is provided a motion amount calculating means for calculating a motion amount between odd and even fields generated by decomposing one interlaced image signal. Determining means for determining the amount of motion based on the amount of motion between the odd and even fields.

According to a fourteenth aspect of the present invention, there is provided a moving image processing method for continuously reproducing right-eye and left-eye image data for three-dimensional image reproduction in units of frames, between the two image data in units of frames. A correlation calculation step for calculating the correlation of the three-dimensional image, a motion amount estimation step for estimating that the motion amount is small when the depth degree of the three-dimensional image is smaller than a preset setting value based on the calculated correlation value, A thinning reproduction step of thinning and reproducing the frame image data when the amount of motion is estimated to be small by the amount estimation step.
According to a fifteenth aspect of the present invention, there is provided a moving image processing method for continuously reproducing right-eye and left-eye image data for three-dimensional image reproduction in units of frames, and estimating vertical and horizontal movements of the image. A motion amount estimation step; and a thinning reproduction step for thinning and reproducing the frame image data when the motion amount in the depth direction is small and the vertical and horizontal motion amounts are smaller than a preset motion amount reference value. It is characterized by that.
According to a sixteenth aspect of the present invention, there is provided an encoding method for encoding frame image data in a moving image processing method for continuously reproducing right-eye and left-eye image data for 3D image reproduction in units of frames. A step of calculating the correlation between the two image data in units of frames, and the amount of motion is small when the depth of the three-dimensional image is smaller than a preset value based on the calculated correlation value And a frame code amount reduction step for reducing a code amount of code data obtained by encoding the frame image data when the motion amount is estimated to be small. To do.
According to a seventeenth aspect of the present invention, there is provided an encoding method for encoding frame image data in a moving image processing method for continuously reproducing right-eye and left-eye image data for three-dimensional image reproduction in units of frames. Step, a motion amount estimation step for estimating the vertical and horizontal motion of the image, and the frame image data when the motion amount in the depth direction is small and the vertical and horizontal motion amount is smaller than a preset motion amount reference value. And a frame code amount reduction step for reducing the code amount of the encoded code data.

  According to the present invention, in a moving image processing apparatus that continuously reproduces image data for right and left eyes for three-dimensional image reproduction in units of frames, the amount of motion is estimated with high accuracy, By reducing the code amount by thinning out the frame or editing the frame image when the moving amount of the moving image including the direction is small, it is possible to prevent image quality deterioration due to editing.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, the frame thinning control based on the depth motion amount will be described.
FIG. 1 is a diagram illustrating a configuration of a processing block that performs frame correction based on a depth motion amount of the moving image processing apparatus according to the first embodiment of the present invention. In the present embodiment, the configuration of a moving image processing apparatus that continuously reproduces image data for right eye and left eye for 3D image reproduction in units of frames will be described as an example.
The moving image processing apparatus shown in FIG. 1 is a diagram illustrating a configuration of a processing block that performs frame correction based on a depth motion amount. Three-dimensional image input unit 101 to which image data for right eye and left eye for each frame is input, image depth calculation unit 102, motion amount evaluation unit 103, frame (code data) editing unit 104, and image data storage unit 105 , An evaluation reference motion amount storage unit 106, a frame-by-frame depth storage unit 107, a frame motion amount evaluation result storage unit 108, and a frame motion amount change evaluation result storage unit 109.
In the moving image processing apparatus configured as described above, the image depth calculation unit 102 that calculates the image depth for each frame calculates a correlation between the two image data for the right eye and the left eye for each frame, The degree of depth of the three-dimensional image is estimated based on the calculated correlation value. The motion amount evaluation unit 103 determines whether the depth degree estimated by the image depth calculation unit 102 is smaller than a preset setting value. When it is determined that the amount of motion is small, the frame code data editing unit 104 commands frame correction. The frame code data editing unit 104 thins out frame image data determined to have a small amount of motion.
That is, in the present embodiment, right-eye and left-eye image data is input by the 3D image input unit 101, and then the depth amount of the image data is estimated for each frame by the image depth calculation unit 102. To do. Subsequently, the reference motion amount stored in the evaluation reference motion amount storage unit 106 in the motion amount evaluation unit 103 is compared with the depth motion amount of the frame estimated in the image depth calculation unit 102, and the frame motion amount evaluation result is displayed as a frame. The result is stored in the motion amount evaluation result storage unit 108. Then, the motion amount evaluation result of the frame stored immediately before is compared with the evaluation result of the motion amount of the new frame, and the frame motion amount change evaluation result is stored in the frame motion amount change evaluation result storage unit 109. The frame code data editing unit 104 edits the frame code data in units of frames using the frame motion amount change evaluation result.

FIG. 2 is a block diagram illustrating a configuration of the image depth calculation unit 102 that estimates the amount of motion in the depth direction for each frame.
The image depth calculation unit 102 shown in FIG. 2 includes an image extraction unit 41 for extracting an image for each region, a depth calculation unit 42 for each image region for calculating a depth for each image region, a frame depth calculation unit 43, and image data. The storage unit 44, the image region data storage unit 45, and the depth storage unit 46 for each region are configured.
The image area depth calculation unit 42 includes an area image extraction unit 51, a line image data extraction unit 52, a correlation calculation unit 53, a depth calculation unit 54, a line image data storage unit 55, and a correlation data storage unit 56. Has been.
The image depth calculation unit 102 reads the three-dimensional image data with the three-dimensional image input unit 101, and calculates the image data of the image region with the image extraction unit 41 for each region. In the depth calculation unit 42 for each image region, the image data of the image region is read by the three-dimensional image input unit 101 for each image region, the image data is read for each line by the image data extraction unit 52 for each line, and correlation calculation is performed. The unit 53 calculates a correlation value for each line. Then, the depth calculation unit 54 calculates depth data for each region. Note that the depth data is typically proportional to the reciprocal of the correlation value.
The frame depth calculation unit 43 uses the stored correlation values in units of lines, totals the reciprocals of the correlation values for each line of the image, calculates the average value for each line, and calculates the depth degree of the entire image. . The frame depth storage unit 107 stores depth information when image data of the frame is reproduced.
In the operation image processing apparatus configured as described above, frame correction is performed to reduce the processing burden or to save the amount of frame code data. I try to do it. For this reason, as described above, the depth motion amount is evaluated as the motion amount.

FIG. 3 is a flowchart showing the frame thinning control process based on the depth motion amount in the moving image processing apparatus according to the first embodiment.
In this case, first, image data for right eye and left eye, which is three-dimensional image data, is input for each line (S1), and a correlation value for each line of the respective image data is calculated (S2). Next, the reciprocal of the correlation value is set as a depth degree for each line of the image (S3), and the reciprocal of the correlation value is totaled for each line of the image to calculate an average value for each line to obtain a depth degree of the region. (S4).
Next, in step S5, it is determined whether or not the processing in steps S1 to S4 has been performed on all the images in the region. If not performed on all the images in the region (N in S5), Returning to step S1, the process is repeated.
On the other hand, if it is determined in step S5 that the processing in steps S1 to S4 has been performed on all the images in the region (Y in S5), it is determined whether or not the processing has been performed on the entire region in subsequent step S6. If the process is not performed for each area (N in S6), the area is changed (S7), and the process returns to step S1 to perform the process again.
If it is determined that the process has been performed for each region (Y in S6), the process proceeds to step S8, and if necessary, the average value of the depth degree of the frame image is compared with a predetermined threshold value. A depth (degree) evaluation value of the frame image is determined (S8). For example, if it is smaller than a certain first threshold, it is determined that the depth evaluation value is small, if it is larger than a certain first threshold, it is determined that the depth evaluation value is large, and otherwise, it is determined that the depth evaluation value is medium. .
Next, the depth evaluation value of the frame image data is stored (S9), and the magnitude of the depth change / increase / decrease in the depth direction is identified as compared with the depth evaluation value of the frame image data (S10).
For example, if N = | depth value evaluation value of the image frame−depth evaluation value of the previous image frame |, and N is smaller than a certain first threshold, it is determined that the depth change = small, and N is larger than a certain second threshold. → Depth change = large, otherwise it is determined → Depth change = medium.
Next, in step S11, it is determined whether or not the change in the amount of motion of the frame image data (the amount of motion in the depth direction) is smaller than a preset amount of change. If the depth change is small (Y in S11), The frame image data is deleted (frame thinning) (S12), and the process proceeds to step S13.
On the other hand, when a negative result is obtained in step S11 (N in S11), the process proceeds to step S13 without deleting the frame image data in step S12.
In step S13, it is determined whether or not all the frames have been processed. If all the frames have not been processed, the process returns to step S1. On the other hand, if all the frame processes have been completed, the process ends.

ここで、画素の（ｘ，ｙ）は座標値であり、Ｉ（ｘ，ｙ）は画像Ｉの座標（ｘ，ｙ）の値であり、Ｊ（ｘ，ｙ）は画像Ｊの座標（ｘ，ｙ）の値である。画像は、ｘの最大値がｌx-1、ｙの最大値がｌy-1よりなる。
なお、相関値は必ずしも上記に示すような算出式（式１）により求めなくてもよく、画像データ間のデータ値の近さが算出できればよく、例えば、あるｙ番目のラインの相関Ｓ（ｙ）は、以下の式２により求めてよい。その場合はより簡潔になる。
相関値Ｓ（ｙ）は、

As described above, in the moving image processing apparatus according to the first embodiment, the correlation between the two image data for the right eye and the left eye is calculated, and the depth degree of the three-dimensional image is estimated based on the calculated correlation value. Then, it is determined whether or not the depth degree is smaller than a preset setting value. When it is determined that the amount of motion is small, frame correction is performed. In this case, frame image data is thinned out and reproduced. In the above-described frame thinning control process, when calculating the amount of motion, the amount of motion can be evaluated by evaluating not only the amount of motion in the depth direction but also the motion in the two-dimensional plane direction.
Here, in the moving image processing apparatus of the present embodiment, the amount of motion is estimated based on the movement in the depth direction when 3D image data is reproduced. The correlation between two image data for the right eye and the left eye is used to estimate the amount of motion in the depth direction.
In the present embodiment, the correlation between the two image data for the right eye and the left eye is calculated according to the following equation.
S (y) is a correlation value of a y-th line of an image (y is a numerical value in the range of 0 to ly-1).

Here, (x, y) of the pixel is a coordinate value, I (x, y) is a value of the coordinate (x, y) of the image I, and J (x, y) is a coordinate of the image J (x , Y). In the image, the maximum value of x is 1x-1, and the maximum value of y is ly-1.
Note that the correlation value does not necessarily have to be obtained by the calculation formula (formula 1) as described above, and it is only necessary to calculate the closeness of the data value between the image data. For example, the correlation S (y of a certain y-th line) ) May be obtained by the following equation 2. In that case it becomes more concise.
The correlation value S (y) is

Further, the depth degree is estimated by the following relational expression.
The formula for calculating the depth is
Correlation value <S → depth degree = large Correlation value> S → depth degree = small Expression 3
In the example shown in FIG. 3, the depth for each image region is calculated, and the average value is estimated as the depth of the entire image. Further, when the correlation value is smaller than the predetermined value S, the depth degree is estimated to be large, and when the correlation value is larger than the predetermined value S, the depth degree is estimated to be small.
Also, the amount of motion in the depth direction is estimated from a change in depth with time, that is, a change in depth in units of frames.
By the way, in the moving image processing apparatus of the first embodiment described above, the estimation of the motion amount only in the depth direction is mentioned, but in reality there is also a motion on a two-dimensional plane.
Therefore, in the second embodiment, the moving image processing that can improve the estimation accuracy of the motion amount in the three-dimensional direction by estimating the motion amount including the motion in the two-dimensional plane direction as well as the depth direction. The apparatus will be described. That is, a moving image processing apparatus that estimates the amount of motion by evaluating not only the amount of motion in the depth direction but also the motion in the two-dimensional plane direction when calculating the amount of motion will be described.

FIG. 4 is a diagram illustrating a configuration of a depth degree calculation unit and a peripheral unit of the moving image processing apparatus according to the second embodiment. In addition, the same code | symbol is attached | subjected to the same site | part as FIG. 2, and description is abbreviate | omitted.
The depth degree calculation unit 102 shown in this figure includes an image extraction unit 41 for each region, a depth calculation unit 42 for each image region, a frame depth calculation unit 43, an image data storage unit 44, an image region data storage unit 45, and a depth storage unit for each region. 46, an intra-screen motion amount calculation unit 81, and a frame depth direction motion amount calculation unit 82.
In this case, the in-screen motion amount calculation unit 81 analyzes the 3D moving image data and estimates the motion in the 2D plane direction. For example, wavelet transform is performed on the three-dimensional moving image data by the wavelet transform processing unit of the encoding processing unit, and the motion amount in the two-dimensional plane direction is determined by the code amount of the 1LH sub-band component and the code amount of the 1HL sub-band component. presume.
The estimation accuracy of the motion amount is improved by evaluating not only the motion amount in the depth direction but also the motion in the two-dimensional plane direction. Of course, the motion amount estimation method may be switched depending on the processing speed request. For example, when high-speed processing is required, the amount of motion is estimated only by movement in the depth direction or only by movement in the two-dimensional plane direction. Otherwise, movement in the depth direction and two-dimensional plane direction is estimated. It is also possible to estimate the amount of motion.

FIG. 5 is a conceptual explanatory diagram of the frame thinning process based on the depth motion amount.
In this embodiment, when the movement in the depth direction in the three-dimensional display is small, the frame is deleted (thinning out the frames) to reduce the processing load or save the storage amount of the frame code data. That is, the present embodiment is characterized in that the code data is edited instead of the image data. In the encoding of moving image data by inter-frame difference such as MPEG, it is not possible to easily realize thinning out frames because the encoding is not performed independently for each frame at the code level. On the other hand, in the present embodiment, it is possible to easily reduce the frame data at the code level by encoding independently between frames by Motion-JPEG2000 encoding.
The depth direction motion amount estimation is described in detail in Japanese Patent Application No. 2005-202358 filed earlier by the present applicant.
Next, the frame code amount reduction control method based on the depth motion amount of the present invention will be described.
Note that the configuration of the moving image processing apparatus in that case can be realized by the moving image processing apparatus shown in FIG. 2, and is not shown in the figure, but the frame image determined that the amount of motion of the frame code data editing unit 104 is small. A feature is that it has a function to thin out and reproduce data.

FIG. 6 is a flowchart illustrating a frame code amount reduction process based on the depth motion amount in the moving image processing apparatus according to the third embodiment. The same processes as those in FIG. 3 are denoted by the same reference numerals, and detailed description thereof is omitted.
Also in this case, first, image data for right eye and left eye, which is three-dimensional image data, is input for each line (S1), and a correlation value for each line of each image data is calculated (S2). Next, the reciprocal of the correlation value is set as a depth degree for each line of the image (S3), and the reciprocal of the correlation value is totaled for each line of the image to calculate an average value for each line to obtain a depth degree of the region. (S4).
Next, in step S5, it is determined whether or not the processing in steps S1 to S4 has been performed on all the images in the region. If not performed on all the images in the region (N in S5), Returning to step S1, the process is repeated.
On the other hand, if it is determined in step S5 that the processing in steps S1 to S4 has been performed on all the images in the region (Y in S5), it is determined whether or not the processing has been performed on the entire region in subsequent step S6. If the process is not performed for each area (N in S6), the area is changed (S7), and the process returns to step S1 to perform the process again.
If it is determined that the process has been performed for each region (Y in S6), the process proceeds to step S8, and if necessary, the average value of the depth degree of the frame image is compared with a predetermined threshold value. A depth (degree) evaluation value of the frame image is determined (S8).
Next, the depth evaluation value of the frame image data is stored (S9), and the magnitude of the depth change / increase / decrease in the depth direction is identified as compared with the depth evaluation value of the frame image data (S10).
Next, in step S21, it is determined whether or not the change in the amount of motion of the frame image data (the amount of motion in the depth direction) is smaller than a preset amount of change. If the depth change is small (Y in S11), The code amount of the frame image data is reduced (S21), and the process proceeds to step S13.
On the other hand, when a negative result is obtained in step S11 (N in S11), the process proceeds to step S13 without deleting the frame image data in step S21.
In step S13, it is determined whether or not all the frames have been processed. If all the frames have not been processed, the process returns to step S1. On the other hand, if all the frame processes have been completed, the process ends.

FIG. 7 is a conceptual explanatory diagram of a process of reducing the frame code amount based on the depth motion amount.
In the third embodiment, when the motion in the depth direction in the three-dimensional display is small, the frame code amount is deleted to reduce the processing burden or save the storage amount of the frame code data.
The third embodiment is characterized in that the code data is edited instead of the image data. In this case, in the encoding of moving image data using inter-frame differences such as MPEG, it is not possible to easily edit the frame code data because it is not encoded independently for each frame at the code level. Therefore, in the present embodiment, it is possible to easily reduce the frame code amount at the code level by encoding independently between frames by Motion-JPEG2000 encoding.
Next, a motion amount estimation method in the depth direction for each region will be described.
In the moving image processing apparatus of the present embodiment, a motion amount is estimated by examining a part of an image area, and frame correction can be performed based on the motion amount. Since only a part of the image area is examined, the processing efficiency can be improved.
In the present embodiment, the above-described calculation of the correlation value for motion amount estimation is performed, for example, in (Expression 1) or (Expression 2) with respect to a line from the first line to several lines (yn) by Σ (y = 0 to yn) S (y) is calculated and estimated as the correlation value of the entire image.
Here, a part of the image regions may be a region of interest where the estimation accuracy is important, or the correlation between the image data of the image regions read in advance may be calculated giving priority to the processing efficiency. Absent. Not having to analyze all the image data has a great effect of improving the processing efficiency.

FIG. 8 is a diagram illustrating a configuration of a calculation processing block that calculates a depth degree for each image attribute in the image processing apparatus according to the fourth embodiment. In addition, the same code | symbol is attached | subjected to the same site | part as FIG. 2, and description is abbreviate | omitted.
The moving image processing apparatus shown in FIG. 8 includes a three-dimensional image input unit 101, an image data storage unit 12, an attribute region extraction unit 13, an attribute image region data storage unit 14, an image attribute depth calculation unit 15, and a depth data storage unit. 16.
Further, the image attribute depth calculation unit 15 includes an image attribute image extraction unit 21, a line image data extraction unit 22, a correlation calculation unit 23, a depth calculation unit 24, a line image data storage unit 25, and a correlation data storage unit 26. It is configured.
The three-dimensional image input unit 101 reads right-eye and left-eye three-dimensional image data, and the attribute-specific region extraction unit 13 performs region extraction for each of the right-eye and left-eye image attributes. In the image attribute depth calculation unit 15, the image attribute image extraction unit 21 reads the image data of the attribute area, the line image data extraction unit 22 reads the image data in units of lines, and the correlation calculation unit 23 performs line units. To calculate the correlation value. The depth calculation unit 23 calculates depth data. When the correlation is calculated, the correlation value for each line is stored in the correlation data storage unit 26.
The depth calculation unit 24 calculates the degree of depth using the correlation value for each line in which the image data of the attribute area is stored. In addition, the depth calculation unit 24 uses the stored correlation values in units of lines to total the reciprocal of the correlation values for each line of the image, calculates an average value for each line, and calculates the depth degree of the entire image. The depth data storage unit 16 stores depth information calculated for each image attribute area.

FIG. 9 is a principle diagram for calculating the depth for each three-dimensional image region described above.
The correlation calculation unit 23 performs correlation calculation processing on the area data for each attribute of the two image data for the right eye and the left eye. In this case, the correlation calculation unit 23 calculates the correlation between the image data in the same region of the two image data for the right eye and the left eye. Therefore, the region for each image attribute may be calculated by calculating the image attribute region for only one image data out of the two image data for the right eye and the left eye.
FIG. 10 is a flowchart showing a calculation process for calculating the depth degree for each image attribute as the fourth embodiment.
In this case, right-eye and left-eye 3D image data is input (S41), and right-eye or left-eye 3D image data is analyzed to calculate a region for each attribute (S42). .
Next, it inputs for every line of the three-dimensional image data for right eyes and left eyes (S43), and calculates a correlation value for each line of each image data (S44). Then, the reciprocal of the correlation value is set as the depth degree for each line of the image (S45), the reciprocal of the correlation value is totaled for each line of the image, and the average value for each line is calculated to obtain the depth degree of the area (S46). .
In subsequent step S47, it is determined whether or not all the images in the region have been performed. If not performed for all the images in the region (N in S47), the process returns to step S43 for processing.
On the other hand, if it is determined in step S47 that the image has been applied to all the images in the region (Y in S47), it is determined in step S48 whether or not the operation has been performed on all regions, and the entire region is determined for each region. If not (N in S48), after changing the area (S49), the process returns to step S43 to perform the process again. On the other hand, if the process is performed for each region with respect to all regions (Y in S48), the process is terminated.

Next, frame correction at a code level in JPEG 2000 will be described.
As described above, when frame image data is encoded or decoded based on the Motion-JPEG2000 (ISO / IEC 15444-1) standard, Motion-JPEG2000 can handle each frame independently. The method of the present invention can be easily applied and is effective. In particular, it is effective to reduce a part of a frame of a moving image at a code level or to reduce a code amount.
Also, when encoding is performed with JPEG2000, the encoding can be easily performed in units of frames by analyzing wavelet coefficient code data when estimating the amount of motion as described later.
The standard called Motion JPEG 2000 is for continuously reproducing still images encoded in the JPEG 2000 format. First, the “JPEG2000 algorithm” will be briefly described.

Encoding according to the JPEG2000 (ISO / IEC 15444-1) standard is performed in the following procedure.
(1) The frame data of the interlaced image is converted into data for each color component of Y, Cr, and Cb.
(2) Two-dimensional discrete wavelet transform is performed on the color data of each color component.
(3) The obtained wavelet coefficient is subjected to a scalar quantization process specified in JPEG2000.
(4) The entropy encoding process (arithmetic encoding process based on so-called coefficient modeling) defined in JPEG2000 is performed on the scalar quantized data.
After performing the processing (2) to (4) on all color data,
(5) A code string defined by JPEG2000 is generated.
The decoding process is the reverse procedure.
Of course, these processes may be realized by a hardware circuit. The processing speed can be increased. Note that there is already a moving image processing apparatus that implements all the encoding processing compliant with JPEG2000 with a hardware circuit.

FIG. 11 is a diagram for explaining a hierarchical encoding algorithm that is the basis of JPEG 2000. The color space transform / inverse transform unit 110, the two-dimensional wavelet transform / inverse transform unit 111, the quantization / inverse quantization unit 112, and the entropy. The encoding / decoding unit 113 and the tag processing unit 114 are configured. One of the biggest differences compared to the JPEG algorithm is the conversion method. Discrete cosine transform (DCT) is used in JPEG, and discrete wavelet transform (DWT) is used in the hierarchical coding compression / decompression algorithm.
The advantage that DWT has better image quality in the high compression region than DCT is one of the major reasons adopted in JPEG2000, which is the successor algorithm of JPEG.
Another major difference is that, in the latter case, a function block called a tag processing unit 114 is added in order to perform code formation at the final stage. In this part, compressed data is generated as a code stream during the compression operation, and a code stream necessary for decompression is interpreted during the decompression operation. And with the code stream, JPEG2000 can realize various convenient functions. For example, as shown in FIG. 15, the compression / decompression operation of a still image can be freely stopped at an arbitrary hierarchy (decomposition level) corresponding to octave division in block-based DWT.
A color space conversion unit is often connected to the input / output portion of the original image. For example, the RGB color system composed of R (red) / G (green) / B (blue) components of the primary color system and the Y (yellow) / M (magenta) / C (cyan) components of the complementary color system This corresponds to the part that performs conversion from the YMC color system consisting of the above to the YUV or YCbCr color system or the reverse conversion.

Hereinafter, the JPEG2000 algorithm will be described in detail.
In general, as shown in FIG. 12, in a color image, each component (in this case, the RGB primary color system) of the original image is divided by a rectangular area (tile). Each tile, for example, R00, R01, ..., R15 / G00, G01, ..., G15 / B00, B01, ..., B15, is a basic unit for executing the compression / decompression process. Therefore, the compression / decompression operation is performed independently for each component and for each tile.
At the time of encoding, the data of each tile 120 of each component is input to the color space conversion unit 110 in FIG. 11 and subjected to color space conversion, and then the two-dimensional wavelet conversion unit 111 performs two-dimensional wavelet conversion (forward conversion). Is applied and spatially divided into frequency bands.
FIG. 13 shows sub-bands at each decomposition level when the number of decomposition levels is three. That is, the tile original image (0LL) (decomposition level 0) obtained by the tile division of the original image is subjected to two-dimensional wavelet transform, and sub-bands (1LL, 1HL, 1LH, 1HH). Subsequently, the low-frequency component 1LL in this hierarchy is subjected to two-dimensional wavelet transform to separate sub-bands (2LL, 2HL, 2LH, 2HH) shown in the decomposition level 2. Similarly, the low-frequency component 2LL is also subjected to two-dimensional wavelet transform to separate the sub-bands (3LL, 3HL, 3LH, 3HH) shown in the decomposition level 3.

Further, in FIG. 15, the sub-bands to be encoded at each decomposition level are represented in gray.
For example, when the number of decomposition levels is 3, the sub-bands shown in gray (3HL, 3LH, 3HH, 2HL, 2LH, 2HH, 1HL, 1LH, 1HH) are to be encoded, and the 3LL sub-band is Not encoded.
Next, the bits to be encoded are determined in the specified encoding order, and the context is generated from the bits around the target bits in the quantization unit 112 in FIG.
The wavelet coefficients that have undergone the quantization process are divided into non-overlapping rectangles called “precincts” for each sub-band. This was introduced to use memory efficiently in implementation. As shown in FIG. 15, one precinct consists of three rectangular regions that are spatially matched. Further, each precinct is divided into non-overlapping rectangular “code blocks”. This is the basic unit for entropy coding.
The entropy encoding unit 113 (see FIG. 11) performs encoding on the tile of each component by probability estimation from the context and the target bit. In this way, encoding processing is performed in tile units for all components of the original image.

The minimum unit of code data formed by the entropy encoding unit 113 is called a packet. The packets are sequenced in progressive order, which is indicated by one of the image header segments. The packets are arranged by some progressive order data, eg, region, resolution, layer, and color component, respectively. That is, in the JPEG2000 standard, by changing the priority order of the four image elements of image quality (layer (L)), resolution (R), component (C), and position (precinct (P)), the following 5 Street progression is defined.
(1) LRCP progression: Since decoding is performed in the order of precinct, component, resolution level, and layer, the image quality of the entire image is improved each time the layer index is advanced, and the progression of image quality can be realized. Also called layer progression.
(2) RLCP progression: Since the decoding is performed in the order of precinct, component, layer, and resolution level, the progression of resolution can be realized.
RPCL progression: Since decoding is performed in the order of layer, component, precinct, and resolution level, it is a progression of resolution as in RLCP, but the priority of a specific position can be increased.
(3) PCRL progression: Since decoding is performed in the order of layer, resolution level, component, and precinct, decoding of a specific part is prioritized, and progression of spatial position can be realized.
(4) CPRL Progression: Since decoding is performed in the order of layer, resolution level, precinct, and component, for example, when a color image is progressively decoded, a component progression that first reproduces a gray image can be realized.
As described above, in the JPEG2000 standard, an image is divided into regions (image constituent elements such as tiles or precincts), resolutions, hierarchies (layers), and color components, and each is independently encoded as a packet. These packets are characterized in that they can be identified and extracted from the code stream without decoding.

Finally, the tag processing unit (code string forming unit) combines all the encoded data from the entropy coder unit into one code stream and performs processing for adding a tag thereto.
FIG. 14 simply shows the structure of the code stream. Tag information called a header is added to the head of the code stream and the head of the partial tiles constituting each tile, followed by the encoded data of each tile. A tag is again placed at the end of the code stream.
On the other hand, at the time of decoding, contrary to the time of encoding, image data is generated from the code stream of each tile of each component.
This will be briefly described with reference to FIG. In this case, the tag processing unit 114 interprets tag information added to the code stream input from the outside, decomposes the code stream into code streams of each tile of each component, and each tile of each component. Decoding processing is performed for each code stream.
The positions of the bits to be decoded are determined in the order based on the tag information in the code stream, and the inverse quantization unit 112 determines from the sequence of the peripheral bits (that has already been decoded) at the target bit position. A context is created.
The entropy decoding unit 113 performs decoding by probability estimation from the context and the code stream, generates a target bit, and writes it in the position of the target bit. Since the data decoded in this way is spatially divided for each frequency band, the two-dimensional wavelet inverse transform unit 111 performs two-dimensional wavelet inverse transform on each of the tiles of each component of the image data. Restored. The restored data is converted into original color system data by the color space inverse conversion unit.
As described above, since the JPEG 2000 code data has a hierarchical structure in units of packets having five kinds of progression, it is possible to easily reduce the code amount by deleting the packets constituting the hierarchy in units of layers. It can be done.
The standard called Motion-JPEG2000 is to continuously reproduce still images encoded in the JPEG2000 format. Since it is encoded independently between frames as in Motion-JPEG2000, deleting a frame code obtained by encoding a frame image can be easily realized.
After the code data is formed by the Motion-JPEG2000 encoding, the frame code data can be deleted at the code level, or the code data can be easily edited.

Next, a method for estimating the amount of motion in the two-dimensional plane direction will be described.
In this embodiment, a plurality of means for efficiently determining the amount of motion in the two-dimensional plane direction are provided. A minimum function for determining whether or not the movement in the two-dimensional plane direction is more than normal is provided.
The determination of the amount of motion in the two-dimensional plane direction is to determine whether the moving speed of the object in the image is high or low. A typical estimation method of the motion amount in the two-dimensional plane direction is a motion amount determination based on a difference between frames. As a means for determining the amount of motion in the two-dimensional plane direction, a difference in image data between frames is used. Typically, the image data of the previous and subsequent frames may be sequentially compared in units of pixels. Alternatively, in the reproduction of an interlaced moving image to be described later, a frame may be decomposed into fields and compared in units of fields. The amount of motion may be determined based on the amount of motion between fields before and after the interlaced image signal is generated by decomposing. It is more efficient by comparing a smaller amount of data than the entire frame data. Moreover, since it is a comparison between every other data, there is sufficient accuracy.
The amount of motion in the two-dimensional plane direction may be easily estimated by comparing the amount of data in the frequency domain within the frame.
According to an empirical rule that most subjects move in the horizontal direction in general shooting, the amount of movement can be determined by detecting the horizontal movement speed (high speed / low speed) of the subject in the frame. The moving speed (high speed / low speed) of the object can be detected by a simple calculation process using a smaller amount of data than the determination using the difference in image data between the former frames.

Here, the determination of the amount of movement in the two-dimensional plane direction by analyzing the comb formation will be described.
FIG. 16A is a diagram for describing frame data composed of interlaced image data captured by a video camera or the like. Each image data is three-dimensional image data for right eye and left eye.
In this way, the frame 0 image is scanned in the interlace format together with the shooting start time t0, and a frame image is formed. After a certain time elapses, for example, 1/60 seconds later, the frame 1 image is scanned in the interlace format. To form a frame image. Interlaced scanning is performed by dividing the scanning line into a plurality of times as will be described later. Then, a total of n frames of images are formed in an interlaced format in a fixed time unit, for example, 1/60 second unit until the end time tn. In general, in frame data, there is a difference in shooting time between an odd field and an even field.
FIG. 16B is a diagram for explaining field data of interlaced frame data.
As shown in FIG. 16B, an interlaced image is obtained by scanning a pixel line (scanning line indicated by a solid line) and then skipping a pixel line immediately below (scanning line indicated by a dotted line) to obtain two pixels. The lower line (scan line indicated by a solid line) is scanned. Next, scanning of pixel lines (scan lines indicated by dotted lines) that were not scanned last time is performed from above. As described above, a certain time, for example, 1/120 second has elapsed since the line of the pixel immediately below after scanning a certain line. Therefore, as will be described later, images having different scan times are formed in one frame data of an interlace format scanned and read.
As shown in FIG. 16B, interlaced frame data can be decomposed into field data having different scan times. The odd field data is data scanned by odd scan lines (scan lines indicated by solid lines), and the even field data is data scanned by even scan lines (scan lines indicated by dotted lines).
Conversely, it goes without saying that one frame data is generated by combining odd field data and even field data. In other words, the image data of the odd field data and the even field data is alternately arranged (supplemented with the data of the unscanned line) for each scanned line (in the above example, for each scanning line of one pixel unit). Data can be formed. At this time, as described above, in the frame data, there is a difference in the scan time between the odd field and the even field, so that when the subject is moving, completely continuous data is not formed.

Thus, even if the frame image data is an interlaced image, the right-eye and left-eye three-dimensional image data is data when the same place is viewed at the same time in line units. It is possible to estimate the amount of motion in the depth direction using the correlation between the right-eye and left-eye 3D image data, whether it is field data or synthesized frame data. is there.
In the odd field data and even field data generated by decomposing the frame data into field data, the subject (image) is almost continuous, but if the subject moves at high speed, the subject in the odd field data There is a shift between the (image) and the subject (image) in the even field data. In general, in the frame data, as the movement in the horizontal direction is larger (the movement in the horizontal direction is faster) due to the difference in the shooting time between the odd field and the even field, the comb shape appears more remarkably as shown in FIG. By adding a new frame when the movement in the lateral direction is large, it can be determined by a simple calculation as described above.

As described above, when the subject moves at a high speed, the subject (image) in the odd field data and the subject (image) in the even field data are shifted due to different scan times. If it moves in the direction, the amount of comb-shaped parts will increase accordingly.
Therefore, the amount of movement in the two-dimensional plane direction can be estimated from the amount of comb formation.
That is, the amount of motion between odd and even fields generated by decomposing one interlaced image signal is obtained by calculating a comb type amount. By providing a function that estimates the amount of motion based on the amount of comb formation, it is possible to estimate the motion between fields within one frame, so that the motion of an image (frame) can be estimated efficiently without examining multiple frames. . A smooth frame that reduces the effects of comb noise can be added.
Now, the comb shift amount L shown in FIG. 18 becomes longer in proportion to the moving speed of the subject in the interlaced image. Here, the 1LH sub-band coefficient value obtained by two-dimensional discrete wavelet transform of image data of a non-interlaced image is proportional to the sum of horizontal edge components, that is, the moving speed of the subject in the interlaced image. Then increase. Also, the coefficient value of the 1HL sub-band increases in proportion to the sum of the vertical edge components, but the subject movement follows the empirical rule that most subjects move in the horizontal direction in general shooting. It can be considered that it hardly changes depending on the speed. Therefore, as a simpler method for determining the amount of motion between frames, the moving speed of the subject in the interlaced image is determined using the above characteristics of the coefficient value of the 1 LH sub-band. That is, the amount of motion between the frames is determined based on the coefficient value of the sub-band among the wavelet coefficients obtained by the wavelet transform unit. In the case of wavelet coding, the amount of motion can be easily estimated from the coefficient value of the 1LH sub-band component. The amount of motion in the two-dimensional plane direction can be determined by comparing the frequency domain data.

By the way, the determination of the moving speed of the subject in the interlaced image may be performed by a sub-block (JPEG 2000 standard code block). Compared to the case where the moving speed of the subject is determined in units of sub-bands, for example, based on the determination result in units of sub-blocks that are smaller than the sub-band, for example, a 32 × 32 pixel matrix, By determining the moving speed, it is possible to correctly recognize a case where only a relatively small subject moves at high speed in a still image (landscape). That is, the frame is divided into sub-bands by wavelet transform, and further divided into sub-blocks, and the amount of motion between frames is determined using coefficient values of arbitrary sub-blocks. The amount of motion can be determined by comparing frequency domain data in finer units.
Therefore, the determination of the amount of motion in the two-dimensional plane direction based on the code amount will be described next.
What has been described above can be determined not by the coefficient value but also by the code amount. That is, from the same idea as described above, the code amount of the sub-band of 1 LH among the wavelet coefficients obtained by performing the two-dimensional discrete wavelet transform on the frame image data of the interlaced image, particularly the subject in the frame to be imaged. Focusing on the fact that the code amount of the 1 HL sub-band shows a substantially constant value, while increasing in accordance with the moving speed in the horizontal direction, the horizontal direction of the subject in the frame is utilized using this characteristic. The amount of movement can be determined.
The code amount in sub-band 1LH is compared with the code amount in sub-band 1HL, or the code amount of a predetermined sub-block in sub-band 1LH and the code amount of a predetermined sub-block in sub-band 1HL And the amount of motion between frames is determined based on the comparison result.
At this time, the code amount in the sub-band 1LH and the code amount in the sub-band 1HL are sequentially compared in a predetermined order in units of sub-band regions, or a predetermined sub-block in the sub-band 1LH And the code amount of a predetermined sub-block in sub-band 1HL are sequentially compared in a predetermined order in units of sub-blocks, and the amount of motion between frames is determined based on the comparison result. It is also possible to efficiently determine the amount of movement by determining that the movement is large when a certain value exceeds a preset value, and stopping the subsequent comparison.

FIG. 17 is a flowchart showing processing for estimating a motion amount in the two-dimensional plane direction according to the fifth embodiment of the present invention.
In this case, first, sum1LH, which is the sum of absolute values of 1LH coefficients, is obtained from the wavelet coefficients obtained by the two-dimensional discrete wavelet transform (S51). Next, sum1HL, which is the sum of absolute values of 1HL coefficients, is obtained (S52). Then, the sum1LH / sum1HL value is obtained as a numerical speed representing the moving speed of the subject in the frame (S53). The speed can be considered to change substantially in proportion to the moving speed of the subject in the frame. This is because, as described above, the value of the 1LH coefficient increases in proportion to the increase in the edge amount in the horizontal direction of the image, that is, the increase in the moving speed of the subject within the frame, whereas the coefficient of 1HL. This value is a value that is proportional to the amount of edge in the vertical direction of the image, and from experience, the subject moves only in the horizontal direction in most cases, and thus takes a relatively stable value.
Next, it is determined whether or not the coefficient speed obtained in step S53 is larger than the experimentally determined threshold value Vth1, and if it is large (Y in S53), the moving speed of the subject within the frame is high. Is determined. On the other hand, when the coefficient speed is smaller than the threshold value Vth1 (N in S53), it can be determined that the moving speed of the subject within the frame is low.

It is the figure which showed the structure of the process block which performs the flame | frame correction | amendment by the amount of depth movements of the moving image processing apparatus of the 1st Embodiment of this invention. It is the block diagram which showed the structure of the image depth calculation part 102 for every flame | frame which estimates the motion amount of a depth direction. It is the flowchart which showed the process of the frame thinning-out control by the depth motion amount in the moving image processing apparatus of 1st Embodiment. It is the figure which showed the structure of the depth degree calculation part of a moving image processing device which concerns on 2nd Embodiment, and a peripheral part. It is a conceptual explanatory view of the frame thinning process based on the depth motion amount. It is the flowchart which showed the reduction process of the frame code amount by the depth motion amount in the moving image processing device of 3rd Embodiment. It is a conceptual explanatory drawing of the process which reduces the frame code amount by a depth motion amount. It is the figure which showed the structure of the calculation process block which calculates the depth degree for every image attribute in the image processing apparatus of 4th Embodiment. It is a principle figure which calculates depth for every three-dimensional image field. It is the flowchart which showed the calculation process which calculates the depth degree for every image attribute as 4th Embodiment. It is a figure for demonstrating the hierarchical encoding algorithm used as the foundation of JPEG2000. It is a figure for demonstrating the basis of tile division | segmentation. It is a figure for demonstrating a decomposition level and a sub band. It is a figure for demonstrating the structure of a code stream. It is a figure for demonstrating a precinct and a code block. (A) is explanatory drawing about frame data, (b) is explanatory drawing about the field data of the frame data of an interlace format. It is the flowchart which showed the process which estimates the motion amount of the two-dimensional plane direction in 5th Embodiment. (A)-(d) is a figure for demonstrating the shift | offset | difference which arises between an odd field and an even field when a to-be-photographed object moves at high speed.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... Three-dimensional image input part, 102 ... Image depth calculation part, 103 ... Motion amount evaluation part, 104 ... Frame code data edit part, 105 ... Image data storage part, 106 ... Evaluation reference | standard motion amount storage part, 107 ... Every frame Depth storage unit 108 ... Frame motion amount evaluation result storage unit 109 ... Frame motion amount change evaluation result storage unit 110 ... Color space conversion / inverse conversion unit 111 ... Two-dimensional wavelet transform / inverse conversion unit 112 ... Quantization Inverse quantization unit 113 113 entropy encoding / decoding unit 114 tag processing unit 12 image data storage unit 13 attribute region extraction unit 14 attribute image region data storage unit 15 image attribute Depth calculation unit, 16 ... Depth data storage unit, 21 ... Image extraction unit for each image attribute, 22 ... Image data extraction unit for each line, 23 ... Correlation calculation unit, 24 ... Depth calculation 25 ... Image data storage unit for each line, 26 ... Correlation data storage unit, 3 ... Correlation calculation unit, 41 ... Image extraction unit for each region, 42 ... Depth calculation unit for each image region, 43 ... Frame depth calculation unit, 44 ... Image Data storage unit 45 ... Image region data storage unit 46 ... Depth storage unit for each region 51 ... Image extraction unit for each region 52 ... Image data extraction unit for each line 53 ... Correlation calculation unit 54 ... Depth calculation unit 55 ... Image data storage unit for each line, 56 ... Correlation data storage unit, 81 ... In-screen motion amount calculation unit

Claims (17)

  Correlation calculation for calculating a correlation between two pieces of image data in units of frames in a moving image processing apparatus that continuously reproduces two pieces of image data for right eyes and left eyes for three-dimensional image reproduction in units of frames Means, a motion amount estimating means for estimating that the amount of motion is small when the depth degree of the three-dimensional image is smaller than a preset setting value based on the correlation value calculated by the correlation calculating means, and the motion amount estimating means And a thinning reproduction means for thinning and reproducing the frame-unit image data when it is estimated that the amount of motion is small.   In a moving image processing apparatus that continuously reproduces right-eye and left-eye image data for three-dimensional image reproduction in units of frames, a motion amount estimation unit that estimates the vertical and horizontal movements of an image, A moving image comprising: thinning reproduction means for thinning and reproducing the frame-unit image data when the movement amount is small and the vertical and horizontal movement amounts are smaller than a preset movement amount reference value Processing equipment.   In a moving image processing apparatus that continuously reproduces two image data for right eye and left eye for three-dimensional image reproduction in units of frames, encoding means for encoding frame image data, A correlation calculating unit that calculates a correlation between two image data, and a motion amount is estimated to be small when the depth degree of the three-dimensional image is smaller than a preset value based on the correlation value calculated by the correlation calculating unit And a frame code amount reducing means for reducing a code amount of code data obtained by encoding the frame image data when the motion amount is estimated to be small. Image processing device.   In a moving image processing apparatus that continuously reproduces right-eye and left-eye image data for three-dimensional image reproduction in units of frames, encoding means for encoding the image data in units of frames, A motion amount estimating means for estimating the motion in the direction, and a code that encodes the image data for each frame when the motion amount in the depth direction is small and the motion amount in the vertical and horizontal directions is smaller than a preset motion amount reference value. And a frame code amount reduction means for reducing the code amount of the data.   5. The moving image processing apparatus according to claim 2, further comprising processing speed request input means for inputting a processing speed request, wherein the motion amount estimation means is arranged in the depth direction when the processing speed request is a high speed request. Estimate the amount of motion with the amount of motion, or estimate the amount of motion with the amount of vertical and horizontal motion that estimates the vertical and horizontal motion of the image. Otherwise, the amount of motion in the depth direction and the vertical and horizontal direction of the image A moving image processing apparatus that estimates a motion amount from a vertical and horizontal motion amount for estimating a motion.   5. The moving image processing apparatus according to claim 1, further comprising a frame correction accuracy request input unit configured to input a frame correction accuracy request, wherein the motion amount estimation unit has a high frame correction accuracy request. In this case, the amount of motion is estimated by the amount of motion in the depth direction, or the amount of motion is estimated by the amount of motion in the vertical and horizontal directions that estimates the vertical and horizontal motion of the image. Otherwise, the amount of motion in the depth direction And an amount of motion estimated from the amount of motion in the vertical and horizontal directions for estimating the vertical and horizontal motion of the image.   7. The moving image processing apparatus according to claim 1, wherein a motion amount estimation unit for each region for estimating a motion amount for each image region, and a code amount reduction unit for reducing a code amount for each of the image regions. And a moving image processing apparatus.   8. The moving image processing apparatus according to claim 1, wherein the calculation of the correlation is a partial area of the image data.   9. The moving image processing apparatus according to claim 1, wherein the frame code data is encoded based on the Motion-JPEG2000 standard.   10. The moving image processing apparatus according to claim 9, wherein data storage means for storing frame code data obtained by encoding image data, and frame code data are reconstructed with respect to code data encoded after encoding of frame image data A moving image processing apparatus.   10. The moving image processing apparatus according to claim 9, wherein sub-band code amount calculation means for calculating a code amount of code data in sub-band units obtained in the process of wavelet transform for wavelet transforming image data, and sub-band A moving image processing apparatus comprising: a motion amount estimating unit that estimates the amount of motion in the vertical and horizontal directions based on a code amount in 1LH or a code amount in sub-band 1HL.   12. The moving image processing apparatus according to claim 1, wherein the frame image data is an interlaced image, and the field dividing unit divides the interlaced image data into odd field image data and even field image data. And a motion amount means for estimating a motion amount using field data of an odd field of an interlaced image and field data of an even field of an interlaced image immediately preceding the interlaced image. A moving image processing apparatus.   13. The moving image processing apparatus according to claim 12, wherein a motion amount calculating means for calculating a motion amount between odd and even fields generated by decomposing one interlaced image signal, and between the odd and even fields. A moving image processing apparatus comprising: determination means for determining the amount of movement based on the amount of movement.   In a moving image processing method for continuously reproducing right-eye and left-eye image data for three-dimensional image reproduction in units of frames, a correlation calculation step of calculating a correlation between the two image data in units of frames; A motion amount estimation step for estimating that the motion amount is small when the depth degree of the three-dimensional image is smaller than a preset setting value based on the calculated correlation value, and a motion amount is small by the motion amount estimation step. A moving picture processing method comprising: a thinning reproduction step of thinning and reproducing the frame image data when estimated.   In a moving image processing method for continuously reproducing right-eye and left-eye image data for three-dimensional image reproduction in units of frames, a motion amount estimation step for estimating vertical and horizontal motion of an image, A moving picture processing method comprising: a thinning reproduction step for thinning and reproducing the frame image data when the movement quantity is small and the vertical and horizontal movement quantity is smaller than a preset movement quantity reference value .   In a moving image processing method for continuously reproducing right-eye and left-eye image data for three-dimensional image reproduction in units of frames, an encoding step for encoding frame image data; A correlation calculating step for calculating a correlation between the image data, a motion amount estimating step for estimating that the amount of motion is small when the depth degree of the three-dimensional image is smaller than a preset setting value based on the calculated correlation value; And a frame code amount reduction step of reducing a code amount of code data obtained by encoding the frame image data when the motion amount is estimated to be small.   In a moving image processing method for continuously reproducing right-eye and left-eye image data for three-dimensional image reproduction in units of frames, an encoding step for encoding frame image data, and vertical and horizontal movements of the image A motion amount estimation step for estimating the amount of code data obtained by encoding the frame image data when the motion amount in the depth direction is small and the motion amount in the vertical and horizontal directions is smaller than a preset motion amount reference value. A moving image processing method comprising: a frame code amount reduction step for reduction.

Images