JP2012004862A - Video processing method, video processing device and video processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable stereoscopic appearance of an in-plane part of a target object to be naturally enhanced.SOLUTION: A video processing method comprises at least: frequency decomposition means for performing spatial frequency decomposition to the spatial frequency component of a two-dimensional image to calculate frequency components; counting means for counting each of the frequency components obtained via the frequency decomposition means; determination means for determining a dynamic property of the scene from the result obtained via the counting means; parallax amount designation means for designating a parallax amount to be given to an image to be three-dimensionalized based on the result obtained via the determination means; parallax amount allocation means for allocating the parallax amount to the frequency components obtained by the decomposition; frequency composition means for combining each frequency component obtained via the parallax amount allocation means; image conversion means for outputting a left eye image and a right eye image which enable a binocular stereoscopic vision from the result obtained by the frequency composition means; and the video output means for outputting a video from plural images.

Description

  The present invention relates to a moving image processing method, a moving image processing apparatus, and a program that enable a stereoscopic effect to be manipulated (emphasized or suppressed) when a moving image is stereoscopically viewed.

In human visual stereoscopic processing, a stereoscopic image is recognized based on an image (video) viewed with the left and right eyes. Therefore, in order to make an image (video) appear three-dimensionally, a set of images (video) for the left eye image (video) and the right eye image (video) is required.
A so-called “stereoscopic” technique is known in which a two-dimensional image is processed to make a planar image appear stereoscopically. Among them, there is a technique for generating two images for the left eye and right eye from one two-dimensional image. This calculates the binocular parallax with respect to the object of interest (the object to be imaged) in the image, and shifts each pixel position of the image in the horizontal direction by an amount based on the parallax. Is a process for generating

As a method for calculating binocular parallax, there is a method of estimating the actual depth (relative distance from the camera) of an object based on color information such as brightness and saturation.
For example, in Patent Document 1, an integrated value of high frequency components, luminance contrast, integrated luminance value, and integrated saturation value for each region are defined as image feature amounts representing the perspective of an image, and any one or combination thereof is determined. A method is disclosed in which the depth correction amount is calculated so that a difference is produced between the regions, and the degree of perspective is determined from the magnitude.
In this method, the depth amount is corrected so as to conveniently give a front-to-back relationship between regions. For this purpose, an appropriate image feature amount is selected, and there is flexibility to cope with various cases. In addition, it lacks decisive power such as automatic determination. Furthermore, it can be said that it is inappropriate for an object in which it is difficult to give a context within a region such as the same plane.

Many methods for adjusting the stereoscopic effect are provided on the premise of binocular stereoscopic vision.
For example, in Patent Document 1, an average value of motion vectors corresponding to the amount of movement of a region of interest in a video is obtained, and whether the region of interest is a subject or a background is determined from the magnitude relationship compared with the average value. A method for adjusting the horizontal phase amount in accordance with the difference between the subject and the background is presented. However, this method cannot give a stereoscopic effect to the background itself or the inside of the subject surface.
Further, in Patent Document 2, edge information of a video signal is obtained, and a front-rear relationship is obtained by regarding a dense area of edge information as a foreground and a sparse area as a background, and a large parallax is given to the foreground and a small background. A method is presented. However, since this method targets the entire region with dense edges, the background cannot be specified because the entire surface is the target of the foreground, and the background cannot be specified for materials with dense structures such as fur and fabric. I can't find it.
Further, in Patent Document 3, since there are generally many images in which the subject is in focus, it is considered that the closer the object is, the higher the high frequency component, contrast, luminance, and saturation, and the higher frequency integrated value, luminance contrast, luminance A method is proposed in which the depth correction amount is determined by determining that an area having a larger integrated value and saturated integrated value is an object existing ahead, and the parallax amount is linearly associated with the depth amount. However, this method is intended to give parallax between areas, and is inappropriate for the purpose of giving parallax inside a small area, for example.

  On the other hand, there is known a method for obtaining a three-dimensional effect by using a perceptual characteristic called a Pulfrich effect, in which a bright stimulus feels as if it is lit earlier than a dark stimulus. For example, Patent Document 4 proposes a method of generating a pseudo three-dimensional video by setting a time difference between a video image projected on the left eye and a video image projected on the right eye. However, this method is effective for objects with different distances from the left and right eyes.For example, it is effective for enhancing the stereoscopic effect of an object that moves left and right. I can not say.

  As described above, the conventional method reproduces the three-dimensional effect (such as the thickness of the material surface) within each surface of the target object, and further emphasizes how the surface is displaced forward and in the depth direction. It was a difficult problem to do.

Japanese Patent No. 3235776 Japanese Patent Laid-Open No. 10-336702 Japanese Patent Laid-Open No. 10-191397 Japanese Patent No. 3249335

“Wavelet Beginners Guide”, Susumu Sugawara, Tokyo Denki University Press, 1995 “Wavelets and Orthogonal Function Systems”, G. G. By Walter Susumu Sugawara, Takeshi Sasayo, Translated by Ryuichi Kanno, Tokyo Denki University Press, 2001

  As described above, the conventional technique enhances the stereoscopic effect by giving a sense of front and back and a sense of depth by giving parallax between regions (for example, individual objects) in an image (video) or between individual surfaces. It does not take into account manipulating the stereoscopic effect inside individual surfaces or areas. Therefore, if the conventional technique is applied to the inside of the surface or the area as it is, there is a problem that a stereoscopic image becomes unnatural and has a strong sense of incongruity, such as a double image or a double layer.

  In view of the above problems, the present invention makes a natural three-dimensional feel when binocularly viewed with the object surface, the area inside, the undulation and thickness of the surface derived from the detailed structure and material as the direct processing target. In addition, it is possible to emphasize the stereoscopic effect when the surface is displaced in the near side or the depth direction with respect to time. Further, the left-eye moving image and the right are obtained from only one two-dimensional moving image. An object of the present invention is to provide a moving image processing method capable of generating an ophthalmic moving image.

As means for solving the above problems, the first invention is a frequency decomposition means for calculating a frequency component by spatial frequency decomposition of a spatial frequency component of one two-dimensional moving image;
A parallax amount designation means for designating a parallax amount to be given to an image to be three-dimensionalized;
Parallax amount allocating means for allocating the parallax amount to the frequency component obtained by the decomposition;
Frequency synthesis means for synthesizing each frequency component obtained through the parallax amount allocation means;
From the result obtained by the frequency synthesizing unit, an image converting unit that outputs a left-eye image and a right-eye image that enables binocular stereoscopic vision, and a moving image that outputs a moving image by combining a plurality of frame images Output means;
Is a moving image processing method characterized by including at least.

  According to a second aspect of the present invention, the dynamic property of the scene is determined from the counting means for counting each frequency component obtained through the frequency resolving means over a plurality of frame images, and the result obtained through the counting means. The moving image processing method according to claim 1, further comprising determination means.

  In the third invention, the amount of parallax is large in a moving image data with respect to a zoom-in scene or a region where the surface is displaced toward the near side, and is small with respect to a zoom-out scene or a region where the surface is displaced toward the depth side. The moving image processing method according to claim 2, wherein the moving image processing method is given gradually.

  According to a fourth aspect of the present invention, the frequency component to which the parallax amount is allocated by the parallax amount allocating means is a frequency component of brightness of the two-dimensional image. This is an image processing method.

  According to a fifth aspect of the present invention, the frequency component to which the parallax amount is allocated by the parallax amount allocating unit is a high frequency component of brightness of the two-dimensional image. This is a moving image processing method.

  Further, according to a sixth aspect of the invention, the frequency component to which the parallax amount is assigned by the parallax amount allocating means is a low frequency component of brightness of the two-dimensional image. The moving image processing method described in the above.

  The seventh invention is the moving image processing method according to any one of claims 1 to 6, wherein the parallax amount is horizontal parallax.

  The eighth aspect of the present invention is the moving image processing method according to claim 7, wherein the horizontal parallax is large for a component having a high lightness and small for a component having a low lightness. is there.

  The ninth aspect of the present invention is the moving image processing according to claim 7, wherein the horizontal parallax is given to a component having a high saturation in an image and small to a component having a low saturation. Is the method.

The tenth invention is a moving image input means for inputting a moving image and outputting a plurality of frame images, and a frequency resolving means for calculating a frequency component by spatially resolving a spatial frequency component of one frame image. And a parallax amount specifying means for specifying the parallax amount to be given to the image to be three-dimensionalized, a parallax amount assigning means for assigning the parallax amount to the frequency component obtained by the decomposition, and the parallax amount assigning means. Frequency synthesis means for synthesizing each frequency component,
From the result obtained by the frequency synthesizing means, an image output means for outputting a left-eye image and a right-eye image capable of binocular stereoscopic vision, and a moving image output means for outputting a moving image by combining frame images When,
A moving image processing apparatus including at least

  According to an eleventh aspect of the present invention, there is provided counting means for counting each frequency component obtained through the frequency resolving means over a plurality of frame images, and determining a dynamic property of the scene from the result obtained through the counting means. The moving image processing apparatus according to claim 10, further comprising a determination unit.

  In a twelfth aspect of the invention, the amount of parallax is large in a moving image data with respect to a zoom-in scene or a region where the surface is displaced to the near side, and is small with respect to a zoom-out scene or a region where the surface is displaced to the depth side. The moving image processing apparatus according to claim 11, wherein the moving image processing apparatus is gradually given so as to satisfy.

  The thirteenth aspect of the present invention is the moving image processing apparatus according to claim 10, wherein the frequency component to which the parallax amount is allocated by the parallax amount allocating unit is a frequency component of brightness of the two-dimensional image. is there.

  The fourteenth aspect of the present invention is the moving image processing apparatus according to claim 10, wherein the frequency component to which the parallax amount is allocated by the parallax amount allocating unit is a high frequency component of brightness of the two-dimensional image. .

  The fifteenth aspect of the present invention is the moving image processing apparatus according to claim 10, wherein the frequency component to which the parallax amount is allocated by the parallax amount allocating unit is a low frequency component of brightness of the two-dimensional image. It is.

  The sixteenth aspect of the present invention is the moving image processing apparatus according to any one of claims 10 to 15, wherein the parallax amount is horizontal parallax.

  The seventeenth aspect of the present invention is the moving image processing apparatus according to claim 16, wherein the horizontal parallax is large for a component having a high lightness and small for a component having a low lightness. It is.

  The eighteenth aspect of the present invention is the moving image according to claim 16, wherein the horizontal parallax is given to a component having a high saturation of an image and is given a small value to a component having a low saturation. It is a processing device.

  According to a nineteenth aspect of the present invention, there is provided a moving image processing program that causes an arithmetic device to execute the image processing method according to any one of the first to eighth aspects.

According to the first aspect of the present invention, the spatial frequency decomposition is applied to one frame image of the surface of the object to be decomposed into individual frequency components, and a predetermined horizontal parallax amount is obtained for the desired spatial frequency component. Given, by generating the left-eye image and right-eye image by frequency synthesis, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity under viewing conditions that allow binocular stereoscopic viewing. It becomes possible.
In addition, the present invention is particularly effective in enhancing the thickness of the object surface, and is particularly effective for materials that humans feel thickness, such as carpets, furs, woolen fabrics, thick fabrics, and clothing. is there. Further, since the parallax can be generated in a pseudo manner even if the video is not originally taken for stereoscopic viewing, it can be utilized when generating a pseudo stereo video from an existing video.
By observing these in a binocular stereoscopic environment, the stereoscopic effect on the surface is enhanced or suppressed without a sense of incongruity.
For example, it is possible to easily realize binocular stereoscopic video or animation video from video captured by a single imaging device or animation video, and to realize realistic pseudo stereoscopic vision in an observation environment where binocular stereoscopic vision is possible. It becomes feasible.

  Further, according to the second invention, the distribution of frequency components among a plurality of frame images is compared to classify video scenes, and the stereoscopic effect of the surface is enhanced or suppressed according to the type of scene. It becomes possible to do.

  According to the third invention, a zoom-in or zoom-out scene or a region where the surface changes to the near side and the depth side along the time axis is detected from the moving image data (a plurality of continuous frame images). By giving a horizontal parallax amount that increases in time to a zoom-in scene or an area that shifts to the near side, and giving a horizontal parallax amount that decreases in time to a zoom-out scene or an area that shifts to the depth side, It is possible to enhance or suppress the stereoscopic effect of the object surface that is displaced in time without any sense of incongruity under viewing conditions that allow binocular stereoscopic viewing.

Further, according to the fourth invention, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity by decomposing the spatial frequency component of brightness.
By separating the lightness component and the chromaticity component and processing only the lightness component, it is possible to effectively enhance or suppress the stereoscopic effect while suppressing the color change to the maximum.

Further, according to the fifth aspect, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity by decomposing and processing the high-frequency component of brightness.
In particular, since human vision is highly sensitive to lightness components mainly for medium and high frequency components, applying the present invention to high frequency components of lightness has an effective effect on stereoscopic effect. Can be obtained.
The present invention is particularly suitable for emphasizing the surface three-dimensional effect of details of materials with dense hair such as thick fur. In addition, it is effective for enhancing and suppressing the stereoscopic effect in a scene where the camera gradually approaches and zooms in and in a zoom-out scene where the camera gradually moves away.

Further, according to the sixth aspect, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity by decomposing and processing the high-frequency component of brightness.
Since the low frequency component of the brightness mainly reflects the surface shape of the object (uneven shape with a large area over the entire surface), the three-dimensional effect of the surface shape can be effectively enhanced or suppressed.
For example, the present invention relates to a moving image of hair of an animal such as a dog or a cat, and is used to relatively detect an object such as a back, an abdomen, a buttocks, or a cloud floating in the air, a sand dune in the air, or a wave of sea waves. It is suitable for emphasizing or suppressing a three-dimensional effect over a wide area. Further, it is effective for projecting a wide area of an object in a zoomed out scene where the camera gradually takes a long distance in a moving image.

  Further, according to the seventh invention, by adopting horizontal parallax as the parallax amount, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity under the observation conditions where binocular stereoscopic vision is possible. Become.

According to the eighth aspect of the invention, horizontal parallax is adopted as the amount of parallax, and the stereoscopic effect on the surface of the object is uncomfortable by giving a large value to a component with high brightness and a small value with respect to a component having low brightness. It is possible to emphasize and suppress without any loss.
In particular, in the case of a material having a multi-layered network structure on the surface structure, such as multi-layer gauze, cotton, woolen fabric, or nylon scrubbing, the reflection is strong and bright as it is closer to the surface, and the reflection is weak and dark as it goes deeper. By giving a large amount of parallax to a component with high brightness and giving a small amount of parallax to a component with low brightness, the depth difference between the front side and the back side can be gradually emphasized, resulting in a natural sense of depth. Can be emphasized.

According to the ninth aspect of the present invention, the horizontal parallax is adopted as the parallax amount, and the stereoscopic effect on the object surface is uncomfortable by giving a large value to a component with high saturation and a small value with respect to a component with low saturation. It is possible to emphasize and suppress without any loss.
This is based on the rule of thumb that the saturation of the surface of the target object in the vicinity is strong and the saturation of the surface of the target object in the distance is weak. For example, costumes and curtains with flashy colors, Persian carpets, etc. This is effective when it is desired to obtain a three-dimensional surface effect of a colorful material, and is effective for reproducing a change in the three-dimensional effect on the object surface during zoom-in / zoom-out in an image.

According to the tenth aspect, the spatial frequency decomposition is applied to one frame image of the surface of the object to be decomposed into individual frequency components, and a predetermined horizontal parallax amount is obtained for the desired spatial frequency component. Given, by generating the left-eye image and right-eye image by frequency synthesis, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity under viewing conditions that allow binocular stereoscopic viewing. It becomes possible.
In addition, the present invention is particularly effective in enhancing the thickness of the object surface, and is particularly effective for materials that humans feel thickness, such as carpets, furs, woolen fabrics, thick fabrics, and clothing. is there. Further, since the parallax can be generated in a pseudo manner even if the video is not originally taken for stereoscopic viewing, it can be utilized when generating a pseudo stereo video from an existing video.
By observing these in a binocular stereoscopic environment, the stereoscopic effect on the surface is enhanced or suppressed without a sense of incongruity.
For example, a binocular stereoscopic image, video, or animation can be easily realized from an image, video, or animation captured by a single imaging device, which is realistic in a binocular stereoscopic viewing environment. Pseudo stereoscopic vision can be realized.

  According to the eleventh aspect, by decomposing the spatial frequency component of the brightness, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity. By comparing the distribution of frequency components among a plurality of frame images, it is possible to classify the scenes of the video and enhance or suppress the stereoscopic effect of the surface according to the type of the scene.

According to the twelfth aspect, a zoom-in or zoom-out scene or a region where the surface changes to the near side and the depth side along the time axis is detected from moving image data (a plurality of continuous frame images). By giving a horizontal parallax amount that increases in time to a zoom-in scene or an area that shifts to the near side, and giving a horizontal parallax amount that decreases in time to a zoom-out scene or an area that shifts to the depth side, It is possible to enhance or suppress the stereoscopic effect of the object surface that is displaced in time without any sense of incongruity under viewing conditions that allow binocular stereoscopic viewing.
In addition, the present invention is particularly effective in enhancing the thickness of the object surface, and is particularly effective for materials that humans feel thickness, such as carpets, furs, woolen fabrics, thick fabrics, and clothing. is there. In addition, since it is possible to generate a pseudo-parallax even if it is not an image or video originally taken for stereoscopic viewing, it is utilized when generating a pseudo-stereo image or stereo video from an existing image or video. be able to.
By observing these in a binocular stereoscopic environment, the stereoscopic effect on the surface is enhanced or suppressed without a sense of incongruity.

According to the thirteenth invention, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity by decomposing the spatial frequency component of brightness.
By separating the lightness component and the chromaticity component and processing only the lightness component, it is possible to effectively enhance or suppress the stereoscopic effect while suppressing the color change to the maximum.

Further, according to the fourteenth aspect, by decomposing and processing the high-frequency component of brightness, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity.
In particular, since human vision is highly sensitive to lightness components mainly for medium / high frequency components and high frequency components, the present invention is applied to high frequency components and high frequency components of lightness. Can be obtained effectively without a sense of incongruity.
The present invention is particularly suitable for emphasizing the surface three-dimensional effect of details of materials with dense hair such as thick fur. Also, it is effective for scenes where the camera gradually approaches and zooms in on the video, and scenes where the surface of the target object is more closely projected.

According to the fifteenth aspect, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity by decomposing and processing the high-frequency component and the high-frequency component of brightness.
Low-frequency component of brightness The low-frequency component mainly reflects the surface shape of the object (uneven shape having a large area over the entire surface), so that the stereoscopic effect of the surface shape can be effectively enhanced or suppressed.
For example, the present invention relates to a moving image of hair of an animal such as a dog or a cat, and is used to relatively detect an object such as a back, an abdomen, a buttocks, or a cloud floating in the air, a sand dune in the air, or a wave of sea waves. It is suitable for emphasizing or suppressing a three-dimensional effect over a wide area. In addition, it is effective in projecting a wide area of the object in a zoom-out scene where the camera gradually takes a long distance in the video.

  Further, according to the sixteenth aspect, by adopting horizontal parallax as the parallax amount, it is possible to enhance or suppress the stereoscopic effect on the object surface without a sense of incongruity under viewing conditions that allow binocular stereoscopic viewing. Become.

Further, according to the seventeenth aspect, by adopting horizontal parallax as the amount of parallax, giving a large value to a component having a high lightness and a small value to a component having a low lightness, the stereoscopic effect on the object surface is uncomfortable. It is possible to emphasize and suppress without any loss.
In particular, in the case of a material having a multi-layered network structure on the surface structure, such as multi-layer gauze, cotton, woolen fabric, or nylon scrubbing, the reflection is strong and bright as it is closer to the surface, and the reflection is weak and dark as it goes deeper. By giving a large amount of parallax to a component with high brightness and giving a small amount of parallax to a component with low brightness, the depth difference between the front side and the back side can be gradually emphasized, resulting in a natural sense of depth. Can be emphasized.

According to the eighteenth aspect of the invention, by adopting horizontal parallax as the amount of parallax, giving a large value for a component with high saturation and a small value for a component with low saturation, the stereoscopic effect on the object surface is uncomfortable. It is possible to emphasize and suppress without any loss.
This is based on the rule of thumb that the saturation of the surface of the target object in the vicinity is strong and the saturation of the surface of the target object in the distance is weak. For example, costumes and curtains with flashy colors, Persian carpets, etc. This is effective when it is desired to obtain a three-dimensional surface effect of a colorful material, and is effective for reproducing a change in the three-dimensional effect on the object surface during zoom-in / zoom-out in an image.

In addition, according to the nineteenth invention, by causing the present invention to be executed by a computing device such as a personal computer, the stereoscopic effect on the object surface can be emphasized or suppressed without a sense of incongruity under viewing conditions that allow binocular stereoscopic viewing. Is possible.
According to the first to nineteenth inventions, in the video, by appropriately selecting and applying the high-frequency component stereoscopic effect enhancement method and the low-frequency component stereoscopic effect enhancement method of the present invention on the time axis, It is possible to emphasize or suppress the movement of viscous objects such as swells on the water surface and the movement of cloths such as flags fluttering in the wind.

The schematic diagram showing an example of the moving picture processing method of the present invention. The processing flowchart which showed an example of the moving image processing method of this invention. The processing flowchart which showed an example of the frequency decomposition process of this invention. The processing flowchart which showed an example of the parallax amount allocation process of this invention. The processing flowchart which showed an example of the frequency synthesis process of this invention. Explanatory drawing which shows the concept of wavelet transformation and reverse transformation. The figure which shows the shape of the wavelet function and scaling function of a Daubechies wavelet (10th order). The conceptual diagram which shows an example of the moving image presentation environment which can display the binocular parallax image of the observation target object which displays the moving image produced | generated by this invention. The conceptual diagram which shows the mode of the single virtual image which an observer can perceive in the environment of FIG.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 8 is an example of a moving image presentation environment in which a binocular differential image of an observation target object can be displayed for displaying a moving image generated using the present invention. FIG. 9 is an example of a conceptual diagram showing a state of a single virtual moving image that can be perceived by an observer in the environment of FIG.

The images displayed in the left-eye image 1 and the right-eye image 2 are reflected by the fusion mirror 3 and are observed with both eyes of the observer. When the same image is displayed on both images, the viewer can perceive as if a single virtual image 4 exists in front as shown in FIG. 9 by appropriate arrangement of the image and the mirror. Bring.
In addition, changing the arrangement of the image and the mirror changes the parallax angle, which changes the sense of distance to the image. In addition, the parallax angle changes and the sense of distance also changes by adding different amounts of translation to the images displayed in both images without changing the arrangement of the images and mirrors.

  FIG. 1 is a schematic diagram showing an example of a moving image processing method of the present invention.

In the moving image input means S1, a moving image to be three-dimensionalized is input to the apparatus of the present invention.
The moving picture input unit S1, and inputs the moving image data V _in the like the video signal line connected like a communication line or a video apparatus via a storage device or a network such as a hard disk device.

Then, the image conversion unit S2, converts the moving image V _in input from said image input means into a plurality of frame images, the frame image I at time t (x, y, t) with respect to the brightness image L (x , Y, t), the chromaticity image C ₁ (x, y, t), C ₂ (x, y, t), and the saturation image S (x, y, t) are output.

  Next, the parallax amount designating unit S3 inputs the parallax amount to be given to the image to be three-dimensionalized to the apparatus of the present invention. The parallax amount input means S3a inputs the parallax amount H. This can be realized, for example, by inputting an arbitrary numerical value from an input screen presented on a computer terminal or by selecting from a plurality of candidate values. Or you may input via the input device etc. which arranged the button, the dial, etc.

  The parallax amount of the present invention assumes binocular parallax, and is intended for parallax in a direction parallel to the direction of both eyes of the observer and horizontal parallax in the horizontal direction of the parallax image (the horizontal direction of the image) Adjust them accordingly.

  For example, the parallax amount input means S3a gives a predetermined numerical value h as the horizontal parallax amount H (H = h). Next, the correction coefficient designating unit S3b receives the lightness component image L (x, y, t) and the saturation component image S (x, y, t) output from the image conversion unit S2, and receives the numerical value k and the lightness. The parallax correction coefficient K is set in any one of the monotonically increasing function f (v) of v and the monotonic increasing function g (s) of the saturation s (K = k, K = f (v), K = g (s)).

  Here, the parallax correction coefficient is given as a monotonically increasing function of brightness, because of the nature of light reflection, the surface layer portion of the object surface appears brighter, and conversely the deeper portion appears darker. Yes.

  Also, the parallax correction factor is given as a monotonically increasing function of saturation because the closer to the human eye looks brighter and the farther away the object appears to be weaker. Further, the surface layer portion is vivid, that is, the saturation is high, and conversely, the deep portion corresponds to the light color, that is, the saturation is low.

  Subsequently, the frequency selection means S4 calculates the maximum decomposition level M of the frame image received from the image conversion means S2, and selects a desired decomposition level from decomposition levels 1 to M.

Here, the decomposition level corresponds to the number of decompositions when an image is decomposed into frequency components. For example, when the frequency is defined in units of pixels, which are the minimum structural unit of an image, the finest, that is, the highest frequency is when two pixels are one wavelength. This corresponds to the frequency 1/2 at the decomposition level 1. Next, at decomposition level 2, the frequency becomes ½ of the frequency, and thereafter, the same is repeated, and at decomposition level n, frequency becomes 1/2 ⁿ .
Note that the frequency value at each decomposition level depends on the unit size when the frequency is defined.

  Next, the frequency decomposition means S5 performs spatial frequency decomposition on the image converted by the image conversion means S2.

Here, when calculating individual frequency components by spatial frequency decomposition of a two-dimensional image, it can be performed on various image signals. For example, there is a method of calculating individual frequency components by spatially resolving lightness components and chromaticity components.
Since the present invention relates to human stereoscopic vision, it is considered appropriate to apply it to signals that are meaningful in terms of visual perception. Regarding the lightness component and chromaticity component, which are image signal components, the sensitivity characteristic for the lightness component shows a bandpass type, while the sensitivity characteristic for the chromaticity component is low-passed, with respect to the spatial frequency perception of human visual perception. Since the lightness component is mainly used for the perception of the medium and high frequency components, the spatial frequency processing in the present invention is preferably performed on the lightness component. Hereinafter, the case where the brightness component is applied will be described.

Next, in the counting means S6, the decomposed frequency components are counted over a predetermined number of frames. Here, it is appropriate that the predetermined number of frames is adaptively determined according to the content of the moving image, but it is preferable to set a small value for an object that changes quickly and a large value for an object that changes slowly.
There are various counting methods. For example, there is a method using a histogram. A histogram of each frequency component is obtained over an image having a predetermined number of frames.

Next, the determination means S7 receives the counting result from the counting means S6, determines whether it is a zoom-in scene, a zoom-out scene, or another scene, and outputs the function K ₂ (f) of the frame f. .
Judgment methods include zoom-in and zoom-out scenes, and other scenes as learning videos. Each histogram is calculated, and the classification threshold is calculated and used by a statistical method such as discriminant analysis. A way to do this is possible
Alternatively, a histogram over a predetermined number of frames is sequentially calculated at each time, and the distribution shape and peak position of the histogram are compared, and the scene where the shape is similar and the peak position shifts to the low frequency side is a zoom-in scene For example, a method of determining a scene where the shape is similar and the peak position shifts to the high frequency side as a zoom-out scene can be considered.
Here, as the function K ₂ (f), in the zoom-in scene, a function that monotonically increases from 0 to 1 over the total number of frames of the scene (start frame f ₁ to end frame f ₂ ), and in the zoom-out scene, As a function that monotonously decreases from 1 to 0 over the total number of frames, K ₂ (f) = 1 is set in other scenes.
The simplest example is
However, the present invention is not limited to these, and various functions can be selected within the range of monotonic increase and monotonic decrease.

Frame At parallax amount correcting means S8, receives predetermined parallax amount H from the parallax amount specifying means S3a, receives the parallax correction factor K ₁ from the correction coefficient specifying means S3b, in response to the further scene from said determination means S7 The function K ₂ (f) is received, and H ′ = K ₁ × K ₂ (f) × H is calculated as the parallax correction amount H ′.
Here, as a function, for example, assuming that the maximum value of the brightness of the brightness image L (x, y, t) is Vmax and the minimum value is Vmin, f = (v) = It can be given linearly as v / (Vmax−Vmin). Similarly, as the saturation function g (s), assuming that the maximum value of the saturation of the saturation image S (x, y, t) is Smax and the minimum value is Smin, s = S (x, y, t ) Can be linearly given as g (s) = s / (Smax−Smin). Of course, the present invention is not limited to these, and various functions can be selected within a monotonically increasing range.

  Next, in the parallax amount assigning unit S9, the parallax correction amount H ′ corrected by the parallax amount specifying unit S8 is assigned to the frequency component obtained by the frequency decomposing unit S5.

  At this time, the parallax correction amount H ′ is a method in which the high frequency component is shifted to the left and right by a predetermined horizontal parallax amount and the other frequency components are not shifted depending on the type of image. A new horizontal parallax amount that is shifted left and right by the amount of parallax and other frequency components are not shifted, or a value obtained by multiplying a predetermined horizontal parallax amount by a constant or a correction coefficient given by a monotonically increasing function of the brightness component Adopting a method of shifting the desired frequency component, adopting a value obtained by multiplying a predetermined horizontal parallax amount by a constant or a correction coefficient given by a monotonically increasing function of the saturation component as a new horizontal parallax amount, A method for shifting the frequency component is appropriately selected, or a combination thereof is used to assign the parallax amount.

Specifically, among the frequency components F _i (x, y, t) received from the frequency resolving means S5, the parallax correction amount as the horizontal parallax amount for the frequency component of decomposition level i to which 1 is assigned. H ′ is assigned symmetrically in the horizontal direction, and the frequency component image F _i ^L (x, y, t) for the left eye is not assigned to the frequency component of decomposition level i to which 0 is assigned without assigning a horizontal parallax amount. , And the frequency component image F _i ^R (x, y, t) for the right eye is generated as follows.

Next, in the frequency synthesizer S10, an image is constructed by synthesizing the decomposed frequency components.
In the frequency synthesizer S10, the process proceeds in the opposite direction to the frequency decomposition method. An inverse wavelet transform is applied to the decomposition level n low frequency image and the decomposition level n wavelet coefficients to obtain a decomposition level n-1 low frequency image. Next, by applying wavelet inverse transformation to the low-frequency image at decomposition level n-1 and the wavelet coefficients, a low-frequency image at decomposition level n-2 is obtained. By sequentially repeating this operation, an image at the decomposition level 0, that is, the original image can be reconstructed.

  By performing frequency decomposition by wavelet transform, wavelet coefficients for each decomposition level are obtained. The decomposition level corresponds to a spatial resolution of one power of 2 due to the two-scale relationship. For example, the decomposition level i is the spatial resolution of the original image.

Twice the spatial resolution, which is

This corresponds to a double spatial frequency component.

  Therefore, by applying the synthesis method without applying horizontal parallax to a desired frequency component, that is, a wavelet coefficient having a desired decomposition level, and applying no horizontal parallax to other wavelet coefficients, only the desired frequency component is obtained. Left eye image with horizontal parallax

And right-eye images are obtained.

Next, the image conversion means S11 outputs the result obtained by the frequency synthesis means S10 as an image of a desired format.
Specifically, the image output unit S11 outputs the frequency composite image for the left eye from the frequency synthesis unit.

And right eye frequency composite image

And chromaticity images C ₁ (x, y, t) and C ₂ (x, y, t) are received from the image conversion means S2, and color conversion is applied from these to the left, which is a desired two-dimensional image. An image for eye I ^L (x, y, t) and an image for right eye I ^R (x, y, t) are output.

The moving image output unit S12 receives the left-eye image I ^L (x, y, t) and the right-eye image I ^R (x, y, t) of each frame from the image conversion unit S11, and receives the left-eye moving image. The image and the moving image for the right eye are output.

  The medium for outputting the result as an image is not particularly limited as long as the image can be displayed, and examples thereof include a liquid crystal display, plasma, CRT, and organic EL.

FIG. 2 is a processing flowchart showing an example of the moving image processing method of the present invention. Hereinafter, an embodiment of the present invention will be described using the processing flow of FIG.
First, the moving image input process P1, corresponding to the three-dimensional moving image data V _in to be processed.

Then, the image conversion processing P2, 2-dimensional moving image data V _in the frame image I read in the moving image input process P1 (x, y, t) is decomposed into further luminance component and a chromaticity of each frame image The two-dimensional image L (x, y, t) of the lightness component, the two-dimensional image C ₁ (x, y, t), C ₂ (x, y, t) of the chromaticity component, and the saturation Each component is converted into a two-dimensional image S (x, y, t).

  As a method for separating the color information into lightness information and chromaticity information, any color system such as Y (R−Y) (BY), CIELAB, or HSV may be used. Here, when the RGB color system is used as a reference for the most commonly used two-dimensional image color system, Y (R−Y) (BY) is used for broadcasting, and brightness Y Is calculated by Equation 1, and the saturation S is calculated by Equation 2.

  Y = 0.299R + 0.587G + 0.114B (Formula 1)

  On the other hand, in the parallax amount input process P3, first, the parallax amount H is input via the parallax amount input means S3a. This can be realized, for example, by inputting an arbitrary value from an input screen presented on a computer terminal or by selecting from a plurality of candidate values. Or you may input from the input device etc. which arranged the button, the dial, etc.

  In the correction coefficient calculation process P4, the brightness image L (x, y, t) and the saturation image S (x, y, t) are received from the image conversion P2, and the function f (v) of the processing numerical value k and the brightness v is received. Then, the correction coefficient K is calculated in one of the functions g (s) of the saturation s (K = k, K = f (v), K = g (s)).

  Here, as the brightness function f (v), for example, when the maximum brightness value of the brightness image L (x, y, t) is Vmax and the minimum value is Vmin, v = L (x, y, t). As f (v) = v / (Vmax−Vmin). Similarly, as the saturation function g (s), assuming that the maximum value of the saturation of the saturation image S (x, y, t) is Smax and the minimum value is Smin, s = S (x, y, t ) Can be linearly given as g (s) = s / (Smax−Smin). Of course, the present invention is not limited to these, and various functions can be selected within a monotonically increasing range.

Next, 1 is substituted into the time variable t. In this embodiment, the time is substituted by the frame number, which means that the first frame number is substituted.
Next, in the maximum decomposition level calculation process P5, the brightness component image L (x, y, t) output at the time t output in the image conversion process P2 is received, the width width and height height are read, and the width and height are read. The maximum power of 2 that does not exceed the smaller value is calculated, and that power is substituted into M as the maximum decomposition level.

  Next, in the frequency selection process P6, the maximum decomposition level M is received from the maximum decomposition level calculation process P5, a desired decomposition level is designated between decomposition levels 1 to M, and the parallax is determined according to each decomposition level. It is stored in list (i) which is a list for designating whether to give or not. Here, a smaller frequency represents a higher frequency, and a larger level represents a lower frequency.

  As for the selection method, in order to instruct the processing according to the nature of the subject of the image, when processing only the high frequency component, select the decomposition level of the high frequency component, and when processing only the low frequency component Selects the decomposition level of the low frequency component, selects only the decomposition level of the desired frequency component when processing only the desired frequency, and selects the entire decomposition level when processing regardless of the frequency.

  As a method for realizing list (i), which is a list for specifying parallax, for example, list (i) = 1 is given when the decomposition level i is selected, and list (i) = 0 is given when not selected. Output. If the frequency component division number is M, list (i) can be realized, for example, as data of length M bits, or can be realized as an array of size M.

On the other hand, a scene over a plurality of frames including time t is analyzed. First, t ₁ is substituted for the time variable t ′. Here, t ₁ represents the _first frame of a plurality of frames to be referred to (t ₁ ≦ t).
In the frequency resolution process P7, the maximum resolution obtained from the brightness level L (x, y, t ′) at the time t ′ from the brightness image calculated in the image conversion process P2 from the resolution level 1 by the maximum resolution level calculation process. The frequency component images F _i (x, y, t ′) of the respective decomposition levels are calculated by sequentially performing spatial frequency conversion up to the level M.

Here, a frequency component decomposition method will be described. FIG. 3 is a process flow diagram showing an example of the frequency decomposition process of the present invention. First, c ⁰ = L (x, y, t ′), the decomposition level is i, 1 is substituted for i (P7b), and the first spatial frequency transformation is applied (P7c). Thereby, decomposition coefficients c ¹ and d ¹ of decomposition level 1 are obtained (P7d).
Next, i is updated with a value obtained by adding 1 to the value of i (P7e), and is compared with the maximum number of decomposition levels M (P7f). If the value of i is M or less, decomposition coefficients of decomposition level 2 by applying processing again from P7c against ^{c 1} until ^P7F, the determined c 2, ^{d 2,} less, the processing from P7c to P7F The decomposition coefficients c ⁱ and d ⁱ are sequentially calculated by repeating. When the value of i exceeds M, the frequency decomposition process is terminated. As a result, a frequency component image F _i (x, y, t ′) = c ^{i at} each decomposition level is obtained.

Next, with regard to the frequency conversion method, many methods have been devised, particularly in the field of acoustic technology. For example, the Fourier transform method and the wavelet transform method are representative. In the present invention, a case where a wavelet transform method with high spatial localization accuracy is applied is introduced here as a more preferable example (although various frequency conversion methods can be used as appropriate).
The wavelet transform is a method for decomposing and generating a signal using two types of functions (scaling function and wavelet function) that satisfy a two-scale relationship and an orthonormal system condition. The basic content of the wavelet transform is described in, for example, “Non-Patent Document 1”. The function system for constructing the wavelet is described in, for example, “Non-Patent Document 2”.
Examples of wavelet functions include Daubechies wavelets and cardinal spline wavelets. A wavelet function that is straightforward and smooth in nature is suitable for the present invention.

From the image to be processed, a region having the maximum area and the width and height of the image are equal to each other and the value is equal to a power of 2 is extracted as a processed image. For this image, a scaling factor and a wavelet coefficient of a desired wavelet function are prepared, and wavelet transformation is performed. As shown in FIG. 6, every time wavelet transform is performed, the frequency is halved and decomposed into a high frequency component and a low frequency component. Here, the state of the original image before conversion is called decomposition level 0, the case where the conversion is applied once is called decomposition level 1, and the state where the conversion is repeated n times is called decomposition level n. Therefore, when the repeated conversion to n times, as the highest frequency spatial frequency of the original image, its 1/2, 1/4-fold, ..., are disassembled into 1/2 ⁿ times the frequency component .

  The following relationship holds between the decomposition component at decomposition level k and the decomposition component at decomposition level k-1 (0 <k <n).

These formulas represent the original signal ^ck and the left eye signal obtained by shifting the original signal to the right by a predetermined horizontal parallax amount H.

And a signal for the right eye obtained by shifting the original signal to the left by a predetermined horizontal parallax amount H

Is a filter:

Depending on the resolution one step lower:

This means that it can be disassembled. Where the function

Is called a wavelet function, also a function

Is called a scaling function.

  FIG. 7 shows waveforms of the Daubechies wavelet (10th order) wavelet function and scaling function.

Next, the frequency counting means P8 receives each frequency component from the frequency decomposing means P7, counts each frequency component of the frame image, and updates the counting result hist (i).
Subsequently, the value of the time variable t ′ is updated by adding the increment time Δt, compared with the final time t _{2 of the} scene (t ≦ t ₂ ), and if smaller than t ₂ , the processes of P7 and P8 are executed again. repeat.

After reaching the final time of the scene, in the scene determination process P9, the distribution shape of the count result hist (i) received from the frequency counting means P8 and the shift amount of the peak position are calculated, and the zoom-in scene / zoom-out scene / Other scenes are determined, and a correction coefficient K ₂ (t) suitable for each scene is set.
For example, hist (i) over a predetermined frame (from time t ₁ to t ₂ ) is calculated at each time, and the distribution shape and peak position of the histogram are compared, the shapes are similar, and the peak position is low A scene shifting to the frequency side is determined as a zoom-in scene, and a scene having a similar shape and a peak position shifting to the high-frequency side is determined as a zoom-out scene.
Here, as a function K ₂ (t) (t ₁ ≦ t ≦ t ₂ ), in a zoomed-in scene, it increases monotonically from 0 to 1 over the total number of frames (start frame t ₁ to end frame t ₂ ). In the zoom-out scene, a function that monotonously decreases from 1 to 0 over the entire number of frames, and K ₂ (t) = 1 is set in other scenes.
The simplest example is
However, the present invention is not limited to these, and various functions can be selected within the range of monotonic increase and monotonic decrease.

Next, in the parallax amount correction processing P10, the lightness component image L (x, y, t) and the saturation component image S (x, y, t) are received from the image conversion processing P2, and from the parallax amount input processing P3. A predetermined parallax amount H is received, a correction coefficient K is received from the correction coefficient calculation process P4, and a frame function K ₂ (t) corresponding to the scene is received from the scene determination process P9, and the parallax correction amount H ′ is H '= K ₁ × K ₂ (t) × H is calculated.

In the parallax amount allocation processing P11, the parallax correction amount H ′ is received from the parallax amount correction processing P10, the instruction list list (i) is received from the frequency selection processing P6, and each frequency component image calculated from the frequency resolution processing P7 is received. F _i (x, y, t) is received, the parallax correction amount H ′ is given as a horizontal parallax to the frequency component image F _i (x, y, t) of the designated decomposition level i, and the frequency for the left eye The component image F _i ^L (x, y, t) and the right-eye frequency component image F _i ^R (x, y, t) are output.

Next, the parallax amount allocation process P11 will be described in detail. FIG. 4 is a process flow diagram showing an example of the parallax amount allocation process P11 of the present invention.
First, the selection list list (i) is received from the frequency selection process P6, the parallax correction amount H ′ is received from the parallax amount correction process P10, and each frequency component image F from the frequency resolution process P7 to the decomposition levels 1 to M is received. _i (x, y, t) is received.
Now, 1 is assigned to the decomposition level i (P11a), the value of list (i) is compared with 1 (P11b), and if they match, the frequency component image F _i (x, y, t) at the decomposition level i is symmetrical. Given the horizontal parallax, the frequency component image F _i ^L (x, y, t) for the left eye and the frequency component image F _i ^R (x, y, t) for the right eye at the decomposition level i are calculated. Specifically, a frequency component image for the left eye at the decomposition level i by giving a parallax H ′ in the horizontal right direction to the frequency component image F _i (x, y, t) at the decomposition level i.

Similarly, the frequency component image _Fi (x, y, t) at the decomposition level i is given a parallax H ′ in the horizontal left direction, and the frequency component image for the right eye at the decomposition level i is calculated.

Is calculated. This process (from P11e to P11i) is repeated while increasing the value of x by 1 from the value of x to 0 to width, and further, while increasing the value of y by 1 from the value of y to 0 to height.
On the other hand, if the value of list (i) does not match 1, the frequency component image F _i (x, y, t) at the decomposition level i is converted to the frequency component image F _i ^L (x, y for the left eye at the decomposition level i). , T) and the frequency component image F _i ^R (x, y, t) for the right eye are substituted into (F _i ^L (x, y, t) = F _i (x, y, t), F _i ^R (x , Y, t) = F _i (x, y, t)). This process (from P11j to P11n) is repeated while increasing the value of x by 1 from the value of x to 0 to width, and is further repeated while increasing the value of y by 1 from the value of y to 0 to height.
Next, 1 is added to the value of i and compared with M. If it is less than or equal to M, the same processing (from P11b to P11o) is repeated again, and the processing ends when M is exceeded.

  Next, the value of the time variable t is updated with a value obtained by adding the time increment Δt and compared with the last frame time T. If the value of t is less than or equal to T, the series of processing from P5 to P11 is repeated. If the value of t exceeds T, the process proceeds to the next process P12.

In the frequency synthesizing process P12, the left-eye frequency component image F _i ^L (x, y, t) to which a predetermined horizontal parallax is given to the desired frequency component in the parallax amount allocation process P11 and the right-eye frequency. From the component image F _i ^R (x, y, t), starting from the decomposition level M, the inverse transform of the frequency decomposition process is applied to calculate a frequency image of the decomposition level M−1 that is one step lower, and this process is decomposed It repeats sequentially until the level becomes 0, and finally the frequency image F ₀ ^L (x, y, t) for the left eye and the frequency image F ₀ ^R (x, y, t) for the right eye at the decomposition level 0 are finally obtained. Is obtained.

Next, the frequency synthesis process P12 will be described in detail.
In FIG. 6, let us consider that the flow of an arrow going from left to right is advanced from right to left in the reverse direction. Using the low-frequency component and the high-frequency component at the maximum decomposition level M, the frequency is doubled each time the wavelet inverse transformation is performed, and the decomposition level is reduced to one less frequency component. Therefore, when the conversion is repeated ^M times, frequency components of the maximum decomposition level are doubled, quadrupled,..., ^2M times. The frequency component of ^2M times represents a level where decomposition is not performed, and the frequency component of the original image is synthesized after all.
Hereinafter, the processing will be specifically described.
FIG. 5 is a process flow diagram showing an example of the frequency synthesis process of the present invention.
From the parallax amount allocation processing P11, the frequency component images F _i ^L (x, y, t) for the left eye and the frequency component images F _i ^R (x, y, t) for the right eye from the decomposition levels 1 to M are obtained. receive.
Now, assuming that the decomposition level is i, first, the maximum decomposition level number M is substituted for i (P12a). Left eye frequency component image

And right eye frequency component images

The frequency component image for the left eye having a level one less than the decomposition level is obtained by applying inverse transformations (Equation 3) and (Equation 4) of the frequency transformation P8c to (P12b).

And right eye frequency component images

Is calculated.

  Next, the value of i is updated with a value reduced by 1 and compared with 1. If it is 1 or more, the inverse transformation process (from P12b to P12e) is applied again, and if it is 0, the process ends. As a result, the frequency composite image for the left eye is finally obtained.

And right eye frequency composite image

Is obtained.

  Next, in the image conversion process P13, the frequency image for the left eye which is the output of the frequency synthesis process P12

And right eye frequency image

And the chromaticity images C ₁ (x, y, t) and C ₂ (x, y, t) separated in the image conversion processing P2 are received and are the same as the original image I (x, y, t). The left-eye two-dimensional image I ^L (x, y, t) and the right-eye two-dimensional image I ^R (x, y, t) are output.

Finally, in the moving image output process P14, the left-eye two-dimensional image I ^L (x, y, t) and the right-eye two-dimensional image I ^R (x, y, t) of each frame image from the image conversion process P13. ) And outputs a left-eye moving image V ^L and a right-eye moving image V ^R.

  This time, a still image is used as an example to explain the present invention. The present invention is intended for general two-dimensional images, and the application target is not limited to still images. For example, the present invention can be similarly applied to each frame image of 2D video, CG (computer graphics) image, and each frame image of 3DCG video.

  The present invention is a method for generating a left-eye image and a right-eye image for binocular stereoscopic viewing by image processing a single two-dimensional image, and is a stereoscopic image represented by a 3D movie that has recently become widespread. It can be applied to images and stereoscopic images in general.

DESCRIPTION OF SYMBOLS 1 ... Left eye image 2 ... Right eye image 3 ... Fusion mirror 4 ... Virtual image 5 ... Parallax angle D1 ... Daubechies wavelet (10th order) scaling Function D2 ... Daubechies wavelet (10th order) wavelet function P1 ... Moving picture input process P2 ... Image conversion process P3 ... Parallax amount input process P4 ... Correction coefficient calculation process P5 ... Maximum decomposition level calculation process P6... Frequency selection process P7... Frequency decomposition process P7a... Set decomposition level to variable i P7b... Set 1 to P7c. Store decomposition factor P7e ... Add 1 to the value P7f ... Compare the value of i with M P8 ... Frequency counting process P9 ... Scene determination process P10 ... Parallax amount correction process P11 ...・ Visual Quantity allocation processing P11a: set 1 to variable i P11b: compare value of list (i) to 1 P11c: set 0 to variables x, y P11d: set 0 to variables x, y Setting P11e: Left eye and right eye frequency components set with frequency components set to be horizontally and parallax H 'symmetrically right and left in the x direction P11f: P11g for adding 1 to the value of variable x・ ・ ・ Compare variable x with width P11h ・ ・ ・ Convert variable y to 1 P11i ・ ・ ・ Compare variable y with height P11j ・ ・ ・ Left and right eye frequency components Set component P11k ... Add 1 to variable x P11l ... Compare variable x to width P11m ... Convert variable y to 1 P11n ... Variable y to height Comparison P11o: Add 1 to the value of variable i P11p ... Compare the value of variable i to M P12... Frequency synthesis processing P12a... M is set to variable i P12b... Frequency inverse transform P12c... Decomposition image at decomposition level i-1 is calculated from decomposition coefficient and decomposition image at decomposition level P12d .. P12e to reduce the value of variable i by 1 ... Compare the value of variable i to 1 P13 ... Image conversion processing P14 ... Moving image output processing

S1 ... Moving image input means S2 ... Image conversion means S3 ... Parallax amount designation means S3a ... Parallax amount input means S3b ... Correction coefficient designation means S4 ... Frequency selection means S5 ... Frequency decomposing means S6 ... counting means S7 ... determining means S8 ... parallax amount correcting means S9 ... parallax amount allocating means S10 ... frequency synthesizing means S11 ... image converting means S12 ... moving image Image output means

Claims (19)

A frequency resolving means for calculating a frequency component by spatially resolving a spatial frequency component of one two-dimensional image; a parallax amount specifying means for specifying a parallax amount to be given to an image to be three-dimensionalized;
Parallax amount allocating means for allocating the parallax amount to the frequency component obtained by the decomposition;
Frequency synthesis means for synthesizing each frequency component obtained through the parallax amount allocation means;
From the result obtained by the frequency synthesizing means, an image converting means for outputting a left-eye image and a right-eye image capable of binocular stereoscopic vision, and a moving image output for outputting a moving image by combining a plurality of images Means,
A moving image processing method comprising:   2. The method according to claim 1, further comprising: counting means for counting each frequency component obtained through the frequency resolving means; and determining means for judging a dynamic property of a scene from a result obtained through the counting means. The moving image processing method described in 1.   The moving image processing method according to claim 1, wherein the frequency component to which the parallax amount is allocated by the parallax amount allocating unit is a frequency component of brightness of the two-dimensional image.   The moving image processing method according to claim 1, wherein the frequency component to which the parallax amount is allocated by the parallax amount allocating unit is a high-frequency component of brightness of the two-dimensional image.   The moving image processing method according to claim 1, wherein the frequency component to which the parallax amount is allocated by the parallax amount allocating unit is a low frequency component of brightness of the two-dimensional image.   6. The moving image processing method according to claim 1, wherein the amount of parallax is horizontal parallax.   The moving image processing method according to claim 6, wherein the horizontal parallax is given to a component having a high lightness of the image and small to a component having a low lightness.   The moving image processing method according to claim 6, wherein the horizontal parallax is given to a component having a high saturation of an image and is given a small value to a component having a low saturation.   In the moving image data, the horizontal parallax is gradually given so as to be large for a zoom-in scene or a region where the surface is displaced toward the front side and to be small for a zoom-out scene or a region where the surface is displaced toward the depth side. The moving image processing method according to claim 6. A frequency resolving means for calculating a frequency component by spatially resolving a spatial frequency component of one two-dimensional image, a counting means for counting each frequency component obtained through the frequency resolving means, and a value obtained via the counting means. Determination means for determining the dynamic properties of the scene from the obtained results, and a parallax amount specifying means for specifying the amount of parallax to be given to the image to be three-dimensionalized based on the result obtained through the determination means;
Parallax amount allocating means for allocating the parallax amount to the frequency component obtained by the decomposition;
Frequency synthesis means for synthesizing each frequency component obtained through the parallax amount allocation means;
From the result obtained by the frequency synthesizing means, an image converting means for outputting a left-eye image and a right-eye image capable of binocular stereoscopic vision, and a moving image output for outputting a moving image by combining a plurality of images Means,
A moving image processing apparatus comprising:   11. The apparatus according to claim 10, further comprising: a counting unit that counts each frequency component obtained through the frequency resolving unit; and a determination unit that determines a dynamic property of the scene from a result obtained through the counting unit. The moving image processing apparatus according to 1.   11. The moving image processing apparatus according to claim 10, wherein the frequency component to which the parallax amount is allocated by the parallax amount allocating unit is a frequency component of brightness of the two-dimensional image.   The moving image processing apparatus according to claim 10, wherein the frequency component to which the parallax amount is allocated by the parallax amount allocating unit is a high frequency component of brightness of the two-dimensional image.   The moving image processing apparatus according to claim 10, wherein the frequency component to which the parallax amount is allocated by the parallax amount allocating unit is a low frequency component of brightness of the two-dimensional image.   The moving image processing apparatus according to claim 10, wherein the amount of parallax is horizontal parallax.   16. The moving image processing apparatus according to claim 15, wherein the horizontal parallax is given to a component having a high lightness of the image and small to a component having a low lightness.   The moving image processing apparatus according to claim 15, wherein the horizontal parallax is given to a component having a high saturation in an image and small to a component having a low saturation.   In the moving image data, the horizontal parallax is gradually given so as to be large for a zoom-in scene or a region where the surface is displaced toward the front side and to be small for a zoom-out scene or a region where the surface is displaced toward the depth side. The moving image processing apparatus according to claim 15.   A moving image processing program that causes an arithmetic device to execute the image processing method according to claim 1.