JP4963124B2 - Video processing apparatus, video processing method, and program for causing computer to execute the same - Google Patents

Description

  The present invention relates to a video processing apparatus, a video processing method, and a program for causing a computer to execute a process for enhancing stereoscopic effect and depth feeling on an input video.

  For flat images such as televisions, various methods have been proposed for enhancing the three-dimensionality and depth of video in order to make moving images and still images appear with higher quality and higher image quality. For example, in stereoscopic viewing using binocular parallax, which is used as a method to enhance stereoscopic effect, send different images with parallax to the left and right eyes, creating an illusion and feeling a sense of depth and depth. Let In order to send different images to the left and right eyes, for example, as shown in Patent Document 1, the left and right shutters are switched to synchronize with the frequency of the display that alternately displays the left eye image and the right eye image, thereby blocking the optical path. An anaglyph method in which two right and left RGB images having parallax are converted into a red image and a blue image, and an image obtained by superimposing the two images is viewed through red and blue glasses, as shown in Patent Document 2. As described above, a polarized glasses method using polarized glasses by outputting left and right parallax images with different polarizations on one display can be raised.

  On the other hand, in the world of painting, three-dimensionality and depth are enhanced by using painting techniques such as perspective, shading, and a combination of forward and backward colors. Trick art (also known as Tromp-Rouille) is a painting created using these methods. Draw an object on a two-dimensional work and use the method described above to describe the appropriate superposition relationship with the object. So, by making the illusion that a part of the picture drawn in two dimensions protrudes into the three-dimensional space of the real world, the three-dimensional effect can be enhanced. For example, works that make a part of the picture protrude from the picture frame depicted in the picture and make the shadow appear as if it protrudes from the picture are shown well. Yes.

JP 60-7291 A JP-A-1-171390

  In a system that emphasizes a three-dimensional effect or a depth sensation as shown in the above-mentioned patent document, a special filter, glasses, and the like are required. Therefore, the current two-dimensional display system (display) has to be replaced. Further, when viewing a video created by the technique described in the above-mentioned patent document, the viewer is required to wear special glasses, and the viewing method is significantly limited.

  In view of the above problems, the present invention imposes a burden on or restrictions on a viewer by performing video processing based on depth information of each object in a video without changing a conventional two-dimensional display system. The purpose is to provide a technique for improving the perceived stereoscopic and depth perception of video.

The present invention is a video processing device that enhances the three-dimensionality and depth of an input video,
A video acquisition unit that acquires an input video, a depth information acquisition unit that acquires depth information of each object in the captured video, and a video in front of a predetermined depth position based on the depth information. Is inserted into a foreground area, a video dividing unit that divides an image deeper than a predetermined depth position into two as a background area, and a depth position between the foreground area and the background area. A depth perception object generation unit that generates a video of an object, a shadow region generation unit that generates a shadow region of the foreground region, the depth information including the foreground region, the background region, the distance perception object, and the shadow region. comprising a video synthesizing unit for creating an image synthesized on the basis, wherein the image combining unit to the shadow area synthesized only in a region overlapping with the sense of distance perception objects, the shadow Frequency creating unit, the position coordinates of the shadow area, x-axis coordinates of the foreground area, the y-axis, and obtains by adding a preset value.

In the video processing apparatus, the video composition unit may change video processing applied to the shadow area according to depth information of the foreground area .
Here, the video processing is transmission processing or blurring processing.

Further, in the image processing apparatus, without increasing deformation of or deformation of the foreground area, and includes an image deformation unit for a reduced deformation of the background area, the shadow area generation unit, after the processing of the image deforming unit and generating the shadow of the foreground area based on expanded the foreground regions of.

In addition, the present invention is a video processing method for enhancing the stereoscopic effect and depth feeling of an input video,
A video acquisition step for acquiring an input video, a depth information acquisition step for acquiring depth information of each object in the captured video, and a video in front of a predetermined depth position based on the depth information. A foreground region, a video dividing step of dividing an image deeper than a predetermined depth position into two as a background region, and a depth position between the foreground region and the background region, for distance perception A depth perception object generation step for generating a video of an object, a shadow region generation step for generating a shadow region of the foreground region, and the foreground region, the background region, the distance perception object, and the shadow region as depth information. A video compositing step for creating a composite video based on the distance perception object, A region only combining the shadow area, in the shadow region generating step, the position coordinates of the shadow area, x-axis coordinates of the foreground area, the y-axis, and characterized by obtaining by adding a preset value To do.

In the video processing method, in the video composition step, video processing applied to the shadow area is changed according to depth information of the foreground area .

The image processing method may further include an image transformation step for enlarging or deforming the foreground area and for reducing or deforming the background area , and the shadow area generation step is performed after the processing of the image transformation unit. and generating the shadow of the foreground area based of the foreground region.

Further, the present invention may be a program for causing a computer to execute each step of the video processing method.

  According to the video processing apparatus of the present invention, since a distance perception object such as a forehead or a curtain is inserted between the foreground area and the background area based on the depth information, a special display system or a special It is possible to acquire an image with a three-dimensional effect and a depth feeling enhanced only by image processing without requiring wearing glasses. In addition, since shadows such as foreground areas and frame objects are generated, it is possible to acquire a video that further enhances the stereoscopic effect and the depth effect.

  Furthermore, if the video transformation is performed to enlarge or reduce the foreground area or the background area, it is possible to obtain a video that emphasizes perspective, and thus it is possible to acquire a video with enhanced stereoscopic effect and depth.

It is a functional block diagram which shows the example of 1 structure of the video processing video processing apparatus by the 1st Embodiment of this invention. It is a functional block diagram which shows the modification of the example of 1 structure of the video processing video processing apparatus by the 1st Embodiment of this invention shown in FIG. It is a flowchart figure which shows the flow of operation | movement in a video processing video processing apparatus. It is a figure which shows an input image | video. It is a figure which shows the depth image corresponding to an input image | video. It is a figure which shows the foreground image and background image which were divided | segmented based on the depth information. It is the figure which showed the frame to synthesize | combine. It is a functional block diagram which shows the modification of the example of 1 structure of the video processing video processing apparatus by the 1st Embodiment of this invention shown in FIG. It is a functional block diagram which shows the modification of the example of 1 structure of the video processing video processing apparatus by the 1st Embodiment of this invention shown in FIG. It is the figure which showed the example of the output image | video after the synthesis | combination by the 1st Embodiment of this invention. It is the figure which showed the shadow area | region to synthesize | combine. It is a figure explaining the synthetic | combination method of the shadow area | region. It is the figure which showed the example of the video processing using the wall by the 2nd Embodiment of this invention. It is the figure which showed the example of the output image | video after a synthesis | combination by the 2nd Embodiment of this invention. It is a functional block diagram which shows the example of 1 structure of the video processing video processing apparatus by the 3rd Embodiment of this invention. It is the figure which showed the example of the output image | video after a synthesis | combination by the 3rd Embodiment of this invention.

  An image processing apparatus according to each embodiment of the present invention will be described with reference to the drawings.

<First Embodiment>
FIG. 1 is a functional block diagram showing a configuration example of a video processing apparatus according to the first embodiment of the present invention.
As shown in FIG. 1, the video processing apparatus 10 according to the present embodiment includes a depth information acquisition unit 11 that acquires depth information, a video acquisition unit 12 that acquires input video, and a depth obtained by the depth information acquisition unit 11. A video dividing unit 13 that divides the video obtained from the video acquisition unit 12 based on the information, a distance-sensitive perception object generating unit 14 that generates an object that defines a pseudo plane in the video according to the divided video, and And a shadow region generation unit 15 that generates a shadow region, and the generated distance perception object, the shadow, and the divided video, and outputs a video in which the stereoscopic effect and the depth feeling are emphasized to the output unit 17 Part 16.

  FIG. 2 is a flowchart showing an operation flow in the video processing apparatus 10. When the process is started (start), the video acquisition unit 12 acquires the input video (step S1). Next, the depth information acquisition unit 11 acquires depth information corresponding to the input video (step S2), and determines a position where the distance perception object is arranged (step S3). In this embodiment, the depth information is acquired from the outside together with the input video. The video dividing unit 13 divides the input video into a portion (foreground) that is arranged in front of the distance-sensitive object and a portion (background) that is arranged behind the distance-sensitive object based on the determined distance-sensitive object. (Step S4). The distance sensation perception object generation unit 14 generates a distance sensation perception object to be synthesized according to the divided video (step S5). On the other hand, the shadow area generation unit 15 creates a shadow applied to the distance perception object from the foreground portion of the divided input video (step S6). Finally, the video synthesis unit 16 synthesizes the divided input video, distance perception object, and shadow after synthesis (step S7), and ends the processing (end).

  In the flowchart described above, the depth information is acquired from the outside together with the input video in step S2, but the depth information may be calculated from the input video. For example, like the video processing device 20 shown in FIG. 3, the depth information acquisition unit 21 may be provided, and the depth information may be calculated and acquired from the input video acquired by the video acquisition unit 12. In addition, the block shown with the same symbol among each part in FIG. 1, FIG. 3 shall have the same function.

  Hereinafter, each component of the video processing apparatus 10 according to the present embodiment will be described in more detail.

(Description of depth information acquisition unit 11)
First, the depth information acquisition unit 11 will be described. The video processing apparatus 10 includes a depth information acquisition unit 11 that acquires depth information corresponding to an input image obtained from the video acquisition unit 12. Here, the depth information is information indicating the distance between each object (for example, a person, a car, a tree, a wall, etc.) reflected in the input video and the photographer (camera). In this embodiment, the depth information is acquired at the time of shooting and is transmitted to the video processing apparatus 10. As a depth information acquisition method at the time of shooting, for example, a stereo method may be applied, and a distance may be obtained based on a positional shift on the imaging surface when the same object is shot with two cameras whose positions are shifted. Further, it may be obtained by measuring the intensity of reflected infrared light irradiated to an object from an infrared irradiation device provided in the camera.

  In this embodiment, the depth information is acquired separately from the input video at the time of shooting. However, as a modification of this embodiment, the depth information may be calculated from the input video as shown in FIG. For example, a technique disclosed in Japanese Patent Laid-Open No. 9-161074 is used to calculate depth information from an input video. Further, when the input video is encoded by some method, depth information may be created from the encoded information. For example, in MPEG-4 (Moving Picture Expert Group phase 4), which is one of the standard moving picture standards created by MPEG (Moving Picture Experts Group), it is possible to encode in units of objects such as backgrounds and persons. If the input video is encoded using this function and the background and the person are encoded separately, this information can be used to make the “camera and person distance” closer than the “camera and background distance”. Create information.

(Description of video acquisition unit 12)
The video acquisition unit 12 will be described. The video acquisition unit 12 acquires a video that is a source for performing video processing for improving a stereoscopic effect and a sense of depth. The video here may be a still image or a moving image, and even if it is compressed by some encoding method, for example, JPEG (Joint Photographic Experts Group) or MPEG-2 (Moving Picture Expert Group phase 2). It may be compressed. If the input video is encoded, it is appropriately decoded (for example, RGB format or YUV format) in this block. Hereinafter, in this embodiment, it is assumed that processing is performed on the decoded image. In addition, the video processing performed in this embodiment has been described as processing on a still image or one frame for the sake of simplicity. However, when the input video is a moving image, the video processing is continuous. The same processing is performed for each frame.

(Description of video dividing unit 13)
Next, the video dividing unit 13 will be described. The video dividing unit 13 uses the distance perception object generation unit 14 (described later in detail) to generate the distance perception object based on the depth information acquired by the depth information acquisition unit 11 based on the input video acquired by the video acquisition unit 12. The foreground to be displayed in front is separated from the background to be displayed behind the same distance perception object.

  Here, it demonstrates still in detail using a figure. Assume that there is an input image 31 shown in FIG. 4 and a depth image 32 having depth information corresponding to the input image 31 shown in FIG. Here, the depth image 32 shows the distance between the photographer (camera) and the object as five levels of depth. As shown in the depth information table 33, the depth image 32 is indicated by 1, 2,... It is assumed that any value of depth 1 to 5 is included corresponding to each pixel of the image 31. Here, for simplicity of explanation, the depth information is shown in five stages, but the depth information may be less than five stages or more than five stages. Of course, there is no problem even if the depth information is shown steplessly.

  At this time, the video dividing unit 13 divides the input video into the foreground and the background based on the threshold Th. In all depth information Depth (n) corresponding to each pixel of the input image 31, when Th> Depth (n), the pixel Px (n) of the corresponding input image 31 is used as a background and Th ≦ Depth (n) In this case, the same pixel Px (n) is used as the foreground. A C language program showing this division determination is shown below.

if (Th> Depth (n)) {
Px (n) = background} else {
Px (n) = foreground}

  Here, if the preset threshold Th is 2, finally, the input image 31 is based on the distance image 32, and the foreground area 34 (a white area surrounded by a solid line in the video 35) shown in FIG. ) And the background area 36 (a white area surrounded by a solid line in the video 37).

  In the present embodiment, the threshold value Th is described as a value recorded in advance in the video processing apparatus 10, but may be a value that can be freely set by a user using the video processing apparatus 10. Further, the threshold Th may be obtained by calculation. For example, the following Equation 1 is an example in which the average of the distance is set as the threshold Th based on the depth image 32.

  Th = (ΣDepth (n)) / n (where n = 0, 1, 2,..., I) (Equation 1)

(Description of Distance Perception Object Generation Unit 14)
The distance perception object generation unit 14 will be described. The distance sensation object generating unit 14 generates a distance sensation object to be combined with the input image based on the foreground area and the background area determined by the image dividing unit A4. Here, the sense of distance object is inserted in the depth position between the foreground and the background in the input video to give a sense of relative distance from the foreground or the background, and the viewer perceives a three-dimensional effect and a sense of depth. It is an object to make it. The depth position to be inserted here refers to a position that is a threshold value of the depth information described above. In the present embodiment, as shown in FIG. 7, a frame 37 that is a frame object surrounding the outer periphery of the video is generated as the distance perception object. Here, the shape, color, and the like of the frame are determined in advance by the video processing apparatus 10, but it is also possible to allow the user to register a frame of an arbitrary shape and color and combine them.

  Further, in this embodiment, the shadow of the frame 37 itself for making the generated frame 37 appear three-dimensionally is determined in advance, but this is based on the information obtained from the input. By combining with the shadow of the avant-garde area in the frame, it is possible to generate a frame that increases the sense of unity. For example, as shown in FIG. 8, a video processing device 40 which is a modified example of the video processing device 10 includes a shadow direction acquisition unit 41, so that a distance sensation object generation unit 14 and a shadow region generation unit 15 (described later). The shadow in the same direction as the input image can be constructed based on the direction of the shadow obtained from the block. The input to the shadow direction acquisition unit 41 here is, for example, the direction of the shadow of each object derived from the illumination conditions at the time of shooting.

  Further, when the shadow direction can be estimated from the input image, as shown in FIG. 9, the video processing video processing device 50, which is a modification of the video processing device 20, uses the input image obtained from the video acquisition unit 12. The shadow direction can be obtained by providing the shadow direction acquisition unit 51 for obtaining the shadow direction by calculation. Of course, it is also possible to add a shadow to the user's favorite direction by making the direction of the shadow designated by the user.

  In the present embodiment, the frame object has been described as an example of the frame object that surrounds the outer periphery of the video, but it goes without saying that this object may be other than the frame. For example, a window frame or a folded curtain 62 as shown in the image 61 of FIG.

(Description of the shadow region generation unit 15)
Next, the shadow area generation unit 15 will be described. The shadow area generation unit 15 generates a shadow area mainly from the foreground part area determined by the video division unit 13. When the foreground area 34 as shown in FIG. 6 is acquired, a shadow area 64 (see FIG. 11) is obtained by filling this area with the color set in the video processing apparatus 10 in advance.

(Description of video composition unit 16)
Finally, the video composition unit 16 will be described. As shown in the video 63 in FIG. 10B, the video composition unit 16 generates the background region 36 obtained by the video division unit 13, the frame 37 generated by the distance perception object generation unit 14, and the shadow region generation unit 15. The generated shadow 64 and the foreground area 34 obtained by the video dividing unit 13 are combined in order. First, the frame 37 is superimposed on the background area 34. Next, the shadow region 64 is superimposed. At this time, the position (x, y) at which the shadow region 64 overlaps is obtained by the following expression 2 relative to the position (u, v) of the foreground object.

x = u + a
y = v + b (Formula 2)

That is, as shown in the image 71 in FIG. 12A, the position (x, y) of the shadow area 64 is in the a and y axis directions from the position (u, v) of the foreground area 34 in the x axis direction. Is a position translated by b. Here, a and b are arbitrary values registered in the video processing apparatus 10 in advance. The video composition unit 16 superimposes the shadow area 64 at the position (x, y), but only the area 74 that overlaps the frame 37 as shown in the video 72 of FIG. Region (73) is not depicted. It should be noted that the shadow region 74 to be drawn may be added with video processing such as transmission processing and blurring processing so as to get closer to the actual shadow and to obtain a three-dimensional effect and a sense of depth. In the present embodiment, a and b are registered in advance in the video processing apparatus 10, but may be set freely by the user, the shadow direction acquisition unit 41 shown in FIG. It is good also as what determines based on the shadow direction information obtained from the shadow direction acquisition part 51 shown in FIG. Finally, the foreground area 34 is synthesized, and the synthesized image 65 (FIG. 10C) is output to the output unit 17.

  In this way, since the frame object for perception of distance such as a forehead or a curtain is inserted between the foreground area and the background area based on the depth information, even in a conventional two-dimensional display system, And images with enhanced depth can be displayed. Furthermore, when a shadow such as a foreground region or a frame object is also generated, the stereoscopic effect and the depth effect are further enhanced. In particular, when the shadow of the foreground area is generated in the frame object, there is an effect that the depth arrangement of the foreground area and the frame object becomes clearer.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. This embodiment is different from the first embodiment in that a wall is combined with an input video instead of a frame.

  The video processing apparatus according to the second embodiment of the present invention is the same as the video processing apparatus 10 of FIG.

  In the video processing apparatus 10 according to the present embodiment, the object generated by the distance perception object generation unit 14 of the video processing apparatus 10 is an object that separates the foreground from the background instead of the frame 37 shown in the first embodiment. It is characterized in that a wall is generated.

  The distance perception object generation unit 14 generates a wall to be combined with the input video 31 based on the foreground area and the background area determined by the video division unit 13. In this embodiment, it is assumed that the depth information 32 as shown in FIG. 5 is acquired, the threshold Th for dividing the foreground and the background is depth 4, and based on this, the input video 31 is shown in FIG. Are divided into a foreground area 82 and a background area 84 shown in FIG.

  As shown in the video 86 shown in FIG. 13C, the video composition unit 16 superimposes the background region 84, the wall region 87, the shadow region 88, and the foreground region 82 in this order as in the first embodiment. A final composite image 91 shown in FIG.

  In the present embodiment, the video composition unit 16 draws a portion of the shadow area 88 that overlaps the wall area 87, and is closer to the actual shadow, such as transmission processing and blurring processing, as in the first embodiment. It is also possible to add video processing for obtaining a three-dimensional effect and a sense of depth. Further, the video processing applied locally to the shadow area may be changed according to the depth information in the foreground area. Based on the depth information in FIG. 5, the synthesized video 92 in FIG. 14B performs blurring processing only on the shadow area 94 corresponding to the foreground area whose depth information is 1, so that the depth information is larger than 1. It is the figure which showed a mode that image processing was not performed to the corresponding shadow area | region 93. FIG. Further, like the wall area 87a in the example of the image 95 shown in FIG. 14C, the wall area is generated with an arbitrary size, and this size is determined by the user or set in the video processing apparatus 10 in advance. It shall be.

  In this embodiment, the description has been given by taking the wall as an example of the distance perception object separating the foreground and the background, but it goes without saying that this object may be other than the wall. For example, poles such as utility poles and objects such as trees may be arranged.

  Thus, since objects for distance perception such as walls and pillars are inserted between the foreground area and the background area based on the depth information, even in a conventional two-dimensional display system, An image with a deep sense of depth can be displayed. Further, when a shadow such as a foreground area or an object is also generated, the stereoscopic effect and the depth effect are further enhanced. In particular, when the shadow of the foreground area is generated on the object, there is an effect that the depth arrangement of the foreground area and the object becomes clearer.

<Third Embodiment>
Next, a third embodiment of the present invention will be described. The present embodiment is characterized in that the input video is deformed based on the depth information.

  FIG. 15 is a functional block diagram showing a configuration example of a system according to the third embodiment of the present invention.

  The video processing apparatus 100 according to the present embodiment is characterized in that a video deformation unit 101 is provided in addition to each block of the video processing apparatus 10 shown in the first embodiment. In FIG. 15, each functional unit having the same function as each functional unit in FIG. 1 is given the name and symbol of the functional unit having the same name.

  The video transformation unit 101 transforms the foreground and background divided by the video division unit 13 using an arbitrary conversion method. For example, after the input video 31 is divided into the foreground 34 and the background 36 as shown in FIG. 6 by the video division unit 13, the video transformation unit 101 reduces the background region 36 at an arbitrary reduction ratio a. Thereafter, generation of a distance perception object, generation of a shadow, and video composition are performed. As shown in FIG. 16A, the transformed background region 112 obtained as a result is used as a new background, and the obtained output video 111 is a video in which the perspective between the foreground and the background is further emphasized. In the output image 65 shown in FIG. 10, it is possible to display an area hidden in the frame. Similarly, after being divided by the video dividing unit 13, as shown in the video 113 in FIG. 16B, the foreground region 114 enlarged by the video transformation unit 101 at an arbitrary magnification b is set as a new foreground region. In this way, it is possible to obtain an image that emphasizes perspective. In the present embodiment, an example in which only either the foreground or the background is deformed has been shown, but deformation processing may be applied to both. In addition, as a conversion method for deformation, perspective conversion may be used based on a conversion method other than simple enlargement / reduction processing, for example, depth information obtained from a depth information acquisition unit.

  In addition, although this embodiment shown in FIG. 15 adds the image | video deformation | transformation part 101 to the image processing apparatus 10 of FIG. 1, it is not restricted to this. The video processing unit 20, 40, 50 shown in FIGS. 3, 8, and 9 may be added with the video transformation unit 101.

  As described above, when the video deformation is performed to enlarge or reduce the foreground area or the background area, it is possible to obtain an image in which the perspective is emphasized, and thus it is possible to display an image with enhanced stereoscopic effect and depth.

  As described above, the functions of the video processing devices according to the above embodiments can be realized by a computer. In that case, a program describing the processing content of such a function is provided, and the processing function is realized on the computer by executing the program on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory.

  It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network. The computer that executes the program stores, for example, the program recorded in the portable recording medium or the program transferred from the server computer in its own storage device, reads the program from its own storage device, and performs processing according to the program Can be executed.

10, 20, 40, 50, 100 Video processing devices 11, 21 Depth information acquisition unit 12 Video acquisition unit 13 Video division unit 14 Distance perception object generation unit 15 Shadow region generation unit 16 Video composition unit 17 Output units 41, 51 Shadow Direction acquisition unit 101 Video transformation unit

Claims (8)

A video processing device that enhances the three-dimensionality and depth of the input video,
A video acquisition unit for acquiring the input video;
A depth information acquisition unit that acquires depth information of each object in the captured video;
A video dividing unit that divides the input video into two based on the depth information, with a video in front of a predetermined depth position as a foreground region and a video deeper than the predetermined depth position as a background region;
A distance sensation perception object generating unit that generates an image of an object for distance sensation inserted into a depth position between the foreground region and the background region;
A shadow area generator for generating a shadow area of the foreground area;
A video synthesizing unit that creates a video obtained by synthesizing the distance sensation values based on the depth information of the foreground area, the background area, the distance perception object, and the shadow area;
With
The video composition unit synthesizes the shadow region only in a region overlapping with the distance perception object ,
The image processing apparatus according to claim 1, wherein the shadow area generation unit obtains the position coordinates of the shadow area by adding preset values to the x-axis and y-axis of the position coordinates of the foreground area .   The video processing apparatus according to claim 1, wherein the video synthesis unit changes video processing to be added to the shadow area in accordance with depth information of the foreground area.   The video processing apparatus according to claim 2, wherein the video processing is transmission processing or blurring processing. A video deformation unit that performs an enlargement deformation of the foreground region or a deformation of the background region and a reduction deformation of the background region;
The shadow area generation unit, a video processing apparatus according to any one of claims 1 to 3, characterized in that to produce a shadow of the foreground area based on the foreground area after processing of the image deforming unit. A video processing method that enhances the three-dimensionality and depth of the input video,
A video acquisition step for acquiring the input video;
A depth information acquisition step of acquiring depth information of each object in the captured video;
A video dividing step of dividing the input video into two based on the depth information, with a video before a predetermined depth position as a foreground region and a video deeper than the predetermined depth position as a background region;
A distance sensation object generating step for generating an image of an object for distance sensation inserted into a depth position between the foreground region and the background region;
A shadow region generating step for generating a shadow region of the foreground region;
A video synthesis step of creating a video in which the foreground area, the background area, the distance perception object, and the shadow area are synthesized based on depth information;
With
In the video composition step, the shadow area is synthesized only in an area overlapping with the sense of distance object ,
In the shadow area generating step, the position coordinates of the shadow area are obtained by adding preset values to the x-axis and y-axis of the position coordinates of the foreground area . 6. The video processing method according to claim 5 , wherein, in the video synthesis step, video processing applied to the shadow area is changed according to depth information of the foreground area. An image transformation step for performing an enlargement deformation of the foreground region or a deformation of the background region and a reduction deformation of the background region,
The video processing method according to claim 5 or 6 , wherein the shadow region generation step generates a shadow of a foreground region based on the foreground region after the processing of the video transformation step. The program for making a computer perform each step of the video processing method in any one of Claim 5 thru | or 7 .

Images