KR101994472B1 - Method, apparatus and recording medium for generating mask and both-eye images for three-dimensional image - Google Patents

Abstract

A method of generating a binocular image for use in solidifying a planar image, comprising: a separation step in which a first image and a second image that are divided or copied from the planar image are obtained; It is applied to the second image, each of the soft image generating step and the first shift image and the second shift image when each is wispy shifted first shift image and the second shift images are generated for the background, the right eye image and the left eye including binocular image confirmation step for confirming as an image, or the left eye image and the right eye image is characterized by true.

Description

[0001] METHOD, APPARATUS AND RECORDING MEDIUM FOR GENERATING MASK AND BOTH-EYE IMAGES FOR THREE-DIMENSIONAL IMAGE [0002]

The present invention relates to a mask for a three-dimensional image and a method, apparatus and recording medium for generating a two-dimensional image, and more particularly, to a dual-aperture single lens 4-color sensor camera A mask for separating a foreground / background of a planar image using a depth image (depth information) together with an RGB image and an IR image which are extracted plane images, a method for producing a binocular image using the mask, an apparatus and a recording medium will be.

Generally, an image is divided into a planar image and a stereoscopic image depending on whether or not a depth sense is provided. The stereoscopic image provides a sense of depth using the parallax of the two eyes. The stereoscopic image can be implemented by a left eye image recognized by the left eye and a right eye image of the right eye image recognized by the right eye, which is a disparity.

A well-known method of stereoscopic image is anaglyph (stereoscopic stereoscopic, stereoscopic stereoscopic) method. This is accomplished by composing left eye and right eye images in different colors, especially complementary colors. At the time of appreciation, the glasses having the left eye lens and the right eye lens of the color opposite to the color of the produced image are worn and watched. For example, if an anaglyph is produced by combining a Cyan left eye image and a Red right eye image, a so-called red-cyan eyeglass equipped with a red left eye lens and a scientific eye lens is worn for viewing the stereoscopic image .

As another example of the stereoscopic image, there is a polarizing glasses method. This is because a filter for outputting a left eye image and a right eye image to the even scan line and the odd scan line among the scan lines of the display and to separately output the left eye image and the right eye image on the front face of the display, And wear glasses to watch.

As another example, a shutter glasses system can be mentioned. This is because the display alternately outputs the left-eye image and the right-eye image in a time-division manner, as shown in the following Patent Document, and the viewer wears shutter glasses for turning on and off the left-eye lens and the right-

And, as another method, there is also a method without glasses (non-glasses type). This is because the display alternately displays the pixels of the left eye image and the right eye image, and by installing a parallax barrier with a hole at the front of the display, the viewer can see through the holes without glasses, And the right eye is configured to view only the right eye image pixels.

As described above, it can be seen that the stereoscopic image methods must be provided with a left eye image and a right eye image in advance.

Conventionally, in the acquisition of the binocular image, at the same time by the stereo camera consisting of two different cameras separated from each other and arranged at different angles toward the object, that is, a left eye camera and a right eye camera, . This imaging apparatus is referred to as a stereoscope (stereoscope, binocular photograph, stereoscopic camera). The two captured images are formed so as to have an angle difference (parallax) with each other, and are referred to as stereo pairs.

However, a method of forming stereoscopic images of a two-dimensional image by adding depth information to a plane image without using a stereo scope has been developed. The depth information is acquired by the stereo camera or acquired by a time-of-flight (TOF) sensor.

Patent Publication 10-2011-0088963 publication

However, the stereo camera for directly acquiring the binocular stereophonic pair is relatively expensive, and both cameras are provided separately. The imaging angle and focus must be separately set and maintained, and the synchronization of the imaging So that it is not easy to handle. Since both of the cameras are RGB cameras, there is a problem that the quality of a captured image is greatly deteriorated, especially at low illuminance. There is a problem that the depth information is inaccurate and the edges of the foreground / background are inaccurate and misunderstood because the image accuracy is degraded at low illuminance.

Since the distance-based sensor measures the distance using the time difference between the reference time of the emitted pulse and the time of the reflected pulse of the object, a separate device for transmitting and receiving pulses is provided in addition to the camera There is a problem that the pulse equipment is a comparatively very expensive equipment and there is a disadvantage that the recognition rate is weak depending on the surrounding environment such as light and sound and the type of object (reflection, absorption, etc.) depending on the type of pulse . In addition, synchronization with the camera is also required, which makes it difficult to handle.

As described above, when the depth information from the stereo camera is used, the detection of the boundary line for extraction and processing of the foreground of the plane image is inaccurate and the image of the binocular stereo pair processed from the plane image is not smooth. Therefore, there is a disadvantage that the resulting stereoscopic images synthesized from these are unnatural. These drawbacks are particularly severe in low-light environments or disturbances.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-described problems, and it is an object of the present invention to provide a planar image display device which is capable of minimizing the cost of equipment and which does not need to set and maintain an imaging angle and focus, And to provide a mask and a binocular image generation method and apparatus for stereoscopic imaging.

It is another object of the present invention to provide a mask and a binocular image generation method and apparatus for stereoscopic image of a planar image, in which not only RGB but also IR image are obtained from the camera, and the quality of the captured image is greatly improved even in low illumination.

To provide a mask and a binocular image generation method and apparatus for stereoscopic image of a planar image, in which the depth information acquired from the camera is kept accurate even at low illumination, and the edges of the foreground / background are correct and the erroneousness is eliminated .

In addition, since it is not a depth information acquisition method by transmission and reception of pulses, a separate device for pulse transmission and reception and a synchronization device with a camera are unnecessary, and it is unnecessary to use a type of object (reflection, absorption, etc.) It is intended to provide a mask and a binocular image generation method and apparatus for stereoscopic image of a flat image which is not affected.

According to another aspect of the present invention, there is provided a mask generation method for separating a foreground from an image, the method comprising the steps of: generating an RGB image simultaneously generated and input at the same angle from the image capture device; (depth) depth times to extract depth depth to remove the background (background) image from the image representing the information around the extraction step and the depth image, RGB image or subtracting the depth of the background image in the boundary (edge) images extracted from the IR image And a mask generation step of generating a mask by deleting the inside of a boundary line of a foreground image of the boundary line formed by the mask line.

Here, the imaging apparatus includes a dual-aperture (dual aperture) is preferred to be a single camera comprises a single lens (single lens) 4- color (4-color) sensor.

The depth background extracting step may be configured to separate the depth background image and the depth foreground image based on a predetermined depth threshold value .

And a boundary image of the mask generation step is extracted from the IR image .

Then, in the mask generation step, the contour lines on the boundary line foreground image are searched, the contour line having the maximum length or the maximum area is left, the remaining contour lines are cleared, and binarization and smooth filtering are performed .

Method for generating a binocular image of the present invention for solving the aforementioned problems is a method for generating a binocular image for use in three-dimensional in the plane image separation step in which the first image and the second image partitioned or replicated from the flat image is obtained A shift image generating step in which the mask generated by claim 1 is applied to the first image to generate a first shift image whose foreground is shifted with respect to the background, And a binocular image determination step of confirming the right eye image, the left eye image, or the left eye image and the right eye image.

A method of generating a binocular image in another embodiment is a method of generating a binocular image used for solidifying a planar image, comprising: a separation step in which a first image and a second image that are divided or copied from the planar image are acquired; A shift image generating step in which the generated mask is applied to the first image and the second image to generate a first shift image and a second shift image whose foreground is shifted with respect to the background respectively; characterized by shifting the image, respectively, yirueojim including binocular image confirmation step for confirming a right eye image and the left eye image or the left eye image and a right eye image.

Here, the shift image generation step may include a foreground separation step in which an image to be subjected to the transition is divided into a foreground image and a background image by the mask, and the foreground image is transited in a predetermined direction and a predetermined amount, and the generated foreground shift steps, including the calculated image shift step in which the shift in the foreground image and the background image is merged image shift calculation is preferably true.

And the planar image is an RGB image, and in the separating step, the planar image is channel-divided into the first image and the second image of different colors .

Here, the different colors are preferably complementary colors .

In the foreground shift step, it is preferable that the shift amount is set to be larger as the depth value is smaller.

In the shift image generation step, the shift directions of the first image and the second image are set to be opposite to each other .

According to another aspect of the present invention, there is provided an apparatus for generating a mask used for separating a foreground from an image, the apparatus comprising: A depth background extracting unit for separating and extracting a depth background image from a depth image representing depth information of the depth image and an edge image extracted from the depth image, And a mask generating unit for generating a mask by deleting a boundary line of a foreground image of the boundary line formed by subtraction.

And an apparatus for generating a binocular image comprises a separating unit in which a first image and a second image obtained by dividing or duplicating from the plane image are obtained, A shift image generation unit in which a mask is applied to the first image to generate a first shift image in which a foreground is shifted with respect to a background, and a second shift image generation unit that shifts the first shift image and the second image into a left eye image, binocular image and the right eye image to the final image And a determination unit .

An apparatus for generating a binocular image in another embodiment is an apparatus for generating a binocular image for use in solidifying a planar image, comprising: a separation unit in which a first image and a second image that are divided or copied from the planar image are acquired; A generated mask is applied to the first image and the second image to generate a shift image in which a foreground shifted first shift image and a second shift image, respectively, An image generating unit for generating a first shift image and a second shift image, respectively, and a binocular image for determining the first shift image and the second shift image as a right eye image and a left eye image, And a determination unit .

According to another aspect of the present invention, there is provided a program stored in a storage medium for generating a mask, the program being stored in a storage medium for executing the method of generating a mask, And a program stored in a storage medium readable by the information processing device for execution by the processing device.

The program stored in the storage medium for the method of generating a binocular image of the present invention is a program stored in a storage medium for executing a method of generating a binocular image in each of the above embodiments, And a program stored in a storage medium readable by the information processing device for execution by the processing device.

According to the present invention, it is possible to minimize the cost of the equipment, and it is possible to provide a mask and a binocular image for stereoscopic image of a flat image, which are constituted by a single camera, do not require setting and maintaining of an imaging angle and focus, A method and apparatus are provided.

A mask and a binocular image generation method and apparatus for stereoscopic image of a planar image are provided by acquiring an RGB image as well as an IR image from a camera, in particular, a quality of a captured image is greatly improved even at low illumination.

A mask and a binocular image generation method and apparatus are provided for a three-dimensional image of a planar image, in which the depth information acquired from the camera maintains accuracy even at low illuminance, and the edges of the foreground / background are correct and the erroneousness is eliminated .

In addition, since it is not a depth information acquisition method by transmission and reception of pulses, a separate device for pulse transmission and reception and a synchronization device with a camera are unnecessary, and it is unnecessary to use a type of object (reflection, absorption, etc.) There is provided a mask and binocular image generation method and apparatus for stereoscopic image of an unaffected, planar image.

1 is a block diagram illustrating data flow and processing in a method of the present invention and in an apparatus to which the method is applied.
Figure 2 shows a depth image (depth map) output of a semiconductor chip using a single camera including a dual aperture single lens 4-color sensor, an RGB image, And an IR image.
3 is an exemplary view showing a process of separating a depth image into a deep foreground (FG) image and a depth background (BG) image.
4 is an exemplary view illustrating a border detection process for extracting an IR border image from an IR image.
5 is an exemplary view showing a process of creating an IR boundary foreground image by subtracting a depth background image from an IR boundary line image.
FIG. 6 is an exemplary view of a mask formed by deleting the inside of the border line of the IR boundary line foreground image. FIG.
FIG. 7 is an exemplary view illustrating a process of separating an RGB image into a red image, which is a complementary color image, and a Cyan image, which is a second image.
8 is a flowchart illustrating a process of applying a mask to a red image to create a red foreground image and a red background image, shifting the red foreground image to produce a shift red foreground image, merging the shift foreground image and the red background image, FIG. 8 is an exemplary diagram showing a process of generating an image. FIG.
FIG. 9 is an exemplary view showing a process of determining a shift red image as a right eye image and a cyan image as a left eye image, respectively, and synthesizing them, for example, to generate a final anaglyphic stereoscopic image.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. However, members that perform the same function by the same structure are denoted by the same reference numerals even if they are different from each other, and detailed description thereof may be omitted. And one step or module may be implemented as two or more steps or modules, and conversely, two or more steps or modules may be merged into one step or module. The apparatus for performing the method may be implemented by hardware and a fixedly formed circuit, or may be implemented by a circuit that can be variably changed by a program installed in the hardware.

≪ Method of generating mask >

The method of generating a mask of the present invention is a method of generating a mask 24 as shown in Fig. 6, which is used for separating a foreground from an image.

As shown in FIG. 1, the mask generation method includes a depth background extraction step and a mask generation step .

The depth background extracting step is a step of extracting the RGB image 40 and IR image 20 simultaneously generated and input at the same angle from the image pickup device 11 and representing depth information calculated by the depth map module 12 Extracting a depth background image 31 from the depth image 30. [

The image pickup device 11 is an apparatus for simultaneously generating the RGB image 40 and the IR image 20 at the same angle. Here, the imaging device 11 is, for example, it is preferred that a single camera, comprising a double aperture (dual aperture), a single lens (single lens) 4- color (4-color) sensor. The double-aperture single-lens four-color sensor is a sensor in which RGB and IR imaging elements are integrated in one imaging element, and light received through one lens is received through two diaphragms to form four colors, i.e., R , G, B, and IR color images. As a result, the RGB image and the IR image are simultaneously captured at the same angle with each other.

Contrastingly, for example, the prior art stereo graph is an apparatus for generating two RGB images, and thus differs from the imaging apparatus of the present invention for generating an RGB image and an IR image. Furthermore, the stereograms are different from the imaging apparatus of the present invention, which generates two angular RGB images with equal parallax, because they generate the same angle.

Since the depth image 30 is generated from the RBG image 40 and the IR image 20, the IR image has a strong effect on the low-illuminance image. For example, even in the dark darkness where no RGB image can be obtained at all, the present invention can acquire the depth information by the IR image.

In comparison, the depth information by the stereo graph obtained from only two RGB images differs from the depth image 30 of the present invention because it is impossible or imprecise to acquire depth information in low light.

In addition, the depth information by the flight time sensor is different from the above-described image pickup apparatus of the present invention because a separate apparatus for transmitting and receiving pulses is additionally required. In addition, depending on the types of pulses, there is a disadvantage in that the recognition rate is weak depending on the surrounding environment such as light, sound, and the type of objects (reflection, absorption, etc.).

The depth background extracting step may be configured to separate the depth background image 31 and the depth forehead image 32 based on a predetermined depth threshold value (D threshold) . The depth threshold value may be set to 50 (Th = 50), for example, an intermediate value, for example, 0 to 100.

The mask generation step may include generating a boundary foreground that is obtained by subtracting the depth background image from the depth image 30, the RGB image 40, or the edge image 21 extracted from the IR image 20, And creating a mask 24 by deleting the inside of the boundary line of the image 23.

The boundary image 21 of the mask generation step may be extracted from the depth image 30 or the RGB image 40 but is preferably extracted from the IR image 20 in terms of sharpness and roughness Do. The IR image 20 is an all in-fucus image and is robust in a low-light environment.

In the mask generation step, the deletion process of the borderline is performed by finding contours on the borderline foreground image 23, deleting contour lines having a maximum length or a maximum area, deleting the remaining contour lines, And smooth filtering is preferably performed.

<Effect>

When the mask is formed in accordance with the above procedure, the image pickup device is simple and high in precision, so that the boundary line of the mask is precise and clear. In addition, since the IR image is directly obtained in the imaging device without a separate IR device, the accuracy is simply improved under low illumination. Since an RGB image and an IR image are simultaneously obtained by one imaging device, synchronization is not required. Furthermore, since the RGB image and the IR image are captured at the same angle by the image pickup apparatus, the center shift of the two images is not generated. Since the depth image is directly extracted from the RGB image and the IR image by one imaging device, the calculation is quick and accurate.

The mask can be used to generate the following binocular image.

&Lt; Method of generating binocular image >

A method of generating a binocular image of the present invention is a method of generating a binocular image, i.e., a right eye image and a left eye image required for implementing a stereoscopic image.

In the anaglyph method, a binocular image is merged into a stereoscopic image, which is viewed as a stereoscopic image through red-eyeglasses.

In the polarizing glasses system, a filter for outputting a left eye image and a right eye image to the even-numbered scanning lines and an odd-numbered scanning line, respectively, is provided on the front surface of the display. To be seen as a three-dimensional body.

In the shutter system, a left-eye image and a right-eye image are alternately outputted in a time-division manner, and the viewer wears shutter glasses for alternately turning on and off the left-eye lens and the right-

In the non-eyeglass mode, the display alternately displays the pixels of the left-eye image and the right-eye image, and a parallax barrier is provided in front of the display with a hole aligned with the pixel interval. Only the left eye image pixels are viewed, and the right eye only sees right eye image pixels and is viewed as a stereoscopic image

This binocular image is generated from a planar image. As this plane image, not only the RGB image 40 but also the IR image 20 can be used. However, for the sake of convenience, the case where the plane image is the RGB image 40 will be described below as an example. However, even when the plane image is the IR image 20, the same applies to the case of the RGB image 40, except for the technique based on color.

&Lt; Method of generating binocular image - First embodiment >

A method of generating a binocular image of the present invention is a method of generating a binocular image used for solidification of a planar image (40).

The method for singularity of a binocular image of the first embodiment is characterized by comprising a separation step , a shift image generation step , and a binocular image determination step .

The separating step is a step in which a first image 41 and a second image 42 obtained from the planar image 40 are obtained.

The step of generating the shift image may include the step of generating a first shift image 46 in which the mask 24 described above is applied to the first image 41 so that the foreground 43 is shifted relative to the background 44 to be.

Here, the shift image generation step may include a foreground separation step , a foreground shift step , and a shift image calculation step .

The foreground separating step separates the images 41 and 42 to be shifted into the foreground image 43 and the background image 44 by the mask 24.

1, when the planar image is an RGB image 40, in the separating step, the planar image 40 includes the first image 41 and the second image 42 of different colors ). &Lt; / RTI &gt; This can produce a traditional anaglyph that separates the binocular image according to color.

Here, there is no limitation on the different colors, but if the colors are similar, the stereoscopic effect is lowered, and as the colors tend to be opposite to each other, visibility and stereoscopic effect are increased. In particular, the different colors are preferably complementary colors . In the case of a complementary color of light, for example, red and cyane, when two colors overlap, it becomes white.

The foreground shift step is a step in which the foreground image 43 is shifted in a predetermined direction and amount, and a shift foreground image 45 is generated.

Here, in the foreground shift step, it is preferable that the shift amount is set larger as the depth value is smaller. This creates a shift image with close-range foreground motion, a far foreground image with a small width, and a background with no moving image.

The shift image calculation step is a step in which the shift foreground image 45 and the background image 44 are merged to generate a shift image 46, 47.

The binocular image fixing step is a step of fixing the first shift image 46 and the second image 42 to the right eye image and the left eye image, or the left eye image and the right eye image, respectively.

<Effect>

According to the method of generating a two-dimensional image of the present invention, a planar image is divided into two images, and then the mask is applied to the first image, which is one of the two images. And the shift image and the original second image become a two-dimensional image. Thus, a binocular image is generated in the planar image. In particular, since the mask is generated using the depth image calculated from the sensor that simultaneously acquires the RGB image and the IR image at the same angle, the precision of the mask, the robustness of the illumination, and the smoothness of the boundary are excellent. Since the shift direction and amount can be controlled, a binocular image of various visibility and stereoscopic effect can be calculated.

The binocular image can be used for each stereoscopic image viewing system such as an anaglyph system, a polarization system, a filter system, and an eyeglass system.

&Lt; Method for Generating Binocular Image-Second Embodiment >

The binocular image generating method of the second embodiment is basically the same as the first embodiment, except that the second image 42 is also shifted. Thus, in the shift image generation step, not only the first shift image 46 but also the second shift image 47 are generated. In the binocular image determination step, the first shift image 46 and the second shift image 47 are determined as the right eye image and the left eye image, or the left eye image and the right eye image, respectively.

<Effect>

According to the second embodiment, both the first image 41 and the second image 42 are shifted, so that the first shift image 46 and the second shift image 47 are generated, , The degree of deviation of the center of the foreground image of the stereoscopic image from the center of the foreground image of the original plane image can be reduced, and various stereoscopic effects can be provided by, for example, diversifying the shift direction.

That is, in the second embodiment, it is preferable that, in the shift image generation step, the shift directions of the first image and the second image are set so as to be opposite to each other . This can reduce center deviation. In addition, if the shift amounts are equal to each other, the degree of deviation of the center becomes zero, thereby making it possible to prevent unreasonable distortion due to solidification.

<Device>

The method of the present invention can be implemented as an apparatus via soft wiring or hard wiring.

&Lt; Mask Generating Apparatus >

An apparatus for generating a mask of the present invention is an apparatus for generating a mask 24 as shown in Fig. 6, which is used for separating a foreground from an image. This corresponds to the mask generation method of the present invention.

The apparatus for generating a mask, as shown in Fig. 1,

From the depth image 30 representing depth information calculated by the depth map module 12 from the RGB image 40 and the IR image 20 simultaneously generated and input from the imaging device 11, A depth background extraction unit for separating and extracting the image 31,

The boundary image of the borderline foreground image 23 made by subtracting the depth background image from the depth image 30, the RGB image 40 or the edge image 21 extracted from the IR image 20, A mask generating unit 24 for generating a mask 24,

.

<Device for Generating Binocular Image - First Embodiment>

An apparatus for generating a binocular image for use in solidification of a planar image (40)

And a separation unit in which the image plane 40 is split or replicated the first image 41 and second image 42 from the pickup,

A shift image generation unit in which the mask 24 generated by claim 13 is applied to the first image 41 to generate a first shift image 46 in which the foreground 43 is shifted relative to the background 44 ,

And a binocular image determining unit 42 for determining the first shift image 46 and the second image 42 as the right eye image and the left eye image or the left eye image and the right eye image,

And a control unit.

&Lt; Binocular image generating device-Second embodiment >

An apparatus for generating a binocular image for use in solidification of a planar image (40)

And a separation unit in which the image plane 40 is split or replicated the first image 41 and second image 42 from the pickup,

The mask 24 generated by claim 13 is applied to the first image 41 and the second image 42 to produce a first shift image 46 in which the foreground 43 is shifted relative to the background 44, and the second shift image 47 is generated which shifts image generation unit,

Said first shift image 46 and the second binocular image for deciding the shift image (47), respectively, the right eye image and the left eye image or the left eye image and the right eye image decision section

And a control unit.

<Program stored on storage medium>

The method of the present invention can be implemented as a program stored in a storage medium readable by an information processing device.

<Storage medium -Mask generation method>

The program for generating the mask is a program stored in a storage medium readable by the information processing apparatus for causing the information processing apparatus to execute the steps of the mask generation method described above.

<Storage medium - Method of generating binocular image>

The program for the method of generating a binocular image is a program stored in a storage medium that can be read by the information processing device to execute the steps of the above-described method of generating a binocular image to the information processing device.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, Should be understood as belonging to the scope.

INDUSTRIAL APPLICABILITY The present invention can be applied to a mask for stereoscopic images and a method for producing a two-dimensional image, an apparatus and an industry of a recording medium.

10: information extraction device
11: DA camera module (image pickup device)
12: DA depth map module
20: IR image
21: IR boundary image
22: Filtering
23: IR Border image in the foreground
24: Mask
30: depth image
31: Depth background image
32: Depth foreground image
40: RGB image
41: 1st image
42: Second image
43: foreground image
44: background image
45: Shift foreground image
46: 1st shift image
47: 2nd shift image
48: stereoscopic image (inner glyph)

Claims (20)

A method of generating a mask for use in separating a foreground from an image,
A depth background extracting step of separating and extracting a depth background image from a depth image representing depth information calculated from an RGB image and an IR image simultaneously generated and input at the same angle from the image pickup apparatus,
A mask generation step of generating a mask by deleting a boundary line of a boundary foreground image obtained by subtracting the depth background image from an edge image extracted from the IR image;
Wherein the mask comprises a mask. A method of generating a mask for use in separating a foreground from an image,
A depth background extracting step of separating and extracting a depth background image from a depth image representing depth information calculated from an RGB image and an IR image simultaneously generated and input at the same angle from the image pickup apparatus,
A mask generation step of generating a mask by deleting a boundary line of a boundary foreground image obtained by subtracting the depth background image from an edge image extracted from the depth image, the RGB image, or the IR image;
And,
In the mask generation step,
After contours on the borderline foreground image are found, contour lines having a maximum length or maximum area are left and the remaining contour lines are erased,
Binarization and smooth filtering are performed.
Wherein the mask is a mask. A method of generating a binocular image for use in solidifying a planar image,
A separation step in which a first image and a second image that are divided or copied from the plane image are obtained;
A shift image generation step in which a mask generated by the first or second aspect is applied to the first image to generate a first shift image whose foreground is shifted with respect to the background,
The first image with the shifted second image, respectively, both eyes image confirmation step for confirming a right eye image and the left eye image or the left eye image and the right eye image
And generating a binocular image based on the generated binocular image. A method of generating a binocular image for use in solidifying a planar image,
A separation step in which a first image and a second image that are divided or copied from the plane image are obtained;
A shift image generation step in which the mask generated by the first or second aspect is applied to the first image and the second image to generate a first shift image and a second shift image,
A binocular image determination step of determining the first shift image and the second shift image as a right eye image and a left eye image, respectively, or a left eye image and a right eye image,
And generating a binocular image based on the generated binocular image. The method of claim 3,
The shift image generation step may include:
A foreground separation step in which an image to be subjected to the transition is separated into a foreground image and a background image by the mask;
A foreground shift step in which the foreground image is shifted in a predetermined direction and amount so that a shift foreground image is generated;
A shift image calculation step in which the shift foreground image and the background image are merged to generate a shift image
And generating a binocular image based on the generated binocular image. The method of claim 4,
The shift image generation step may include:
A foreground separation step in which an image to be subjected to the transition is separated into a foreground image and a background image by the mask;
A foreground shift step in which the foreground image is shifted in a predetermined direction and amount so that a shift foreground image is generated;
A shift image calculation step in which the shift foreground image and the background image are merged to generate a shift image
And generating a binocular image based on the generated binocular image. The method of claim 3,
Wherein the planar image is an RGB image,
In the separating step, the planar image is channel-divided into the first image and the second image of different colors
Wherein the first and second images are displayed on the display unit. The method of claim 4,
Wherein the planar image is an RGB image,
In the separating step, the planar image is channel-divided into the first image and the second image of different colors
Wherein the first and second images are displayed on the display unit. The method of claim 7,
The different colors are complementary colors.
Wherein the first and second images are displayed on the display unit. The method of claim 8,
The different colors are complementary colors.
Wherein the first and second images are displayed on the display unit. The method of claim 5,
In the foreground shift step, the shift amount is set larger as the depth value is smaller
Wherein the first and second images are displayed on the display unit. The method of claim 6,
In the foreground shift step, the shift amount is set larger as the depth value is smaller
Wherein the first and second images are displayed on the display unit. The method of claim 4,
In the shift image generation step, the shift directions of the first image and the second image are set so as to be opposite to each other
Wherein the first and second images are displayed on the display unit. An apparatus for generating a mask for use in separating a foreground from an image,
A depth background extracting unit for separating and extracting a depth background image from a depth image representing depth information calculated from an RGB image and an IR image generated and input simultaneously from the imaging apparatus at the same angle;
A mask generating unit for generating a mask by deleting a boundary line of a border foreground image obtained by subtracting the depth background image from an edge image extracted from the IR image ,
Wherein the mask is a mask. An apparatus for generating a mask for use in separating a foreground from an image,
A depth background extracting unit for separating and extracting a depth background image from a depth image representing depth information calculated from an RGB image and an IR image generated and input simultaneously from the imaging apparatus at the same angle;
A mask generating unit for generating a mask by deleting a boundary line of a boundary foreground image obtained by subtracting the depth background image from an edge image extracted from the depth image, the RGB image, or the IR image;
And,
In the mask generator,
After contours on the borderline foreground image are found, contour lines having a maximum length or maximum area are left and the remaining contour lines are erased,
Binarization and smooth filtering are performed.
Wherein the mask is a mask. An apparatus for generating a binocular image, which is used for solidification of a planar image,
A separating section in which a first image and a second image that are divided or copied from the plane image are obtained,
A shift image generation unit in which a mask generated by the method of claim 14 or 15 is applied to the first image to generate a first shift image whose foreground is shifted with respect to the background,
And a binocular image determining unit for determining the first shift image and the second image as a right eye image and a left eye image or a left eye image and a right eye image,
And an image generating unit for generating a binocular image based on the generated binocular image. An apparatus for generating a binocular image, which is used for solidification of a planar image,
A separating section in which a first image and a second image that are divided or copied from the plane image are obtained,
A shift image generation unit in which masks generated by the fourteenth aspect of the present invention are applied to the first image and the second image to generate a first shift image and a second shift image,
And a binocular image determination unit for determining the first shift image and the second shift image as a right eye image and a left eye image or a left eye image and a right eye image,
And an image generating unit for generating a binocular image based on the generated binocular image. A program stored in a storage medium for causing an information processing apparatus to execute each step of a method for generating a mask used for separating a foreground from an image,
A program stored in a storage medium readable by an information processing device for causing each of the steps of the method for generating a mask of claim 1 or 2 to be executed by the information processing device. A program stored in a storage medium for causing an information processing apparatus to execute each step of a method of generating a binocular image used for stereoscopic image formation,
A program stored in a storage medium readable by an information processing apparatus for causing each of the steps of the method of generating a binocular image of claim 3 to be executed by the information processing apparatus. A program stored in a storage medium for causing an information processing apparatus to execute each step of a method of generating a binocular image used for stereoscopic image formation,
A program stored in a storage medium readable by an information processing apparatus for causing each of the steps of the method for generating a two-dimensional image of claim 4 to be executed by the information processing apparatus.

Images