JP5782761B2 - Parallax image generation device, parallax image generation method, program, and storage medium - Google Patents

Description

  The present invention relates to a parallax image generation device that generates a parallax image for realizing stereoscopic vision, and more particularly to a parallax image generation device that corrects a hidden surface generated in two-dimensional / three-dimensional conversion.

  Conventional three-dimensional video display systems are roughly classified into a two-parallax system and a multi-parallax system having three or more parallaxes. In either method, it is necessary to prepare a video in which the scene is viewed from the required number of viewpoints.

  As the most direct video production method, there can be considered a method in which cameras for the required number of parallaxes are prepared and the same scene is shot with a plurality of cameras arranged at predetermined intervals. Recently, cameras that can directly shoot two parallaxes are commercially available, but cameras that can directly shoot multiple parallaxes are not common.

A method of creating a three-dimensional CG (Computer Graphics) image and setting a plurality of camera positions for rendering is also conceivable. The three-dimensional CG image includes depth information, and a parallax image corresponding to the multi-parallax method can be created from the three-dimensional image. However, when the number of camera positions is large, there is a problem that the number of renderings increases, which takes time and cost.
Furthermore, most videos are not CG images but are real photographs, and it is required to create a three-dimensional video from the real photographs.

There are several methods (so-called two-dimensional / three-dimensional conversion methods) for converting a multi-parallax image based on a flat image to produce a multi-parallax image based on a live-action image (Patent Document 1, Patent Document 2). ).
Patent Document 1 discloses a two-dimensional / three-dimensional conversion method in which a planar image is separated into a near view and a distant view, and only the near view region is shifted by a predetermined distance and enlarged to generate a parallax image.
On the other hand, the present applicant receives a parallax from a relationship between a depth image and a virtual camera position by inputting a two-dimensional image and an image having depth information (hereinafter referred to as “depth image”) in Patent Document 2. A method for generating video is proposed.

JP 2010-233391 A Japanese Patent Application No. 2010-176864

However, the method described in Patent Document 1 separates the entire screen into two levels of depths of foreground and distant views, and there is a problem that the depth impression received from the generated image is poor with only the two levels of depth. is there. Further, since the inside of the foreground has no parallax, there is a problem that the impression of depth is further poor.
In addition, the technique described in Patent Document 1 avoids the appearance of a blank portion by shifting and enlarging the foreground because a blank portion of the video is generated when the foreground and the distant view are separated and shifted. However, since the subject in the foreground is originally photographed large, there is a problem that a further enlargement causes a contradiction in the scene on the screen and a great sense of discomfort.

On the other hand, the technique described in Patent Document 2 has the advantage that the overall depth impression is enriched because the shift amount due to parallax is determined for each pixel rather than for each subject in the image from the relative relationship between the depth image and the camera position. There is.
However, since it is not possible to specify whether two pixels belong to the same subject or another subject only by information on the depth amount between two adjacent pixels, it is a hidden surface when two pixels belong to another subject. There is a problem that cannot be dealt with.

Here, the “hidden surface” will be described with reference to FIG. The “hidden surface” is an area where pixel information at the position is missing in the original planar image, and is caused by parallax.
FIG. 15A shows the original planar image. The scene recorded in the original planar image is composed of a background 100, a front object 101 that is a circular object, and a back object 102 that is a rectangular object.
FIG. 15B shows a state of looking into from the A direction in FIG. In FIG. 15B, a hidden surface 103 a (black region) is generated between the near object 101 and the background 100.
FIG. 15 (c) shows a state of looking into from the B direction in FIG. 15 (a). In FIG. 15C, a hidden surface 103b (black region) is generated between the near object 101 and the deep object 102.
If no action is taken against such a hidden surface, pixels at the border of the front subject or the back subject are displayed in the area where the back subject should be properly compensated. May result in an unnatural image.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a parallax image generation device and the like that can generate a parallax image in which a hidden surface is corrected without a sense of incongruity. .

In order to achieve the above-described object, a first invention is a parallax image generation device that generates a parallax image for realizing stereoscopic vision, and includes a desired two-dimensional image, from a camera related to the two-dimensional image to a subject. An image input means for inputting a depth image that is an image in which the depth value is replaced with a pixel value, and a label image that is an image in which a label value that identifies a subject constituting the scene of the two-dimensional image is replaced with a pixel value , number of cameras, camera separation, and the parameter input means for inputting a parameter consisting of the depth values from the camera to the screen surface, and the sampling point of the screen surface, the camera position is set based on the previous Kipa parameter, the A line of sight line passing through is calculated, and any one of the passing sections, which are sections through which the line of sight line passes, among the sections of the depth values is adopted as the section adopted. Among the first depth value that is the depth value on the first direction side and the second depth value that is the depth value on the second direction side that is different from the first direction, refer to the position that indicates the back side. When the first label value and the second label value of the label image corresponding to the position of the first depth value and the second depth value are different from the reference position specifying unit as the position, the reference position is a hidden surface. When the reference position is determined not to be a hidden surface by the hidden surface determination means that determines that the pixel is a hidden surface, the pixel value of the reference position in the two-dimensional image is the pixel value of the sampling point. And when the reference position is determined to be a hidden surface by the hidden surface determination means, the determination result by the hidden surface determination means is determined as a deeper one of the first depth value and the second depth value. The pixel value of the sampling point is determined by referring to the first direction side or the second direction side which is a direction indicating the hidden surface boundary position, and by performing a folded copy of the pixel value with the hidden surface boundary position as the folded position. And a pixel value assigning unit for assigning a compensation pixel value which is a pixel value for compensating for the parallax image generation apparatus.
According to the first invention, it is possible to generate a parallax image in which the hidden surface is corrected without a sense of incongruity. This also makes it possible to assign compensation pixel values that are considered reasonable in many cases.

In the compensation processing in the pixel value assigning means according to the first aspect of the invention, for example, it is determined whether or not the label value of the pixel position to be copied by copying matches the label value of the subject that should appear at the reference position. If they match, the compensation pixel value is assigned by folding copy of the pixel value with the hidden surface boundary position as the folding position, and if they do not match, it is the same as the label value of the subject that should appear at the reference position A pixel value at a reference position related to the farthest point in a group of sampling points is assigned as the compensation pixel value.
Accordingly, the pixel value of another subject is not compensated as the compensation pixel value, and in any case, it is possible to assign a compensation pixel value that is considered to be reasonable.

In the first invention, when the reference position is a hidden surface, and when the first depth value indicates a value on the back side with respect to the second depth value, the hidden surface determination unit is configured to perform the line of sight. It is determined that a first direction hidden surface viewed from the first direction is generated by passing a straight line between the subject in front and the subject in the back from the first direction, or the second depth value is In the case where a value on the back side from the first depth value is indicated, the second line viewed from the second direction by passing the line of sight between the subject in the foreground and the subject in the back from the second direction. It is determined that a direction hidden surface has occurred, and the compensation processing in the pixel value assigning unit refers to the determination result by the hidden surface determination unit in the first direction in the case of the first direction hidden surface. To identify the hidden surface boundary position Alternatively, in the case of the hidden surface in the second direction, the hidden surface boundary position is specified by referring to the determination result by the hidden surface determination means in the second direction, and the hidden surface boundary position is used as a reference. It is desirable to specify the compensation pixel value as follows.
Thereby, a parallax image in which the hidden surface is corrected without a sense of incongruity can be generated.

A second invention is a parallax image generation method for generating a parallax image for realizing stereoscopic vision, wherein a desired two-dimensional image and a depth value from a camera to a subject related to the two-dimensional image are replaced with a pixel value An image input step of inputting a depth image that is a captured image, and a label image that is an image obtained by replacing a label value that identifies a subject constituting the scene of the two-dimensional image with a pixel value, the number of cameras, a camera interval, and calculates a parameter input step for inputting a parameter consisting of the depth values from the camera to the screen surface, and the sampling point of the screen surface, the camera position is set based on the previous Kipa parameter, the sight line passing through the One of the passing sections that are the sections through which the line of sight passes among the sections of depth values is defined as the employed section, and the first direction in the employed section Of the first depth value, which is the depth value of the second depth value, and the second depth value, which is the depth value on the second direction side different from the first direction, is a reference position specifying the position that indicates the back side as the reference position Hiding and determining that the reference position is a hidden surface when the first label value and the second label value of the label image respectively corresponding to the position of the first depth value and the second depth value are different from each other When it is determined by the surface determination step and the hidden surface determination step that the reference position is not a hidden surface, the pixel value of the reference position in the two-dimensional image is set as the pixel value of the sampling point, and the hidden surface determination is performed. If it is determined in the step that the reference position is a hidden surface, the determination result in the hidden surface determination step is determined as a deeper one of the first depth value and the second depth value. The pixel value of the sampling point is determined by referring to the first direction side or the second direction side which is a direction indicating the hidden surface boundary position, and by performing a folded copy of the pixel value with the hidden surface boundary position as the folded position. And a pixel value assigning step for assigning a compensated pixel value that is a pixel value for compensating for a parallax image.
According to the second invention, it is possible to generate a parallax image in which the hidden surface is corrected without a sense of incongruity.

According to a third aspect of the present invention, a computer forms a desired two-dimensional image, a depth image that is an image obtained by replacing a depth value from the camera related to the two-dimensional image to a subject with a pixel value, and a scene of the two-dimensional image An image input means for inputting a label image, which is an image obtained by replacing a label value for identifying a subject to be replaced with a pixel value, and a parameter input for inputting a parameter consisting of the number of cameras, a camera interval, and a depth value from the camera to the screen surface. It means, and the sampling point of the screen surface, before the camera position is set based on Kipa parameters, calculates the line of sight line passing through a passage which is the line of sight straight line passing through the section in the depth values between the interval Any one of the sections is an adopted section, and is different from the first depth value, which is the depth value on the first direction side in the adopted section, and the first direction. Of the second depth values that are the depth values on the second direction side, corresponding to the position of the reference position specifying means that uses the position indicating the deeper as the reference position, and the positions of the first depth value and the second depth value, respectively. When the first label value and the second label value of the label image are different, the reference position is not a hidden surface by the hidden surface determination means that determines that the reference position is a hidden surface, and the hidden surface determination means. If it is determined, the pixel value of the reference position in the two-dimensional image is set as the pixel value of the sampling point, and when the reference position is determined to be a hidden surface by the hidden surface determination unit, The determination result by the hidden surface determination means is referred to the first direction side or the second direction side which is the direction indicating the far side of the first depth value and the second depth value, and the hidden surface boundary position is determined. Identify, This is a program for functioning as pixel value assigning means for assigning a compensation pixel value that is a pixel value for compensating the pixel value of the sampling point by folding copy of the pixel value with the hidden surface boundary position as the folding position. .
By installing the program according to the third invention in the computer, the parallax image generating device according to the first invention can be obtained.

  A fourth invention is a computer-readable storage medium storing a program according to the third invention.

  According to the present invention, it is possible to provide a parallax image generation device that can generate a parallax image in which a hidden surface is corrected without a sense of incongruity.

Hardware configuration diagram of parallax image generation device The figure which shows the input-output data of a parallax image generation apparatus Diagram explaining parameters Flow chart showing details of parallax image generation processing The figure explaining reference position specific processing The figure explaining reference position specific processing The figure which shows the 1st process example The figure which shows the 2nd processing example The figure explaining modification 1 The figure which shows an example of a two-dimensional image Diagram showing an example of a depth image The figure which shows an example of a label image The figure which shows an example of the parallax image from which the eyes | visual_axis is from the left direction The figure which shows an example of the parallax image from which a gaze is right Illustration explaining the hidden surface

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a hardware configuration diagram of the parallax image generation device 1. Note that the hardware configuration in FIG. 1 is an example, and various configurations can be adopted depending on applications and purposes.
In the parallax image generation device 1, a control unit 11, a storage unit 12, a media input / output unit 13, a communication control unit 14, an input unit 15, a display unit 16, a peripheral device I / F unit 17, etc. are connected via a bus 18. Is done.

The control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Y Memory Y), a RAM (Random Access Memory Y), and the like.
The CPU calls and executes a program stored in the storage unit 12, ROM, recording medium, or the like to a work memory area on the RAM, drives and controls each device connected via the bus 18, and the parallax image generation device 1. The process to be described later is realized.
The ROM is a non-volatile memory, and permanently stores a boot program of the parallax image generation device 1, a program such as BIOS, data, and the like.
The RAM is a volatile memory, and temporarily stores programs, data, and the like loaded from the storage unit 12, ROM, recording medium, and the like, and includes a work area used by the control unit 11 for performing various processes.

The storage unit 12 is an HDD (hard disk drive) or the like, and stores a program executed by the control unit 11, data necessary for program execution, an OS (operating system), and the like. As for the program, a control program corresponding to an OS (operating system) and an application program for causing the parallax image generating device 1 to execute processing described later are stored.
Each of these program codes is read by the control unit 11 as necessary, transferred to the RAM, read by the CPU, and executed as various means.

The media input / output unit 13 (drive device) inputs / outputs data, for example, media such as a CD drive (-ROM, -R, -RW, etc.), DVD drive (-ROM, -R, -RW, etc.) Has input / output devices.
The communication control unit 14 includes a communication control device, a communication port, and the like, is a communication interface that mediates communication between the parallax image generation device 1 and the network, and performs communication control between other devices via the network. . The network may be wired or wireless.

The input unit 15 inputs data and includes, for example, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad.
An operation instruction, an operation instruction, data input, and the like can be performed on the parallax image generation device 1 via the input unit 15.
The display unit 16 includes a display device such as a CRT monitor and a liquid crystal panel, and a logic circuit (such as a video adapter) for realizing the video function of the parallax image generation device 1 in cooperation with the display device.

The peripheral device I / F (interface) unit 17 is a port for connecting a peripheral device to the parallax image generation device 1, and the parallax image generation device 1 transmits data with the peripheral device via the peripheral device I / F unit 17. Send and receive. The peripheral device I / F unit 17 is configured by USB, IEEE 1394, RS-235C, or the like, and usually includes a plurality of peripheral devices I / F. The connection form with the peripheral device may be wired or wireless.
The bus 18 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

FIG. 2 is a diagram showing input / output data of the parallax image generating device 1. As shown in FIG. 2, the parallax image generating device 1 uses a two-dimensional image 2, a label image 3, a depth image 4, and a parameter 5 as input data. Further, the parallax image generation device 1 uses the parallax image 6 as output data.
The two-dimensional image 2, the label image 3, and the depth image 4 may be generated by the control unit 11 of the parallax image generation device 1, or may be acquired from the outside via the media input / output unit 13, the communication control unit 14, and the like. May be. The parameter 5 may be input via the input unit 15 of the parallax image generation device 1 or a parameter file in which the parameter 5 is defined in advance may be acquired from the outside.
The parallax image 6 may be stored in the storage unit 12 of the parallax image generation device 1, displayed on the display unit 16, or output to the outside via the media input / output unit 13, the communication control unit 14, and the like. You may do it.

The two-dimensional image 2 is a frame image or a still image for one moving image (planar image). A moving image (planar image) includes, for example, a color still image of a predetermined number of frames per unit time (usually 30 frames per second), and each pixel has 256 gradations of RGB.
Since all the processes of the parallax image generation process to be described later are executed independently for each frame, the following description will describe the process of one frame image (still image).

The label image 3 is associated with each two-dimensional image 2 and is divided into regions by unique numerical values (label values) for identifying subjects constituting the scene recorded in the two-dimensional image 2, and the label values are converted into pixel values. It is an image replaced with. For example, in a scene composed of a background, a building, and a person, L0, L1, and L2 are assigned as label values corresponding to the background, the building, and the person, respectively, and each pixel of the corresponding two-dimensional image 2 is assigned to which subject. Whether it belongs or not is expressed by replacing it with pixel values of L0, L1, and L2.
The label image 3 may be generated manually using photo retouching software or the like based on the two-dimensional image 2, or may be automatically generated using separate software.

The depth image 4 is an image in which the depth value from the camera to the subject related to the two-dimensional image 2 is replaced with a pixel value. Here, the camera means a virtual camera defined in the internal processing of the parallax image generation device 1.
The depth image 4 is, for example, a gray scale with a pixel value range of 0 to 255 (8 bits), and 0 (black) is the farthest and 255 (white) is often the foremost. There may be. In the following, for ease of explanation, it is assumed that the depth image 4 has a pixel value range of 0 to 255, 0 (black) being the deepest and 255 (white) being the foremost.
The depth image 4 may be generated manually using photo retouching software or the like based on the two-dimensional image 2, or may be automatically generated using separate software.

The parameter 5 is a value used for a parallax image generation process to be described later. Parameter 5 includes the number of cameras n, the camera interval, the depth value from the screen surface to the camera, and the like.
Furthermore, the parameter 5 includes the horizontal width, vertical width, pixel pitch, and the like of the two-dimensional image 2. In the two-dimensional image 2, the number of pixels in the horizontal direction (X-axis direction) and the number of pixels in the vertical direction (Y-axis direction) are determined by the horizontal width, vertical width, and pixel pitch of the parameter 5.

FIG. 3 is a diagram for explaining the parameter 5.
In the parallax image generation processing, as shown in FIG. 3, the screen surface S and the cameras C _{1 to} C ₈ are arranged in a virtual three-dimensional space (XYZ space).
The screen surface S is an area where the two-dimensional image 2 is projected, and each pixel of the parallax image 6 is included. In the example shown in FIG. 3, the screen surface S coincides with the XY plane.

The cameras C _{1 to} C ₈ are arranged based on the number of cameras n of parameter 5, the camera interval, and the depth value from the screen surface S to the camera. In the example illustrated in FIG. 3, the camera number n is “8”, the camera interval is “equally spaced C _{camera_X} ”, and the depth value is “C _{camera_Z} ”. C _{camera_X} is a binocular parallax (about 60 mm to about 70 mm).
In the example illustrated in FIG. 3, the camera installation line 30 is a straight line parallel to the screen surface S and having a distance of “C _{camera_Z} ” from the screen surface S. When C _i is a camera ID (C _i = 1,..., 8), the camera positions (C _Xi , C _Zi ) (i = 1,..., 8) of the cameras C _{1 to} C ₈ are as follows. Based on the formula of W is the width of the two-dimensional image 2.

In the example shown in FIG. 3, the camera interval is set to be equal, but this depends on how the image is displayed on the display. When the camera interval is set to be equal, the parallax images 6 adjacent to each other can be seen from any position. However, if another way of viewing is desired, the camera interval may not be equal.
In the example shown in FIG. 3, the camera installation line 30 is a straight line, but this also depends on how the image is shown on the display. For example, when the display is a curved surface, the camera installation line 30 may be a curved line.
A farthest plane P shown in FIG. 3 is a position farthest from the camera C among subjects represented in the parallax image 6 and is a plane parallel to the screen plane S.

Returning to the description of FIG.
The parallax image 6 is an image for realizing stereoscopic viewing. In the embodiment of the present invention, the parallax image 6 is an image for realizing stereoscopic vision by naked-eye observation, particularly using binocular parallax (about 60 mm to about 70 mm).
Although not described in detail in the embodiment of the present invention, a plurality of parallax images 6 are synthesized in accordance with the specifications of the display and displayed on the display, thereby making it possible to realize stereoscopic vision by naked eye observation. . A known technique may be used for the synthesis process of the plurality of parallax images 6.

  In the reference position specifying process 21, the depth image 4 and the parameter 5 are used as input data. In the reference position specifying process 21, a line of sight that passes through the sampling point on the screen surface and the camera position set based on a predetermined parameter is calculated, and the line of sight in the interval between the depth values in the depth image 4 is calculated. Any one of the passing sections that are passing sections is taken as the adopted section, and the first depth value that is the depth value on the first direction (for example, “left”) side in the adopted section and the second different from the first direction Of the second depth values that are depth values on the direction (eg, “right”) side, the position that indicates the depth is the reference position. The identified reference position is stored in the RAM or the storage unit 12 as a “reference position buffer”.

The size of the reference position buffer may be the number of pixels in the horizontal direction of the two-dimensional image 2 × the number of pixels in the vertical direction, or may be the number of pixels in the horizontal direction of the two-dimensional image 2. The same applies to the sizes of other buffers to be described later. Note that the number of pixels in the horizontal direction and the number of pixels in the vertical direction of the two-dimensional image 2, the label image 3, the depth image 4, and the parallax image 6 are all the same.
In the following description, the first direction is left and the second direction is right. Similarly, the first depth value will be described as the left depth value, and the second depth value will be described as the right depth value.

  The reference position can be specified by the depth amount between two pixels. On the other hand, only by the depth amount between two pixels, it cannot be specified whether the two pixels belong to the same subject or another subject. In addition, when two pixels belong to different subjects, a hidden surface is generated due to the parallax between the two pixels, and if nothing is dealt with, an area that should be displayed by appropriately compensating for the rear subject. In addition, the result is that the pixels at the boundary of the front subject or the rear subject are copied, which is not preferable. Therefore, in the embodiment of the present invention, the label image 3 is also used as input data, and an appropriate pixel value is compensated for the hidden surface by the hidden surface determination process 22, the label value assignment process 23, and the pixel value assignment process 24.

  In the hidden surface determination process 22, the label image 3 and the reference position buffer are used as input data. In the hidden surface determination process 22, when the left label value and the right label value of the label image 3 corresponding to the positions of the left depth value and the right depth value are different, the reference position is determined to be a hidden surface. Moreover, in the hidden surface determination process 22, when the left label value and the right label value are the same, it determines with a reference position not being a hidden surface. The hidden surface determination result is stored in the RAM, the storage unit 12 or the like as a “hidden surface buffer”.

  Further, in the hidden surface determination process 22, when the reference position is a hidden surface, and further, when the left depth value indicates a value on the back side with respect to the right depth value, the line of sight line is compared with the object on the front side from the left direction. It is determined that a left hidden surface viewed from the left is generated by passing between the subject and the subject. Further, in the hidden surface determination processing 22, when the right depth value indicates a value on the back side with respect to the left depth value, the line of sight passes between the subject on the near side and the subject on the back from the right, and from the right. It is determined that the right hidden surface that you looked into has occurred. This determination result is also stored in the RAM or the storage unit 12 as a “hidden surface buffer”.

  In the label value assignment process 23, the label image 3 and the hidden surface buffer are used as input data. In the label value assignment process 23, when it is determined in the hidden surface determination process 22 that the reference position is not a hidden surface, the left label value or the right label value of the label image 3 is assigned to the sampling point to be processed (in this case, Since the left label value and the right label value are the same, either may be used.) Also, in the label value assignment process 23, when the hidden surface determination process 22 determines that the reference position is a hidden surface, which subject appears by the hidden surface and which subject hides the hidden surface. Identify what was happening. In other words, in the label value assignment process 23, the back-side subject and the near-side subject relating to the hidden surface are specified. In the label value assigning process 23, the label values of both of the specified subjects are assigned to the sampling points to be processed. The assigned label value is stored as “label buffer” in the RAM, the storage unit 12 or the like.

In the pixel value assignment process 24, the two-dimensional image 2, the label buffer, the reference position buffer, and the like are input data. In the pixel value assignment process 24, when it is determined in the hidden surface determination process 22 that the reference position is not a hidden surface, the pixel value at the reference position in the two-dimensional image 2 is set as the pixel value of the sampling point, and the hidden surface determination process If it is determined in 22 that the reference position is a hidden surface, a compensation process that is a process of specifying a pixel value (compensation pixel value) for compensating the pixel value of the sampling point is executed.
In the compensation processing, for example, in the case of the left hidden surface, the control unit 11 specifies the hidden surface boundary position by referring to the determination result by the hidden surface determination processing 22 to the left. In the case of the right hidden surface, the control unit 11 identifies the hidden surface boundary position by referring to the determination result of the hidden surface determination process 22 to the right. And the control part 11 specifies a compensation pixel value on the basis of a hidden surface boundary position.
A set of pixel values of sampling points assigned by the pixel value assignment process 24 is a parallax image 6.

Below, the detail of the process of the parallax image generation apparatus 1 is demonstrated, referring FIGS. 4-8.
FIG. 4 is a flowchart showing details of the parallax image generation processing.

The control unit 11 of the parallax image generation device 1 determines the Y coordinate on the screen surface S in order to specify the XZ plane to be processed (S201). Hereinafter, the Y coordinates determined in S201 and _{Y s.} Parallax image generating device 1 repeats the process of S202~S208 each _{Y s.} By executing the processing of S202 to S208 for all the Y coordinates, one parallax image 6 of the two-dimensional image 2 is obtained for one camera C.

The control unit 11 determines a pixel to be processed on the screen surface S, that is, a sampling point (S202). However, Y coordinates of the sampling points is the _{Y s} determined in S201.
Next, the control part 11 performs the process which calculates the pixel value of the parallax image 6 by the process of S203-S206 with respect to the sampling point determined in S202.

  The control unit 11 performs the reference position specifying process 21 (S203). Here, the reference position indicates which position in the two-dimensional image 2 corresponds to each pixel in the generated parallax image 6. For each pixel in the parallax image 6, the control unit 11 calculates a reference position in the two-dimensional image 2 corresponding to the pixel from geometric information such as the depth value of the corresponding point and the camera position.

  5 and 6 are diagrams for explaining the reference position specifying process. 5 and 6, the depth distribution curve 32 connecting the depth distribution information to a plurality of black circles with the pixel position of each depth value (the pixel value of the depth image 4) on the screen surface S as the X coordinate and the depth value itself as the Z coordinate. It is illustrated by On the screen surface S, sampling points are determined by a predetermined interval. It is assumed that the two-dimensional image 2 is at the position of the farthest surface P.

First, a description will be given with reference to FIG.
Control unit 11 calculates the X-coordinate _{X s} of the sampling points, the camera position to be processed _{(C Xi, C Zi) (} i = 1, ···, n) and the line of sight straight 31a passing. As described above, the camera position (C _Xi , C _Zi ) is set based on the number of cameras n of parameter 5, the camera interval C _{camera_X,} and the depth value C _{camera_Z} from the camera C _i to the screen surface S. In the example shown in FIG. 5, and the camera C ₃ processed.

Next, the control unit 11 specifies a passage section 33a that is a section through which the line of sight line 31a passes among sections of depth values (black circles) in the depth distribution curve 32. And the control part 11 makes the passage area 33a an adoption area, and shows the back | inner side among the left depth value 34a which is the depth value of the left side in the adoption area, and the right depth value 35a which is a right depth value ( The X coordinate position of the smaller Z coordinate value is set as the reference position. In the example shown in FIG. 5, the X coordinate position of the left depth value 34a is the reference position. When the X-coordinate of the left depth values 34a and _{X k,} the difference between _{X s} and _{X k} is the shift amount 36. The shift amount 36 is a difference between the pixel position of the generated parallax image 6 and the corresponding pixel position of the two-dimensional image 2.
Then, the control unit 11, the value X _k of the X-coordinate of the left depth value 34a, as a reference position for the sampling point X _s, and stores the reference position buffer.
The control unit 11 performs the above processing on all the cameras C _{1 to} C ₈ and calculates the shift amount of each pixel (X _s , Y _s ) in each parallax image 6.

FIG. 6 shows an example in which a plurality of passage sections 33b, 33c, and 33d exist. In this case, the control unit 11 sets the one closest to the cameras C _{1 to} C ₈ as the adopted section. In the example shown in FIG. 6, the passage section 33 b is an adopted section.
And the control part 11 makes the X coordinate position which shows the back | inner side among the left depth value 34b and the right depth value 35b in an employment area the reference position. In the example shown in FIG. 6, the X coordinate position of the right depth value 35b is the reference position.

If multiple pass section 33 is present, the closest to the camera C ₁ -C ₈ is the closest to the object. Therefore, by using the section closest to the cameras C _{1 to} C ₈ as the adopted section, it is possible to generate the parallax image 6 in which the arrangement relationship of a plurality of subjects is faithfully reproduced.

FIG. 7 is a diagram illustrating a first processing example. In Figure 7, hidden plane buffer 42 as the processing result with respect to each sampling point _{X s,} processing the results reference position buffer 41 of the reference position specifying process 21 (S203), the hidden surface determination process 22 (S204), the label A label buffer 43 that is the processing result of the value assignment processing 23 (S205) and a generation result buffer 44 that is the processing result of the pixel value assignment processing 24 (S206) are shown. In the example of FIG. 7, there are seven subjects that form the scene of the two-dimensional image 2 including the background. The label value “L0” is associated with the background, and the label values “L1” to “L6” are associated with the other objects.

The reference position buffer 41 in FIG. 7, the sampling point _{X s} is "1,2,3,4,5,6,7,8, ..." for each of the reference position buffer 41 is "3, 4, 5, 7, 8, 9, 11, 12,... "Are stored.

Returning to the description of FIG.
The control unit 11 performs a hidden surface determination process 22 (S204). The control unit 11 refers to the label image 3 and both ends (left depth value 34a and right depth value 35a) of the adopted section (33a in the example of FIG. 5) specified in S203 belong to the same subject, or It is determined whether the subject belongs to a different subject.
Specifically, the control unit 11 sets the pixel value of the pixel position of the label image 3 corresponding to the position of the left depth value 34a (pixel position of the depth image 4) as the left label value, and the position of the right depth value 35a (depth). The pixel value at the pixel position of the label image 3 corresponding to the pixel position of the image 4 is set as the right label value. Then, the control unit 11 compares the left label value with the right label value. When both are the same, it means that the line of sight (31a in the example of FIG. 5) intersects with a single subject represented by the label value, so the control unit 11 determines that the reference position is not a hidden surface. To do. On the other hand, if they are different, it means that the line of sight (31a in the example of FIG. 5) passes between different objects, and the control unit 11 determines that the reference position is a hidden surface.

  The control unit 11 stores the hidden surface determination result in the hidden surface buffer 42. More specifically, the control unit 11 (1) determines whether or not the reference position is a hidden surface, and (2) when the reference position is a hidden surface, the reference position looks into the subject in front from the left. The hidden surface buffer 42 stores whether it is a hidden surface (= left hidden surface) caused by the above or a hidden surface (= right hidden surface) generated by looking into the right.

For example, in the example of FIG. 5, since the left depth value 34a (depth information of the reference position) is behind the right depth value 35a, the position (reference position) of the left depth value 34a is a hidden surface. The control unit 11 determines that this hidden surface is a left hidden surface.
For example, in the example of FIG. 6, the right depth value 35b (depth information of the reference position) is behind the left depth value 34b, so the position (reference position) of the right depth value 35b is a hidden surface. If there is, the control unit 11 determines that this hidden surface is a right hidden surface.

  The hidden surface buffer 42 of FIG. 7 stores that the reference position buffer 42 is a right hidden surface (= “right”) for “3, 4, 5”. The reference position buffer 42 stores “7, 8, 9, 11, 12” indicating that it is not a hidden surface (= “non”).

Returning to the description of FIG.
Next, the control unit 11 performs a label value assignment process 23 (S205). The control unit 11 assigns a label value to each reference position and stores it in the label buffer 43. At this time, when the reference position is a hidden surface, the control unit 11 determines the label value of the subject (= subject subject) that has hidden the reference position and the subject (= subject subject) that should appear at the reference position. Stores the label value.

For example, in the example of FIG. 5, when the position (reference position) of the left depth value 34a is a hidden surface in the hidden surface determination result, this hidden surface is determined to be the left hidden surface. Accordingly, the control unit 11 determines that the subject with the left label value corresponding to the position of the left depth value 34a is the subject on the back side, and uses it as the label value of the subject (= the subject on the back side) that should appear at the reference position. Stores the left label value. Further, the control unit 11 determines that the subject with the right label value corresponding to the position of the right depth value 35a is the subject on the near side, and the label value of the subject (= the subject on the near side) that has hidden the reference position. As the right label value.
For example, in the example of FIG. 6, when the position (reference position) of the right depth value 35 b is a hidden surface in the hidden surface determination result, this hidden surface is determined to be the right hidden surface. . Therefore, the control unit 11 determines that the subject with the right label value corresponding to the position of the right depth value 35b is the subject on the back side, and uses it as the label value of the subject (= the subject on the back side) that should appear at the reference position. Stores the right label value. Further, the control unit 11 determines that the subject of the left label value corresponding to the position of the left depth value 34b is the subject on the near side, and the label value of the subject (= the subject on the near side) that has hidden the reference position. As the left label value.

The label buffer 43 in FIG. 7 stores the label value at each reference position. When the reference position buffer 42 is “3, 4, 5”, the label value of the subject whose reference position is hidden = L6 / the label value of the subject to appear at the reference position = L0 is stored. Further, since the reference position X _k = 7, 8, 9, 11, 12 is not a hidden surface, only the label value = L0 is stored.

Next, the control unit 11 performs a pixel value assignment process 24 (S206). The control unit 11 confirms the hidden surface buffer 42 for each sampling point X _s , and if the result of the hidden surface determination is not a hidden surface (in the case of “non”), confirms the reference position buffer 41 and samples the sampling point X _s. relative, assigning the pixel values of the reference position X _k in the two-dimensional image 2.

In the example shown in FIG. 7, since the value of the hidden surface buffer 42 at the sampling point X _s “4, 5, 6, 7, 8” is “non-”, the control unit 11 is stored in the reference position buffer 41, respectively. The reference positions X _k = 7, 8, 9, 11, and 12 are stored in the generation result buffer 44. That is, the control unit 11, the pixel values of the reference position X _k stored in the reference position buffer 41, as it is assigned to the sampling point X _s.

On the other hand, the control unit 11 checks the hidden plane buffer 42 for each sampling point X _s, (in the case of "left") As a result of the determination hidden surface when left hidden face, with reference to the hidden plane buffer 42 to the left The hidden surface boundary position is specified by going, or when the result of the hidden surface determination is the right hidden surface (in the case of “right”), the hidden surface boundary position is determined by referring to the hidden surface buffer 42 to the right. Identify.
And the control part 11 specifies the pixel value (compensation pixel value) which compensates the pixel value of a sampling point on the basis of a hidden surface boundary position.

For example, the control unit 11 generates a value of the reference position buffer 41 for calculating the distance d to the hidden surface boundary position from the sampling point X _s, the sampling point X _s with the sampling point X _s at a distance of 2d-1 _' Store in the result buffer 44. That is, the control unit 11, the pixel values of the reference position for the sampling points X _s' with the sampling point X _s to 2d-1 of the distance is assigned as the compensation pixel value.
In this method, a point folding hidden surface boundary positions by copying folded pixel values, compensates for the pixel value of the sampling point X _s.

In the example shown in FIG. 7, the value of the hidden surface buffer 42 whose sampling point X _s is “1” is “right”, and therefore the control unit 11 refers to the hidden surface buffer 42 to the right from the sampling point X _s. To specify that the boundary position of the hidden surface is “4”. Then, the control unit 11 calculates a distance L = 4-1 = 3 from the sampling point X _s “1” to the boundary position “4” of the hidden surface, and 2d−1 = 2 from the sampling point X _s “1”. The value “9” of the reference position buffer 41 for the sampling point X _s ′ “6” at the distance of × 3−1 = 5 is stored in the generation result buffer 44.
Further, since the value of the hidden surface buffer 42 with the sampling point X _s being “2” is also “right”, the control unit 11 refers to the hidden surface buffer 42 to the right from the sampling point X _s , thereby It is specified that the boundary position is “4”. Then, the control unit 11 calculates a distance L = 4-2 = 2 from the sampling point X _s “2” to the boundary position “4” of the hidden surface, and 2d−1 = 2 from the sampling point X _s “2”. The value “8” of the reference position buffer 41 for the sampling point X _s ′ “5” at the distance of x2−1 = 3 is stored in the generation result buffer 44.
Further, since the value of the hidden surface buffer 42 with the sampling point X _s “3” is also “right”, the control unit 11 refers to the hidden surface buffer 42 to the right from the sampling point X _s , thereby It is specified that the boundary position is “4”. Then, the control unit 11 calculates a distance L = 4-3 = 1 from the sampling point X _s “3” to the boundary position “4” of the hidden surface, and 2d−1 = 2 from the sampling point X _s “3”. The value “7” of the reference position buffer 41 for the sampling point X _s ′ “4” at the distance of x1-1 = 1 is stored in the generation result buffer 44.
In the case where it is determined as a hidden surface by the above processing, it is possible to assign a pixel value that is considered to be reasonable in many cases by returning the pixel value with the hidden surface boundary position as the return point.

Example shown in FIG. 7, the label value of the sampling point X _s' with the sampling point X _s at a distance of 2d-1 "L0" is was identical with the label value of the sampling point X _s "L0", It is not always the same.
Therefore, with reference to FIG. 8, and the label value of the sampling point X _s' with the sampling point X _s at a distance of 2d-1, the exception process when the label value of the sampling point X _s are different will be described.

In the example illustrated in FIG. 8, the label value of the sampling pixel position X _s ' “6” at a distance of 2d−1 = 5 from the sampling point X _s “1” is “L3”, and the sampling point X _s This is different from the label value “L0”. In such a case, the above-described method is irrational because the pixel values of other subjects are compensated as hidden surface compensation.
Therefore, the control unit 11 determines whether or not the label value of the pixel position to be copied by copying matches the label value of the subject that should appear at the reference position of the sampling point. More specifically, the control unit 11, the label values of pixels at the sampling point X _s at a distance of 2d-1 (the value of the label buffer 43), the subject of the label value should appear in the reference position of the sampling point X _s (label It is determined whether or not it matches the second label value of the buffer 43. Then, the control unit 11 executes the hidden surface compensation process by the above-described folded copy when they match, and executes the exception process described below when they do not match.

In exception processing, the control unit 11, the label value of the object to appear at the reference position of the sampling point X _s (2-th label value of the label buffer 43) farthest sampling point among the same single monolithic sampling point group the pixel values of X s _'see reference stored in the position buffer 41 position of' assigned as the pixel values of the sampling point X _s.
In the example shown in FIG. 8, the label value sampling point group of the same single monolithic with "L0" of the object to the sampling point X _s appears at the reference position "1", is "2" to "5" . Therefore, the control unit 11 uses the reference position “8” stored in the reference position buffer 41 of the farthest sampling point X _s ″ “5” in the sampling point groups “2” to “5” as the sampling point. The generation result buffer 44 of X _s “1” is stored.
Through the above exception processing, the pixel value of another subject is not compensated as the pixel value of the hidden surface, and in any case, it is possible to assign a reasonable pixel value.

Then, by assigning the generated results final pixel values of corresponding two-dimensional image 2 to the reference position stored in the buffer 44 as the pixel value of each sampling point X _s, hidden surface is compensated for by appropriate pixel value The parallax image 6 can be obtained.

Returning to the description of FIG.
Control unit 11, all the sampling points (where, Y coordinates _{Y s} only) process to determine whether finished for (S207). If the process has not ended (No in S207), the process is repeated from S202. If the process has been completed (Yes in S207), the process proceeds to S208.

  Next, the control part 11 confirms whether the process was complete | finished about all the Y coordinates (S208). If the process has not ended (No in S208), the process is repeated from S201. If the process has been completed (Yes in S208), the process proceeds to S209.

Next, the control unit 11 collects the pixel values calculated in S206 for each camera C and saves them in the storage unit 12 as the parallax image 6 (S209). When the camera number n is “8”, the control unit 11 collects the pixel values calculated in S206 for each of the cameras C _{1 to} C ₈ and stores them in the storage unit 12 as eight parallax images 6.

  As described above, the parallax image generation device 1 according to the embodiment of the present invention can generate the parallax image 6 in which the hidden surface is corrected without a sense of incongruity.

<Modification 1>
Modification 1 of the pixel value assignment process 24 (S206) will be described.
In the above method, the point folding hidden surface boundary positions by copying folded pixel values, compensates for the pixel value of the sampling point X _s. Since this method considers only the pixel values in the horizontal direction, the subject that should appear at the reference position (= subject object) has a large gradation change, and the subject that has hidden the reference position (= front side) When the contour of the subject on the side is a curve, a streak line along the contour of the subject on the near side may appear in the hidden surface area. In the first modification, this phenomenon can be avoided.

FIG. 9 is a diagram for explaining the first modification. In the example shown in FIG. 9, the subjects constituting the scene of the two-dimensional image 2 are the object 51 and the background 52. In this example, the camera position is to the right of the front. A hidden surface is generated between the object 51 and the background 52.
First, the control unit 11 includes a plurality of continuous Y coordinates Y _s , Y _s ′, and Y _s ′.
,..., A reference position specifying process 21 (S203), a hidden surface determining process 22 (S204), and a label value assigning process (S205) are executed.
Next, the control unit 11 extracts a region where a plurality of reference positions determined to be hidden surfaces are connected as a hidden surface region 53 (“black” in FIG. 9).
Next, the control unit 11 extracts a circumscribing straight line extending in the Y direction of the hidden surface area 43 as a folding line 54. In this example, since the hidden surface is the right hidden surface, a circumscribed straight line on the right side of the hidden surface region 53 is extracted. On the other hand, if the hidden surface is the left hidden surface, a circumscribed straight line on the left side of the hidden surface region 53 may be extracted.
Then, the control unit 11, based on the fold line 54, by copying folded pixel value to compensate for the pixel values of the sampling point X _s.

  According to the first modification, the gradation of the subject that should appear at the reference position (= subject subject) is severe, and the contour of the subject (= subject subject) that has hidden the reference position is curved. Even if it exists, a streak line does not appear in the horizontal direction in the area of the hidden surface.

<Modification 2>
A second modification of the pixel value assignment process 24 (S206) will be described. In the second modification, various methods that are not return copy will be described.

Control unit 11, in order to compensate for the pixel values of the sampling point X _s, and calculates or acquires one of the values shown below.
(1) Overall average value of subjects that should appear at the reference position (= subject subject) (2) Among subjects (= subject subjects) that should appear at the reference position, subjects (= Average value of pixel group adjacent to (front subject) (3) Representative value of subject (= subject subject) that should appear at reference position (represented value that is considered appropriate as compensation pixel value in advance) (It is stored in the storage unit 12 etc.)
And the control part 11 allocates the value of (1)-(3) calculated or acquired as a compensation pixel value.

<Modification 3>
In Modification 3, a method of processing the two-dimensional image 2 so as not to cause a hidden surface before executing the flowchart shown in FIG. 4 will be described. The method of the third modification is effective for a two-dimensional image 2 of a scene where there is almost no overlap between subjects except for the background.
The control unit 11 extracts a background region from the two-dimensional image 2 and generates a processed background region having a pixel value at the pixel position of the entire two-dimensional image 2 by a texture generation technique. Then, the control unit 11 stores the pixel value of the corresponding processed background region in association with each pixel included in the subject region excluding the background in the two-dimensional image 2 in the storage unit 12 or the like.
Then, the control unit 11, the hidden surface determination process 22 (S204), if a reference position is determined to be hidden face, the pixel values of the corresponding processed background area is assigned as the pixel values of the sampling point X _s.

  FIG. 10 is a diagram illustrating an example of the two-dimensional image 2. In the example illustrated in FIG. 10, the two-dimensional image 2 is a grayscale image, and is configured by a circular object (= light gray), a rectangular object (= dark gray), and a grid-like background (= white and black). Has been.

FIG. 11 is a diagram illustrating an example of the depth image 4 for the two-dimensional image 2 of FIG. In the example illustrated in FIG. 11, the depth image 4 is a grayscale image having a gradation of 0 to 255.
In the depth image 4 shown in FIG. 11, the depth value of the circular object in FIG. 10 is gradation “0”, the depth value of the grid-like background is gradation “255”, and the depth value of the rectangular object is gradation “128”. Is. That is, the circular object is in the foreground and the lattice-shaped background is in the back.
For easy understanding, the depth image 4 shown in FIG. 11 has a single depth value for each subject. For general subjects, the depth value is often different for each part.

FIG. 12 is a diagram illustrating an example of a label image 3 for the two-dimensional image 2 of FIG. In the example illustrated in FIG. 12, the label image 3 is a grayscale image having a gradation of 0 to 255.
In the label image 3 shown in FIG. 12, for example, the label value of the circular object in FIG. 10 is gradation “0”, the label value of the grid-like background is gradation “255”, and the label value of the rectangular object is gradation. It is “128”.
For ease of explanation, FIGS. 11 and 12 are the same images. As described above, in the case of a general subject, the depth value is often different for each part, and the depth image 4 and the label image 3 are different images.

  In this embodiment, the parallax image 6 is generated using the two-dimensional image 2 shown in FIG. 10, the depth image 4 shown in FIG. 11, and the label image 3 shown in FIG. 12 as input images.

FIG. 13 is a diagram illustrating an example of the parallax image 6 when the line of sight is from the left direction. In the example shown in FIG. 13, the camera position is the position of the camera C ₃ or C ₄ in FIG.
When generating the parallax image 6 with the line of sight from the left direction, a hidden surface is generated between the left outline of the circular object and the background, or between the left outline and the background of the rectangular object.
In the example shown in FIG. 13, the hidden surface is compensated with an appropriate pixel value and corrected without a sense of incongruity.

FIG. 14 is a diagram illustrating an example of the parallax image 6 when the line of sight is from the right direction. In the example shown in FIG. 14, the camera position is the position of the camera C ₇ and C ₈ in FIG. In the example shown in FIG. 13, since the camera position is the position of the camera C ₃ or C ₄ in FIG. 3, the hidden surface area shown in FIG. 14 is larger than the hidden surface area shown in FIG. It has become.
When generating the parallax image 6 with the line of sight from the right direction, a hidden surface is generated between the right outline of the circular object and the background, or between the right outline and the background of the rectangular object.
In the example illustrated in FIG. 14, the hidden surface between the right outline of the rectangular object and the background is compensated by an appropriate pixel value and corrected without a sense of incongruity.
On the other hand, the hidden surface between the right outline of the circular object and the background is not correctly compensated. This is because the gradation of the subject (= background) that should appear at the reference position is sharp, and the contour of the subject (= circular object) that has hidden the reference position is curved. A streak line along the contour has appeared. This phenomenon can be avoided by the first modification as described above.

  The preferred embodiments of the parallax image generating device and the like according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. Understood.

DESCRIPTION OF SYMBOLS 1 ......... Parallax image generation apparatus 2 ......... 2D image 3 ......... Label image 4 ......... Depth image 5 ......... Parameter 6 ......... Parallax image

Claims (7)

A parallax image generation device that generates a parallax image for realizing stereoscopic vision,
A desired two-dimensional image, a depth image that is an image obtained by replacing the depth value from the camera to the subject related to the two-dimensional image with a pixel value, and a label value that identifies a subject constituting the scene of the two-dimensional image An image input means for inputting a label image which is an image replaced with a value;
Parameter input means for inputting parameters consisting of the number of cameras, the camera interval, and the depth value from the camera to the screen surface;
The sampling point of the screen surface, both before and camera position is set based on Kipa parameters, it calculates the line of sight line passing through a passage section which is the line of sight straight line passing through the section in the depth values between the interval One of the adopted sections is a first depth value that is a depth value on the first direction side in the adopted section, and a second depth value that is a depth value on the second direction side different from the first direction. , A reference position specifying means for setting the position indicating the back as a reference position;
Hidden surface determination means for determining that the reference position is a hidden surface when the first label value and the second label value of the label image respectively corresponding to the positions of the first depth value and the second depth value are different. When,
When the hidden surface determination unit determines that the reference position is not a hidden surface, the pixel value of the reference position in the two-dimensional image is set as the pixel value of the sampling point, and the reference position is determined by the hidden surface determination unit. Is determined to be a hidden surface, the determination result by the hidden surface determination means is the first direction side that is the direction indicating the far side of the first depth value and the second depth value, or A compensation pixel value that is a pixel value that compensates for the pixel value of the sampling point by referring to the second direction side, specifying a hidden surface boundary position, and returning a pixel value with the hidden surface boundary position as a return position A pixel value assigning means for assigning
A parallax image generating apparatus comprising: The pixel value allocating means determines whether or not a label value of a pixel position to be copied by copying matches a label value of a subject to appear at the reference position;
If they match, assign the compensation pixel value by folding copy of the pixel value with the hidden surface boundary position as the folding position,
If they do not match, the pixel value of the reference position relating to the farthest point in the same sampling point group that is the same as the label value of the subject that should appear at the reference position is assigned as the compensation pixel value. The parallax image generation device according to claim 1. When the reference position is a hidden surface, and when the first depth value indicates a value on the back side with respect to the second depth value, the hidden surface determination unit determines that the line of sight line is the first line. It is determined that a first direction hidden surface viewed from the first direction is generated by passing between the front subject and the back subject from the direction, or the second depth value is the first depth value. When the value on the far side is shown, the second direction hidden surface viewed from the second direction is generated by passing the line of sight between the subject in the foreground and the subject in the back from the second direction. The parallax image generation device according to claim 1, wherein the parallax image generation device is determined.   In the case of the first direction hidden surface, the pixel value assigning unit specifies a hidden surface boundary position by referring to the determination result by the hidden surface determination unit in the first direction, or In the case of a two-way hidden surface, the hidden surface boundary position is specified by referring to the determination result by the hidden surface determination means in the second direction, and the compensation pixel value is determined based on the hidden surface boundary position. The parallax image generation device according to claim 3, wherein the parallax image generation device is specified. A parallax image generation method for generating a parallax image for realizing stereoscopic vision,
A desired two-dimensional image, a depth image that is an image obtained by replacing the depth value from the camera to the subject related to the two-dimensional image with a pixel value, and a label value that identifies a subject constituting the scene of the two-dimensional image An image input step for inputting a label image which is an image replaced with a value;
A parameter input step for inputting parameters consisting of the number of cameras, the camera interval, and the depth value from the camera to the screen surface;
The sampling point of the screen surface, both before and camera position is set based on Kipa parameters, it calculates the line of sight line passing through a passage section which is the line of sight straight line passing through the section in the depth values between the interval One of the adopted sections is a first depth value that is a depth value on the first direction side in the adopted section, and a second depth value that is a depth value on the second direction side different from the first direction. , A reference position specifying step with the position indicating the back as a reference position;
A hidden surface determination step of determining that the reference position is a hidden surface when the first label value and the second label value of the label image corresponding to the positions of the first depth value and the second depth value are different. When,
When it is determined by the hidden surface determination step that the reference position is not a hidden surface, the pixel value of the reference position in the two-dimensional image is set as the pixel value of the sampling point, and the reference position is determined by the hidden surface determination step. Is determined to be a hidden surface, the determination result of the hidden surface determination step is a first direction side that is a direction indicating a deeper side of the first depth value and the second depth value, or A compensation pixel value that is a pixel value that compensates for the pixel value of the sampling point by referring to the second direction side, specifying a hidden surface boundary position, and returning a pixel value with the hidden surface boundary position as a return position A pixel value assigning step for assigning
A parallax image generating method characterized by comprising: Computer
A desired two-dimensional image, a depth image that is an image obtained by replacing the depth value from the camera to the subject related to the two-dimensional image with a pixel value, and a label value that identifies a subject constituting the scene of the two-dimensional image An image input means for inputting a label image which is an image replaced with a value;
Parameter input means for inputting parameters consisting of the number of cameras, the camera interval, and the depth value from the camera to the screen surface;
The sampling point of the screen surface, both before and camera position is set based on Kipa parameters, it calculates the line of sight line passing through a passage section which is the line of sight straight line passing through the section in the depth values between the interval One of the adopted sections is a first depth value that is a depth value on the first direction side in the adopted section, and a second depth value that is a depth value on the second direction side different from the first direction. , A reference position specifying means for setting the position indicating the back as a reference position;
Hidden surface determination means for determining that the reference position is a hidden surface when the first label value and the second label value of the label image respectively corresponding to the positions of the first depth value and the second depth value are different. When,
When the hidden surface determination unit determines that the reference position is not a hidden surface, the pixel value of the reference position in the two-dimensional image is set as the pixel value of the sampling point, and the reference position is determined by the hidden surface determination unit. Is determined to be a hidden surface, the determination result by the hidden surface determination means is the first direction side that is the direction indicating the far side of the first depth value and the second depth value, or A compensation pixel value that is a pixel value that compensates for the pixel value of the sampling point by referring to the second direction side, specifying a hidden surface boundary position, and returning a pixel value with the hidden surface boundary position as a return position A pixel value assigning means for assigning
Program to make it function.   A computer-readable storage medium storing the program according to claim 6.