JP4155401B2 - 3D image generation method, 3D image generation apparatus, 3D image generation program, and recording medium - Google Patents

Description

  The present invention relates to a three-dimensional image generation method, a three-dimensional image generation apparatus, a three-dimensional image generation program, and a recording medium, and more particularly, to a three-dimensional image generation method used in stereoscopic image communication in fields such as the video industry and communication industry. And a technique effective when applied to a three-dimensional image generation apparatus.

  In recent years, mobile devices such as mobile phones and PDAs (Personal Digital Assistants) have been displaying stereoscopic images (three-dimensional images), and as a means for inputting the stereoscopic images, small size, low cost, and low consumption. There is a need for a three-dimensional shape measuring apparatus that is not limited in electrical power, measurement environment, or measurement range.

  However, an input method and an input device for a three-dimensional image of an object are currently developing technologies, and a three-dimensional shape input device that can be incorporated into a portable device has not yet been put into practical use.

  When displaying a stereoscopic image with the portable device or the like, it is often sufficient to obtain a stereoscopic effect of the displayed object, and it is not always necessary to obtain an accurate three-dimensional shape. In other words, even if the three-dimensional shape of the photographic object is generated in a pseudo manner by any method, it is sufficiently practical if the shape is visually natural and a stereoscopic effect is felt when displayed on the stereoscopic display device. .

  Various three-dimensional image display apparatuses for generating and displaying a stereoscopic image have been proposed, but the expression capability in the depth direction is not necessarily the same as the expression capability in the horizontal or vertical direction. Here, the expression capability in the depth direction will be described by taking a DFD display device (for example, see Non-Patent Document 1) as an example of a three-dimensional image display device as an example.

  The DFD method is a method of displaying a three-dimensional image by overlapping and displaying two display screens, and the displayable range in the depth direction is limited between the front display screen and the rear display screen. For this reason, when there is a high demand for thickness reduction as in the portable device and the distance between the two display screens cannot be sufficiently secured, the displayable range in the depth direction is remarkably limited.

  If an accurate three-dimensional shape is to be displayed with such a display device, only a part of the depth range can be stereoscopically displayed. In addition, it is conceivable to display a compressed three-dimensional shape in the depth direction in order to display all, but if it is simply compressed, there is a problem that an image with poor stereoscopic effect is generated. Therefore, for example, if a three-dimensional shape is converted into a relief-like three-dimensional expression that can be seen on a pattern on the surface of a coin, a three-dimensional display can be achieved even with a three-dimensional image display device having a small displayable range in the depth direction. .

  However, in this method, two steps of three-dimensional measurement and relief-like three-dimensional shape conversion are required for the process of generating a three-dimensional image, and the calculation cost increases. For this reason, this problem is particularly serious in a portable device embedded CPU having a small processing capability.

  As an existing three-dimensional shape input technique for solving these problems, a technique for generating a three-dimensional shape from one image has been proposed (see, for example, Patent Document 1). However, the technique described in Patent Document 1 or the like generates a three-dimensional shape based on the premise that the image center is a distant view and the image periphery is a close view. Therefore, unless it was originally an image of such a distance distribution, a meaningful three-dimensional shape could not be generated, and there was a drawback that the application range was limited.

  As a three-dimensional shape input technique for solving this problem, a technique for artificially generating a three-dimensional shape from a binary distance image of a distant view and a close view has also been proposed (see, for example, Japanese Patent Application No. 2002-275471). . However, the technology for generating the three-dimensional shape in a pseudo manner has a problem that only a shape having almost no depth unevenness in the depth direction can be generated because the three-dimensional shape is generated in a kamaboko shape using the foreground region as one region.

S. Suyama, M. Date and H. Takeda, "Three-Dimensional Display System with Dual-Frequency Liquid-Crystal Varifocal Lens", Jpn. J. Appl. Phys., 39 (Pt. 1, 2A), 480 (2000 ) JP-A-10-307926

  As described in the background section, the conventional three-dimensional image generation apparatus has a problem that it is difficult to reduce the size, reduce the power consumption, reduce the cost, etc., and it is difficult to incorporate it into a small and thin portable device. It was.

  In addition, for example, a conventional three-dimensional image generation apparatus to which the DFD method is applied has a low representation capability in the depth direction when the distance between two display screens is small. Therefore, there was a problem that it was difficult to apply to the portable device.

  In addition, the conventional three-dimensional image generation apparatus has a problem that a high calculation processing capability is required, and there is a problem that it is difficult to apply and embed to a portable device that has been reduced in size and thickness.

  In addition, there are problems that an accurate stereoscopic image cannot be generated when the distance distribution is different from the assumed distance distribution, and that the ability to express in the depth direction is low.

  It is an object of the present invention to provide a highly feasible three-dimensional shape input means by acquiring only auxiliary information necessary for generating a three-dimensional shape and generating a three-dimensional shape of a photographic object based on the auxiliary information. And If only the auxiliary information is acquired, it is easy to realize small-sized, low-cost and highly reliable 3D image generation compared with the case of obtaining an accurate 3D shape. A cost-effective three-dimensional image generation apparatus can be realized.

  Another object of the present invention is to solve the problem of calculation cost by directly generating a relief-like solid shape, rather than a two-step process of converting a relief-like solid shape after obtaining an accurate three-dimensional shape. . As a result, a relief-like 3D shape that can be used even in a 3D image display device with a small displayable range in the depth direction can be efficiently generated at a low calculation cost, and even with a CPU with low processing performance incorporated in a portable device, An image capable of obtaining a sufficient stereoscopic effect can be easily generated.

  Another object of the present invention is to provide a high-quality color image simultaneously with the generation of three-dimensional shape information by simultaneously generating an image that cannot be shaded by illumination during photographing.

  In the three-dimensional image generation method and the three-dimensional image generation apparatus of the present invention, a three-dimensional shape (three-dimensional image) is generated based on auxiliary information. Therefore, a meaningful three-dimensional shape can be generated without depending on the distribution of the distant view area and the foreground area in the image. Further, according to the present invention, objects in the context can be distinguished from each other even in the foreground area, so that a three-dimensional shape with more details can be generated.

  The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

The outline of the invention disclosed in the present application will be described as follows.
(1) A first image taken by illuminating from a position away from the photographing means by a predetermined distance in the horizontal direction, and a second image taken by illuminating from a position away from the photographing means by a predetermined distance in the direction opposite to the horizontal direction. A second step of generating a difference image between the first image and the second image, and a right end of an area having a pixel value equal to or greater than a certain positive threshold k1 on the difference image. A third step of extracting a pixel and a pixel at the left end of a region having a negative threshold value −k1 or less for each horizontal line, a region on the right side of the right end pixel, and a left side of the left end pixel A fourth step of generating a distance image by performing an operation for giving a distance to the region according to a predetermined distance function for each horizontal line, and a fifth step of smoothing the distance image in the vertical direction 3. Dimensional image It is a generation method.

  (2) If the absolute value of each pixel value of the difference image is larger than a predetermined threshold value k2 between the second step and the third step in the means of (1), a difference A step A in which the absolute value of the quantity is replaced with a threshold value k2, a step B1 in which a parameter of an approximate expression determined in advance to approximate the distribution shape of the pixel value is approximated for each horizontal line of the difference image, and each pixel Step B2 for subtracting the approximate value at the pixel position from the pixel value, and once in the order of Step A, Step B1, Step B2, or Step B1, Step B2, and Step A. Or a step of generating a new difference image by performing a plurality of times while decreasing k2 for each iteration.

( 3 ) Between the third step and the fourth step in the means (1) or ( 2 ), for each of the rightmost pixel and the leftmost pixel in the image obtained in the third step. When pixels are continuously present in the vertical direction and there is a local discontinuity, the discontinuous portion is interpolated from the upper and lower pixels to form the right end pixel or the left end pixel. It is a three-dimensional image generation method which has a step which cancels.

( 4 ) After the first step in the means of (1) to ( 3 ), by comparing pixels at the same position in the first image and the second image and selecting a pixel having a large luminance value A three-dimensional image generation method including a step of generating a color image of a photographing object.

( 5 ) A first image that is photographed by illuminating from a position that is a predetermined distance in the horizontal direction from the photographing means, and a second image that is photographed by illuminating from a position that is a predetermined distance away from the photographing means in the direction opposite to the horizontal direction. An image acquisition means for acquiring a three-dimensional shape using the first image and the second image acquired by the image acquisition means, the three-dimensional shape generation means, Difference image generating means for generating a difference image between the first image and the second image, a rightmost pixel of a region having a pixel value equal to or larger than a certain positive threshold k1 on the difference image, and a certain negative threshold −k1 The pixel extraction means for extracting the leftmost pixel of the region having the following pixel values for each horizontal line, and the distance according to a predetermined distance function in the region on the right side of the rightmost pixel and the region on the left side of the leftmost pixel Give The operation that is a three-dimensional image generating apparatus comprising: a distance image generating means for generating distance image by performing for each horizontal line, and a smoothing means for smoothing said range image in the vertical direction.

( 6 ) The three-dimensional shape generation means in the means of ( 5 ), in addition to the means, a difference image that generates a new difference image by converting a distribution shape of pixel values for each horizontal line of the difference image. A three-dimensional image generation apparatus including a conversion unit.

( 7 ) In the means of ( 5 ) or ( 6 ), for each of the rightmost pixel and the leftmost pixel in the image, the pixels are continuously present in the vertical direction and are locally discontinuous. In this case, the three-dimensional image generation apparatus includes a pixel interpolation unit that interpolates the interrupted portion from the upper and lower pixels.

( 8 ) In the means of ( 5 ) to ( 7 ), the pixels at the same position in the first image and the second image are compared, and a color image of the photographing object is generated by selecting a pixel having a large luminance value. A three-dimensional image generation apparatus including color image generation means for performing the above-described operation.

( 9 ) A three-dimensional image generation program for causing a computer to execute each step of the three-dimensional image generation method according to any one of the means ( 1 ) to ( 4 ).

(1 0 ) A recording medium on which the three-dimensional image generation program of ( 9 ) is recorded in a computer-readable state.

  The three-dimensional image generation method according to the present invention can provide relief-like solid shape input that is simple, low-cost, and highly reliable by using the means (1). At this time, a relief three-dimensional shape is directly generated instead of a two-step process of obtaining an accurate three-dimensional shape and converting it to a relief three-dimensional shape as in the prior art, so that the calculation cost can be reduced. Therefore, it is particularly excellent as a three-dimensional image generation method incorporated in a portable device having a low calculation processing capability.

  Further, by using the means (2), it is possible to input a stable relief-like three-dimensional shape even from an image taken under illumination with uneven illumination, and to generate a highly reliable three-dimensional image. it can.

In addition, by using the means ( 3 ), even when the contour line of the object (subject) in the image cannot be clearly acquired due to factors such as noise, a stable and high-quality relief-like solid shape can be generated. This makes it possible to generate a highly reliable three-dimensional image.

Further, by using the means ( 4 ), it is possible to input a high-quality color image that is not shaded by illumination when a color image is also acquired.

Further, the three-dimensional image generation apparatus of the means ( 5 ) to ( 8 ) is an apparatus having means for executing each step of the means (1) to ( 4 ). Therefore, it is possible to provide a three-dimensional image generation apparatus that is small in size, low in cost, and excellent in reliability.

Further, as in the above-mentioned means ( 9 ), if a program for causing the computer to execute each step of the means (1) to ( 4 ) is not limited to a dedicated three-dimensional image generation apparatus, an existing mobile phone Even portable devices such as PDAs can easily generate highly reliable three-dimensional images.

Further, if the three-dimensional image generation program of the means ( 9 ) is recorded on a recording medium such as a CD-ROM, as in the means (1 0 ), the program can be easily provided. it can. The three-dimensional image generation program is not limited to the recording medium, and can be provided by recording the program on a terminal on a network and using the Internet or the like.

  The three-dimensional image generation method of the present invention includes a first image that is photographed by being illuminated from a position that is a predetermined distance in the horizontal direction from the photographing means, and a light that is illuminated from a position that is a predetermined distance from the photographing means in the direction opposite to the horizontal direction. A first step of acquiring a second image photographed in a second step, a second step of generating a difference image between the first image and the second image, and a pixel on the difference image that is equal to or greater than a certain positive threshold k1 A third step of extracting, for each horizontal line, a pixel at the right end of the region having a value and a pixel at the left end of the region having a pixel value equal to or less than a certain negative threshold −k1, a region on the right side of the right end pixel, And a fourth step of generating a distance image by performing an operation for giving a distance to a region on the left side of the leftmost pixel according to a predetermined distance function for each horizontal line, and a step of smoothing the distance image in the vertical direction. 5 steps And a flop.

  FIG. 1 is a schematic diagram for explaining the principle of the three-dimensional image generation method of the present invention.

  In the three-dimensional image generation method of the present invention, first, in the first step, two images of the first image 101 and the second image 102 are acquired as shown in FIG. At this time, the first image 101 is an image photographed by illuminating a photographing object (subject) with a lighting device arranged on the right side of the photographing means (camera). For this reason, in the first image 101, as shown in FIG. The second image 102 is an image obtained by illuminating a photographic object with a lighting fixture arranged on the left side of the photographing means (camera). Therefore, as shown in FIG. 1, the second image 102 has an illumination shadow 104 on the right side of the photographed object 103.

In the next second step, as shown in FIG. 1, a difference image 105 between the first image 101 and the second image 102 is generated. Here, the value (pixel value) of each pixel of the difference image 105 is obtained by subtracting the pixel value of the first image 101 from the pixel value of the second image 102. At this time, on the difference image 105, an area 106 corresponding to an area where both the first image 101 and the second image 102 are illuminated is such that the pixels on the images 101 and 102 have substantially the same luminance value. is there. Therefore, the difference pixel value is almost 0 (zero). On the other hand, on the difference image 105, an area 10 07 corresponding to the shadow 104 of the first image 101 is an area having a positive pixel value. An area 108 corresponding to the shadow 104 of the second image 102 is an area having a negative pixel value. That is, as shown in FIG. 1, when the pixel value of each pixel of the horizontal line 109 of the difference image 105 is scanned, a pixel value graph 110 is obtained.

  In the next third step, as shown in FIG. 1, on the difference image 105, the rightmost pixel of a region having a pixel value greater than or equal to a certain positive threshold value k1 and a pixel equal to or less than a certain negative threshold value −k1. The pixel at the left end of the area having a value is extracted for each horizontal line. At this time, for example, when the pixel value of the horizontal line 109 shown in the difference image 105 is extracted, the pixel value graph 110 is shown. Therefore, in the horizontal line 109 shown in the pixel value graph 110, a region having a pixel value greater than or equal to a certain positive threshold k1, that is, the rightmost pixel 111 of the region 107 corresponding to the shadow 104 of the first image 101, A region having a pixel value equal to or less than a certain negative threshold value −k1, that is, the leftmost pixel 112 of the region 108 corresponding to the shadow 104 of the second image 102 is extracted. When this process is performed for all the horizontal lines of the difference image 105 and these are combined, an image 113 as shown in FIG. 1 is obtained.

  In the next fourth step, in the image 113 obtained in the third step, a distance defined in advance to the right side of the extracted right end pixel 111 and the left side of the extracted left end pixel 112 is determined. The operation for giving the distance according to the function is performed for each horizontal line. By this processing, the distance image 114 as shown in FIG. 1 is obtained. At this time, for example, a range from the right end pixel 111 to the right side until the left end pixel 112 or another right end pixel appears is raised according to a predefined distance function, and then the left end pixel 112 The relief solid three-dimensional shape 115 of the photographic object 103 is generated by superimposing and enlarging the range until the rightmost pixel 110 or other leftmost pixel appears on the left side from the left side in accordance with a predefined distance function. The detailed procedure for generating the distance image 114 will be described in a later embodiment.

  Note that there are various variations in the distance function definition formula and distance function application rules in the fourth step, and the generation method of the distance image 114 is not limited to the above example, and the gist of the present invention. Any processing that does not deviate from the above is included.

  In the next fifth step, as shown in FIG. 1, the distance image 114 obtained in the fourth step is smoothed in the vertical direction to generate a three-dimensional image 116. In the three-dimensional image generation method of the present invention, the distance image 114 obtained in the fourth step discontinuously changes between different horizontal lines even if the distance change of the photographed object is continuous. There is. This is a factor that visually perceives large image quality degradation. Therefore, the effect of the fifth step is to eliminate the discontinuous change by smoothing in the vertical direction and suppress visual image quality deterioration. On the other hand, the three-dimensional image 116 generated in the fifth step is generated so that a portion where the distance change in the vertical direction of the photographing object is discontinuous also changes continuously. The fifth step has an effect of visually converting to a higher-quality distance image when viewed comprehensively because the continuous distance change is inherently less discontinuous.

  There are various kinds of variations in the smoothing method in the fifth step, and the present invention is not limited to smoothing using a specific function.

  2 and 3 are schematic diagrams for explaining the principle of the three-dimensional image generation method of the present invention, and are diagrams for explaining an example of a method for improving the reliability of the generated three-dimensional image. is there. In FIG. 2, reference numeral 2 denotes a photographing object (subject), 3 denotes photographing means (camera), 4A denotes first illumination means, 4B denotes second illumination means, and 5 denotes a back surface.

  When photographing the first image 101 and the second image 102 acquired in the first step, for example, as shown in FIG. 2A, the first illumination is provided on the right side of the photographing means 3 for photographing the photographing object 2. The means 4A is arranged, and the second illumination means 4B is arranged on the left side of the photographing means 2. When the first image 101 is photographed, the photographing object 2 is illuminated by the first illumination means 4A, and when the second image 102 is photographed, the photographing object 2 is photographed by the second illumination means 4B. Illuminate.

  At this time, when the first illuminating means 4A and the second illuminating means 4B are, for example, as shown in FIG. 2B, the illuminance at the center of the irradiation direction is large, and the illuminance decreases as the irradiation direction moves away from the center, The illumination in the part away from the center of the irradiation direction becomes dark. For this reason, the first image 101 and the second image 102 obtained by photographing using such illumination means 4A and 4B with the arrangement shown in FIG. 2A are shadows of the contour of the photographing object. In addition to this, an area darkened due to low illumination occurs. At this time, the darkened areas are generated in opposite directions. For example, as shown in FIG. 2C, a darkened area 117 is generated on the left side of the screen as shown in FIG. Produces a darkened area 117 on the right side of the screen. Therefore, when the difference image 105 is obtained from the first image 101 and the second image 102, as shown in FIG. 2 (c), the difference image 105 is the original three-dimensional image generation method of the present invention. In addition to the areas 107 and 108 corresponding to the shadows of the outline of the necessary photographic object, an area 118 corresponding to the area darkened due to low illumination is also extracted. As described above, when a region other than the region corresponding to the shadow of the contour portion of the photographed object necessary for generating the three-dimensional image is extracted from the difference image 105, the object is also included in the region in a later step. As a result, the reliability of the generated three-dimensional image is reduced. Therefore, the following processing is performed so that such a problem does not occur.

  First, as shown in FIG. 3, in the first step, the first image 101 and the second image 102 in which the illuminance distribution is photographed under the illumination means 4A and 4B as shown in FIG. get. At this time, the pixel value scanned along the horizontal line 119 a in which the first image 101 is present becomes a pixel value graph 120. Further, the pixel value scanned along the horizontal line 109 b of the second image 102 that is the same height as the horizontal line 109 a of the first image 101 becomes a pixel value graph 121. At this time, the pixel values of the horizontal lines having the same height as the horizontal lines 109a and 109b in the difference image 105 obtained from the first image 101 and the second image 102 are, for example, the pixel value graph shown in FIG. 122, indicating a change in which the pixel value of the region 123 excluding the regions 107 and 108 corresponding to the shadow of the contour portion of the photographic object increases from the left end of the screen toward the right end.

  Thus, if the pixels in the region 123 other than the region corresponding to the shadow of the contour portion of the photographic object have a large pixel value, the reliability of the generated three-dimensional image is lowered. Therefore, before performing the third step, the pixel values in the region 123 other than the region corresponding to the shadow of the contour of the photographic object in the pixel value graph 122 shown in FIG. It is preferable to remove the unnecessary components on the difference image 122 by performing a process of making the value almost 0 as in the value graph 110.

  In the process of removing unnecessary components on the difference image 122, first, as Step A, when the absolute value of each pixel value of the difference image 122 is larger than a predetermined threshold value k2, the absolute value of the difference amount is calculated. The pixel value graph 122 is replaced like a pixel value graph 124 by replacing with the threshold value k2. This means that clipping processing is performed within a range of ± k2.

  The effect of step A will be described. The subsequent step B is a process of bringing the area where the difference amount should originally be 0 close to 0, but the area exceeding the threshold is an area that is likely to be a shadow whose difference amount is not 0. If the process of step B including the region is performed, the process cannot be performed with sufficient accuracy. The step A has an effect of reducing this adverse effect by suppressing the size of the pixel value in the region exceeding the threshold value.

  Next, as step B, the process according to the two steps of step B1 and step B2 described below is performed on each horizontal line of the difference image.

  In step B1, a parameter of an approximate expression designated in advance is determined so as to approximate the distribution shape of pixel values of each horizontal line of the difference image 105. At this time, for example, when the approximate expression is a straight line of y = ax + b, the approximate expression 125 is obtained by obtaining the parameters a and b by the least square method from the arrangement (distribution shape) of the pixel values of the pixel value graph 124. Generate. This approximation has various variations other than the linear approximation, and the approximation used in the three-dimensional image generation method of the present invention is not limited to the linear approximation or the moving linear approximation shown in the following embodiments.

  In Step B2, the approximate value at the pixel position is subtracted from the pixel value for each pixel on the horizontal line. As a result, as shown in FIG. 3, a pixel value graph in which the pixel value of the region 123 other than the regions 107 and 108 corresponding to the shadow of the outline of the photographic object is close to 0, as is unnecessary for generating the three-dimensional image. 126 is obtained. By the action of the step B (B1, B2), it becomes possible to remove the influence of the illuminance unevenness of the illumination means 4A, 4B.

  The series of processing of Step A, Step B1, and Step B2 described here is set as one step, and the processing is performed on all the horizontal lines of the difference image 105, thereby darkening due to low illuminance. The difference image 105 generated from the first image 101 and the second image 102 having the region 117 can be replaced with a difference image from which the influence of illuminance unevenness is removed. Thereafter, if the third and subsequent steps are executed, a highly reliable three-dimensional image can be generated.

  In FIG. 3, the pixel value graph 127 is shown to be obtained in one process. However, the illumination means 4A and 4B have large illuminance unevenness. For example, the slope of the region 123 in the pixel value graph 122 is If the pixel value of the areas 107 and 108 corresponding to the shadow of the contour portion of the photographic object in the pixel value graph 122 is not sufficient in the case where the amount of illumination is small or the pixel value graph 122 is small, a large threshold k2 is given in advance. A series of processes of Step A, Step B1, and Step B2 are repeated, and a difference image that looks like the pixel value graph 126 is obtained by decreasing the value of the threshold value k2 each time it is repeated. In the case of repetition, the execution order of the steps within one step may be the order of step A, step B1, step B2, or the order of step B1, step B2, step A.

  With such a method, even when an image taken under illumination with uneven illumination is acquired, unnecessary components of the difference image 105 can be removed. Therefore, a reliable distance image (three-dimensional image) can be generated in a later step.

  Further, the process of removing unnecessary components from the difference image 105 is not limited to the process by the step A, the step B1, and the step B2, and may be another process.

FIG. 4 is a schematic diagram for explaining the principle of the three-dimensional image generation method of the present invention, and is a diagram for explaining an example of a method for removing the influence of noise.

The three-dimensional image generation method of the present invention, the third image 113 generated in step, as shown in FIG. 4 (a), pixel and right of the contour of a region 111 corresponding to the left side of the contour of the shooting object Pixels in the region 112 corresponding to are extracted. At this time, the left side of the pixels extracted in the area 111 corresponding to the contour of the shooting object is mainly, as shown in FIG. 4 (b), corresponds to a shadow 104a occurring on the left side of the imaging object 103 in the image 139 This is the pixel at the right end of the area to be processed. Also, the right side of the pixels extracted in the area 112 corresponding to the contour of the shooting object is mainly equivalent to the shadow 104b caused the right side of the imaging object 103 in the image 139 shown in FIG. 4 (b) region Is the leftmost pixel. Therefore, if the original pixel of the respective regions 111 and 112, but it should continuously lined up in the vertical direction, the influence of noise, for example, as shown in FIG. 4 (a), a local break 140 may occur and may be partially missing.

Therefore, in order to eliminate the partial omission, for each of the rightmost pixel and the leftmost pixel in the image 113 obtained in the third step, a collection 111 of pixels that are generally connected in the vertical direction; When a local discontinuity 140 exists at 112, the discontinuous portion is interpolated by the upper and lower pixels to perform processing to obtain a right end pixel or a left end pixel. As a result, for example, as shown in FIG. 4C, an image 141 from which the influence of noise is removed is generated. Therefore, even when noise is present, a high-quality three-dimensional image can be generated.

  The details of the method for interpolating the interrupted portion in the image obtained in the third step will be described in a later embodiment. However, the present invention does not limit the method for interpolating the interrupted portion, and the interrupted portion is not limited. Any method may be used as long as it is a method of interpolating the portion from the upper and lower pixels.

FIG. 5 is a schematic diagram for explaining the principle of the three-dimensional image generation method of the present invention, and is a diagram for explaining the color image generation method.

  In the three-dimensional image generation method of the present invention, in the first step, the first image 101 obtained by illuminating from a position that is a certain distance in the horizontal direction from the photographing unit is obtained, and the horizontal direction is obtained from the photographing unit. A second image 102 obtained by illuminating from a position separated by a certain distance in the opposite direction is acquired.

In general, when generating a three-dimensional image, in the first step, often taking a color information of the photographing object at the same time, when actually displaying three-dimensional images, as shown in FIG. 5 (a), A color image 142 using the color information of the photographed object is generated, and is displayed in a three-dimensional form. However, if the color information of the first image 101 or the second image 102 is used as it is, the right side or the left side of the photographic object 103 becomes an image with a sharp border 104. Furthermore, just in the method of imaging using the first image 101 obtains an average of the second image 102, or two lighting simultaneously, as shown in FIG. 5 (b), shaded portions 104c remained thin color It becomes an image 143, and the shadow cannot be completely removed. In addition, the method of making a shadow inconspicuous by applying a plurality of indirect illuminations from a wide range of positions as in general photography cannot achieve downsizing and portability of illumination means.

Therefore, the color image 141 of the shooting object is generated by comparing the pixels at the same position in the first image 101 and the second image 102 acquired in the first step and selecting the pixel having the larger luminance value. To do. At this time, for example, for a pixel in a shadowed position in either image, a pixel having a large luminance value is always selected, so that the generated image 142 is an image without a shadow. That is, for example, as shown in FIG. 5 (a), was compared with the luminance value of the pixel area 144a of the first image 101, the luminance value of the pixels in the region 144b of the second image 102 corresponding to the area 144a In this case, since the luminance value of the pixel on the second image 102 that is illuminated is larger than the luminance value of the pixel of the first image 101 that is in shadow, the pixel of the region 145 of the color image 142 As the luminance value, the luminance value of the pixel on the second image 102 is adopted. Therefore, a color image without a shadow can be easily generated.

  This process is efficient when combined with the three-dimensional image generation method of the present invention because the same image can be used, but it is also useful when used alone for normal image shooting.

FIG. 6 is a schematic diagram showing a schematic configuration of the three-dimensional image generation apparatus of the present invention. In FIG. 6 , reference numeral 2 denotes a photographic object, 3 denotes a photographic means (camera), 4A denotes a first illuminating means (first light) located at a fixed distance in the right horizontal direction of the photographic means, and 4B denotes a first illumination. Second illumination means (second light) located at a fixed distance in the opposite direction (left horizontal direction) to the means, 6 is a three-dimensional image generation device, 601 is an image acquisition means, 602 is a three-dimensional shape generation means, Reference numeral 603 denotes camera / light control means, and 7 denotes image display means.

For example, as shown in FIG. 6 , the 3D image generation apparatus 6 that implements the 3D image generation method of the present invention includes an image acquisition unit 601 that acquires an image captured by an imaging unit (camera) 3, and the image acquisition unit. Using the image acquired by the means 601, a three-dimensional shape generating means for generating a three-dimensional shape of the photographed object 2 shown in the image, and a first illumination arranged on the left and right of the photographing means 3 and the photographing means 3 And a camera / light control means 603 for controlling the means 4A and the second illumination means 4B.

  At this time, the illuminating means includes a first illuminating means 4A at a position separated by a certain distance on the right side of the photographing means and a second illuminating means at a position separated by a certain distance on the opposite side (left side) from the first illuminating means 4B. 4B is arranged and is controlled by the camera / light control means 603 so that one of them is lit.

  The photographing means 3 is a normal camera, and the camera / light control means 603 lights the photographing object 2 in a state where either the first illumination means 4A or the second illumination means 4B is turned on. The photographed object is photographed, and the photographed image is transferred to the image acquisition means 601.

Further, the three-dimensional shape forming unit 602, FIG. 1 to in the manner described with reference to FIG. 5, we obtain a three-dimensional shape of the imaging object 2 is reflected in the image acquired by the image acquisition unit 601, A means for generating a three-dimensional image. The three-dimensional image generated by the three-dimensional shape generation unit 602 is displayed on the image display unit 7 such as a liquid crystal display, for example.

  The three-dimensional image generation apparatus 6 can generate a three-dimensional image from two images captured using the three units of the imaging unit 3, the first illumination unit 4A, and the second illumination unit 4B. Therefore, there is an advantage that it can be configured to be extremely low cost and small and can be easily mounted on a portable terminal. Moreover, since illumination does not require a special light source such as a laser, it has an advantage of high safety.

  Embodiments relating to a 3D image generation method and a 3D image generation apparatus according to the present invention will be described below.

7 to 1 3 is a schematic diagram for explaining the three-dimensional image generation method of Example 1 according to the present invention, FIG. 7 is a flow diagram showing an overall processing procedure, Figure 8 is step 803 of FIG. 7 flow diagram showing a specific example of the process of the flow diagram FIG. 9 to FIG. 1 1 is showing a specific example of the processing of step 805 of FIG. 7, diagram for explaining the operation of Figure 1 2 and 1 3 step 805 It is.

As shown in FIG. 7 , the three-dimensional image generation method according to the first embodiment includes a step 801 for acquiring a first image P1 and a second image P2, a first image P1 (101), and a second image P2 ( 102) for generating the difference image P3 (105) in step 102, the rightmost pixel of the region having a pixel value greater than or equal to a certain positive threshold value k1, and a pixel value equal to or less than a certain negative threshold value −k1. Step 803 for extracting the pixel on the left side of the region having the pixel value for each horizontal line, Step 804 for preparing the image P4 with the pixel value initialized to 0, and the pixel on the right side of the pixel on the right side extracted in Step 803 A step 805 of generating a distance image by performing a process for giving a distance to the region and a region on the left side of the leftmost pixel according to a predefined distance function for each horizontal line; and the distance image And a step 806 of generating a 3-dimensional image is smoothed in a vertical direction. Here, the step 801 corresponds to the first step, the step 802 corresponds to the second step, the step 803 corresponds to the third step, and the step 805 corresponds to the fourth step. Step 806 corresponds to the fifth step.

  Hereinafter, the processing of each step will be described in detail. The size of each image is X rows and Y columns. In the following description, the pixel value of the coordinates (x, y) on each image Pr (r = 1, 2, 3,..., N) is described as Pr (x, y), and there is an image Pr. The horizontal line image is described as Lr, and the pixel value of the x coordinate of the line image Lr is described as Lr (x). From this definition, the pixel value Lr (x) on the line image in the y-th column is equal to Pr (x, y). In the first embodiment, it is assumed that a monochrome image is a target and the pixel value is a scalar value.

  First, in step 801, a first image P1 obtained by illuminating from a position that is a certain distance in the horizontal direction from the photographing means is obtained. At this time, the first image P1 acquired from the photographing unit may be a monochrome image or an RGB color image. However, when the color image is acquired, the first image P1 is converted into a monochrome image according to Equation 1 below.

  Here, P (x, y) Red, P (x, y) Green, and P (x, y) Blue are color information recorded in each pixel of the color image.

  In addition, when the first image P1 is acquired, the second image P2 captured by illuminating from a position away from the imaging unit by a certain distance in the direction opposite to the horizontal direction is acquired. Similarly to the first image P1, the second image P2 to be acquired may be a monochrome image or a color image. However, when the color image is acquired, it is converted into a monochrome image according to the equation 1.

  Next, in step 802, a difference image P3 is generated from the first image P1 and the second image P2 according to the following mathematical formula 2.

Next, in step 803, a process as shown in FIG. 8 is performed on each horizontal line of the difference image P3. Here, the pixel value of the x coordinate of the line image L of the y-th line of the image P is described as L (x). At this time, L (x) = P (x, y).

In the process of step 803, first, as shown in FIG. 8 , the parameter (y) relating to the horizontal line L3 of the difference image P3 is initialized (step 803a).

  Next, the value of each pixel of the horizontal line L3 is converted according to the following mathematical formula 3 to generate a line image L31 (step 803b).

  Here, k1 is a positive threshold value given in advance.

  Next, the value of each pixel of the line image L31 generated in the step 803b is converted according to the following mathematical formula 4 to generate a line image L32 (step 803c).

  As a result of the operation of Equation 4, the rightmost pixel of the pixel column having a pixel value greater than or equal to the positive threshold value k1 has a value of 1, and the leftmost pixel of the pixel column having a pixel value of a certain negative threshold value −k1 or less has a value of −1. Extracted as

  When the line image L32 for the horizontal line L3 is generated, the parameter (y) of the horizontal line L3 is updated (step 803d), and it is determined whether or not the process of generating the line image L32 for all horizontal lines has been performed (step). 803e). If there is an unprocessed horizontal line L3, the process returns to step 803b and the process is repeated. If all the horizontal lines L3 have been processed, each horizontal line L3 in the difference image P3 is replaced with the line image L32 (step 803f), and the process in step 803 is terminated. As a result of the processing in step 803, an image such as the image 113 shown in FIG. 1 is obtained.

  Next, in step 804, a distance image P4 with all pixel values set to 0 is prepared. Here, the pixel value 0 represents the farthest distance, and the pixel value increases as the distance decreases.

Next, in Step 805, the difference image P3 (hereinafter, the image P32 to) substituted on the line image L32 to the line image L32 of each horizontal line of the processing shown in FIGS. 9 to 1 1 Do.

In step 805, first, as shown in FIG. 9 , the parameter (y) relating to the line image L32 of the image P32 is initialized (step 805a).

  Next, pixel values are examined in order from the leftmost pixel of the line image L32, and when a pixel with L32 (i) = 1 appears, its coordinate i is recorded (step 805b).

  Subsequently, the pixel value of the pixel on the right side of the coordinate i in the line image L32 is checked, and when a pixel with L32 (j) ≠ 0 appears, the coordinate j is recorded (step 805c).

  Next, the distance value z (x) given by the following equation 5 is added to the pixel value of the pixel from the coordinate i to the coordinate j of the line image L4 on the horizontal line having the same height as the line image L32 of the distance image P4. ) Are added (step 805d).

  Here, k (0 <k <1) and z0 are parameters given in advance.

An example pixel values of the line image L4 range image P4 generated by the Equation 5 shown in FIG. 1 2 (a). Figure 1 2 (a) is a line image L4 range image P4 which is generated in accordance with Equation 5, z0 is sized to pop out to the front, k represents the position of the center jump out. Note that various formulas other than the formula 5 can be applied to the distance function, and the present invention does not limit the distance function to the formula 5.

  Next, it is determined whether the right end pixel of the line image L32 has been examined (step 805e). If the right end pixel of the line image L32 has not been examined, it is determined whether L32 (j) = 1 ( Step 805f). If L32 (j) = 1, i = j is set (step 805g), and the process returns to step 805d. On the other hand, if L32 (j) ≠ 1 in the determination in step 805f, the process returns to step 805c.

Further, in step 805e, if it is determined that the examined and the right edge of the line image L32, the process proceeds to step 805h shown in FIG. 1 0.

  In step 805h, pixel values are examined in order from the rightmost pixel of the line image L32, and when a pixel with L32 (i) = − 1 appears, its coordinate i is recorded. Thereafter, the pixel value of the pixel on the left side of the coordinate i of the line image L32 is checked, and when a pixel with L32 (j) ≠ 0 appears, the coordinate j is recorded (step 805i).

Next, the distance value z (x) given by the equation 5 is added to the pixel value of the pixel from the coordinate i to the coordinate j of the line image L4 on the horizontal line having the same height as the line image L32 in the distance image P4. ) Are added (step 805j). The distance value z is added to each pixel of the line image L4 range image P4 generated (x) is, for example, as shown in FIG. 1 2 (b).

  Next, it is determined whether the leftmost pixel of the line image L32 has been checked (step 805k). If the rightmost pixel of the line image L32 has not been checked, it is determined whether L32 (j) = 1 ( Step 805l). If L32 (j) = 1, i = j is set (step 805m) and the process returns to step 805j. On the other hand, if L32 (j) ≠ 1 in the determination in step 805m, the process returns to step 805i.

Further, in step 805K, if it is determined that the examined and the right edge of the line image L32, the process proceeds to step 805n shown in FIG 1.

  In step 805n, the line image L32 of the image P32 is updated, and then it is determined whether all the line images L32 have been processed (step 805o). Here, if there is an unprocessed line image L32, the process returns to step 805b to execute the processing of each step.

  If it is determined in step 805o that all the line images L32 have been processed, the line image of each horizontal line in the distance image 4 is replaced with the line image L4 generated from the line image L32, for example, FIG. Is generated (step 805p), and the step 805 is terminated.

  In step 805, the distance image P4 is obtained by performing the processing from step 805b to step 805m on all horizontal lines, and the processing in the fourth step is realized. However, the processing by each step described in the first embodiment is only a specific example for realizing the fourth step. Therefore, even if an algorithm other than the algorithm (processing procedure) described here is used, an area on the right side of the rightmost pixel and an area on the left side of the leftmost pixel as described in the first embodiment are preliminarily provided. Any algorithm that provides an effect of giving a distance according to a defined distance function may be used.

For range image P4 generated by the step 805 will be described in more detail by way of example image 146 illustrated in FIG 3. When attention is paid to a certain horizontal line 147 in the image 146, the pixel value of each pixel of the line image L 32 in the horizontal line 147 becomes a pixel value graph 148 by performing the processing in step 803.

Distance Next, the process of step 805d from the step 805b, shown in line image L4 of the distance image P4 of all pixel values of the zero state, the distance value graph 149 line image L4 in FIG 3 by a white area The value 150 is added. In addition, the distance value 150 indicated by the hatched area is added to the pixel value graph 149 to which the distance value 150 has been added by the processing from Step 805h to Step 805j. As a result, a distance distribution 152 as shown by a bold line is obtained. It can be seen that the distance distribution 152 is a distance image in the horizontal line 147 when the image 146 is expressed in a relief solid shape.

The processing at step 805, After generating the range image P4, as shown in FIG. 7 or 1 1, smoothing the range image P4 in the vertical direction (step 806). In step 806, a distance image P5 is prepared, and the pixel value of each pixel of the distance image P5 is determined according to the following formula 6. Then, the distance image P5 composed of these pixel values is output as a three-dimensional image and displayed on the image display means 7 such as a liquid crystal display.

  In Equation 6, P4 (x, -2) = P4 (x, 0), P4 (x, -1) = P4 (x, 0), P4 (x, Y) = P4 (x, Y− 1), P4 (x, Y + 1) = P4 (x, Y−1). Moreover, although the said Numerical formula 6 is a smoothing filter by the simple average of five pixels, this is an example and this invention does not limit the kind and smoothing range of a smoothing filter to the said Numerical formula 6. .

  As described above, according to the three-dimensional image generation method of the first embodiment, the first image P1 illuminated from the right side of the imaging unit and the second image P2 illuminated from the left side of the imaging unit. A three-dimensional image can be generated from two images. Therefore, it is possible to easily generate a three-dimensional image with little visual image degradation. Further, at this time, a relief three-dimensional shape (distance image P4) can be directly generated, instead of a two-step process in which an accurate three-dimensional shape is obtained and then converted into a relief three-dimensional shape. The calculation cost can be reduced. Therefore, it is particularly excellent as a three-dimensional image generation method incorporated in a portable device having a low calculation processing capability.

Figure 1 4 is a flowchart for explaining the three-dimensional image generation method according to a second embodiment of the present invention.

  In the three-dimensional image generation method described in the first embodiment, the arrangement of the illumination units 4A and 4B and the illuminance characteristics as shown in FIGS. When a region 117 darkened due to low illuminance as shown in FIG. 2C occurs in (P1) and the second image 102 (P2), the difference image 105 (P3) has a three-dimensional image. A component 118 is generated that is unnecessary for production. In the second embodiment, an example of a specific processing method in the case where unnecessary components on the difference image P3 are removed by the procedure shown in FIG. 3 will be described.

  The process described in the second embodiment is a process performed between step 802 for generating the difference image P3 and the next step 803 in the three-dimensional image generation method described in the first embodiment. Therefore, in the following description, the description of the portion that performs the same process as each step described in the first embodiment will be omitted, and only the step related to the process of removing the unnecessary portion on the difference image P3 will be described.

  First, as described in the first embodiment, after obtaining the first image P1 and the second image P2, a difference image P3 between the first image P1 and the second image P2 is generated (steps 801 and 802). ). At this time, it is assumed that the difference image P3 is, for example, an image 105 as shown in FIG.

Then, the processing shown in FIG. 1 4, removes unnecessary components on the difference image P3. At this time, first, a predetermined threshold value k2 is substituted for K (step 807a).

  Next, parameters relating to the horizontal line of the difference image P3 are initialized (step 807b).

  Next, the processing from the next step 807c to step 807e is performed on the horizontal line image L3 of the difference image P3.

  In step 807c, for each pixel L3 (x) of the line image L3, an approximate expression y = a (x) · x + b for linearly approximating the distribution of pixel values from the section x−M / 2 to the section x + M / 2. The parameters a (x) and b (x) of (x) are determined using Equation 7 and Equation 8 below.

  Here, the addition range in the formulas 7 and 8 is defined as follows.

(1) If xM / 2 <0, Σ addition range: i = 0 to x + M / 2: M ′ = x + M / 2
(2) If xM / 2 ≧ 0 and x + M / 2 <X, the addition range of Σ: i = xM / 2 to x + M / 2: M ′ = M
(3) If x + M / 2 ≧ X, addition range of Σ: from i = xM / 2 to X: M ′ = XxM / 2

  In either case, xi = i and yi = L3 (i).

  By performing the processing in step 807c, the processing in step B1 is realized. In the second embodiment, a moving average approximation expression using a linear expression is used. However, an approximation expression that approximates a distribution shape of pixel values may be used. The method for obtaining the approximate expression is not limited.

  Next, in step 807d, the value of each pixel of the line image L3 is converted in accordance with the following formula 9 to generate a line image L31.

  By performing the processing in step 807d, the processing in step B2 is realized.

  Next, in step 807e, the value of each pixel of the line image L31 is converted according to the following formula 10, and a line image L32 is generated.

  By performing the process of step 807e, the process of step A is realized.

  As described above, the processing shown in FIG. 3 is realized by performing the processing from step 807c to step 807e on all horizontal lines. At this time, as shown in FIG. 3, in the line image L32, the values of pixels other than the area unnecessary for generating the three-dimensional image, that is, the area corresponding to the shadow area of the outline of the photographing object are almost zero. become.

  Next, the line image L3 is updated (step 807f), and it is determined whether or not processing has been performed for all horizontal line line images (step 807g). At this time, if there is an unprocessed line image L3, the process returns to step 807c and the process is repeated. If processing has been performed for all the line images L3, the line image L3 of the original difference image P3 is replaced with the line image L32 to generate a new difference image P3 (step 807h).

  Thereafter, it is determined whether or not the iteration is completed (step 807i). If it is determined that the iteration is terminated, the process proceeds to step 803. If it is determined that the process has not been completed, the value of K is updated using the following formula 11 (step 807j), and then the process returns to step 807b.

  Here, the operation symbol% in the equation 11 represents an operation in which, when K is divided by 2, the decimal part is rounded down to extract only the integer part.

  The steps 807i and 807j may be processes that update K to a value smaller than the current value, and are not limited to the equation 11.

  In the determination in step 807i, for example, it is determined that the iteration is completed when any of the following conditions or a combination of a plurality of conditions is satisfied.

(Condition 1) A certain number of iterations is exceeded.
(Condition 2) The value of K falls below a certain value.
(Condition 3) In step 807e, the number of pixels satisfying L31 (x) ≠ L32 (x) is greater than or equal to a certain value.
(Condition 4) In the method of counting pixels where L31 (x) ≠ L32 (x) in step 807e and the number of connected pixel columns is 1, the number of pixels is greater than a certain value.

  When the processing from step 807b to step 807h is repeated, each time the number of iterations increases, the pixel value of the difference image gradually approaches 0, but the criterion K for determining whether or not it is 0 also becomes smaller. The number gradually increases. Then, as the value of K approaches the noise level of the image, the number of unequal pixels starts to increase rapidly. Therefore, the conditions 3 and 4 for the end of the repetition substantially represent that the end is made when the value of K approaches the noise level of the image. Therefore, by using the condition 3 or condition 4 for the end of repetition, there is an advantage that the processing can be automatically performed without knowing the noise level of the image in advance.

  If it is determined in step 807i that the iteration has ended, the new difference image P3 generated in the immediately preceding step is used to perform step 803 described in the first embodiment, that is, the processing after the third step. .

  As described above, according to the three-dimensional image generation method of the second embodiment, the first image P1 and the second image P2 acquired in the first step are darkened due to the illuminance characteristics of the illumination unit. Even in the case where there is, the component of the region corresponding to the darkened region can be removed from the difference image P3. Therefore, a stable three-dimensional image can be generated even from an image taken under illumination with uneven illumination.

15 and 16 are schematic diagrams for explaining the three-dimensional image generation method according to a third embodiment of the present invention, FIG. 15 is a flow diagram of a process of interpolating a local break, 16 an interpolation method It is a figure for demonstrating.

In the three-dimensional image generation method described in the first embodiment and the second embodiment, the right end pixel and the left end of the region corresponding to the contour of the photographic object on the image generated in the third step (step 803). It is assumed that the pixels are extracted continuously. However, in actual image generation processing, first, image P1 and the second image P2 obtained, the influence of noise generated in the difference image P3, as shown in FIG. 4 (a), the third step In the image 113 generated in step 1, a local discontinuity 140 may occur in a portion that is originally continuous. Therefore, in the third embodiment, the process of interpolating a local discontinuity 140 such as shown in FIG. 4 (a) will be described.

The process for interpolating the local break 140 is a process performed between the third step (step 803) and the fourth step (step 805). Therefore, in the following description, description of each step described in the first embodiment and the second embodiment is omitted, and only processing for interpolating the local break 140 is described.

  The image generated in the third step is described as a distance image P33. At this time, the size of the distance image P33 is assumed to be X pixels × Y pixels.

The three-dimensional image generation method of the third embodiment, after generating the image P33 in the procedure as described in Example 1 or Example 2, as shown in FIG. 1 5, the X pixel × Y pixels An image P34 and variables x and y are prepared (step 809a).

  Next, in step 809b, the pixel values of all the pixels adjacent to the four sides of the distance image P33 are copied to the same position of the image P34, and y = 1 is set.

  Next, in step 809c, x = 1 is set.

The processing described in the third embodiment cannot be applied to an area having a width of one pixel around the image. This is because the processing described in the third embodiment is performed using the pixel of interest and its neighboring pixels, and a pixel whose coordinate value of the variable x or y is 0 has no part of the neighboring pixels. So it can not be processed. Therefore, in the processing of the third embodiment, the initial values of the variables x and y are set to 1.

Then, in step 809D, each pixel P34 (x, y) of the image P34 value of is determined based on the condition set forth in Equation 1 below 2.

However, in the equation 1 2 of each of the conditional expressions, v is assumed to take the values of either all 1 or -1.

Geometric meaning the conditional expression shown in each of Equation 1 2 is as follows. First, for example, as shown in FIG. 16 (a), the first conditional expression is the value of the pixel P33 (x, y + 1) below the center pixel P33 (x, y) of the distance image P33 and the upper left. If both the values of the pixel P33 (x-1, y-1) are v, this indicates that the value of the pixel P34 (x, y) of the image P34 is v. Similarly, the second condition, as shown in FIG. 1 6 (b), the center pixel of the distance image P33 P33 (x, y) pixel under P33 (x, y + 1) pixel of the upper-value P33 If the values of (x, y−1) are both v, it means that the value of the pixel P34 (x, y) of the image P34 is v. Hereinafter, the third to fifth conditional expressions mean the same operation, and when the five conditional expressions are not satisfied, the sixth conditional expression P34 (x, y) = P33 (x, y) is set.

Using Equation 1 2, pixel P34 (x, y) of the image P34 be determined pixel values, on the image P34 is lacking of one pixel present in the segment of consecutive on the distance image P33 Is repaired.

In the conditional expression mentioned Equation 1 2, as shown in FIG. 1 6 (c), the value of the central pixel P33 of the range image P33 (x, y) pixel on the P33 (x, y-1) When the value of the lower right pixel P33 (x + 1, y + 1) is v, the center pixel P34 (x, y) is not restored. However, this is as shown in FIG. 1 6 (d), around the neighboring pixel pixel P33 (x, y) when a is always processed because a positional relationship shown in FIG. 1 6 (a) The In other words, the sequence as shown in FIG. 1 6 (c), the not put in Equation 1 2 To save duplication of processing. Also, in the conditional expression listed in Equation 1 2, as shown in FIG. 1 6 (e), the central pixel P33 (x, y) right pixel of P33 (x + 1, y) values and the left pixel P33 (x −1, y) is not processed even when the value of v is v. However, in the fourth step (step 805), the distance in the horizontal direction is calculated by interpolation processing. Since there is not much necessity to make it continuous, it is not necessary to carry out. However, if you do not focus on computational cost, it may be added to the conditional expression to be processed in case of FIG. 1 6 (c) and FIG. 1 6 (e) in Equation 1 2.

Further, the step 809d, the formula in place of the conditional expression shown in Equation 1 2, advance to create a lookup table in advance, by subtracting the reference table to generate a reference address from the distribution of the peripheral pixels it is also possible to perform 1 2 equivalent process. In this step, if the discontinuous portion on the distance image P33 is interpolated from the upper and lower pixels to be the right end pixel or the left end pixel, the expression is given if it is an expression that eliminates the local discontinuity. It is not limited to 1 or 2 .

  Next, the variable x is updated to x = x + 1 (step 809e), and it is determined whether x <X−1 (step 809f). If x <X-1, the process returns to step 809d.

  If x = X-1, the variable y is updated to y = y + 1 (step 809g), and it is determined whether y <Y-1 (step 809h). If y <Y-1, the process returns to step 809c and the process is repeated.

  If it is determined in step 809h that y = Y−1, the distance image P33 is replaced with an image P34 including the pixel P34 (x, y) whose pixel value has been determined in step 809d (step 809i), and processing is performed. finish. Thereafter, the fourth step (step 805) and the fifth step (step 806) are executed.

As described above, according to the three-dimensional image generation method of the third embodiment, on the third image generated in step, pixels representing the contour of the shooting object is caused in the portion which is continuous, Local breaks can be interpolated. Therefore, it is possible to prevent a reduction in visual accuracy of the generated three-dimensional image.

FIG. 17 is a flowchart for explaining the three-dimensional image generation method according to the fourth embodiment of the present invention.

In the above Examples 1 to Example 3, as the first image P1 and the second image P2, has been described a method for generating a 3-dimensional image by using a monochrome image, In general, when generating a three-dimensional image In many cases, when the first image P1 and the second image P2 are acquired, the color information of the photographing object is often photographed at the same time, and when the three-dimensional image is actually generated and displayed, the color of the photographing object is often obtained. A color image using information is three-dimensionally displayed. However, in the conventional color image generation method, the shadow of the contour portion of the photographed object remains, which causes deterioration in visual accuracy. In the fourth embodiment, a method for generating a color image that prevents deterioration in visual accuracy will be described.

The color image generation process described in the fourth embodiment is a process performed after step 801 for acquiring the first image P1 and the second image P2 in the three-dimensional image generation method described in the first embodiment. is there. Therefore, in the following description, each step described in the above Example 1 to Example 3 will be omitted of description, only the steps relating to processing for generating a color image.

In the fourth embodiment, the first color image C1 and the second color image are used to distinguish the image acquired in the first step to be processed from the first image P1 and the second image P2, respectively. Let C2. An image generated by the method of the fourth embodiment is a generated color image C3. The sizes of the first color image C1, the second color image C2, and the generated color image C3 are each assumed to be X pixels × Y pixels.

  When the first color image C1 and the second color image C2 are acquired, first, x = 0 and y = 0 are set in step 810a.

Next, at step 810b, in accordance with the following Equation 1 3, and calculates the pixel value of each pixel C3 (x, y) on the generated color image C3.

Here, the function MAX of the right side in Equation 1 3 is intended to mean a function that selects the larger value of the argument.

  Next, the variable x is updated to x = x + 1 (step 810c), and it is determined whether x <X (step 810d). If x <X, the process returns to step 810b and the process is repeated.

  If x = X, then the variable y is updated to y = y + 1 (step 810e), and it is determined whether y <Y (step 810f). Here, if y <Y, x = 0 (step 810g) and then the process returns to step 810b to repeat the process.

  If it is determined in step 810f that y = Y, an image C3 composed of C3 (x, y) is generated, and the process ends.

The above process, as shown in FIG. 4 (c), color picture 141 (C3) removed shadow caused by the illumination is generated.

Incidentally, the color image C3 is, when generating a three-dimensional image described in the first embodiment to the third embodiment is an image that does not directly necessary. Therefore, the processing as shown in FIG. 17 may be performed at any stage after the first step is completed. At this time, if the processing of the fourth embodiment is performed in parallel with the processing after the second step, an increase in processing time can be prevented.

As described above, according to the three-dimensional image generation method of the fourth embodiment, when a color image of the photographic object is also acquired at the same time, a high-quality color image without shadows due to illumination can be generated. .

The generation processing procedure of the color image described in the fourth embodiment, although the combination with the three-dimensional image generation method described in the above Example 1 to Example 3 is the most effective, not limited to this It is also useful when used alone for normal image photography.

FIG. 18 is a schematic diagram illustrating a schematic configuration of the three-dimensional image generation apparatus according to the fifth embodiment of the present invention.

In FIG. 18 , 2 is a photographic object, 3 is a photographic camera, 4A is first illumination means (right illumination), 4B is second illumination means (left illumination), 6 is a three-dimensional image generation device, 6A is an image input memory, 6B is a CPU, 6C is a three-dimensional shape recording memory, and 6D is a color image recording memory.

As the three-dimensional image generation apparatus 6 of the fifth embodiment, for example, an apparatus such as a computer can be used. As shown in FIG. 18 , an image input that acquires and stores an image captured by the imaging camera. A memory 6A, a CPU 6B that performs processing for generating a three-dimensional image using an image stored in the image input memory 6A, and a three-dimensional shape (distance image) and a color image obtained by the processing by the CPU 6B are recorded. A three-dimensional shape recording memory 6C and a color image recording memory 6D to be stored. At this time, the image input memory 6A, the three-dimensional shape recording memory 6C, and the color image recording memory 6D are realized by using physically divided semiconductor memories such as DRAM and EEPROM, for example. It doesn't matter.

Although not shown, the three-dimensional image generating device has the possible program to the steps described in Examples 1 to Example 4 run on the CPU6B is recorded memory The CPU 6B generates the distance image and the color image according to the program.

  Further, the CPU 6B also performs control such as timing of photographing with the photographing camera 3 and switching of the illumination means 4A and 4B.

When an image is generated using the three-dimensional image generation apparatus according to the fifth embodiment, the first illumination unit 4A is turned on under the control of the CPU 6B, the shooting object 3 is shot with the shooting camera 3, and the image is stored in the image input memory 6A. Enter. This image is defined as a first image P1 (101).

  Next, the CPU 6B turns off the first illuminating means 4A, turns on the second illuminating means 4B, shoots the photographic object 2 with the photographic camera 3, and inputs an image to the image input memory 6A. This image is defined as a second image P2 (102). The above operation corresponds to the first step (step 801) described in the first embodiment.

Then, CPU 6b generates a distance image (3-dimensional image) using the method described in the first image P1 from the second image P2, to said third embodiment from the first embodiment, the three-dimensional shape recording Record in the memory 6C.

When acquiring a color image of the photographic object at the same time, a color image C3 is generated from the first image P1 and the second image P2 using the method described in the fourth embodiment, and is recorded in the color image recording memory 6D. .

  The distance image (three-dimensional image) recorded in the distance image recording memory 6C and the color image recorded in the color image recording memory 6D are read by the operation of the user of the three-dimensional image generation device 6, etc. It is displayed on display means (not shown) such as a liquid crystal display.

In this case, generation processing of the distance image performed by the CPU 6b (3-dimensional image), as described in the Example 1 to Example 3, since generated using two images, calculation cost And a highly reliable three-dimensional image can be generated. Therefore, not only a dedicated computer (computer) with high processing capability but also a general-purpose computer, PDA, mobile phone, etc. can be used to generate and display a three-dimensional image with sufficiently high visual reliability. Can do.

  In the three-dimensional image generation method of the present invention, as described in the first embodiment, the first image illuminated from the right side of the photographing means (camera) and the second image illuminated from the left side of the photographing means are two. Since a three-dimensional image is generated from a single image, it is easy to incorporate a photographing means (camera) and an illuminating means for photographing an acquired image into the three-dimensional image generation apparatus. Therefore, for example, if the photographing means and the illumination means are incorporated into a small portable device such as a mobile phone or a PDA, a small, low-cost and highly reliable three-dimensional image generation apparatus is provided. Can do.

Moreover, the process as described in Examples 1 to Example 4 by programmed, not be a dedicated device, as long as practicable device the program, excellent reliability 3 A dimensional image can be easily generated. At this time, if the program is recorded on a recording medium such as a CD-ROM or recorded on a terminal on a network, it is used not only by a specific user device but also by many user devices. It becomes possible.

  The present invention has been specifically described above based on the above-described embodiments. However, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention. is there.

It is a schematic diagram for demonstrating the principle of the three-dimensional image generation method of this invention. It is a schematic diagram for demonstrating the principle of the three-dimensional image generation method of this invention, and is a figure for demonstrating an example of the method for improving the reliability of the three-dimensional image to produce | generate. It is a schematic diagram for demonstrating the principle of the three-dimensional image generation method of this invention, and is a figure for demonstrating an example of the method for improving the reliability of the three-dimensional image to produce | generate. It is a schematic diagram for demonstrating the principle of the three-dimensional image generation method of this invention, and is a figure for demonstrating an example of the method of removing the influence of noise. It is a schematic diagram for demonstrating the principle of the three-dimensional image generation method of this invention, and is a figure for demonstrating the generation method of a color image. It is a schematic diagram which shows schematic structure of the three-dimensional image generation apparatus of this invention. It is a schematic diagram for demonstrating the three-dimensional image generation method of Example 1 by this invention, and is a flowchart which shows the whole process sequence. Is a schematic diagram for explaining the three-dimensional image generation method of Example 1 according to the present invention, it is a flow diagram showing a specific example of the processing of step 803 in FIG. Is a schematic diagram for explaining the three-dimensional image generation method of Example 1 according to the present invention, it is a flow diagram showing a specific example of the processing of step 805 in FIG. Is a schematic diagram for explaining the three-dimensional image generation method of Example 1 according to the present invention, it is a flow diagram showing a specific example of the processing of step 805 in FIG. Is a schematic diagram for explaining the three-dimensional image generation method of Example 1 according to the present invention, it is a flow diagram showing a specific example of the processing of step 805 in FIG. It is a schematic diagram for demonstrating the three-dimensional image generation method of Example 1 by this invention, and is a figure for demonstrating the effect | action of step 805. FIG. FIG. 6 is a schematic diagram for explaining the three-dimensional image generation method according to the first embodiment of the present invention, and is a diagram for explaining the operation of Step 805. It is a flowchart for demonstrating the three-dimensional image generation method of Example 2 by this invention. It is a schematic diagram for demonstrating the three-dimensional image generation method of Example 3 by this invention, and is a flowchart of the process which interpolates a local interruption. It is a schematic diagram for demonstrating the three-dimensional image generation method of Example 3 by this invention, and is a figure for demonstrating the interpolation method. It is a flowchart for demonstrating the three-dimensional image generation method of Example 4 by this invention. It is a schematic diagram which shows schematic structure of the three-dimensional image generation apparatus of Example 5 by this invention.

Explanation of symbols

101, P1 ... first Ima <br/> 102, P2 ... second Ima <br/> 105, P3 ... differential picture image 2 ... shooting object body 3 ... photographing means (camera)
4A ... first illumination hand stage 4B ... second illumination hand stage 6 ... 3-dimensional image generating equipment 601 ... image acquisition hand stage 602 ... three-dimensional shape generation hand stage 603 ... camera / write control hand stage 6A ... image input memory 6B ... CP U
6C ... 3-dimensional shape recording memory 6D ... color image recording memory 7 ... image display hand stage

Claims (10)

A first image that is illuminated and photographed from a position that is a predetermined distance in the horizontal direction from the photographing means and a second image that is illuminated and photographed from a position that is a predetermined distance in the opposite direction to the horizontal direction from the photographing means are acquired. A first step;
A second step of generating a difference image between the first image and the second image;
A pixel on the right side of a region having a pixel value greater than or equal to a certain positive threshold value k1 and a pixel on the left end of a region having a pixel value equal to or less than a certain negative threshold value −k1 are extracted for each horizontal line. 3 steps,
A fourth step of generating a distance image by performing an operation for giving a distance according to a predetermined distance function to a region on the right side of the right end pixel and a region on the left side of the left end pixel for each horizontal line;
And a fifth step of smoothing the distance image in the vertical direction. If the absolute value of each pixel value of the difference image is greater than a predetermined threshold value k2 between the second step and the third step, the absolute value of the difference amount is replaced with the threshold value k2. A and
A step B1 of determining a parameter of an approximate expression determined in advance so as to approximate a distribution shape of pixel values for each horizontal line of the difference image;
Subtracting the approximate value at the pixel position from the pixel value for each pixel, and step B2.
A new value is obtained by repeating the steps A, B1, B2, or the steps B1, B2, and A for a number of times while decreasing k2 once or repeatedly. The three-dimensional image generation method according to claim 1, further comprising a step of generating a difference image. Between the third step and the fourth step, for each of the rightmost pixel and the leftmost pixel in the image obtained in the third step, the pixels are continuously continuously in the vertical direction. 2. The method according to claim 1, further comprising the step of eliminating the discontinuity by interpolating the discontinuous portion from the upper and lower pixels into a right end pixel or a left end pixel when there is a discontinuity locally. Or the three-dimensional image generation method of Claim 2 . After the first step, there is a step of comparing the pixels at the same position in the first image and the second image and generating a color image of the photographic object by selecting a pixel having a large luminance value. The three-dimensional image generation method according to any one of claims 1 to 3 . A first image that is illuminated and photographed from a position that is a predetermined distance in the horizontal direction from the photographing means and a second image that is illuminated and photographed from a position that is a predetermined distance in the opposite direction to the horizontal direction from the photographing means are acquired. Image acquisition means;
Three-dimensional shape generation means for generating a three-dimensional shape using the first image and the second image acquired by the image acquisition means,
The three-dimensional shape generation means includes a difference image generation means for generating a difference image between the first image and the second image, and a rightmost pixel of a region having a pixel value equal to or greater than a certain positive threshold k1 on the difference image. And a pixel extraction means for extracting a leftmost pixel of a region having a pixel value equal to or less than a certain negative threshold value −k1 for each horizontal line, a region on the right side of the rightmost pixel, and a region on the left side of the leftmost pixel A distance image generating means for generating a distance image by performing an operation for giving a distance in accordance with a predetermined distance function for each horizontal line, and a smoothing means for smoothing the distance image in the vertical direction. A three-dimensional image generating apparatus. The three-dimensional shape generation means includes difference image conversion means for converting the distribution shape of pixel values for each horizontal line of the difference image to generate a new difference image in addition to the means described above. The three-dimensional image generation apparatus according to claim 5 . In addition to the above-described means, the three-dimensional shape generation means has pixels that are continuously present in the vertical direction in each of the rightmost pixel and the leftmost pixel in the image and are locally discontinuous. If the three-dimensional image generating apparatus according to claim 5 or claim 6, characterized in that it comprises a pixel interpolating means for interruption portion interpolated from upper and lower pixels. In addition to the above means, a color image generating means for comparing the pixels at the same position in the first image and the second image and generating a color image of the photographing object by selecting a pixel having a large luminance value is provided. The three-dimensional image generation apparatus according to any one of claims 5 to 7 . A three-dimensional image generation program for causing a computer to execute each step of the three-dimensional image generation method according to any one of claims 1 to 4 . A recording medium on which the three-dimensional image generation program according to claim 9 is recorded in a computer-readable state.

Images