JP2000292131A - Three-dimensional information input camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To project pattern light without using any flash light, reduce the influence of the move of an image due to the move of a subject, and improve the accuracy in three-dimensional information input by equipping this camera with a projection means that uses a light-emitting diode as a light source for projecting the pattern light. SOLUTION: A light-emitting diode is used as a projection means for projecting pattern light to an image pickup region. The switching of the presence or absence of a pattern by the emission and non-emission of the light-emitting diode can be made faster than the mechanical switching of a mask pattern. Three-dimensional information can be obtained from a subject image with the pattern of the light-emitting diode and the subject image of only steady light without using any flash light source. In a light-emitting diode array 560, a mask 550 for forming a fringe pattern is arranged in front of the light-emitting diode array 560 so that the period coincides with that of a row of emission elements of R, G, and B. Light from each emission element is transmitted through the mask 550 and is projected from a pattern projection window 553.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional information input camera.

[0002]

2. Description of the Related Art Conventionally, as three-dimensional information input, a method of obtaining three-dimensional information from two images passing through a plurality of photographing lenses, or a principle of triangulation by projecting light onto an object as shown in FIG. There is known a method of detecting a distance distribution by using the method.

Further, as disclosed in, for example, JP-A-6-249624, there is a method of projecting a fringe pattern, inputting the pattern with another camera, and detecting a distance distribution by so-called triangulation. Also, when the grid pattern is projected on the object and observed from different angular directions,
A method has also been proposed in which the projected grid pattern obtains deformation data according to the undulation of the object, thereby obtaining the undulation of the object. (Journal of the Japan Society for Precision Engineering, 55, 10, 85 (198
9)). In addition, as shown in FIG. 24, a method in which a gray code pattern is projected instead of the grid pattern projection, and the optical distribution is measured by a CCD camera.

[0004] In order to obtain three-dimensional information by these methods, it is necessary to take a plurality of images or the processing of the image information is troublesome. Therefore, although there is no problem as a measuring instrument, it is considered that it is not suitable for use in a camera.

As a method for obtaining three-dimensional information with high accuracy by short-time photographing and post-calculation, there are the following proposals.

For example, FIG. 25 (Source: “optical three-dimensional measurement”)
As shown in Toru Yoshizawa, New Technology Communications, p. 89, Fig. 5.2.12 (a)), a stripe pattern is projected, and the projected stripe pattern is projected from the subject at an angle determined by design. And the distance distribution of the subject is detected from the deformed image of the stripe due to the unevenness of the subject. That is, a phase shift from the original fringe with respect to the phase of the image measured at each image point is calculated. This phase shift also includes information on the height of the subject. Therefore, the distance distribution of the subject is obtained based on the phase information and the information obtained by triangulation. However, detection requires high accuracy. Since there is a limit in the density distribution and luminous intensity of the stripe pattern, a method has been adopted in which the distance distribution of the subject is obtained from a plurality of captured images in which the position of the stripe pattern is slightly shifted. For example, four phase shift patterns of 0 °, 90 °, 180 ° and 270 ° are projected.

As a measuring device, a device that scans slit light has been commercialized, but input time of three-dimensional information including scanning for inputting three-dimensional information is several hundred ms.
It is also hanging. Further, in a conventional digital camera, in the continuous shooting mode, continuous shooting is performed for a plurality of images. In some cameras, recording on a memory card is performed after shooting.
It was not a dimension information input camera.

[0008]

By the way, in order to obtain three-dimensional information, it is necessary to project a pattern light and photograph an image with a pattern. It is conceivable to use it first. However, the flash light requires a booster circuit, which requires a circuit configuration. Further, photographing cannot be performed during charging. In addition, when flash light is projected, a strong shadow is formed on a subject, and three-dimensional information may not be obtained for that part.

Therefore, a first technical problem to be solved by the present invention is to provide a three-dimensional information input camera capable of projecting pattern light without using a flash light emitting device.

Further, the above-mentioned devices are intended to be used in a fixed manner indoors and the like, and are dedicated measuring machines, so that the operation is complicated. In addition, in order to realize a device that is easy to use as a three-dimensional input camera as a portable device such as a general digital camera, it is necessary to use the device by holding it by hand. It takes time,
Cannot be operated with hand. This is due to the presence of camera shake and subject shake.

Accordingly, a second technical problem to be solved by the present invention is to reduce the influence of image movement due to camera shake and image movement due to movement of a subject, and to increase the accuracy of inputting three-dimensional information. It is to provide a dimensional information input camera.

[0012]

In order to solve the above technical problems, the present invention provides a three-dimensional information input camera having the following configuration.

The three-dimensional information input camera includes photographing means for photographing a photographing area, and projecting means for projecting pattern light onto the photographing area, and is formed on a subject in the photographing area by the pattern light projected by the projecting means. The projection pattern is photographed by the photographing means. The projection means has a light emitting diode, and uses the light emitting diode as a light source for pattern light projection.

In the above arrangement, when inputting three-dimensional information, the projecting means projects predetermined pattern light onto a photographing area by using light emission of a light emitting diode (LED) as a light source. Take an image with the attached pattern. If a light emitting diode is used, it is not necessary to charge as in a flash light emitting device, light emission can be started immediately at any time, and imaging for inputting three-dimensional information can be performed quickly. Further, if a light emitting diode in which a large number of light emitting elements are arranged on a substrate is used, the light source becomes a surface light source, so that it is easy to prevent a strong shadow from being cast on a subject. Furthermore, as will be described later, pattern light can be projected without using a mask, and the stripe pattern cycle and pattern color distribution of the pattern light can be changed.

Therefore, pattern light can be projected without using a flash light emitting device.

In the case of switching between pattern light and flat auxiliary light having no pattern, or in the case of selectively switching and projecting a plurality of types of pattern light, the configuration of mechanically moving the mask is eliminated. It is possible to switch the pattern light in a short time.

Therefore, the influence of the movement of the image due to camera shake or the movement of the image due to the movement of the subject can be reduced, and the accuracy of inputting three-dimensional information can be improved.

The pattern light can be projected using a mask.

That is, the projection means further includes a mask having a stripe-like transmittance or reflectance distribution, and projects the light emitted from the light emitting diode as pattern light through the mask.

In this case, the light from the light emitting diode passes through the mask or is reflected by the mask, is provided with a stripe pattern, and is projected as pattern light. An image with a pattern can be taken under the pattern light projection.

Preferably, the photographing means can photograph a photographing area when the light emitting diode emits light and projects pattern light, and when the light emitting diode does not emit light and does not project pattern light. .

That is, the light emitting diode is turned off when an image without a pattern is photographed (normal photographing). In this case, if there is no other device for providing the auxiliary light, an image without a pattern is taken under the constant light without the auxiliary light.

As another configuration, pattern light is projected without using a mask by the light emission distribution of the light emitting diode itself.

That is, the light emitting diode has a stripe light emission distribution, and the projection means projects the light emitted from the light emitting diode as pattern light.

In this case, the light emitting diode itself emits light with a stripe light emission distribution and projects pattern light without passing through a mask. If the stripe light emission distribution of the light emitting diode can be changed, it is possible to select and project pattern light having a stripe cycle and color suitable for the subject.

Also, if the light emitting diode can switch between the stripe light emission distribution and the uniform white light emission distribution, the light emission of the uniform white light emission distribution of the light emitting diode can be used as auxiliary light to produce an image without a pattern. You can also shoot.

[0027] Preferably, the light emitting diode is R
(Red), G (green), B (blue), or W (white).

In this case, the light emitting diode may emit light of a single color as a whole, or may emit light with two or more colors distributed. Further, the light emission color may be able to be switched entirely or partially. There are various types of light-emitting diodes having various colors, but it is preferable to select one of the above-mentioned colors because the light emission luminance is higher than those of other colors.

[0029]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a three-dimensional information input camera (hereinafter, referred to as a 3D camera) according to an embodiment of the present invention will be described with reference to the drawings.

As shown in the front view of FIG. 1, the 3D camera is composed of a stripe pattern projection unit 1, a box-shaped camera body 2, and a rectangular parallelepiped imaging unit 3 (shown by a thick line). . The imaging unit 3 is detachable from the right side of the camera body 2 when viewed from the front.

The image pickup section 3 is provided with an image pickup circuit having a CCD color area sensor 303 (see FIG. 5) at an appropriate position behind a zoom lens 301 having a macro function as a photographing lens. In addition, similarly to the silver halide lens shutter camera, a light control circuit 305 including a light control sensor 305 for receiving the reflected light of the flash light from the subject at an appropriate position in the imaging unit 3.
Reference numeral 04 (see FIG. 5) further includes a distance measuring sensor AF for measuring the distance to the subject and an optical finder 31.

On the other hand, a zoom motor M1 (see FIG. 5) for changing the zoom ratio of the zoom lens 301 and moving the lens between the accommodation position and the photographing position and focusing is provided inside the imaging section body 3. M2 (see FIG. 5).

On the front surface of the camera body 2, a grip portion 4 is provided at an appropriate position on the left end, a built-in flash 5 is provided at an appropriate upper portion on the right end, and a 3D camera and an external device (for example, another 3D camera or the like). An IRDA port for performing infrared communication with a personal computer) is provided.
A shutter button 9 is provided on the upper surface of the camera body 2.

The stripe pattern projection unit 1 is located between the camera body 2 and the imaging unit body 3 and has a stripe pattern projection unit 5.
01 is arranged. The projection unit 501 includes the photographing lens 30
The center of the stripe pattern is arranged so as to be substantially at the same height as the center of the optical axis of No. 1. The stripe pattern is arranged so that the pattern direction is perpendicular to the direction away from the optical axis. Since it is fundamental to obtain three-dimensional information based on the principle of triangulation, it is necessary to increase the so-called base line length and secure the accuracy, and to provide an offset or relative to an arrangement with an angle other than vertical. The objective is to cover the subject with a small stripe pattern.

The projection of the stripe pattern uses flash light here. As will be described in another embodiment, the projection may be lamp light instead of flash light.

A film is used for the stripe pattern. The stripe pattern is not limited to a film, but may be a glass substrate provided with a pattern such as a pigment or a dye.

As shown in the rear view of FIG. 2, on the back of the camera body 2, an LCD display section 10 for displaying a monitor of a captured image (corresponding to a viewfinder) and displaying and reproducing a recorded image is provided. Is provided. Key switches 521 to 526 for operating the 3D camera, and a power switch P of the camera main body are located below the LCD display unit 10.
S is provided. On the left side of the power switch PS, there are provided an LED 1 which is lit when the power is ON, and a BUSY display LED 2 which indicates a state in which an input to the camera is not accepted because the memory card is being accessed or necessary for photographing preparation.

Further, on the back of the camera body 2, there is provided a photographing / reproduction mode setting switch 14 for switching between a "photographing mode" and a "reproduction mode". The photographing mode is a mode in which a photograph is taken, and the reproduction mode is a mode in which a photographed image recorded on the memory card 8 (see FIG.
This is a mode for reproducing and displaying on the D display unit 10. The shooting / playback mode setting switch 14 also includes a two-contact slide switch. For example, when the switch is slid down, the playback mode is set, and when the switch is slid up, the shooting mode is set.

Further, a four-way switch Z is provided on the upper right side of the back of the camera. By pressing the buttons Z1 to Z2, the zoom motor M1 (see FIG. 5) is driven to perform zooming. Exposure compensation is performed by pressing Z4.

An LCD button for turning on and off the LCD display is provided on the back side of the image pickup unit 3. Each time the button is pressed, the LCD display switches between on and off states. For example, when photographing is performed exclusively using the optical viewfinder 31, the LCD display is turned off for the purpose of saving power. At the time of macro shooting, by pressing the MACRO button, the focus motor M2 is driven and the shooting lens 301 enters a state where macro shooting is possible.

On the back side of the stripe pattern projection unit 1,
A flash power supply for projecting a stripe pattern, that is, a 3D flash power switch Z5 is provided.

As shown in the side view of FIG. 3, a DC input terminal and a Vi for outputting the content displayed on the liquid crystal to an external video monitor are provided on the side of the main body 2 of the 3D camera.
A deo output terminal is provided.

As shown on the bottom of FIG.
A battery loading chamber 18 and a card loading chamber 17 for the memory card 8 are provided on the bottom surface of the device, and the loading port is closed by a clamshell type lid 15. The drive source of the 3D camera in the present embodiment is a power battery formed by connecting four AA batteries in series. Further, on the bottom surface, a release lever Rel for releasing the engagement between the imaging unit 3 and the main unit 2 connected by the connector and the hook-shaped connecting tool is provided.

The battery loading chamber 518 and the lid 515 are provided on the bottom of the stripe pattern projection unit 1 in the same manner as the camera body 2.
And a flash battery different from the camera body 2 is used. A tripod screw 502 is provided on the bottom surface of the stripe pattern projection unit 1. The tripod screw 502 is provided in the stripe pattern projection unit 1 located relatively at the center in view of the balance of the camera.

Next, the internal blocks of the imaging section 3 will be described with reference to the blocks of FIG.

The CCD 303 is a macro zoom lens 30
The light image of the object formed by the step 1 is represented by R (red), G
(Green) and B (blue) color component image signals (signals composed of a signal sequence of pixel signals received by each pixel) are photoelectrically converted and output. The timing generator 314 includes the CCD 30
3 to generate various timing pulses for controlling the driving of the third driving circuit.

The exposure control in the image pickup section 3 is performed by adjusting the exposure amount of the CCD 303, that is, the charge accumulation time of the CCD 303 corresponding to the shutter speed, since the stop is a fixed stop. If an appropriate shutter speed cannot be set when the subject brightness is low, the CCD 3
By performing the level adjustment of the image signal output from the device 03, the improper exposure due to the insufficient exposure is corrected. That is, when the luminance is low, the exposure control is performed by combining the shutter speed and the gain adjustment. The level adjustment of the image signal is performed in gain adjustment of an AGC circuit described later in the signal processing circuit 313.

The timing generator 314 generates a drive control signal for the CCD 303 based on a reference clock transmitted from the timing control circuit 202 of the main unit 2. The timing generator 314 includes, for example, a timing signal of integration start / end (exposure start / end),
A clock signal such as a read control signal (horizontal synchronization signal, vertical synchronization signal, transfer signal, etc.) for the light receiving signal of each pixel is generated, and C
Output to CD303.

The signal processing circuit 313 performs predetermined analog signal processing on an image signal (analog signal) output from the CCD 303. The signal processing circuit 313
It has a DS (correlated double sampling) circuit and an AGC (auto gain control) circuit. The CDS circuit reduces the noise of the image signal, and the level of the image signal is adjusted by adjusting the gain of the AGC circuit.

The light control circuit 304 controls the amount of light emitted from the built-in flash 5 during flash photography by the overall control unit 2 of the main unit 2.
11 to control the light emission amount to a predetermined value. In flash photography, the reflected light of the flash light from the subject is received by the light control sensor 305 at the same time as the start of exposure, and when the amount of received light reaches a predetermined light emission amount, the light control circuit 304 is provided in the control unit 211. The FL control circuit emission stop signal is output. The FL control circuit forcibly stops the light emission of the built-in flash 5 in response to the light emission stop signal, whereby the light emission amount of the built-in flash 5 is controlled to a predetermined light emission amount.

The 3D information input will be described later in the sequence, but is obtained from two flash images. One is an image with stripe pattern projection and the other is an image without stripe pattern projection. Ideally, the basic luminous intensity (see FIG. 27) is constant between the two images. When extracting phase information from fringe pattern information, basic luminous intensity information must be removed. Therefore, in the case of taking two pictures, the flash emission time is kept constant without performing separate light control. Note that the flash light control itself is controlled by the overall control unit 211 of the camera body 2.

As described above, the image pickup section 3 and the main body section 2 are provided on the mounting surface 334 of the image pickup section 3 by 334a to 334.
g, and the imaging unit 3 and the main body 2 pass through the stripe pattern projection unit 1 by the seven groups of connection terminals including the groups 234a to 234g provided on the connection surface 233 of the main body 2. And are electrically connected.
Further, the stripe pattern projection unit 1 and the main body 2 are
It is electrically connected by the connection terminal of 4h.

Next, the internal blocks of the camera body 2 will be described.

In the camera body 2, the A / D converter 205 converts each pixel signal of the image signal into a 10-bit digital signal.

A timing control circuit 202 for generating a reference clock, a timing generator 314, and a clock for the A / D converter 205 is provided in the camera body 2. The timing control circuit 202 includes the control unit 2
11.

The black level correction circuit 206 corrects the black level of the A / D converted pixel signal (hereinafter referred to as pixel data) to a reference black level. The WB circuit 207 performs level conversion of pixel data of each of the R, G, and B color components so that the white balance is also adjusted after the γ correction. The WB circuit 207 converts the level of the pixel data of each of the R, G, and B color components using the level conversion table input from the overall control unit 211. The conversion coefficient (gradient of the characteristic) of each color component in the level conversion table is set by the overall control unit 211 for each captured image.

The γ correction circuit 208 corrects the γ characteristics of the pixel data.

The image memory 209 is a memory for storing the pixel data output from the gamma correction circuit 208. The image memory 209 has a storage capacity for one frame. That is, in the image memory 209, the CCD 303
In the case of having pixels in rows and m columns, it has a storage capacity for pixel data of n × m pixels, and each pixel data is stored in a corresponding pixel position.

The VRAM 210 is a buffer memory for image data reproduced and displayed on the LCD display unit 10. VR
The AM 210 has a storage capacity for image data corresponding to the number of pixels of the LCD display unit 10.

In the photographing standby state, the A / D converters 205 to γ correction circuit 208 perform predetermined signal processing on each pixel data of an image captured every 1/30 (second) by the imaging unit 3. Thereafter, the image data is stored in the image memory 209, transferred to the VRAM 210 via the overall control unit 211, and displayed on the LCD display unit 10 (live view display). Thus, the photographer can visually recognize the subject image from the image displayed on the LCD display unit 10. Further, in the reproduction mode, the image read from the memory card 8 is subjected to predetermined signal processing by the overall control unit 211 and then transferred to the VRAM 210, where
Will be displayed.

The card I / F 212 is an interface for writing image data to the memory card 8 and reading image data.

The flash control circuit 216 is a circuit for controlling light emission of the built-in flash 5. The flash control circuit 216 controls the presence / absence of light emission of the built-in flash 5, the amount of light emission, the light emission timing, and the like based on the control signal of the overall control unit 211, and based on the light emission stop signal STP input from the dimming circuit 304. Is controlled.

The RTC 219 is a clock circuit for managing the shooting date and time. It is driven by another power source (not shown).

The operation section 250 is provided with the various switches and buttons described above.

The shutter button 9 is in a half-pressed state (S1) as employed in a silver halide camera and in a depressed state (S1).
2) is a two-stage switch that can be detected. When the shutter button is set to the state S1 in the standby state, the distance information is transmitted to the overall control unit 2 based on the distance measurement information from the distance measurement sensor AF.
Input to 11. According to an instruction from the overall control unit 211, A
By driving the F motor M2, the photographing lens 301 is moved to the in-focus position.

The overall control unit 211 is composed of a microcomputer, and includes the inside of the image pickup unit 3 and the camera main unit 2 described above.
The driving of each of the members is organically controlled to totally control the photographing operation of the 3D camera 1. This will be described with reference to the block diagram of FIG.

The overall control unit 211 includes a luminance determination unit 211a for setting an exposure control value (shutter speed (SS)) and a shutter speed setting unit (SS setting unit 2).
11b).

The brightness determination unit 211a determines the brightness of the subject using an image captured by the CCD 303 every 1/30 (second) in the shooting standby state. That is, the luminance determination unit 211a determines whether the image memory 2
The brightness of the subject is determined using the image data that is renewed and stored in step 09.

The shutter speed setting section 211b sets the shutter speed (integration time of the CCD 303) based on the result of the brightness determination of the subject by the brightness determination section.

Further, the overall control unit 211 includes a filter unit 211f for performing a filtering process and a recorded image generating unit 211g for generating a thumbnail image and a compressed image in order to perform the recording process of the photographed image. In order to reproduce the image recorded on the LCD display unit 10, a reproduced image generation unit 211h that generates a reproduced image is provided.

The filter section 211f corrects high-frequency components of an image to be recorded by a digital filter, thereby correcting image quality related to contours.

The recorded image generating section 211f is provided in the image memory 2
The pixel data is read from 09 and a thumbnail image and a compressed image to be recorded on the memory card 8 are generated. The recording image generation unit 211h reads pixel data for every eight pixels in both the horizontal direction and the vertical direction while scanning from the image memory 209 in the raster scanning direction, and sequentially transfers the pixel data to the memory card 8, thereby obtaining the thumbnail image. Is recorded on the memory card 8 while generating.

The recording image generation unit 211f reads out all pixel data from the image memory 209 and converts these pixel data into JPE data such as two-dimensional DCT transform and Huffman coding.
A predetermined compression process according to the G method is performed to generate image data of a compressed image, and the compressed image data is recorded in the main image area of the memory card 8.

In the case of the 3D information input mode, JP
Since it is desirable not to perform EG compression, it is treated as 1/1 compression when passing through the recording image generation unit 211f.

In the photographing mode, when the photographing is instructed by the shutter button 9 in the photographing mode, the thumbnail image of the image taken into the image memory 209 after the photographing instruction and the compression ratio set by the compression ratio setting switch 12 Generates a compressed image compressed by the JPEG method, and generates tag information (frame number, exposure value, shutter speed, compression ratio, shooting date, flash on / off data at shooting, scene information, image determination result regarding the shot image) ) Are stored in the memory card 8 together with information such as

In the case of the 3D information input mode, as shown in FIG. 7, 3D information of one subject is obtained for the first time in the first and second frames. That is, the first image is an image with a stripe pattern and the second image is a normal image without a stripe pattern and b. Normally, if a card can take 40 shots, 20
This is a 3D image of the scene.

Each frame of the image recorded by the 3D camera has a tag portion, high-resolution image data ((1600 × 1200) pixels) compressed in JPEG format, and image data for thumbnail display ((80 × 60) Pixel) is recorded.

When the photographing / reproduction mode setting switch 14 is set to the reproduction mode, the image data having the largest frame number in the memory card 8 is read out, the data is expanded by the reproduction image generation section 211h, and this is expanded into the VRAM 21.
By being transferred to 0, the image with the largest frame number, that is, the most recently captured image is displayed on the display unit 10. By operating the UP switch Z3, an image having a large frame number is displayed, and the DOWN switch Z4 is displayed.
By pressing, an image with a small frame number is displayed.
However, when the image is captured in the 3D mode, if the 3D mode information is recorded, the image is a 3D captured image, so that the second image of the set of two images, that is, the image b in FIG. 7 is output. This is to prevent an image with a stripe pattern from being displayed.

Next, the portion of the stripe pattern projection unit 1 will be described. The internal circuit of the stripe pattern projection unit 1 operates when the switch of the 3D flash power switch Z5 is ON. If it is ON, the flash control circuit 216 of the camera body and the built-in flash 5 enter a non-operation state.
The control circuit 514 of the stripe pattern projection unit 1 includes a circuit for operating the flash 505 of the stripe pattern projection unit 501 and a circuit for switching the stripe pattern. To switch the mask, a signal is sent to the mask motor M3 to operate the mask unit 530. A power supply circuit and a battery (not shown) are additionally arranged in the stripe pattern projection unit 1.

The inside of the stripe pattern projection unit 1 is shown in FIG.
It is like. FIG. 8A is a front perspective view,
(B) is a perspective plan view. Stripe pattern projection unit 1
Inside, a xenon tube 531 that emits flash light, a concave lens 532 for projecting a pattern onto a subject wide, a mask unit 530 in which two masks are combined in an L shape, and a mask unit 530 And a motor (not shown) for rotating the shaft 534.

When the mask unit 530 is at the position shown by the solid line in FIG. 8A, one mask is projected from the projection window 533. When the mask unit 530 rotates 90 degrees counterclockwise from the solid line position in the figure, the other mask is projected from the projection window 533. When the mask unit 530 rotates 90 degrees clockwise from the solid line position in the figure, both masks are completely retracted from the projection window 533.

The control circuit 514 includes a capacitor for storing electric energy for flash light and a light control sensor 305.
Switch IGB that stops flash emission upon receiving a signal
Although there is T or the like, since the configuration is the same as that of the conventional flash circuit, the description is omitted.

The two masks of the mask unit 530 are
As shown in (s) and (t) in FIG. 11, the patterns have different frequencies. The pattern of each mask is in a gradation state.

The detailed distribution of the pattern is shown in FIGS.
It has become. The number of stripe patterns is, for example, 10 to 30 periods (14 in FIG. 12A), and each stripe has a density distribution as shown in FIG. 12B.
Each concentration shows a triangular wave with a distribution of, for example, 20% to 70%. That is, it is in a gradation state. Based on this density distribution, a phase shift when light is received can be detected, and a phase image, that is, a distance distribution image (three-dimensional image) can be obtained. In principle, any combination of a monotonically increasing portion and a monotonically decreasing portion is sufficient, and each of them may have a sine wave or a Gaussian distribution. In addition, the mask pattern may be in a state in which the density is changed stepwise, in addition to the gradation state.

In FIGS. 12 (a) and 12 (b), the portion K whose color has been changed in order to specify which frequency (the number of stripes) is to be specified.
have. The portion K in FIG.
It has color.

FIG. 12C shows an arrangement in which the pattern of FIG. 12A is arranged at the center and gray codes are arranged at both ends. Edge of the stripe pattern (below or above the stripe pattern,
Or top and bottom) with a gray code pattern. The code has, for example, three lines of codes, and the position can be specified by a combination of three data. Further, by providing a stripe pattern having changed color information or density, the accuracy of position identification can be increased.

FIG. 13 shows another embodiment. FIG. 13 (a)
In one projection fringe pattern, a low frequency at which the position of the fringe pattern is easily specified is placed at a peripheral portion, and a high frequency is placed at a central portion to improve accuracy. The position of the fringe pattern can be specified from the low-frequency fringe pattern or at both low and high frequencies. For example, if one frequency is f, the other is set to 1.4f and 2f. Each stripe pattern has a density of a sine wave or a triangular wave, and is in a gradation state. In addition to the gradation state, a state in which the density changes stepwise may be used. FIG. 13B shows a case of a sine wave, and FIG. 13C shows a case of a triangular wave.

Further, in order to increase the accuracy of specifying the position,
For example, a colored pattern is placed in the center, and not only a stripe with gradation but also a marker using color information can be placed to improve the accuracy of position information. Here, the one having a concentration of 30% to 80% is used.
The density of the colored portion is 50%. The entire density distribution requires a contrast of about 50%. If the CCD sensor can detect a change of (SN) of 5% from the detection ability, it is possible to distinguish 10 levels of density here. As the contrast increases, the resolution increases, and the accuracy in obtaining 3D information improves.

Taking these as an example, a pattern in which the respective frequencies are set low is t. As shown in FIG.
It is possible to switch between pattern projection using s, pattern projection using t, and projection using only flash light without a pattern. This is because when the spatial frequency of the subject is high or when the frequency is similar to the projected fringe pattern, it may be difficult to specify the fringe position. In such a case, the frequency of the fringe pattern is switched. This is to prevent erroneous detection.

The switching condition is based on the frequency of the photographed image. That is, when a captured image is input,
A frequency check is performed. If the same frequency as the stripe pattern is included, the stripe pattern is switched and the subject image with the stripe pattern is captured again without storing the image. Thus, even if the spatial frequency of the subject is a repetition like a stripe pattern, three-dimensional information can be input correctly without lowering the accuracy of specifying the position of the stripe pattern.

Another example of the mask unit is shown. FIG. 14 shows an s-pattern. FIG. 15 and FIG.
It is a pattern.

FIG. 14 shows a pattern similar to that shown in FIG. 12A, but there are a plurality of colored portions. Here, cyan (C) is used. The reason for using a single color is that mask production is ready. That is, it can be performed at low cost. When the cyan color is projected, information is obtained with high sensitivity to the flesh color and is suitable as a marker.

However, the position cannot be specified depending on the color of the subject or the color of the light source illuminating the subject, and it is difficult to specify the stripe position. Therefore, even if the color of the subject or the color of the light source illuminating the subject is biased toward a specific color, the stripe pattern is switched in order to correctly input the three-dimensional information without lowering the accuracy of specifying the position of the stripe pattern. Here, cyan is set to magenta (M) as in the pattern of FIG.

The condition for this switching is based on the color of the photographed image. That is, when a captured image is input, a color check is performed, and if the same color as the stripe pattern is included, the stripe pattern is switched and the subject image with the stripe pattern is re-recorded without memorizing the image. Take an image. Thus, even when the color of the subject or the light source color is a repetition like a stripe pattern, three-dimensional information can be input correctly without lowering the accuracy of specifying the position of the stripe pattern.

As another switching example, there is a pattern as shown in FIG. This is because R (red), G
(Green), B (blue), C (cyan), M (magenta), Y
(Yellow). When a colored pattern can be created at low cost, using a pattern as shown in FIG. 16 reduces the error probability.

Here, another embodiment of the stripe pattern projection will be described.

In this method, the switching between the imaging of the subject image when projecting the fringe pattern and the imaging when the pattern is not projected is performed only electrically and without any moving part without performing mechanical switching. The purpose of this is to shorten the time between the imaging of a subject image when projecting a pattern and the imaging when not projecting a pattern, to reduce the effect of image movement due to camera shake and the movement of the image due to movement of the subject, and to reduce the effect of three-dimensional information. It is to increase the input accuracy.
In an image with and without a pattern, a phase image cannot be obtained correctly unless the subject and the camera have the same positional relationship. In the case of mask pattern switching using a spatially modulating element that can be electromagnetically changed, the mask pattern can be switched in a shorter time than mechanical switching, and the three-dimensional information input accuracy increases.

FIG. 9 shows an embodiment in which a liquid crystal element is used as an electromagnetically variable spatial modulation element.

FIG. 9A shows a stripe pattern projection unit 1.
(B) is a plan perspective view, 541 is a xenon tube, 540 is a liquid crystal panel (LCD), 54
2 is a lens for pattern projection, and 543 is a pattern projection window. In this case, the mask motor M3 is unnecessary in the circuit in FIG.

The liquid crystal itself of the LCD 540 is shown in FIGS.
In the case of the mask pattern of No. 3, for example, it can be constituted by a single color such as white and black, and in the case of FIGS.
It will be multicolored. There are various types of liquid crystal, a type that uses a polarizing plate and a type that does not look like a guest host,
Although there is an organic liquid crystal type or the like, any type may be used for the LCD 540. The LCD 540 is different from a general LCD that simply switches between display and non-display in a binary manner.
It is possible to display a halftone, thereby forming a pattern in a gradation state, and to increase the three-dimensional information input accuracy as compared with a general LCD which can only perform a binary display.

Instead of projecting the pattern by the light transmitted through the liquid crystal panel 540, a configuration in which the pattern is projected by the light reflected by the liquid crystal panel 545, for example, as shown in a plan perspective view of FIG. is there. That is, the light of the xenon tube 541 is
In step 4, the liquid crystal panel 545 is irradiated, and the reflected light is projected from the pattern projection window 543.

In addition to the liquid crystal, a transparent electrochromic element or electroluminescent element is projected from the backside with a flash light, and the light transmitted through the electrochromic element or electroluminescent element is used to form a stripe pattern and color marker on the subject. It is possible to add

FIG. 10 shows still another embodiment using a light emitting diode. For this purpose as well, the time between the imaging of the subject image when projecting the pattern and the imaging when the pattern is not projected is shortened, and the effect of image movement due to camera shake or object movement is reduced, thereby reducing the effect of three-dimensional information. It is to increase the input accuracy. The configuration does not use a flash light source, but instead uses a light emitting diode (LED).

In the case of switching the presence or absence of a pattern by light emission and non-light emission of the light emitting diode, the mask pattern can be switched in a shorter time than mechanical switching. In this case, without a flash light source, three-dimensional information is obtained from a subject image with a light emitting diode pattern and a subject image with only steady light.

FIG. 10A shows a stripe pattern projection unit 1.
(B) is a perspective plan view. The light emitting diode array 560 has R (red), G (green), and B (blue) light emitting elements arranged in a row, and each light emitting element is provided with a microlens. A mask 550 for forming a stripe pattern is arranged in front of the light emitting diode array 560 so that the period of the light emitting elements matches the cycle.
The mask 550 has a configuration as shown in FIG.
And the portion K where the density of stripes changes
May not be required. Light from each light emitting element of the light emitting diode array 560 passes through the mask 550 and is projected from the pattern projection window 553. Therefore, R (red), G
Pattern light in which (green) and B (blue) stripes are repeated is projected.

The light emitting elements of the light emitting diode array 560 include W (white), R (red), and W as shown in FIG.
(White), G (green), W (white), B (blue), and W (white). Color separation correction with white becomes easy.

In the configuration shown in FIGS. 10A, 10B, and 10C, only the pattern light with stripes can be projected from the projection window of the stripe pattern projection unit 1, so that an image without a stripe pattern can be displayed. In order to obtain it, shooting is performed in a natural light state without auxiliary light such as a flash, or the built-in flash 5 of the camera body 2 is used.

The pattern light can be projected directly from the light emitting diode array 560 without using the mask 550. In particular, in this case, as shown in FIG. 10D, if two light emitting elements are arranged as one set corresponding to one microlens and the light emitting diode array 570 capable of switching colors can be used, When one of the light-emitting elements (R, W, G, W, and B on the left side in the figure) emits light and projects striped pattern light, an image with a pattern is taken, and each set of light-emitting elements When the other (the right side W, W, W, W, W in the figure) emits light to emit flat white light without stripes, the white light is used as auxiliary light to capture an image without a pattern. It can be carried out.

Next, the operation using the 3D camera will be described with reference to FIG.
This will be described with reference to FIG.

First, the main switch PS of the camera is turned on.
After that, the 3D flash switch Z5 is turned on (U
1). Next, the 3D mode is set (U2). Here, the mode is set using the switch keys 521 to 526. This may be automatically set simultaneously with Z5ON. If the circuit type and the power source type are supplied only from the camera body, the setting may be made only by the switch keys 521 to 526.

When the mode is set, the 3D input area becomes LC
Is displayed on the D monitor 10 (U3) and the BUSY display (LE
D2) (U4), and the stripe pattern t is set.
(U5). A live image of the subject is displayed on the LCD display 10 (U6). Then, charging of the capacitor (not shown) of the 3D flash is started (U7). Wait for charging to end (U8), and when it is finished, the BUSY display disappears (U8).
9). Then, the control waits for a release signal (ON of the shutter button 9) at U11.

For 3D photographing, two continuous shots are required.
One image is obtained with a stripe pattern, and one image is obtained without a stripe pattern. When the release signal is input, the first image capturing is started and the integration of the image sensor is started (U12). During this integration, the flash with the stripe pattern emits light, and a stripe pattern image is obtained (U13). In U13, the image with the stripe pattern is the first image, but may be the second image.

Before storing the input image, the frequency of the subject image is checked (U14). If the frequency is different from the stripe pattern frequency and three-dimensional information can be input, the process proceeds to U17 and U19. However, when it is determined that three-dimensional information cannot be input, the stripe pattern is switched from the t pattern to the s pattern (U16). Then, the process returns to U12 for re-imaging.

The frequency can be changed not only by the mechanical switching method but also by the LCD pattern switching method.

Further, instead of checking the frequency in U14, the method of the modification shown in FIG. 21 for checking the image color is also possible. That is, it is determined whether three-dimensional information can be input based on the subject color and the light source color. This color check is based on the auto white balance (AW
B) The function is used for checking (U34). And
In U35, if it is determined that the marker can be correctly confirmed by the color and the phase image in which the position of the stripe pattern can be specified is obtained, that is, if it is determined that three-dimensional information can be input, the process proceeds to U17 and U19. If it is determined that it is impossible, the stripe pattern is switched from the t pattern to the s pattern in U36, the process proceeds to U12, and a re-striking image is performed without storing the already captured image. The patterns t and s in FIG. 21 are the patterns in FIG. 14 and FIG.

Next, the image data a
(Image with stripe pattern) is stored (U17). At this time, since the captured image is a stripe pattern, a live image is displayed on the LCD monitor 10 instead of the subject with the stripe pattern (U18).

On the other hand, in the fringe pattern projection unit 1, unlike a general flash, it is prohibited to start charging after flash emission (U19), and the pattern is saved (U20).

The mask is evacuated by the mask motor M3 as shown in FIG. The evacuation time is shortened, and the photographing interval between the two images is shortened as much as possible. The displacement of the image is negligible even if the subject moves. For example, the target is set within 100 ms including the bound of the mask. When this saving is performed by a motor, a large current consumption is required. Therefore, if the flash charging is started at the same time, a large current is required for both, so that the motor may not operate, so that the motor cannot be evacuated, and an image without a stripe pattern cannot be obtained in the second photographing. Therefore, simultaneous operation of flash capacitor charging and motor energization is avoided.

After the pattern is switched, the second image pickup is started (U21). In the same way, a flash fires (U2
2) An image without a stripe pattern is obtained.

Then, the image b without stripe pattern is stored in U23, and the image pickup result, that is, the subject image without stripe pattern is displayed on the LCD monitor 10 in U24.

On the other hand, in parallel with this, the striped pattern projection unit 1 restores the saved pattern (U2
5). Then, charging of the 3D flash is restarted for the first time (U26). Turn on the BUSY display again (U2
7).

After the end of U24 and U27, at U28,
The image a with the stripe pattern and the image b without the stripe pattern are written in the memory card 8. Here we write 2
This is for shortening the photographing time interval between frames. This is because it takes time to write data one by one. That is, when the 3D mode is set, the mode is a mode in which data is written to the memory card 8 two by two. Hereinafter, if the 3D flash switch Z5 remains on, the display is switched to the live image display at U30, and the U8
Return to

FIG. 21 shows a sequence in which a pattern is switched according to the color of a photographed image described with reference to FIGS. 14 and 15. Steps other than U34, U35 and U36 are the same as those in FIG. 19 in the above-described embodiment.
Note that the sequence after U35 (and thereafter) is the same as that in FIG.

Here, a sequence when the LCD is used for projecting the stripe pattern will be described.

In this case, U14, U15,
U16 is omitted. Then, in U5, the "stripe pattern t
“LCD stripe pattern is turned on” instead of “set”. In U20, instead of "save the pattern", "L
Turn off CD stripe pattern ". Then, in U25, instead of "restoring the pattern", "LCD stripe pattern is turned on". The above is an LCD type sequence.

In the case of the LED type sequence, FIG.
It looks like 2. After turning on the main switch PS of the camera, the 3D mode is set (V1). Here, the mode is set using the switch keys 521 to 526.
This may be automatic setting at the same time when Z5 is ON.
Further, if the circuit type and the power source type are supplied only from the camera body 2, the setting may be made only by the switch keys 521 to 526.

When the mode is set, the 3D input area becomes LC
The image is displayed on the D monitor 10 (V2), and the live image of the subject is displayed on the LCD display 10 (V3). Then, the control waits for a release signal (the shutter button 9 is turned on) at V4.

When the release signal is input, the stripe pattern becomes LE.
The light is projected by D (V5), the first image is taken, integration of the image sensor starts (V6), and a stripe pattern image is obtained. In V6, the image with the stripe pattern is the first image, but may be the second image.

Next, in the camera body 2, image data a
(Image with stripe pattern) is stored (V7). At this time, since the captured image is a stripe pattern, the LCD monitor 10 displays not the subject with the stripe pattern but a live image stored in the VRAM immediately before the release (V8).

Next, with the LED pattern turned off (V9), the process enters the integration of the next image sensor (V10).
Since only the LED is turned on and off, the photographing interval between the two images is short. Even if the subject moves, the displacement of the image can be ignored. An image without a stripe pattern is obtained by the second imaging. At this time, if the LED type is (d) in FIG. 10, only the white LED emits light to perform illumination without a pattern, and an image without a pattern is obtained. Then, the image b without a stripe pattern is stored in V11, and the imaging result, that is, the subject image without the stripe pattern is displayed on the LCD monitor 10 in V12.

At V13, the image a with the stripe pattern and the image b without the stripe pattern are written to the memory card 8. The collective writing is performed here in order to shorten the time interval between two images. This is because it takes time to write data one by one. That is, when the 3D mode is set, the mode is a mode in which the memory card 8 is written two by two.

Thereafter, if the 3D mode is continued, the flow returns to V3 (V14) to switch to live image display.

The above is the operation of the camera. Data for obtaining 3D information is in the memory card 8. To reproduce the data into a 3D image, this data is subjected to post-processing by a computer such as a personal computer. This process is performed according to the procedure shown in FIG.

That is, after the memory card 8 is set in the personal computer (not shown), data of the image a with the stripe pattern and the image b without the stripe pattern are input from the memory card 8 (D1, D2). The basic luminous intensity information is extracted from the image a, and a basic luminous magnification n for the image b is obtained (D3).
The basic luminous intensity is image data that does not depend on the stripe pattern as shown in FIG.

Next, the basic luminous intensity levels of the image a and the image b are matched to obtain only the stripe pattern information c (D4). Then, based on the stripe pattern information c, a phase image in which the gain is normalized is extracted (D5).

Then, at D6, the distance distribution of the subject is calculated from the phase image. At this time, since the positions of the stripe patterns can be distinguished, it is possible to accurately specify the order of the stripe position corresponding to the phase position. In other words, the position of the projection pattern and the position of the reflection pattern from the subject can be accurately matched. In this way, the distance to the subject,
And the distance distribution can be obtained as accurate information.
When obtaining a three-dimensional image, only information on the distance distribution may be used.

The above-described 3D camera enables input of three-dimensional information without any problem of camera shake or subject shake.

The present invention is not limited to the above embodiment, but can be implemented in other various modes.

For example, while the embodiment of the digital camera has been described, a silver halide camera may similarly capture two images, one with a stripe pattern and the other without a stripe pattern, on a silver halide film and create a 3D image by post-processing. Is possible. In this case, if the film is developed, digitized by a film scanner, and loaded into a computer such as a personal computer, the post-processing becomes the same. Further, instead of the liquid crystal panels 540 and 545, an electrochromic element, an electroluminescent element, or the like may be used.

[Brief description of the drawings]

FIG. 1 is a front view of a three-dimensional information input camera according to an embodiment of the present invention.

FIG. 2 is a rear view of the camera of FIG. 1;

FIG. 3 is a left side view of the camera shown in FIG. 1;

FIG. 4 is a bottom view of the camera of FIG. 1;

FIG. 5 is a circuit block diagram of the camera of FIG. 1;

FIG. 6 is a detailed block diagram of a main part of FIG. 5;

FIG. 7 is an explanatory diagram of a data array.

FIG. 8 is a configuration diagram of a main part of a flash unit.

FIG. 9 is a configuration diagram of a main part of a flash unit using an LCD.

FIG. 10 is a main part configuration diagram of a flash unit using an LED.

FIG. 11 is an explanatory diagram of a stripe pattern.

FIG. 12 is an explanatory diagram of a stripe pattern.

FIG. 13 is an explanatory diagram of a stripe pattern.

FIG. 14 is an explanatory diagram of a stripe pattern.

FIG. 15 is an explanatory diagram of a stripe pattern.

FIG. 16 is an explanatory diagram of a stripe pattern.

FIG. 17 is an explanatory diagram of stripe pattern projection.

FIG. 18 is a flowchart of post-processing of a captured image.

FIG. 19 is a flowchart of a shooting operation. The case where the image frequency is checked will be described.

FIG. 20 is a continuation of the flowchart in FIG. 19;

FIG. 21 is a flowchart for checking with image colors.

FIG. 22 is a flowchart of a photographing operation when pattern projection is performed by an LED.

FIG. 23 is an explanatory diagram of a conventional example.

FIG. 24 is an explanatory diagram of a conventional example.

FIG. 25 is an explanatory diagram of a conventional example.

FIG. 26 is an explanatory diagram of a conventional example.

FIG. 27 is an explanatory diagram of a conventional example.

[Explanation of symbols]

Reference Signs List 1 stripe pattern projection unit 2 camera body unit 3 imaging unit 4 grip unit 5 built-in flash 8 memory card 9 shutter button 10 LCD display unit 14 mode setting switch 15 lid 17 card loading room 18 battery loading room 31 optical viewfinder 210 VRAM 211 overall control Unit 301 zoom lens 302 imaging circuit 303 CCD color area sensor (photographing means) 304 dimming circuit 305 dimming sensor 501 stripe pattern projection unit 502 tripod screw 515 lid 518 battery loading chamber 521 to 526 key switch 530 mask unit 531 xenon tube 532 Concave lens 533 Projection window 534 Axis 540 Liquid crystal panel 541 Xenon tube 542 Lens 543 Pattern projection window 544 Cross prism 545 Liquid crystal panel 550 Mask (projection Means) 553 pattern projection window 560,570 Light emitting diode array (projection means, light emitting diode) AF distance measuring sensor M1 zoom motor M2 focus motor M3 mask motor PS power switch Rel release lever Z quadruple switch Z1-Z4 button Z5 3D flash power switch

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06F 15/64 325G F-term (Reference) 2F065 AA04 AA06 AA53 BB05 BB29 DD11 DD14 EE00 EE09 FF06 FF09 GG03 GG07 GG08 GG14 GG21 GG24 HH06 HH07 JJ03 JJ26 LL04 LL06 LL30 LL41 LL53 NN02 NN11 NN12 NN13 NN17 QQ00 QQ01 QQ03 QQ23 QQ24 QQ32 QQ33 SS02 SS13 5B047 AA07 AB02 BC07 BC11 CA19 5C054 AA05 CA03 EG03 CC04 ED03 CC04 ED03

Claims (5)

[Claims] 1. An image pickup device for photographing a photographing area, and a projecting means for projecting a pattern light onto the photographing region, wherein a projection pattern formed on a subject in the photographing region by the pattern light projected by the projecting means is formed. Shooting by shooting means 3
In the three-dimensional information input camera, the projection means includes a light emitting diode, and the light emitting diode is used as a light source for pattern light projection. 2. The apparatus according to claim 1, wherein said projection means further comprises a mask having a stripe-like transmittance or reflectance distribution, and projects the light emitted from said light-emitting diode as pattern light through said mask. 3. The three-dimensional information input camera according to 1. 3. The photographing means can photograph a photographing area when the light emitting diode emits light and projects pattern light, and when the light emitting diode does not emit light and does not project pattern light. The three-dimensional information input camera according to claim 2, characterized in that it is characterized by: 4. The three-dimensional information input camera according to claim 1, wherein said light-emitting diode has a stripe-shaped light-emission distribution, and said projecting means projects light emission of said light-emitting diode as pattern light. . 5. The light-emitting diode comprises R (red), G
The three-dimensional information input camera according to claim 1, wherein the three-dimensional information input camera has one or two or more emission colors of (green), B (blue), and W (white).