JP2003281509A - Photographing control method and three-dimensional shape input device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To minimize time for pattern photographing and time for texture photographing, and minimize influence such as movement of an object or shaking of a three-dimensional shape input device. <P>SOLUTION: In the photographing control method when photographing the object at least twice by a photographing device provided with an image pickup element, first photographing RK1 is carried out right before transfer of an accumulated charge in the image pickup element and second photographing RK2 is carried out right after the transfer of the accumulated charge of the image pickup element achieved by the first photographing. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a photographing control method for photographing an object at least twice by a photographing device equipped with an image pickup device, and such a photographing device and a projector for projecting pattern light. And a three-dimensional shape input device including.

[0002]

2. Description of the Related Art Conventionally, there has been known a three-dimensional shape input device which optically measures a three-dimensional shape of an object by projecting pattern light on the object. Usually, such a three-dimensional shape input device performs measurement of a three-dimensional shape by pattern projection (hereinafter sometimes referred to as “pattern imaging”) and imaging of a texture image (hereinafter sometimes referred to as “texture imaging”). Is also possible. An imaging device having an imaging device such as a CCD and a light projecting device for projecting pattern light are provided for pattern imaging and texture imaging. Usually, texture photography is performed after pattern photography. By pasting the texture image on the three-dimensional shape data, higher quality three-dimensional information about the object can be obtained.

FIG. 8 and FIG. 9 are diagrams for explaining the photographing timing according to the conventional photographing control method. FIG. 8 shows the case of a progressive type CCD, and FIG. 9 shows the case of an interlaced type CCD.

As shown in FIG. 8, the light receiving surface of the CCD is 2
The charges exposed during one vertical synchronization signal VD and accumulated in the CCD by the exposure are transferred to the registers of the CCD all at once at the timing based on the vertical synchronization signal VD immediately thereafter.

That is, immediately before one vertical synchronizing signal VD1, exposure is performed for an appropriate time T1, and pattern light is projected onto the object during the exposure. The accumulated charge by this is transferred to the register by the vertical synchronization signal VD1. This is pattern photography.

Immediately before the next vertical synchronizing signal VD2, exposure is performed for an appropriate time T2, and normal illumination light (for example, flash light) is projected onto the object during the exposure. . The accumulated charge by this is transferred to the register by the vertical synchronization signal VD2. This is texture photography.

As shown in FIG. 9, in the case of the interlace type CCD, after the exposure by the pattern projection is completed, the accumulated charge of the pixels on the even lines is the vertical synchronizing signal VD.
1, and the charges accumulated in the pixels on the odd lines are further transferred by the vertical synchronizing signal VD2. After that, exposure for texture photographing is performed, and similarly, the vertical sync signals VD3, 4 sequentially transfer the accumulated charges on the even lines and the odd lines.

[0008]

As described above, in the related art, the time T11 of one cycle or more of the vertical synchronizing signal VD is required from the start of pattern shooting to the end of texture shooting. Interlaced CCD
In the case of, the time T12 of 2 cycles or more is required. Therefore, the movement or change of the object during that period and the movement or shake of the three-dimensional shape input device pose a problem.

That is, when the object is, for example, a person or an animal, and the object moves between the timings of pattern photographing and texture photographing, a positional deviation occurs between the three-dimensional shape data and the texture image. However, the mapping accuracy of the texture image is reduced. Even if the object is a landscape, the object inside the object may move due to wind or vibration. In addition, when the three-dimensional shape input device is shaken, the positional displacement similarly occurs, and particularly in the case of the handheld three-dimensional shape input device, the positional displacement is likely to occur. As a result, the resulting three-dimensional model may be unnatural.

The present invention has been made in view of the above-mentioned problems, and it is possible to perform pattern photography and texture photography in the shortest possible time, and to avoid the influence of movement of an object or blurring of a three-dimensional shape input device. The aim is to suppress as much as possible.

[0011]

According to the method of the present invention, at least two objects are picked up by an image pickup device equipped with an image pickup device.
A photographing control method for photographing once, wherein the first photographing is performed immediately before the transfer of the accumulated charge in the image sensor, and the transfer of the accumulated charge of the image sensor obtained by the first photographing is completed. Immediately after taking the second shot.

A device according to the present invention is a three-dimensional shape input device equipped with an image pickup device having an image pickup device and a light projecting device for projecting pattern light onto an object. Immediately before, a first photographing control unit that controls the pattern light to be projected or to perform the first photographing without projecting light, and transfer of the accumulated charge of the image sensor obtained by the first photographing Immediately after the above, the second photographing control unit is controlled to perform the second photographing without or with projecting the pattern light.

[0013] Preferably, in the first photographing,
The pattern light is projected, and in the second shooting,
Illumination light is projected without projecting the pattern light. Further, the photographing apparatus is provided with a shutter that blocks light incident on the image sensor, and the first photographing control unit controls to open the shutter in the first photographing, and The 2-photographing control unit controls to open the shutter in the second photographing.

[0014]

1 is a block diagram showing a functional configuration of a three-dimensional shape input device 1 according to an embodiment of the present invention, and FIG. 2 is a photographing device 11 and a projection device of the three-dimensional shape input device 1. It is a perspective view which shows the structure of 12.

In FIG. 1, the three-dimensional shape input device 1 is
The imaging device 11, the projection device 12, the storage unit 13, the imaging control unit 14, the light projection control unit 15, the processing unit 16, and a display device, an operation input device, and the like (not shown).

As shown in FIG. 2, the projection device 12 includes a light source 121, a condenser lens system 122, a mask 123, a projection lens system 124 and the like. The light emitted from the light source 121 is projected on the object Q through the condenser lens system 122, the mask 123, and the projection lens system 124. A pattern for three-dimensional measurement is formed on the mask 123, and the pattern light forms an image on the object Q.

That is, the mask 123 has a pattern in which regions having a characteristic of transmitting only one or more specific wavelength bands are arranged according to a predetermined rule. The pattern is for determining the spatial position of the surface of the object Q by projecting it onto the object Q.
The content of the projected pattern can be changed by moving the pattern of the mask 123, exchanging a plurality of different patterns, or changing the image of the transmissive liquid crystal panel. The three-dimensional shape of the target object Q can be measured by capturing an image of the pattern-projected target object Q with the image capturing device 11. As such a pattern, various conventionally known patterns can be used.

The normal illumination light for simply illuminating the object Q is projected by allowing the mask 123 to transmit light uniformly over the entire surface or by retracting the mask 123 from the optical path. Is possible.

Similarly, in FIG. 2, a photographing device 11 includes a CCD (Charge Coupled Device) 11 as an image pickup device.
1, a light receiving lens system 112, and the like. CCD1
Reference numeral 11 receives the image light of the object and converts it into an electric signal corresponding to the image. A pattern image of the object Q is obtained when pattern projection is performed by the projection device 12, and a texture image of the object Q is obtained when pattern projection is not performed or simple illumination light is projected. .

The photographing device 11 and the projection device 12 are arranged at positions separated from each other by a predetermined distance d, and are fixed to the main body housing of the three-dimensional shape input device 1. Therefore, distortion corresponding to the three-dimensional shape of the target object Q appears in the pattern image captured by the imaging device 11. The three-dimensional shape of the object Q is measured by detecting the position and magnitude of the distortion generated in the pattern image. That is, based on the position on the pattern image, the projection angle of the fringes forming the pattern, the distance d, etc., the distance (3
The dimensional coordinate) is calculated. Thereby, three-dimensional shape data of the object Q is obtained. By pasting the texture image on the obtained three-dimensional shape data, high-quality three-dimensional information about the object Q can be obtained.

The photographing device 11 has a mechanical shutter 11 for mechanically blocking light incident on the CCD 111.
3 is provided. However, other types or types of shutters can be used as long as the exposure of the CCD 111 can be controlled. The operation of the CCD 111 in the photographing device 11 will be described in detail later.

Returning to FIG. 1, the storage unit 13 stores the photographing device 1
The image (image data) taken by 1 and other data are stored. The photographing control unit 14 controls the photographing device 11. For example, the exposure timing of the CCD 111, the transfer timing of the accumulated charge, the read timing of data, and the like are controlled. The light projection control unit 15 controls the projection device 12. For example, the presence / absence of light emission, the timing of light emission, the position of the pattern on the mask 123, the content of the pattern, the presence / absence of a pattern, etc. are controlled. The processing unit 16 controls the entire three-dimensional shape input device 1 and performs various calculations based on the data stored in the storage unit 13 to generate distance data of the object Q or its original data. To do. The generated data can be output to the outside through an interface (not shown) or a recording medium, and can be displayed by a display device (not shown). The processing unit 16 can be configured by a CPU, ROM, RAM or the like, or a personal computer or the like.

Based on the data output from the three-dimensional shape input device 1, the three-dimensional shape data of the object Q is calculated by a personal computer or the like in which an appropriate program is installed. Note that it is also possible to calculate the three-dimensional shape data in the three-dimensional shape input device 1 by installing a known program for calculating the three-dimensional shape data in the processing unit 16.

Next, the operation of the CCD 111 will be described. FIG. 3 is a diagram showing an outline of a general CCD circuit, and FIG.
Is a diagram showing the relationship between the vertical synchronizing signal VD of the CCD and the horizontal synchronizing signal HD, FIG. 5 is a diagram showing the relationship between the horizontal synchronizing signal HD of the CCD and the clock signal CLK, and FIG. 6 is a diagram showing the shooting control timing of the CCD. FIG. 7 is a diagram showing the photographing control timing of the CCD of another embodiment.

The CCD is classified into a frame transfer type and an interline type, but the interline type is commonly used at present. Due to structural restrictions, the interline CCD is a progressive CCD for low resolution and an interlaced CCD for high resolution. The CCD 111 shown in this embodiment is an interline type, and FIG. 6 shows a case where the CCD 111 is a progressive CCD, and FIG. 7 shows a case where the CCD 111 is an interlaced CCD.

As shown in FIG. 3, the CCD 111 comprises a photoelectric conversion element 111a arranged on the array, a vertical transfer path 111b and a horizontal transfer path 111c for carrying photoelectrically converted charges (stored charges). .

Referring also to FIGS. 4 to 6, when the photoelectric conversion element 111a receives the light from the object Q, charges corresponding to the intensity of the received light are generated, and the generated charges are exposed. Accumulates over time. The accumulated charges (accumulated charges) are simultaneously transferred to the registers of the vertical transfer path 111b in synchronization with the vertical synchronization signal VD. In that case, it may be synchronized with the vertical synchronization signal VD itself or may be synchronized with a synchronization signal or a trigger signal generated based on the vertical synchronization signal VD.

The charges transferred to the vertical transfer path 111b are
By the next horizontal synchronizing signal HD, it is shifted toward the horizontal transfer path 111c by one line in the horizontal direction. The charges that have moved to the horizontal transfer path 111c are sequentially shifted to the left in the drawing by the clock signal CLK, and an electric signal (exposure signal) for pixels for one line is extracted.

Until the next horizontal synchronizing signal HD arrives, that is, during the time Tc, the horizontal synchronizing signals HD are input by the number of lines of the CCD 111, and the charge of the horizontal transfer path 111c is repeatedly read by the clock signal CLK. Then, the electric signals for all the pixels of the CCD 111 are extracted. Controls per se for the transfer of such accumulated charges are known per se.

As shown in FIG. 6, the CCD 111 is provided between one vertical synchronizing signal VD and the next vertical synchronizing signal VD.
There is a time Tc during which exposure can be performed to capture an image of one frame. Although the exposure may be performed over the entire time Tc, normally, the exposure is performed for a time corresponding to the brightness of the object Q in the time Tc. In the present embodiment, the time T that is a part of the time Tc and is adjacent to the vertical synchronization signal VD.
Only in No. 1, the first actual exposure RK1 is performed for pattern imaging. The pattern light is projected from the projection device 12 during the time T1 during which the exposure RK1 is performed. An accumulated charge corresponding to the pattern image is generated in each pixel of the CCD 111. This accumulated charge is transferred to the register by the next vertical synchronization signal VD1.

Immediately after the transfer of the accumulated charges by the vertical synchronizing signal VD1 is finished, the second actual exposure RK2 for texture photographing is performed only for the time T2. Exposure RK
Illumination light is projected from the projection device 12 during the time T2 during which 2 is performed. When the object Q is bright, the illumination light may not be projected. The CCD 111 has an object Q
An accumulated charge corresponding to the natural image of is generated. This accumulated charge is transferred to the register by the next vertical synchronizing signal VD2. The mechanical shutter 113 controls the time of the exposures RK1 and RK2.

The time T3 from the start of pattern shooting to the end of texture shooting is the sum of the time T1 required for pattern shooting, the time T2 required for texture shooting, and the transfer time Tt due to the vertical synchronizing signal VD. , Much shorter than the conventional one.

That is, to give a specific numerical example, for example, when a progressive CCD of 1.5 million pixels is used, it takes about 140 ms in the prior art to shoot the entire image, but in the present embodiment, up to about 40 ms. It can be shortened. As a result, when a person's face is photographed as the object Q, the 3D shape data and the texture image do not match due to the person moving or the facial expression changing, for example, the eyes are open in the 3D shape data. In the texture image, the possibility that the eyes are closed is greatly reduced. In addition, the three-dimensional shape input device 1
It is possible to suppress the occurrence of similar positional displacement due to camera shake of the three-dimensional shape input device 1 when the user holds an image with his / her hand and takes an image. This makes it possible to generate a natural three-dimensional model with no discomfort.

As shown in FIG. 7, in the case of the interlace type CCD 111, after the first exposure RK1 for pattern photographing is completed, the accumulated charge of the pixels on the even lines is changed by the vertical synchronizing signal VD1. The charges accumulated in the pixels on the odd lines are further transferred by the vertical synchronizing signal VD2. After that, the second for texture shooting
The exposure RK2 is performed twice, and similarly, the vertical synchronizing signal VD3,
By 4, the accumulated charges of the even line and the odd line are sequentially transferred.

The time T4 from the start of the pattern photographing to the end of the texture photographing is the time T1 required for the pattern photographing, the time T2 required for the texture photographing, the transfer time Tt twice for the vertical synchronizing signal VD, and one field. The exposure time Tc is added, which is considerably shorter than the conventional one.

Therefore, the interlaced CCD
Even in the case of 111, it is possible to reduce the positional deviation between the three-dimensional shape data and the texture image due to the movement of the object Q and the shake of the three-dimensional shape input device 1.

In FIG. 6 and FIG. 7 showing the above embodiment, the timing of ending the exposure RK1 is the vertical synchronizing signal VD.
Although it coincides with the timing of the start of 1, this does not necessarily have to coincide. For example, the exposure RK1 and the vertical synchronizing signal VD1 may be in a state of being slightly apart from each other, or may be in a state of overlapping with each other. The same applies to the start timing of the exposure RK2 and the end timing of the vertical synchronization signal VD1. The detailed relationship between these timings can be determined according to the specifications or operating characteristics of the CCD 111. Further, if the exposure may be performed during the transfer time Tt by the vertical synchronizing signal VD, the mechanical shutter 113 may also be performed during the transfer time Tt.
It is possible to control so that the exposures RK1 and RK2 can be made continuous by opening.

In the above embodiment, C is used as the image sensor.
Although the CD is used, a CMOS sensor can be used similarly to the CD and the same effect can be obtained. Other types of image sensor configured to transfer the accumulated charge based on the vertical synchronization signal VD can also be used. CCD1
Although the exposure time of 11 is controlled by the mechanical shutter 113, the mechanical shutter 113 may be omitted when, for example, photographing is performed in a pitch dark room.

The pattern exposure was performed in the first exposure RK1 and the texture exposure was performed in the second exposure RK2, but the order may be reversed. In addition, the configuration, operation, timing, etc. of the whole or each part of the imaging device 11, the projection device 12, and the three-dimensional shape input device 1 can be appropriately changed in accordance with the spirit of the present invention.

[0040]

According to the present invention, the pattern photographing and the texture photographing can be performed in the shortest possible time, and the influence of the movement of the object and the blurring of the three-dimensional shape input device can be suppressed as much as possible.

[Brief description of drawings]

FIG. 1 is a block diagram showing a functional configuration of a three-dimensional shape input device according to an embodiment of the present invention.

FIG. 2 is a perspective view showing configurations of a photographing device and a projection device.

FIG. 3 is a diagram showing an outline of a general circuit of a CCD.

FIG. 4 is a diagram showing a relationship between a vertical synchronizing signal and a horizontal synchronizing signal of a CCD.

FIG. 5 is a diagram showing a relationship between a horizontal synchronizing signal of a CCD and a clock signal.

FIG. 6 is a diagram showing CCD image capturing control timing.

FIG. 7 is a diagram showing a CCD image pickup control timing of another embodiment.

FIG. 8 is a diagram illustrating a shooting timing according to a conventional shooting control method.

FIG. 9 is a diagram illustrating a shooting timing according to a conventional shooting control method.

[Explanation of symbols]

1 3D shape input device 11 Imaging device 12 Projector 14 Shooting control unit (first shooting control unit, second shooting control unit) 15 Projection control unit (first shooting control unit, second shooting control unit) 111 CCD (imaging device) 113 Mechanical shutter Q target

Continued front page    F term (reference) 2F065 AA53 DD14 FF04 FF09 HH07                       JJ26 LL30 QQ24                 5B047 AA07 BB04 BC11 CB15                 5C022 AB15 AB55 AC42 AC52                 5C054 AA05 CB03 CH02 FC15 HA05

Claims (4)

[Claims] 1. A photographing control method for photographing an object at least twice with a photographing device having an image pickup device, wherein the first image pickup is performed immediately before the transfer of accumulated charges in the image pickup device. The image capturing control method, wherein the second image capturing is performed immediately after the transfer of the accumulated charges of the image sensor obtained by one image capturing is completed. 2. A three-dimensional shape input device comprising an image pickup device having an image pickup device and a light projecting device for projecting pattern light onto an object, wherein the pattern is provided immediately before the transfer of accumulated charges in the image pickup device. Immediately after the transfer of the accumulated charge of the image pickup device obtained by the first image capturing, a first image capturing control unit that controls to perform the first image capturing with or without projecting light. And a second image capturing control unit that controls the second light image capturing without or with projecting the pattern light, and a three-dimensional shape input device. 3. The pattern light is projected in the first shooting, and the illumination light is projected in the second shooting without projecting the pattern light. 3D shape input device. 4. The shutter is provided in the image pickup device to block light incident on the image pickup device, and the first image pickup control unit controls to open the shutter in the first image pickup. The three-dimensional shape input device according to claim 2 or 3, wherein the second photographing control unit controls to open the shutter in the second photographing.