JP4451968B2 - 3D image input device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional image input apparatus that detects a three-dimensional shape or the like of an object to be measured using a light propagation time measurement method.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, there has been known a three-dimensional image input apparatus that detects a three-dimensional image of a subject by irradiating a subject with pulse-modulated laser light and converting reflected light according to the distance to the subject into an electrical signal by a two-dimensional CCD sensor. It has been. According to this apparatus, only a three-dimensional shape from one direction of the subject can be detected in one detection operation. Therefore, in order to detect the three-dimensional shape of the entire circumference of the subject, a plurality of three-dimensional images detected from different directions are synthesized. There is a need to.
[0003]
Conventionally, synthesis of a three-dimensional image is performed by inputting a three-dimensional image detected by a three-dimensional image input device into a computer and displaying it on a computer display. In other words, the synthesized three-dimensional image is displayed for the first time on the computer display, and the three-dimensional image that is insufficient for synthesis cannot be recognized during the detection operation, and any portion of the three-dimensional image is insufficient on the display screen. It was difficult to confirm.
[0004]
[Problems to be solved by the invention]
An object of the present invention is to provide a three-dimensional image input device that synthesizes and displays a three-dimensional image so that a three-dimensional image necessary for detecting a three-dimensional shape of a subject can be easily confirmed.
[0005]
[Means for Solving the Problems]
The three-dimensional image input apparatus according to the present invention is a camera capable of photographing a three-dimensional image formed by distance data indicating the distance to each point on the surface of the subject, and obtained by photographing the subject from different directions. Coordinate conversion means for performing coordinate conversion to synthesize a plurality of three-dimensional images to form one three-dimensional image, and image display means for displaying the plurality of three-dimensional images on the screen, It is controlled that the image display means is controlled by a real-time mode in which captured 3D images are combined for each shooting operation or in a batch connection mode in which captured 3D images are displayed simultaneously and selected images are combined. Features.
[0006]
Preferably, the image display means displays a three-dimensional image on the screen as if the subject is viewed along the optical axis direction of the camera in the first photographing operation.
[0007]
Preferably, the image display means includes means operable to display a three-dimensional image displayed according to the optical axis direction of the camera on the screen as if the subject is viewed from a direction different from the optical axis direction of the camera. .
[0008]
For example, the coordinate conversion means uses the three-dimensional image in the first photographing operation and the three-dimensional image in the second photographing operation as the origin of the camera position in the first photographing operation, and the optical axis direction of the camera as one coordinate axis. Convert to a coordinate system.
[0009]
Preferably, the coordinate conversion means detects the angular velocity of the camera from the position of the camera in the first shooting operation to the position of the camera in the second shooting operation, and the position of the camera in the first shooting operation. And an acceleration detecting means for detecting the acceleration of the camera from the camera position to the camera position in the second photographing operation.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a perspective view of a camera provided with a three-dimensional image input apparatus according to an embodiment of the present invention.
[0011]
On the front surface of the camera body 10, a finder window 12 is provided at the upper left of the taking lens 11, and a strobe 13 is provided at the upper right. On the upper surface of the camera body 10, a light emitting device (light source) 14 that irradiates laser light that is distance measuring light is disposed directly above the photographing lens 11. A release switch 15 and a liquid crystal display panel 16 are provided on the left side of the light emitting device 14, and a real-time mode setting dial 17 and an operation switch 18 are provided on the right side. A card insertion slot 19 for inserting a recording medium such as an IC memory card is formed on the side surface of the camera body 10, and a video output terminal 20 and an interface connector 21 are provided.
[0012]
FIG. 2 is a rear view of a camera provided with the three-dimensional image input apparatus of FIG. A screen display LCD panel 37 is provided on the back of the camera body 10. An operation button 61, a menu button 62, and a set button 63 are disposed on the right side of the screen display LCD panel 37, and a finder eyepiece 12b is provided on the upper portion of the screen display LCD panel 37.
[0013]
FIG. 3 is a block diagram showing a circuit configuration of the camera shown in FIGS.
A diaphragm 25 is provided in the photographic lens 11. The opening degree of the diaphragm 25 is adjusted by the iris drive circuit 26. The focus adjustment operation and zooming operation of the photographic lens 11 are controlled by the lens driving circuit 27.
[0014]
An imaging device (CCD) 28 is disposed on the optical axis of the photographing lens 11. A subject image is formed on the CCD 28 by the photographing lens 11 and a charge corresponding to the subject image is generated. Operations such as charge accumulation operation and charge read operation in the CCD 28 are controlled by the CCD drive circuit 30. The charge signal read from the CCD 28, that is, the image signal is amplified by the amplifier 31 and converted from an analog signal to a digital signal by the A / D converter 32. The digital image signal is subjected to processing such as gamma correction in the imaging signal processing circuit 33 and temporarily stored in the image memory 34. The iris drive circuit 26, lens drive circuit 27, CCD drive circuit 30, and imaging signal processing circuit 33 are controlled by a system control circuit 35.
[0015]
The image signal is read from the image memory 34 and supplied to the LCD drive circuit 36. The LCD drive circuit 36 operates in accordance with the image signal, whereby an image corresponding to the image signal is displayed on the image display LCD panel 37.
[0016]
Further, the image signal read from the image memory 34 is sent to the TV signal encoder 38 and can be transmitted to the monitor device 39 provided outside the camera body 10 via the video output terminal 20. The system control circuit 35 is connected to the interface circuit 40, and the interface circuit 40 is connected to the interface connector 21. Therefore, the image signal read from the image memory 34 can be transmitted to the computer 41 connected to the interface connector 21. Further, the system control circuit 35 is connected to the image recording device 43 via the recording medium control circuit 42. Therefore, the image signal read from the image memory 34 can be recorded on a recording medium M such as an IC memory card attached to the image recording device 43.
[0017]
A light emitting element control circuit 44 is connected to the system control circuit 35. The light emitting device 14 is provided with a light emitting element 14 a and an illumination lens 14 b, and the light emitting operation of the light emitting element 14 a is controlled by a light emitting element control circuit 44. The light emitting element 14a irradiates laser light that is distance measuring light, and this laser light is irradiated to the object to be measured through the illumination lens 14b. The light reflected from the measurement object enters the photographing lens 11. By detecting this light by the CCD 28, a three-dimensional image of the object to be measured is measured as will be described later.
[0018]
The camera body 10 is provided with an angular velocity sensor 46 and an acceleration sensor 48. The angular velocity sensor 46 detects rotational angular velocities about three axes (X axis, Y axis, Z axis) orthogonal to each other, and the acceleration sensor 48 detects movement acceleration of each of the three orthogonal axes. Data detected by the angular velocity sensor 46 and the acceleration sensor 48 are input to an angular velocity detection circuit 47 and an acceleration detection circuit 49, respectively, and movement of the camera body 10 is detected as three-dimensional angular acceleration data. The three-dimensional angular acceleration data is input to the system control circuit 35, and changes in the camera position, posture, and lens orientation are determined based on this data.
[0019]
Connected to the system control circuit 35 are a switch group 45 including a release switch 15, a real-time mode setting dial 17, an operation switch 18, an operation button 61, a menu button 62, and a set button 63, and a liquid crystal display panel (display element) 16. ing.
[0020]
Next, the principle of distance measurement in this embodiment will be described with reference to FIGS. In FIG. 5, the horizontal axis represents time t.
[0021]
The distance measuring light output from the distance measuring device B is reflected by the subject S and received by a CCD (not shown). The distance measuring light is pulsed light having a predetermined pulse width H. Therefore, the reflected light from the subject S is also pulsed light having the same pulse width H. The rise of the reflected light pulse is delayed by a time δ · t (δ is a delay coefficient) from the rise of the ranging light pulse. The distance measuring light and the reflected light have traveled twice as much distance r between the distance measuring device B and the subject S, so that the distance r is r = δ · t · C / 2.
Is obtained. However, C is the speed of light.
[0022]
For example, when the reflected light is detected from the rising edge of the ranging light pulse and switched to the undetectable state before the reflected light pulse falls, that is, when the reflected light detection period T is provided, The received light amount A in the reflected light detection period T is a function of the distance r. That is, the received light amount A decreases as the distance r increases (the time δ · t increases).
[0023]
In the present embodiment, by utilizing the above-described principle, each point on the surface of the subject S is detected from the camera body 10 by detecting the received light amount A in each of a plurality of photodiodes provided in the CCD 28 and two-dimensionally arranged. 3D image data indicating a three-dimensional shape of the subject S is input in a lump.
[0024]
Next, a description will be given of a synthesis mode in which a subject is photographed from a plurality of directions and distance data detected by photographing from each direction is synthesized.
[0025]
FIG. 6 shows an example of the photographing operation in the composition mode, and a building as a subject is photographed from three places using one camera. First photographing operation is taken in front of the building from the point G _1. Second imaging operation has taken the right side of the building from the point G _2, the third photographing operation are photographing after left oblique building from point G _3. The half lines Lp ₁ , Lp ₂ , and Lp ₃ represent the direction in which the camera lens is directed in photographing the points G ₁ , G ₂ , and G ₃ , and coincide with the optical axis of the camera at the time of photographing. The points G ₁ , G ₂ , and G ₃ coincide with the focal point of the photographing optical system of the camera at the time of photographing.
[0026]
The three-dimensional shape of the subject detected by photographing from one point (one direction) is only for a part of the subject. For example, in the first photographing operation, only the three-dimensional shape of the front part of the building is detected. In order to detect the three-dimensional shape of the entire building, the building is photographed from a plurality of directions as shown in the figure, and the three-dimensional shape of the building detected by photographing each point G ₁ , G ₂ , G ₃ is combined into one. Must be synthesized. In the synthesis mode, the position of each point G ₂ , G ₃ , the posture of the camera, and the direction in which the optical axis is directed are detected using the angular velocity sensor 46 and the acceleration sensor 48 with the point G ₁ as a reference, and these data are obtained. Based on the points G ₁ , G ₂ , and G ₃ , the coordinate data relating to the three-dimensional shape of the building detected is converted into one coordinate system. As a result, the three-dimensional shape of the subject detected separately at each of the points G ₁ , G ₂ , and G ₃ is integrally represented by one coordinate system. In the present embodiment, the coordinate system when the first shooting is performed after the synthesis mode is set, that is, the coordinate system having the point G ₁ as the coordinate origin and the optical axis direction Lp ₁ as the Z axis is adopted as the world coordinate system. .
[0027]
With reference to FIGS. 7 to 11, operations of photographing, combining, and displaying a three-dimensional image according to the present embodiment will be described.
FIG. 7 is a diagram showing a menu screen displayed on the screen display LCD panel 37. The menu screen is displayed on the screen display LCD panel 37 automatically when the camera is turned on or by turning on the menu button 62 even after the start of detection of the three-dimensional image.
[0028]
On the menu screen, the characters “2D shooting” 37a indicating the two-dimensional image shooting mode, the characters “3D shooting” 37b indicating the three-dimensional image shooting mode, the characters “setting” 37c indicating the image setting mode, and the characters “setting” indicating the batch connection mode are displayed. “3D batch connection” 37d is displayed. These items are selected by operating the operation button 61 and are determined by the set button 63. The two-dimensional image photographing mode is a mode for photographing a two-dimensional image obtained through the photographing lens 11. The three-dimensional image photographing mode is a mode for photographing a three-dimensional image using the above-described principle of distance measurement. The image setting mode is a mode for adjusting the resolution of the image. The collective connection mode is a mode related to synthesis of a three-dimensional image described later.
[0029]
FIGS. 8 and 9 are flowcharts of the photographing operation that is executed when two-dimensional image photographing or three-dimensional image photographing is selected on the menu screen.
[0030]
In step 100, the variable n is set to the initial value 0. If it is confirmed in step 101 that the release switch 15 is fully pressed, step 102 is executed, the normal photographing operation (CCD video control) by the CCD 28 is set to the on state, and a two-dimensional image of the subject is photographed. It is detected as image data. The detected image data is temporarily stored in the image memory 34 in step 103. When the two-dimensional image shooting mode is selected, the image is recorded in the data file of the image recording medium M via the image recording device 43, and the shooting operation ends.
[0031]
After the CCD video control is set to the OFF state in step 104, the vertical synchronizing signal is output and the distance measuring light control is started in step 105. That is, the light emitting device 14 is driven, and pulsed ranging light is intermittently output. Next, step 106 is executed, and detection control by the CCD 28 is started.
[0032]
In step 107, it is determined whether one field period has ended since the start of the distance information detection operation, that is, whether a new vertical synchronization signal has been output. When one field period ends, the process proceeds to step 108, and the signal charge of the integrated distance information is output from the CCD 28. This signal charge is temporarily stored in the image memory 34 in step 109.
[0033]
In step 110, the distance measuring light control is switched to the off state, and the light emitting operation of the light emitting device 14 is stopped. In step 111, distance data calculation processing is performed and temporarily stored in the image memory 34.
[0034]
In step 112, for example, distortion correction is performed as calibration of the acquired distance data, and distortion of the data of the three-dimensional image is corrected.
[0035]
In step 113, the coordinates of the position of the subject with the shooting point as the origin are calculated from the distance data calculated in step 111, and are temporarily stored in the image memory 34 as coordinate data.
[0036]
In step 114, coordinate conversion parameters relating to the camera position, posture, and lens direction are calculated from the angular velocity and acceleration data detected by the angular velocity sensor 46 and the acceleration sensor 48 and temporarily stored in the image memory 34.
[0037]
In step 115, the variable n is incremented by one. That is, it is set what number each detected data is related to the three-dimensional image taken.
[0038]
In step 116, the image data, the coordinate data, the coordinate conversion parameter, and the numerical value of the variable n are recorded in the data file of the image recording medium M via the image recording device 43.
[0039]
In step 117, it is determined whether the real-time mode is set. The real-time mode is a mode type related to the synthesis of a three-dimensional image, and is a mode in which detected coordinate data is synthesized and displayed for each photographing operation. The real-time mode is set by manually setting the real-time mode setting dial 17 to the on state.
[0040]
When it is determined that the real-time mode is not set, the process proceeds to step 118 and it is determined whether or not to end the shooting. The end of shooting is selected by setting the operation switch 18 to OFF. If the end of shooting is not selected, the process returns to step 101 and waits until the next shooting is executed, that is, until the release switch is pressed.
[0041]
When it is determined in step 117 that the real-time mode is set, the process proceeds to step 201, and the process proceeds to real-time mode processing related to the synthesis of a three-dimensional image.
[0042]
In step 201, it is determined whether n = 1. That is, it is determined whether or not the currently performed 3D image capturing operation is based on the first (first) capturing operation. When n = 1, the process proceeds to step 202, and the coordinate data obtained in step 113 is determined as the first coordinate data.
[0043]
If it is determined in step 201 that n = 1 is not satisfied, step 203 is executed, the coordinate data obtained in step 113 is determined as second coordinate data, and the coordinate transformation parameters obtained in step 114 are used. , Converted to the world coordinate system. The world coordinates are based on the position of the camera when the first coordinate data is obtained. In step 204, the second coordinate data converted into the world coordinate system is combined with the first coordinate data. One of the overlapping portions of the first coordinate data and the second coordinate data is deleted.
[0044]
In step 205, the synthesized coordinate data is displayed on the screen display LCD panel 37 as a three-dimensional image with a line of sight in the optical axis direction at the time of the first photographing operation. In step 206, it is determined whether or not to end shooting. When the end is selected, the real-time mode routine ends. On the other hand, if the end of shooting is not selected in step 206, the process returns to step 101 and waits until the next shooting is executed, that is, until the release switch is pressed.
[0045]
Next, the case where the collective connection mode is selected will be described with reference to FIGS. In the synthesis of the 3D image in the present embodiment, in addition to the above-described real-time mode, a list of 3D images detected by shooting from each shooting point is displayed at the same time and selected from the displayed 3D images. There is a batch connection mode that combines only the selected images.
[0046]
When the batch connection mode is selected on the menu screen (see FIG. 7), a list of three-dimensional images is displayed in step 301. As shown in FIG. 11, the list display of 3D images displays all 3D images obtained at each shooting point from the start of 3D image shooting until the batch connection mode is selected. In the example of FIG. 11, the three-dimensional image capturing program of FIG. 8 is executed six times, and the first to sixth three-dimensional images are detected and displayed on the menu screen. Numbers {circle around (1)} to {circle around (6)} corresponding to the order of photographing are given to the first to sixth three-dimensional images.
[0047]
In step 302, a three-dimensional image to be synthesized is selected by an operator's operation. A three-dimensional image to be synthesized can be selected from among the three-dimensional images displayed in a list by operating the operation button 61, and a three-dimensional image to be synthesized can be determined by operating the set button 63. In the example of FIG. 11, the three-dimensional images (1) and (5) are selected. The selected three-dimensional image is displayed surrounded by a thick line.
[0048]
In step 303, the selected 3D image is synthesized. That is, the selected three-dimensional image is synthesized by being coordinate-converted into world coordinates in the same manner as in the synthesis in the real-time mode. Here, in the world coordinates, the position of the camera in the first shooting operation is the origin, and the optical axis of the camera in the first shooting operation is the Z axis.
[0049]
In step 304, the list display of 3D images is erased from the screen display LCD panel 37 screen, and the synthesized 3D images are displayed. The synthesized three-dimensional image is standardized to world coordinates, and is displayed with a line of sight in the optical axis direction in the first photographing operation.
[0050]
In step 305, it is determined whether to end the batch connection mode. To end the collective connection mode, the operation switch 18 is set to OFF. If the end of the collective connection mode is not selected, the process returns to step 301 to delete the composite 3D image display and display a list of 3D images captured at each imaging point again.
[0051]
As described above, the three-dimensional image synthesized in the real-time mode and the collective connection mode is displayed on the screen display LCD panel 37 so as to coincide with the camera optical axis direction line of sight when the first three-dimensional image is captured. The Further, the synthesized three-dimensional image can be rotated by operating the operation button 61 and displayed from another direction. By displaying the synthesized image from a different angle, it is possible to discriminate between a portion where data has already been captured and a portion where data has not been captured. That is, since a portion not photographed is not displayed, it is easy to check a portion lacking data.
[0052]
In this embodiment, the angular velocity / acceleration sensor is used to detect the position, orientation, and orientation of the camera. However, for example, a magnetic sensor or a multi-axis arm may be used.
[0053]
The three-dimensional image synthesized in the real-time mode is displayed with the world coordinates as the origin, that is, displayed on the screen display LCD panel 37 in the optical axis direction line of sight of the camera in the first shooting operation, but the last (latest) shooting operation May be displayed with a line of sight in the optical axis direction of the camera. In this case, after the three-dimensional image is converted into world coordinates and synthesized, it is converted into a coordinate system in which the camera position in the last shooting operation is the origin and the optical axis direction of the camera in the last shooting operation is the Z axis. Displayed on the screen.
[0054]
The synthesis and display of the tertiary image may be displayed on the monitor device 39 by the operation of the computer 41.
[0055]
【The invention's effect】
As described above, according to the present invention, it is possible to obtain a three-dimensional image input apparatus that synthesizes and displays so that a three-dimensional image necessary for detecting a three-dimensional shape of a subject can be easily confirmed.
[Brief description of the drawings]
FIG. 1 is a perspective view of a camera provided with a three-dimensional image input apparatus according to an embodiment of the present invention.
FIG. 2 is a rear view of a camera provided with a three-dimensional image input apparatus according to an embodiment of the present invention.
3 is a block diagram showing a circuit configuration of the camera shown in FIGS.
FIG. 4 is a diagram for explaining the principle of distance measurement using distance measuring light.
FIG. 5 is a diagram showing a light amount distribution received by ranging light, reflected light, gate pulse, and CCD.
FIG. 6 is a diagram for describing a method of photographing a subject from a plurality of directions in a synthesis mode and detecting an overall three-dimensional shape of the subject.
FIG. 7 is a diagram showing a menu screen.
FIG. 8 is a flowchart of a three-dimensional image detection operation.
FIG. 9 is a flowchart of a three-dimensional image composition operation in a real time mode.
FIG. 10 is a flowchart of a 3D image synthesis operation in a batch connection mode.
FIG. 11 is a diagram showing a screen for displaying a list of three-dimensional images in the collective connection mode.
[Explanation of symbols]
10 Camera body 37 Screen display LCD panel 61 Operation buttons

Claims (5)

A camera- type three-dimensional image input device capable of photographing a three-dimensional image formed by distance data indicating a distance to each point on the surface of a subject,
Coordinate transformation means for performing coordinate transformation to synthesize a plurality of three-dimensional images obtained by photographing the subject from different directions to form one three-dimensional image;
Image display means for displaying on the screen a 3-dimensional image,
A real-time mode for performing the coordinate transformation and synthesizing the photographed three-dimensional image for each photographing operation ;
A plurality of three-dimensional image shooting displayed simultaneously on the screen, three-dimensional image input apparatus, characterized in that it comprises a collective connection mode for synthesizing subjected to the coordinate transformation to the three-dimensional image selected. Wherein the real-time mode, according to claim 1, wherein the camera of the 3-dimensional image as seen through the subject along the optical axis direction in the first photographing operation is characterized Rukoto displayed on the screen 3D image input device. In the real-time mode, the three-dimensional image displayed in accordance with the optical axis direction before Symbol camera, operable to display on the screen as viewed the object from a direction different from the optical axis of the camera The three-dimensional image input device according to claim 2, further comprising: It said coordinate transformation means, a three-dimensional image image in the second shooting operation, the position of the camera in the first imaging operation as an origin, the first one coordinate axis direction of the optical axis of the camera in photographing operation and The three-dimensional image input apparatus according to claim 1, wherein the three-dimensional image input apparatus is converted into a coordinate system.   In the first imaging operation, the coordinate conversion unit detects an angular velocity of the camera from the position of the camera in the first imaging operation to the position of the camera in the second imaging operation. The three-dimensional image input device according to claim 1, further comprising acceleration detection means for detecting acceleration of the camera from the position of the camera to the position of the camera in the second photographing operation.

Images