JP4306006B2 - Three-dimensional data input method and apparatus - Google Patents

Description

【００６１】
回転マトリックスＲはＪを最小にするように決定する。
【００６２】
【数２】

【００６３】
この計算方法については、文献「Ｆａｕｇｅｒａｓ ａｎｄ Ｈｅｂｅｒｔ，１９８６ Ｏ．Ｆａｕｇｅｒａｓ ａｎｄ Ｍ．Ｈｅｂｅｒｔ．Ｔｈｅ ｒｅｐｒｅｓｅｎｔａｔｉｏｎ，ｒｅｃｏｇｎｉｔｉｏｎ ａｎｄ ｌｏｃａｔi ｎｇｏｆ ３−Ｄ ｏｂｊｅｃｔｓ．Ｉｎｔｅｒｎａｔｉｎｏａｌ Ｊｏｕｒｎａｌ ｏｆ Ｒｏｂｏｔｉｃｓ Ｒｅｓｅａｒｃｈ ５（３）：２７−５２，１９８６．」を参照することができる。
【００６４】
以上のようにして求められた平行移動量Ｔ及び回転マトリックスＲを用いて第２の３次元データの各点Ｐｉの座標変換を次の（１０）式により行う。
Ｐ’ｉ＝Ｒ×（Ｐｉ一Ｔ） …（１０）
上述の座標変換を行なうことにより、第１の３次元データと第２の３次元データとの位置合わせが可能となる。
【００６５】
上述の実施形態において、案内画像ＧＰ及びモニタ画像ＱＰを３次元カメラ２と一体となったファインダー２１上に表示させたが、３次元カメラ２と別体化したファインダーやホスト３におけるディスプレイ３ｂに表示させることも可能である。
【００６６】
上述の実施の形態において、ファインダー２１として、ハーフミラーなどを用いたシースルー方式の表示装置を用いてもよい。これにより、３次元カメラ２の消費電力が節約できる。
【００６７】
上述の実施の形態においては、撮影した被写体Ｑの３次元データに基づいて案内画像ＧＰが生成されるため、案内画像ＧＰを予め作成する手間が省けるとともに案内画像ＧＰを格納しておくためのメモリが最小限で済む。
【００６８】
上述の実施形態において、計測システム１、３次元カメラ２の各部又は全体の構成、形状、配置、回路、処理形態などは、本発明の主旨に沿って適宜変更することができる。
【００６９】
【発明の効果】
本発明によると、被写体の３次元データの入力を簡単に迅速に且つ正確に行うことができ、被写体の全周又は所定範囲における欠落のない合成画像を容易に取得することが可能となる。
【図面の簡単な説明】
【図１】本発明に係る３次元カメラを用いた計測システムの構成図である。
【図２】３次元カメラの外観を示す図である。
【図３】３次元カメラの機能構成を示すブロック図である。
【図４】３次元カメラによって被写体の全周又は所定範囲の３次元データを入力する具体的な方法を示す図である。
【図５】重合わせ画像発生部によるモニタ画像及び案内画像の生成の過程を説明するための図である。
【図６】ファインダー上に表示された被写体のモニタ画像及び案内画像を示す図である。
【図７】３次元データの入力動作及び処理を示すフローチャートである。
【図８】スリット光投影法を適用した３次元カメラの入力原理を示す図である。
【符号の説明】
２ ３次元カメラ（３次元データ入力装置）
２１ ファインダー（モニタ画面）
６３ メモリ（記憶手段）
７３ 重心演算回路（３次元形状モデル画像生成手段）
７６ 表示制御部（表示指示手段）
Ｑ 被写体
ＧＰ 案内画像
ＱＰ モニタ画像（画像）[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional data input method and apparatus for inputting three-dimensional data of a subject.
[0002]
[Prior art]
Optical three-dimensional data input devices (three-dimensional cameras) can be used for data input to CG and CAD systems, body measurements, robot visual recognition, etc. Has been. As a measurement method suitable for such a three-dimensional data input apparatus, a slit light projection method (also called a light cutting method) is known.
[0003]
FIG. 8 is a diagram showing the input principle of the three-dimensional camera 80 to which the slit light projection method is applied.
In FIG. 8, the three-dimensional camera 80 includes a light projecting unit 81 and a light receiving unit 82. The light projecting unit 81 irradiates slit light S having a linear cross section. The light receiving unit 82 includes an imaging surface 83 and an imaging lens (not shown). The light projecting unit 81 and the light receiving unit 82 are normally integrated into one housing in a state where they are separated from each other by a predetermined dimension.
[0004]
The subject Q1 is irradiated with the slit light S from the light projecting unit 81, and the reflected light is captured as a slit image on the imaging surface 83. The spatial coordinates of the point p on the subject Q1 corresponding to one point p ′ in this slit image are the plane formed by the slit light S and the straight line L connecting the point p ′ and the center point O of the imaging lens. It is obtained as the coordinates of the intersection. Therefore, from one slit image obtained with the slit light S, the spatial coordinates of the point group on the surface of the subject Q1 corresponding to each point on the slit image are obtained. By moving the slit light S in the horizontal direction, the subject Q1 is scanned, and by inputting a slit image at each scanning position, three-dimensional data of the front side portion of the subject Q1, that is, the portion irradiated with the slit light S is obtained. Entered.
[0005]
In order to obtain three-dimensional data for the entire circumference of the subject Q1, it is necessary to input the subject Q1 from a plurality of directions. Two methods are known for this purpose. In the first method, the three-dimensional camera 80 is moved on a predetermined trajectory centering on the subject Q1 in a state where the photographing direction faces the subject Q1, and the subject Q1 is photographed from a plurality of directions. In the second method, the subject Q1 is placed on the rotary stage and rotated, and the subject Q1 is photographed from a plurality of directions by the three-dimensional camera 80 installed at a predetermined position.
[0006]
The three-dimensional data of the subject Q1 input from a plurality of directions is subjected to alignment processing using conversion parameters calculated based on the position on the trajectory where the three-dimensional camera 80 moves or the position of the rotary stage. As a result, three-dimensional data on the entire circumference of the subject Q1 is obtained.
[0007]
However, in the above-described method, it is necessary to detect the position of the three-dimensional camera 80 or the angular position of the rotary stage with high accuracy in order to improve the alignment accuracy, resulting in high cost.
[0008]
In addition, since the three-dimensional camera 80 is set in a moving device or the like, the three-dimensional camera 80 cannot be taken with the hand. As a result, subjects that can be input are limited. That is, for example, an object that cannot move, such as an existing stone image or bronze image, cannot input three-dimensional data by this method.
[0009]
In order to solve such a problem, an apparatus described in Japanese Patent Laid-Open No. 9-5051 has been proposed. According to this conventional apparatus, the subject Q1 is photographed from an arbitrary plurality of directions, and three-dimensional data is input. After the input, a three-dimensional shape input from a plurality of directions and a color image input from the same field of view and associated with the three-dimensional data at the same time are displayed. While viewing the displayed color image, the user manually designates corresponding points in the three-dimensional shape based on a color change or the like. Based on the corresponding points designated by the user, the input three-dimensional data are aligned with each other.
[0010]
[Problems to be solved by the invention]
However, according to the conventional apparatus described above, it is necessary for the user to specify the corresponding points in order to align the positions of the plurality of three-dimensional data, which is extremely troublesome.
[0011]
Further, when inputting three-dimensional data, there is no method for confirming whether or not shooting has been successfully performed for the entire circumference of the subject Q1. For this reason, when a plurality of three-dimensional data are not continuous with appropriate overlap, or when the data is insufficient, the corresponding points between the three-dimensional data cannot be taken well. In such a case, the accuracy of the alignment between the three-dimensional data is lowered, or if it is severe, it is necessary to perform photographing again.
[0012]
The present invention has been made in view of the above-described problems, and can easily and quickly and accurately input 3D data of a subject, and can easily produce a composite image free from omission in the entire circumference or a predetermined range of the subject. It is an object of the present invention to provide a three-dimensional data input method and apparatus that can be acquired in a simple manner.
[0013]
3-dimensional data input method according to claim 1, as shown in FIGS. 5 and 6, has a monitor screen 21 for confirming the object Q, the object Q by taking the object Q by shooting A three-dimensional data input method in the three-dimensional data input device 2 configured to input three-dimensional data, the three-dimensional shape corresponding to the shape based on the three-dimensional data input from a part of the subject Q A model image is generated, and the image of the three-dimensional shape model is displayed on the monitor screen 21 as a framing guide image GP. At this time, the guide image GP is converted into an image from the viewpoint direction at the time of the next shooting. It converted to display performs framing so that the image corresponding overlaps the guide image GP of the image QP of the object Q and the guide image GP, the framing Performing the next photographing of the object Q in made state.
[0014]
As shown in FIGS. 2 to 6, the three-dimensional data input device according to claim 2 has a monitor screen 21 for confirming the subject Q. By photographing the subject Q, the three-dimensional data of the subject Q is obtained. A three-dimensional data input device 2 configured to input data, and generates an image of a three-dimensional shape model corresponding to the shape based on three-dimensional data input from a part of the subject Q by photographing. 3D shape model image generating means 73, display instruction means 76 for displaying the image of the 3D shape model on the monitor screen 21 as a framing guide image GP, and the guide image GP when the guide image GP is displayed. means for converting the GP to the image from the viewing direction at the time of the next shooting, 3-dimensional data inputted by next photographing an object Q which framing is made It includes a storage means 63 for storing to, a.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a configuration diagram of a measurement system 1 using a three-dimensional camera 2 according to the present invention, and FIG. 2 is a diagram showing an appearance of the three-dimensional camera 2.
[0016]
As shown in FIG. 1, the measurement system 1 includes a three-dimensional camera 2 and a host 3.
The three-dimensional camera 2 is a portable three-dimensional data input device that performs three-dimensional measurement by the slit light projection method, and inputs three-dimensional data (distance data) by photographing a subject Q to be input. Based on the three-dimensional data, data serving as a basis for obtaining the three-dimensional shape of the subject Q is calculated and output.
[0017]
The host 3 is a computer system including a CPU 3a, a display 3b, a keyboard 3c, a mouse 3d, and the like. Between the three-dimensional camera 2 and the host 3, both online and offline data transfer using the portable recording medium 4 is possible. Examples of the recording medium 4 include a magneto-optical disk (MO), a mini disk (MD), and a memory card.
[0018]
Based on the three-dimensional data sent from the three-dimensional camera 2, the host 3 performs arithmetic processing, pasting processing (compositing processing), and the like for obtaining the coordinates of the sampling points using the triangulation method. Software for this purpose is incorporated in the CPU 3a.
[0019]
As shown in FIG. 2A, the three-dimensional camera 2 has a light projection window 20 a and a light reception window 20 b on the front surface of the housing 20. The light projecting window 20a is located above the light receiving window 20b. The slit light (band-shaped laser beam with a predetermined width w) U emitted from the internal optical unit OU travels toward the subject Q through the projection window 20a. The radiation angle φ in the length direction M1 of the slit light U is fixed. A part of the slit light U reflected by the surface of the subject Q enters the optical unit OU through the light receiving window 20b. The optical unit OU includes a biaxial adjustment mechanism for optimizing the relative relationship between the light projecting axis and the light receiving axis.
[0020]
Zooming buttons 25a and 25b, manual focusing buttons 26a and 26b, and a shutter button 27 are provided on the upper surface of the housing 20.
Further, as shown in FIG. 2B, on the rear surface of the housing 20, a finder 21, a cursor button 22, a select button 23, a cancel button 24, an analog output terminal 32, a digital output terminal 33, and a recording medium 4 are attached. A vent 30a is provided.
[0021]
The viewfinder 21 is an image display device having a monitor screen such as a liquid crystal display. On the viewfinder 21, a monitor image QP and a guide image GP which is a feature of the present invention are displayed. Details of the monitor image QP and the guide image GP will be described later. In addition, operation procedure information for instructing an operation to be performed next by the user at each operation stage is displayed, and a distance image (grayscale image) in which the three-dimensional data of the photographed portion is expressed by shading.
[0022]
The buttons 22 to 24 on the back are for setting the shooting mode and the like.
From the analog output terminal 32, a two-dimensional image signal of the subject Q is output, for example, in the NTSC format. The digital output terminal 33 is, for example, a SCSI terminal.
[0023]
Next, functions of the three-dimensional camera 2 will be described.
FIG. 3 is a block diagram showing a functional configuration of the three-dimensional camera 2. The solid line arrows in FIG. 3 indicate the flow of electrical signals, and the broken line arrows indicate the flow of light.
[0024]
As shown in FIG. 3, the three-dimensional camera 2 includes two optical systems 40 and 50 on the light projecting side and the light receiving side that constitute the above-described optical unit OU. In the optical system 40, the laser beam with a wavelength of 685 nm emitted from the semiconductor laser (LD) 41 passes through the light projecting lens system 42 to become slit light U, and is deflected by the galvanometer mirror (scanning means) 43. The driver 44 of the semiconductor laser 41, the drive system 45 of the light projection lens system 42, and the drive system 46 of the galvano mirror 43 are controlled by a system controller 61.
[0025]
In the optical system 50, the light collected by the zoom unit 51 is split by the beam splitter 52.
Light in the oscillation wavelength band of the semiconductor laser 41 enters the measurement sensor 53. The visible band light is incident on the monitor color sensor 54. Both the measurement sensor 53 and the monitor color sensor 54 are CCD area sensors. The measurement sensor 53 and the monitor color sensor 54 each output shooting information or imaging information of the subject Q as an electrical signal.
[0026]
The zoom unit 51 is an in-focus type and is provided with a zoom lens (not shown). By moving this zoom lens between the long focus side and the short focus side along the shooting direction, three-dimensional data can be input at various resolutions. A part of incident light is used for autofocusing (AF). The AF function is realized by an AF sensor 57, a lens controller 58, and a focusing drive system 59. The zooming drive system 60 is provided for electric zooming.
[0027]
Next, the main flow of electrical signals in the three-dimensional camera 2 will be described.
First, the photographing information from the measurement sensor 53 is transferred to the output processing circuit 62 in synchronization with the clock from the driver 55. The output processing circuit 62 includes an amplifier that amplifies the photoelectric conversion signal of each pixel output from the measurement sensor 53, and an AD conversion unit that converts the photoelectric conversion signal into 8-bit light reception data. The received light data obtained by the output processing circuit 62 is temporarily stored in the memory 63 and then sent to the gravity center calculation circuit 73. Address designation at that time is performed by the memory control circuit 63A. The center-of-gravity calculation circuit 73 calculates data serving as a basis for calculating a three-dimensional shape based on the received light reception data, and outputs the data to the output memory 64. The data stored in the output memory 64 is output from the digital output terminal 33 via the SCSI controller 66 or output to the recording medium 4.
[0028]
The center-of-gravity calculation circuit 73 generates subject shape data TD that is three-dimensional data corresponding to the shape of the subject Q, and outputs it to the display memory 74. The subject shape data TD stored in the display memory 74 is displayed on the viewfinder 21 as a guide image GP after image processing and the like described later are performed by the display control unit 76.
[0029]
On the other hand, imaging information from the monitor color sensor 54 is transferred to the color processing circuit 67 in synchronization with the clock from the driver 56. The imaging information subjected to the color processing is output online via the NTSC conversion circuit 70 and the analog output terminal 32, or is quantized by the digital image generation unit 68 and stored in the color image memory 69. The imaging information stored in the color image memory 69 is output online from the digital output terminal 33 or written to the recording medium 4 via the SCSI controller 66. The imaging information is also displayed as a monitor image QP on the viewfinder 21 via the display control unit 76.
[0030]
The system controller 61 gives an instruction for displaying operation procedure information on the viewfinder 21 to the display control unit 76. The circuits from the optical systems 40 and 50 for generating the subject shape data TD to the centroid calculation circuit 73, the display memory 74, the color image memory 69, the display control unit 76, the system controller 61, and the viewfinder 21 The overlapping image generating unit 7 is configured.
[0031]
The three-dimensional camera 2 having the above-described configuration and functions inputs the entire circumference of the subject Q or a predetermined range of three-dimensional data.
FIG. 4 is a diagram showing a specific method of inputting the entire circumference of the subject Q or a predetermined range of 3D data by the 3D camera 2.
[0032]
In the method shown in FIG. 4A, the user moves the three-dimensional camera 2 around the subject Q and adaptively changes the shooting conditions and performs input. In the method shown in FIG. 4B, input is performed while adaptively changing the imaging conditions while moving the subject Q itself. Alternatively, the three-dimensional camera 2 may be attached to a predetermined arm, and the position where the subject Q is photographed may be changed by moving the arm.
[0033]
Next, the monitor image QP and the guide image GP will be described.
FIG. 5 is a diagram for explaining the process of generating the monitor image QP and the guide image GP by the superimposed image generator 7. FIG. 6 shows the monitor image QP and the guide image GP of the subject Q displayed on the viewfinder 21. FIG.
[0034]
5 and 6, the monitor image QP is a color image for monitoring a portion of the subject Q where the user is about to input the 3D data with the line of sight of the 3D camera 2, and the color image memory 69. Is generated from the imaging information stored in.
[0035]
The guide image GP is an image that gives information to the user from which position and how to photograph the subject Q when inputting the entire circumference of the subject Q or three-dimensional data of a predetermined range. The guide image GP is generated from the subject shape data TD as shown in FIG.
[0036]
Here, the subject shape data TD is three-dimensional data of the input portion when the subject Q is input. In the present embodiment, the left half of the heel is shown as an example.
[0037]
Processing such as wire frame, shading, texture mapping, or coloring is performed on the subject shape data TD. Coloring is performed to facilitate identification of the wire frame, the shading, or the texture mapping image from the monitor image QP. By coloring, for example, colors such as blue, green, and red are obtained. It is also possible to make the state most visible to the user by changing the mixing ratio of the monitor image QP and the guide image GP, that is, by arbitrarily changing the density ratio of these two overlapping images. . By performing these processes, the guide image GP and the monitor image QP can be easily superimposed. It is possible to select which processing is performed according to the user's wishes. In this embodiment, an example in which texture mapping is performed is shown.
[0038]
The user can rotate (posture change) or zoom the subject shape data TD so that the guide image GP is in a state when the subject is viewed from a desired position. As a result, the guide image GP changes. The user can determine the next shooting position with reference to the guide image GP. Hereinafter, when it is necessary to distinguish the guidance image GP from others depending on its display shape, a number is added at the end such as “GP1” and “GP2”.
[0039]
A method for inputting three-dimensional data using the monitor image QP and the guide image GP will be described with reference to FIGS.
The user first images the subject Q from one direction. Here, the left half of the frog is taken from the right side. By taking a picture, three-dimensional data of the left half of the cocoon is obtained. The obtained three-dimensional data is displayed on the finder 21 as a guide image GP1, as shown in FIG.
[0040]
Next, for example, when the user performs an operation to rotate the guide image GP1 (subject shape data TD) in the right direction when viewed from above, as shown in FIG. The guide image GP2 in the state of is displayed. At this time, the right end region GPR of the subject shape data TD is displayed on the viewfinder 21. This is to leave a margin for joining the subject shape data TD obtained by the first photographing and the subject shape data TD obtained by the second photographing. It should be noted that if the color of the end region GPR that becomes the margin is different from that of other portions, the user can easily see.
[0041]
Next, as shown in FIG. 6C, framing is performed so that the guide image GP2 and the portion of the monitor image QP corresponding to the guide image GP2 overlap.
In framing, as shown in FIG. 4, the three-dimensional camera 2 may be moved or the subject Q may be moved. By performing zooming or moving the three-dimensional camera 2, it is possible to input at various resolutions. For example, a portion requiring fine data such as a face can be input at a high resolution, and a portion such as a back where the shape change is not so severe can be input at a low resolution. This can prevent unnecessary increase in data.
[0042]
After framing, the shutter button 27 is pressed to perform the second shooting. Here, the second time is taken from behind.
By a similar operation, the guide image GP2 is sequentially changed to guide images GP3, GP4... (Not shown), and shooting is performed over the entire circumference of the subject. Thereby, three-dimensional data of the entire circumference of the subject is input.
[0043]
For example, the following method can be used to generate the third and subsequent guidance images GP3, GP4,.
(1) A guide image GP is generated from an image captured last time.
(2) A guide image GP is generated using all the images taken so far.
(3) An arbitrary image is selected from images captured so far, and a guide image GP is generated from the selected image.
[0044]
As described above, the guide images GP that are sequentially changed can be changed either in the manual mode performed according to the user's request or in the automatic mode performed in the order determined by the preprogram. When the manual mode is selected, it can be operated with a mouse or the like.
[0045]
By inputting with reference to the guide image GP, it is possible to input the three-dimensional data of the subject easily and quickly and accurately. In addition, since a plurality of photographed three-dimensional data can be easily aligned and can be accurately aligned, it is easy to obtain a composite image that is free from omission in the entire circumference or a predetermined range of the subject. be able to.
[0046]
FIG. 7 is a flowchart showing the input operation and processing of three-dimensional data.
The first three-dimensional data is input by shooting the subject at an appropriate position as the first viewpoint (# 1). The viewfinder display viewpoint position is determined in the coordinate system at the time of inputting the first three-dimensional data (# 2). From the first viewpoint, the guide image GP1 based on the first three-dimensional data is displayed (# 3). The guide image GP1 becomes the guide image GP2 automatically or by rotation or zooming specified by the user. The guide image GP2 and the monitor image QP are superimposed from the second viewpoint, which is the next viewpoint (# 4). If superimposition is successful, the second 3D data is input (# 5). Parameters for converting the second three-dimensional data into the coordinate system of the first three-dimensional data are obtained. The three-dimensional data input in a superimposed manner on the finder 21 is data of each coordinate system, and conversion between these coordinate systems is necessary. Performing alignment of the three-dimensional data means converting to one coordinate system. As a method of performing alignment, the following two methods are possible (# 6). Based on the obtained parameters, coordinate transformation is performed on the second three-dimensional data (# 7), and the initial alignment is completed (# 8). In order to perform alignment with higher accuracy, fine adjustment is performed by ICP (# 9).
[0047]
ICP is a method of minimizing the total distance between corresponding points with the closest point as the corresponding point. For ICP, refer to “A method of registration of 3-D shapes.” (P. Besl and N. McKay., See IEEE Transactions on Pattern Analysis and Machine Intelli. 6, 19-2, 25-39). Can do.
[0048]
Next, two methods for performing the alignment described in (# 6) will be described.
[First alignment method]
The following parameters are used for the description.
[0049]
Pt: viewpoint position at the time of creating the guide image GP Vt: line-of-sight direction at the time of creating the guide image GP Ut: upward direction vector at the time of creating the guide image GP Ft: focal length at the time of creating the guide image GP St: note at the time of creating the guide image GP Distance to viewpoint Fa: focal length Sa when shooting Sa: shooting distance Pa: viewpoint position when shooting Va: line of sight direction when shooting Ua: upward vector Pa ′ when shooting Pa: position of the viewpoint after coordinate conversion Va ′: Gaze direction after coordinate conversion Ua ′: Upward vector after coordinate conversion In the method described here, the viewpoint position Pa at the time of shooting is obtained and alignment is performed. When the guide image GP displayed on the viewfinder 21 is created, the viewpoint position Pt and the line-of-sight direction Vt are set.
[0050]
The line-of-sight direction Va and viewpoint position Pa when the subject Q is photographed by superimposing the monitor image QP on the guide image GP on the finder 21 are as follows. In other words, the line-of-sight direction Va is the same direction as the line-of-sight direction Vt [Equation (3) below], and the viewpoint position Pa is a position moved back and forth in the same direction as the line-of-sight direction Vt.
[0051]
Here, since lens information (focal length) at the time of photographing is obtained, the moving distance before and after the visual line direction Va (= Vt) is obtained by obtaining the photographing distance Sa so that the image magnification is the same as that of the guide image GP. (St 1 Sa) can be calculated, and the viewpoint position Pa is obtained as in the following equation (4). That is,
Ft / St = Fa / Sa (1)
Therefore, the shooting distance Sa is expressed by the following equation (2).
[0052]
Sa = (Fa × St) / Ft (2)
Also,
Va = Vt, Ua = Ut (3)
Therefore, the viewpoint position Pa when the image is taken is expressed by the following equation (4).
[0053]
Pa = Pt + (St 1 Sa) × Vt (4)
The three-dimensional data obtained from each direction is translated based on the viewpoint position Pa, and is rotated based on the line-of-sight direction Va, thereby converting the guide image GP into the original three-dimensional data coordinate system created. It becomes possible. The translation amount T for coordinate conversion for converting the original three-dimensional data into the coordinate system is expressed by the following equation (5).
[0054]
T = Pa′−Pa (5)
The line-of-sight direction Va ′ after the coordinate conversion and the upward vector Ua ′ after the coordinate conversion are expressed by the following equations (6) and (7) using Va and Ua, where R is the rotation matrix. .
[0055]
Va ′ = R × Va (6)
Ua ′ = R × Ua (7)
The alignment is performed by converting the three-dimensional data obtained from each direction by the translation amount T and the rotation matrix R into data in the coordinate system of the original three-dimensional data.
[0056]
By performing superposition, a relative movement amount between the three-dimensional camera 2 and the subject Q is obtained. Based on the amount of movement, the three-dimensional data input from a plurality of positions is converted into data in the coordinate system when first photographed. Thereby, alignment can be performed.
[0057]
As described above, the three-dimensional data input from a plurality of positions is input in a state where the alignment is performed with high accuracy by superimposing the monitor image QP and the guide image GP. Therefore, there is no need to perform alignment afterwards, which is easy.
[0058]
The bonding process is performed by the host 3 based on the three-dimensional data from a plurality of positions that have been accurately aligned as described above. Therefore, the bonding process at the host 3 can be performed at high speed and with high accuracy.
[Second alignment method]
In this method, alignment is performed using the fact that the overlapping portion of the three-dimensional data is known.
[0059]
The following parameters are used for the description.
Pti: Corresponding point Pai of the first photographed data Pai: Corresponding point of data photographed after alignment on the finder T: Parallel movement amount R: Point cloud in the overlapping area on the rotation matrix finder 21 indicating rotational movement On the other hand, the closest point on this two-dimensional image is taken as the corresponding point. Position alignment is performed by performing coordinate transformation that minimizes the three-dimensional distance between the corresponding points. The coordinate transformation that minimizes the distance between corresponding points can determine the parallel movement amount so that the barycentric positions of the corresponding points coincide with each other, and then determine the rotational movement using the least square method or the like. At that time, the following equations (8) and (9) are used.
[0060]
[Expression 1]

[0061]
The rotation matrix R is determined so as to minimize J.
[0062]
[Expression 2]

[0063]
This calculation method is described in the literature “Faugeras and Hebert, 1986 O. Faugeras and M. Hebert. The representation, recognition and locino ngofol 3-D objects. Can be referred to.
[0064]
Using the parallel movement amount T and the rotation matrix R obtained as described above, the coordinate transformation of each point Pi of the second three-dimensional data is performed by the following equation (10).
P′i = R × (Pi 1 T) (10)
By performing the above-described coordinate conversion, the first three-dimensional data and the second three-dimensional data can be aligned.
[0065]
In the above-described embodiment, the guide image GP and the monitor image QP are displayed on the finder 21 integrated with the three-dimensional camera 2. However, the guide image GP and the monitor image QP are displayed on the finder separated from the three-dimensional camera 2 or the display 3 b in the host 3. It is also possible to make it.
[0066]
In the above-described embodiment, a see-through display device using a half mirror or the like may be used as the finder 21. Thereby, the power consumption of the three-dimensional camera 2 can be saved.
[0067]
In the above-described embodiment, since the guide image GP is generated based on the three-dimensional data of the photographed subject Q, the memory for storing the guide image GP is saved while saving the trouble of creating the guide image GP in advance. Is minimal.
[0068]
In the above-described embodiment, the configuration, shape, arrangement, circuit, processing form, and the like of each part or the whole of the measurement system 1 and the three-dimensional camera 2 can be appropriately changed in accordance with the gist of the present invention.
[0069]
【The invention's effect】
According to the present invention, it is possible to input three-dimensional data of a subject easily and quickly and accurately, and it is possible to easily acquire a composite image that is free from omission in the entire circumference or a predetermined range of the subject.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a measurement system using a three-dimensional camera according to the present invention.
FIG. 2 is a diagram illustrating an appearance of a three-dimensional camera.
FIG. 3 is a block diagram illustrating a functional configuration of a three-dimensional camera.
FIG. 4 is a diagram illustrating a specific method of inputting the entire circumference of a subject or a predetermined range of three-dimensional data with a three-dimensional camera.
FIG. 5 is a diagram for explaining a process of generating a monitor image and a guide image by a superimposed image generation unit.
FIG. 6 is a diagram illustrating a monitor image and a guide image of a subject displayed on the viewfinder.
FIG. 7 is a flowchart showing input operation and processing of three-dimensional data.
FIG. 8 is a diagram illustrating an input principle of a three-dimensional camera to which a slit light projection method is applied.
[Explanation of symbols]
2 3D camera (3D data input device)
21 Viewfinder (monitor screen)
63 Memory (storage means)
73 Center of gravity calculation circuit (three-dimensional shape model image generation means)
76 Display control unit (display instruction means)
Q Subject GP Guidance image QP Monitor image (image)

Claims (2)

A three-dimensional data input method in a three-dimensional data input device having a monitor screen for confirming a subject and configured to input the three-dimensional data of the subject by photographing the subject,
Generating an image of a three-dimensional shape model corresponding to the shape based on three-dimensional data input from a part of the subject by photographing ;
The image of the three-dimensional shape model is displayed on the monitor screen as a framing guide image. At that time, the guide image is converted into an image from the viewpoint direction at the time of the next shooting and displayed.
Framing is performed so that the guide image and the image corresponding to the guide image among the images of the subject overlap.
The next shooting of the subject is performed with the framing performed.
A three-dimensional data input method. A three-dimensional data input device having a monitor screen for confirming a subject and configured to input the three-dimensional data of the subject by photographing the subject;
3D shape model image generation means for generating an image of a 3D shape model corresponding to the shape based on 3D data input from a part of the subject by photographing ;
Display instruction means for displaying an image of the three-dimensional shape model on the monitor screen as a framing guide image;
Means for converting the guide image into an image from the viewpoint direction at the time of the next shooting when displaying the guide image;
Storage means for storing three-dimensional data that is input by capturing the next framed subject;
A three-dimensional data input device.

Images