JP2005338977A - Three-dimensional image processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately obtain a three-dimensional model and three-dimensional image data of an object by a simple and inexpensive composition in a three-dimensional image processing system that takes a three-dimensional image and executes image processing. <P>SOLUTION: A stereo camera 20 is attached to the end of an arm 17 that is supported on and rotates about a rotational axis A which almost coincides with the central position of an object H, and the arm 17 is rotation driven by a pulse motor 15. The rotation driving of the arm 17 and the photography processing of the stereo camera 20 are controlled by a control system 40 that includes a PC main body 43. In photographing, the stereo camera 20 is moved to a plurality of photographing positions by rotating the arm 17 so that the stereo camera 20 takes a stereo image of the object H at each photographing position. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a stereoscopic image processing system for capturing a stereoscopic image of a subject and performing image processing.

2. Description of the Related Art Conventionally, a technique for measuring the position and coordinates of a subject from images photographed through a plurality of photographing optical systems having different optical axes has been known (for example, Patent Document 1 below). This type of shooting method is called stereo shooting or stereoscopic shooting, and the camera used is sometimes called a stereo camera or a stereoscopic camera.
JP 11-37720 A

  In addition, for the purpose of obtaining a three-dimensional image or a three-dimensional model of a subject, a technique is known in which a plurality of (not limited to two) cameras are used to perform shooting a plurality of times from different shooting positions. In addition, in order to perform the photographing a plurality of times, a system in which a large number of cameras are fixed at a predetermined position and a subject is placed at the center thereof is also considered. However, the structure in which a large number of cameras are arranged in order to obtain a plurality of different photographing positions has a problem that the apparatus is expensive and is likely to increase in size.

  On the other hand, in the medical field, techniques for obtaining a patient's skeleton, a built-in three-dimensional model and a three-dimensional image using a CT (Computed Tomography) scan or an MRI (Magnetic Resonance Imaging) diagnostic device are already known. The devices are extremely expensive, and these devices have been developed mainly to be able to determine the internal state of the body, and are not suitable for acquiring a three-dimensional model or a three-dimensional image outside the body.

  Therefore, in the medical field, the beauty field, and the like, there is a demand for a technique that can obtain a three-dimensional model and a three-dimensional image of an object inexpensively and accurately. For example, it is preferable if there is a technique that can obtain a three-dimensional model or a three-dimensional image of a target object such as a predetermined part of a patient's body or a head at a low cost and accurately in a medical / beauty field. For example, in the field of dentistry, the state of the external appearance of the patient's current jaw is obtained as a three-dimensional model or three-dimensional image data by stereoscopic imaging, and predetermined dentistry (for example, loading of a denture) is started from these data. It is preferable to simulate what kind of improvement can be obtained when performing the above. Alternatively, the difference between the three-dimensional model and the three-dimensional image data relating to the appearance of the target object (patient) obtained by the three-dimensional imaging and the three-dimensional image data and the three-dimensional image data of the target object (patient) obtained from CT or MRI is taken. The shape of the tissue that exists between the skin and the skeleton, such as the muscles and teeth of the target object (patient), is obtained, or how the shape of the tissue that exists between the skin and the skeleton is further changed by prescribed medical treatment. It is preferable that simulation can be performed.

  In view of the above problems, an object of the present invention is to accurately obtain a three-dimensional model and three-dimensional image data of an object with a simple and inexpensive hardware configuration in a three-dimensional image processing system that performs three-dimensional image shooting and image processing. There is to be able to do it.

  In order to solve the above-described problems, the present invention provides an arm that is pivotally supported on a rotation axis that substantially coincides with the center position of the subject, a stereo camera that is mounted on the tip of the arm, and a drive that rotationally drives the arm. And control means for controlling the driving means, rotating the arm to move the stereo camera to a plurality of shooting positions, and shooting a stereo image of the subject by the stereo camera at each shooting position. Adopted.

  According to the above configuration, the stereo camera attached to the tip of the arm that is pivotally supported on the rotation axis that substantially coincides with the center position of the subject is rotated to move to a plurality of shooting positions, and the subject at each shooting position is moved. Since a stereo image is taken, it is possible to provide an excellent stereoscopic image processing system capable of accurately obtaining a stereoscopic model and three-dimensional image data of an object with a simple and inexpensive hardware configuration.

  Examples relating to a stereoscopic image processing system suitable mainly in the medical or beauty field will be described below.

  FIG. 1 shows the overall configuration of a stereoscopic image processing system employing the present invention. The stereoscopic image processing system of FIG. 1 does not use a large number of cameras, but rotates one stereo camera 20 having lenses 21 and 22 with different left and right optical axes about a rotation axis (rotation center) A. It is arranged and configured to be able to.

  That is, the stereo camera 20 is mounted on the tip of an L-shaped arm 17 with the lenses 21 and 22 facing the subject H, and this arm 17 is connected to the end of the inverted L-shaped column 12 via the support 16. Is rotatably supported on the rotation axis A.

  In FIG. 1, symbol H is a human subject, and the subject H is seated on a stool 18 whose center coincides with the rotation center A of the stereo camera 20. The stool 18 is mounted on the gantry 11 together with the column 12.

  In order to rotate the arm 17 to a predetermined photographing position, the rotation axis A is coupled to the pulse motor 15 via a drive system 14 such as a gear train (however, the drive system 14 may be omitted and a direct connection method may be used). .

  As shown in FIG. 6, the stereo camera 20 is composed of lenses 21 and 22 having optical axes arranged in parallel at a predetermined distance and CCDs disposed behind the lenses 21 and 22 in order to perform stereoscopic shooting. The image pickup devices 201 and 202, and the CPU 203 that controls the focusing operation and exposure control of the lenses 21 and 22, the shooting timing and image output processing of the image pickup devices 201 and 202, and the like. As the lenses 21 and 22, lenses having the same focal length and aperture ratio are usually used (when the left and right of a stereo image are photographed under the same conditions). The CPU 203 is connected to the control system 40 of FIG.

  The control system 40 in FIG. 1 is configured using hardware such as a PC. The control system 40 includes a display (CRT, LCD, etc.) 41, a keyboard 42, a mouse (or other pointing device) 47, a PC main body 43, and the like.

  The pulse motor 15 and the stereo camera 20 are connected to the PC main body 43 via a predetermined interface, for example, an interface such as a USB, and are subjected to photographing control by the control system 40. It should be noted that the rotation amount or rotation position of the pulse motor 15 or the current rotation position of the arm 17 can be detected by a sensor means such as an optical encoder (not shown). The detection amount data of the rotation amount or rotation position of the pulse motor 15 or the current rotation position of the arm 17 is also transmitted to the PC main body 43 via an interface such as a USB.

  The control system 40 can be configured to connect to other devices via the network 45 rather than stand alone. In the case of the present embodiment, the control system 40 can be connected to other devices via a network, and in FIG. 1, it can be connected to a CT scanner (or MRI) 46.

  The configuration of the PC main body 43 is shown in FIG. The PC main body 43 is composed of almost common hardware of an existing PC. That is, the PC main body 43 includes a CPU 51 for controlling the entire apparatus and image processing of the present invention, a ROM 52 and a RAM 53 for storing programs and data, an HDD 54 for storing control programs and photographing data described later, It comprises an NIC (network interface card) 56 for connecting to the network 45, a pulse motor 15 and an interface 55 such as a USB for connecting to the stereo camera 20.

  Next, the operation in the above configuration will be described with reference to FIGS.

  When the arm 17 holding the stereo camera 20 in FIG. 1 is driven to rotate around the rotation axis A by the pulse motor 15 as indicated by reference symbol P2 in FIG. 1, this causes the stereo camera 20 to rotate along the trajectory indicated by reference symbol P1. Can be moved.

  If the positional relationship between the rotation axis A and the support column 12 is configured so that the arm 17 and the support column 12 do not interfere with each other, the stereo camera 20 can have positions Q0 to Q1, Q2, Q3 on the trajectory P1 as shown in FIG. The stereo images R0, R1, R2, and R3 of the subject H can be sequentially taken at these positions. In the case of FIG. 3, four sets of stereo images are shot while changing the shooting direction by 90 °. However, if more images are required, select shooting positions such as Q5, Q6, Q7, and Q8. Then, the stereo camera 20 may be photographed (in this case, photographing every 45 °).

  As in the case of known stereo photography, the stereo images R0, R1, R2, and R3 photograph the subject H from different directions depending on the parallax of the lenses 21 and 22 of the stereo camera 20. Therefore, coordinate (distance) measurement as shown in, for example, Patent Document 1 can be performed using the left and right images of the stereo images R0, R1, R2, and R3. For example, a coordinate data group of a specific point of the nose and chin of the subject H can be calculated from a set of left and right stereo images, and four sets of stereo images while changing the shooting direction by 90 ° as shown in FIG. Can be obtained from each of the four sets of stereo images, a coordinate data group of a specific point of a specific part (for example, nose or chin) of the subject H in the shooting direction.

  Then, by combining the coordinate data groups obtained from these stereo images, a stereo model (or a three-dimensional image) of the subject H can be obtained. In that case, by using a plurality of stereo images, Accurate data can be obtained. For example, the coordinate data in the elevation direction such as the nose of the subject H naturally fits on the screen in the elevation direction such as the nose from the left and right positions Q1 to Q3 rather than the stereo image R2 taken from the position Q2 (front) in FIG. The stereo images R1 to R3 thus photographed have less errors, so that the stereo model data of the part photographed as the side surface portion of the subject H in a certain stereo image is not the stereo image but the stereo image data. The data of the stereo image taken from the side (from the optical axis different by 90 ° in the case of FIG. 3) is used for the stereo image. By combining such coordinate data, the accuracy of the three-dimensional model (or three-dimensional image) can be corrected.

  In addition, in the combination of the left and right data of the stereo image, or the combination of the data of a plurality of stereo images, the determination process such as whether a certain pixel position in each image corresponds to a specific part such as the nose or jaw of the subject H is performed. In addition to automatically performing a known image recognition process, it is performed by a method specified by the user using the mouse 47 or the like.

  4 shows an image processing procedure in the stereoscopic image processing system of FIG. 1, and FIG. 5 shows an example of a control procedure of the imaging control in FIG. 4 and 5 are stored as a program to be executed by the CPU 51 in a storage medium of the PC main body 43, for example, the HDD 54, and are executed by the CPU 51 of the PC main body 43.

  In step S11 of FIG. 4, for example, the photographing process is performed by a technique as schematically shown in FIG. Details of this photographing process are performed as shown in FIG. In step S 21 of FIG. 5, a predetermined command is transmitted to the pulse motor 15 via the interface 55 to drive the pulse motor 15. In the example of FIG. 5, the PC main body 43 can detect the rotation angle of the pulse motor 15 (or arm 17), and the rotation angle of the pulse motor 15 (or arm 17) detected in step S22 reaches a predetermined rotation amount. The driving of the pulse motor 15 in step S21 is controlled while determining whether or not it has been performed.

  When the rotation angle of the pulse motor 15 (or arm 17) reaches a predetermined rotation amount and the stereo camera 20 reaches a predetermined photographing position (for example, Q1 in FIG. 3), the interface 55 is connected to the CPU 203 of the stereo camera 20 in step S23. A predetermined command is transmitted through the camera and shooting is performed. The CPU 203 of the stereo camera 20 controls the focusing operation and exposure control of the lenses 21 and 22, the shooting timing of the image sensors 201 and 202, the image output process, and the like. The left and right image data output from the image sensors 201 and 202 are converted into a predetermined format (for example, a format such as JPEG or Exif) and transmitted to the PC main body 43 via the interface 55.

  In step S24, it is determined whether or not shooting of all the sheets has been completed (in the case of FIG. 3, four sets of stereo images are shot at the shooting positions Q1 to Q4), and if step S24 is affirmed, the shooting process is ended. .

  In FIG. 4 again, when the photographing process (step S11) is completed, predetermined image processing is performed on the plurality of stereo image data transmitted from the stereo camera 20 in step S12. In this image processing, for example, post-processing necessary for subsequent stereo modeling (or generation of a three-dimensional image) such as sharpening processing, contrast adjustment, and color correction is performed.

  In step S13, a three-dimensional modeling (or three-dimensional image generation) process is performed. In this three-dimensional modeling (or three-dimensional image generation) process, for example, in the case of FIG. 3, coordinates such as those disclosed in Patent Document 1 are used using the left and right images of the stereo images R0, R1, R2, and R3. (Distance) is measured, and a process of obtaining a three-dimensional coordinate data group constituting the three-dimensional model of the subject H is performed.

  At that time, when obtaining the three-dimensional coordinate data group as described above, correction processing using a plurality of stereo images, that is, stereo model data of a part photographed as a side portion of the subject H in a certain stereo image A combination process of a plurality of stereo images such as using data of a stereo image taken from the side of the stereo image instead of the stereo image (in the case of FIG. 3, from an optical axis different by 90 °). By performing the above, accuracy correction is possible, and a more accurate three-dimensional coordinate data group can be obtained.

  Further, the stereo model data (or 3D image data) relating to the appearance of the subject H obtained as described above is the skeleton stereo model acquired from the CT scanner (or MRI) 46, as shown in step S15. For example, the shape of the tissue existing between the skin and the skeleton, such as the muscle and teeth of the subject H, is obtained, or the tissue existing between the skin and the skeleton by predetermined medical treatment. It is also possible to perform simulations such as how the shape of the image can be changed.

  After performing the data processing as described above, output processing of the stereo model data (or three-dimensional image data) of the subject H obtained in step S14 is performed. For example, using the three-dimensional model data obtained as a three-dimensional coordinate group, a three-dimensional display (for example, a wire frame display or a three-dimensional display using animation) is performed on the display 41. In addition, this output process is performed through a process of outputting the three-dimensional model data (or three-dimensional image data) to another processing apparatus (such as another PC or medical device) via the network 45, or via a printer (not shown). This also includes processing for printing output images and numerical data.

  As described above, according to the present embodiment, the stereo camera attached to the tip of the arm that is pivotally supported on the rotation axis that substantially coincides with the center position of the subject is rotated and moved to a plurality of shooting positions. Since it adopts a configuration that captures a stereo image of a subject at each shooting position, it has excellent stereo image processing that can accurately obtain a stereo model and 3D image data of an object with a simple and inexpensive hardware configuration. A system can be provided.

  The present invention can be implemented in any stereoscopic image processing system that performs stereoscopic image capturing and image processing. In the above, the configuration in which shooting control and image processing are performed by PC hardware is illustrated, but the apparatus for performing shooting control or image processing is not limited to a PC, and an image processing device configured as a dedicated machine, image data, and the like. May be an image forming apparatus (printer or the like) having a function of inputting the image, or a stereo camera itself. The program for controlling the image processing of the present invention can be introduced into the target image processing apparatus (the above-mentioned PC, stereo camera, etc.) via an appropriate storage medium or via a network.

It is explanatory drawing which showed the structure of the stereo image processing system which employ | adopted this invention. It is the block diagram which showed the structure of the control system in the stereo image processing system of FIG. It is explanatory drawing which showed the imaging | photography process in the stereo image processing system of FIG. It is the flowchart figure which showed the image processing procedure in the stereo image processing system of FIG. It is the flowchart figure which showed the imaging | photography control procedure in the stereo image processing system of FIG. It is explanatory drawing which showed the structure of the stereo camera of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Base 15 Pulse motor 16 Support part 17 Arm 18 Stool 20 Stereo camera 21, 22 Lens 40 Control system 41 Display 42 Keyboard 43 PC main body 45 Network 47 Mouse 51 CPU
54 HDD
55 Interface 56 NIC
201, 202 Image sensor 203 CPU

Claims (2)

An arm pivotally supported on a rotation axis substantially coincident with the center position of the subject;
A stereo camera attached to the tip of the arm;
Drive means for rotationally driving the arm;
Control means for controlling the driving unit, rotating the arm to move the stereo camera to a plurality of shooting positions, and shooting a stereo image of a subject by the stereo camera at each shooting position. Stereoscopic image processing system.   The stereo model data of the subject is obtained from the stereo image of the photographed subject, and the accuracy of the stereo model data of the subject is corrected by combining the stereo images of the plurality of subjects photographed at that time. The stereoscopic image processing system according to 1.

Images