JP5685516B2 - 3D shape measuring device - Google Patents

Description

  The present invention relates to a three-dimensional shape measuring apparatus that measures a three-dimensional shape of an entire circumference of a target object.

  In recent years, an apparatus for measuring the three-dimensional shape of a target object over the entire circumference has been proposed in order to make the target object recognizable in three dimensions. This device shoots while rotating the target object, shoots while moving the measurement device itself around the target object, and shoots from multiple directions at the same time with multiple imaging devices. And a three-dimensional model is created from the entire circumference image (see, for example, Patent Document 1).

JP 2001-243497 A

  In the conventional three-dimensional shape measuring apparatus, the three-dimensional model obtained at a time is only the surface facing the photographing apparatus. To obtain the three-dimensional model of the entire circumference, the target object is rotated or the photographing apparatus There is a problem that it takes time to measure.

  The present invention has been made in view of the above problems, and an object of the present invention is to rotate the target object or move the photographing apparatus for information necessary for measurement of the entire three-dimensional model of the target object. It is an object of the present invention to provide a three-dimensional shape measuring apparatus that can be acquired in a short time and thereby can display a three-dimensional model of the entire circumference of a target object in a short time.

The three-dimensional shape measuring apparatus according to the present invention has the following configuration.
(1) A casing having a shape in which a panel is disposed on a side surface and an upper surface and joined and an open bottom surface is formed; photographing means disposed on each inner surface of the panel for photographing the inside of the housing; and a plurality of the interiors of the housing A lighting device that selectively illuminates from a direction, and a studio apparatus that is used by covering the target object from the bottom surface side, and the illuminating unit selectively irradiates the inside of the housing from a plurality of directions to switch the irradiation. In response to the above, a captured image of a plurality of imaging means arranged on each surface of the panel is acquired, and 3D information indicating the shape of the target object is calculated from these captured images, and all the 3D information is calculated based on the 3D information. comprising an arithmetic control means for creating a circumferential 3D model, said operation control means, by photographing the target object viewed from different directions by the plurality of imaging means under different illumination conditions, the Target object Means for obtaining three-dimensional shape information viewed from a plurality of directions, and means for acquiring texture information obtained by viewing the target object from a plurality of directions by photographing each of the plurality of photographing means under all illumination lighting conditions. , Means for restoring a three-dimensional shape of the target object viewed from a plurality of directions based on the acquired three-dimensional shape information of the target object, and the acquired texture at the coordinates of the restored three-dimensional shape And a means for calculating an all-round three-dimensional coordinate by mapping information and integrating the information, and creating an all-around three-dimensional model of the target object .

(2) In the configuration of (1), the photographing unit includes a lens that is magnified at a predetermined magnification and an image sensor that acquires an image magnified by the lens.
(3) In (1), the photographing means is provided on the inner surface side of the panel and transmits a pinhole array in which a plurality of pinholes are formed at a predetermined interval and the pinholes of the pinhole array. It is set as the aspect provided with the image sensor which acquires the image projected on a light-receiving surface.

(4) In (1), the studio apparatus includes a detecting unit that detects that the object is placed on the placement surface of the target object, and the arithmetic control unit executes the processing based on the detection by the detecting unit. It is set as the aspect which starts.
(5) In (1), the imaging means is sensitive to different wavelengths, and the illumination means includes an illumination group that emits light at different wavelengths.

  According to the present invention, the combination of the photographing means and the illumination means is controlled at high speed by the arithmetic and control means, and the photographing means is used to shoot while each illumination is turned on, so that it can be efficiently performed in a short time under each illumination condition. Shading information of the target object of front / rear / left / right information, that is, information necessary for measuring the entire three-dimensional shape is obtained. In addition, since the photographing means is arranged on the side surface and the upper surface of the box-shaped housing toward the inside of the housing so that photographing can be performed simultaneously from the entire periphery of the target object, the target object is covered with the studio apparatus. By a simple operation, information necessary for measuring the three-dimensional shape of the entire circumference of the target object can be acquired in a short time.

  Further, by using a lens and an image sensor as photographing means, the photographing means is miniaturized because the reflected light from the object to be measured is condensed on the image sensor by the lens, and both the photographing means and the illumination means are small optical components. Thus, it is possible to freely design the shape of the housing within the range in which the three-dimensional shape can be measured.

In addition, by using a pinhole array and an image sensor as photographing means, it is not necessary to separate the photographing means from the center of the casing in order to widen the photographing range, and the width of the measuring device can be reduced.
In addition, it is possible to start 3D shape measurement with an easy-to-understand operation for the user to cover the target object with a casing by equipping the studio device with detection means for detecting that the target object is placed on the mounting surface. To.

Further, in a studio apparatus, by using an illumination unit that emits light having a different wavelength and an imaging unit having a peak sensitivity at this wavelength, it is possible to perform measurement at different wavelengths simultaneously, and to perform three-dimensional shape measurement in a shorter time. Make it possible.
As described above, according to the present invention, it is possible to acquire information necessary for measurement of the entire three-dimensional shape model of the target object in a relatively short time and display the three-dimensional shape model of the target object. A three-dimensional shape measuring apparatus that can be provided can be provided.

It is a block diagram which shows the structure of the three-dimensional shape measuring apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the specific structure of the arithmetic and control apparatus of 1st Embodiment. It is a flowchart which shows the measurement process which acquires three-dimensional shape information and texture information in the studio apparatus of 1st Embodiment. It is a conceptual diagram which shows an example of the three-dimensional shape information and texture information acquired by the measurement process of 1st Embodiment. It is a flowchart which shows the decompression | restoration process which decompress | restores a three-dimensional model in the arithmetic and control apparatus of 1st Embodiment. In 2nd Embodiment, it is the figure which showed the relationship between the size of the target object which can be image | photographed, and the angle of view of an imaging device. It is a block diagram which shows the structure of the three-dimensional shape measuring apparatus which concerns on the 3rd Embodiment of this invention. . In 3rd Embodiment, it is a figure for demonstrating the relationship between the diameter of a pinhole, and the thickness of a pinhole array surface.

Embodiments of the present invention will be described with reference to the accompanying drawings. The embodiment described below is an example of the configuration of the present invention, and the present invention is not limited to the following embodiment.
(First embodiment)
FIG. 1 is a diagram showing an overall configuration of a three-dimensional shape measuring apparatus according to the first embodiment. The three-dimensional shape measurement apparatus includes a studio device 10 for acquiring three-dimensional shape information and texture information of a target object, an arithmetic control device 20 that generates a three-dimensional model from the acquired three-dimensional shape information and texture information, A display device 30 is provided for displaying the three-dimensional model.

  As shown in FIG. 2, the studio apparatus 10 is formed by arranging five panels 101 to 105 on four side surfaces and an upper surface and joining them into a box shape. Illumination devices 111 to 118 are arranged so as to illuminate, and photographing devices 121 to 125 for photographing the inside direction of the studio from four directions and above are arranged on the side and top panels as indicated by arrows A1 to A5 in the figure. . The studio apparatus 10 is used so as to cover a target object (not shown) for which a three-dimensional model is desired. The lighting devices 111 to 118 and the photographing devices 121 to 125 are driven and controlled by the arithmetic control device 20. Further, the image data obtained by the photographing devices 121 to 125 is sent to the arithmetic control device 20.

  The arithmetic and control unit 20 takes in image data obtained by program-controlling lighting of the lighting devices 111 to 118 and shooting of the imaging devices 121 to 125 as 3D shape information and texture information, and these 3D shape information and texture A three-dimensional model of the target object is created based on the information and displayed on the display device 30.

  FIG. 2 is a block diagram showing a specific configuration of the arithmetic and control unit 20. In the arithmetic and control unit 20, the lighting devices 111 to 118, the photographing devices 121 to 125, and the display device 30 are connected to a bus 201 via interfaces (not shown). A CPU (arithmetic processing unit) 202 for executing control and information processing of the apparatus, a program memory 203 for storing a processing program of the CPU 202, a work space for information processing, acquired data, and a storage unit 204 for storing calculation results. 3 is executed, and the three-dimensional shape information and texture information acquisition process (measurement process) shown in FIG. 3 and the three-dimensional model restoration process (calculation process) shown in FIG. 4 are executed. These processes will be described later.

  When measuring a target object using the measuring apparatus having the above-described configuration, the studio apparatus 10 is placed on the target object T from the opening on the bottom surface, and the target object T is disposed at substantially the center inside the studio. When the target object T covered with the studio apparatus 10 is illuminated by lighting one of the lighting apparatuses 111 to 118, the target object T is photographed by the photographing apparatus in an appropriate direction among the photographing apparatuses 121 to 125. . A combination of the lighting devices 111 to 118 and the photographing devices 121 to 125 is controlled at high speed by the arithmetic control device 20.

  When the studio device 10 obtains 3D shape information and texture information necessary for 3D model restoration, the arithmetic and control device 20 creates a 3D model based on the 3D shape information and texture information, Information on the obtained three-dimensional model is sent to the display device 30 and displayed in a predetermined format.

  FIG. 3 is a flowchart showing a measurement process for acquiring three-dimensional shape information and texture information in the studio apparatus 10. First, the illumination device 111 is turned on (step S1), and photographing is performed with the photographing devices 121, 124, and 125 (step S2). When the photographing is finished, the lighting device 111 is turned off and the lighting device 112 is turned on (step S3), and photographing is performed by the photographing devices 121 and 124 (step S4). When shooting is finished, the lighting device 112 is turned off and the lighting device 113 is turned on (step S5), and shooting is performed with the shooting devices 121, 122, and 125 (step S6). When the photographing is finished, the illumination device 113 is turned off and the illumination device 114 is turned on (step S7), and photographing is performed by the photographing devices 121 and 122 (step S8). When the photographing is finished, the illumination device 114 is turned off and the illumination device 115 is turned on (step S9), and photographing is performed with the photographing devices 122, 123, and 125 (step S10). When the photographing is completed, the illumination device 115 is turned off and the illumination device 116 is turned on (step S11), and photographing is performed by the photographing devices 122 and 123 (step S12). When shooting is finished, the lighting device 116 is turned off and the lighting device 117 is turned on (step S13), and shooting is performed with the shooting devices 123, 124, and 125 (step S14). When shooting is finished, the lighting device 117 is turned off and the lighting device 118 is turned on (step S15), and shooting is performed with the shooting devices 123 and 124 (step S16). When shooting is completed, the lighting device 118 is turned off and all the lighting devices 111 to 118 are turned on (step S17), and shooting is performed with all the shooting devices 121 to 125 (step S18). When shooting is completed, all the lighting devices 111 to 118 are turned off (step S19). Finally, the obtained image data group is stored in the storage device 204 as 3D shape information and texture information (step S20).

  FIG. 4 shows an example of three-dimensional shape information and texture information acquired by the measurement process. In the measurement process, the imaging devices 121 to 125 capture the target object viewed from the directions A1 to A5 under four different illumination conditions. The four images taken by each of the photographing devices 121 to 125, that is, a total of 20 images are assumed to be three-dimensional shape information viewed from each direction. In addition, it is assumed that the images under the all-illumination lighting conditions photographed by each of the photographing devices 121 to 125, that is, a total of five images are texture information viewed from each direction. As an example, FIG. 4A shows the three-dimensional shape information when the target object is a cube, and FIG. 4B shows the texture information.

  FIG. 5 is a flowchart showing a restoration process for restoring the three-dimensional model in the arithmetic and control unit 20. First, three-dimensional shape information and texture information are read from the storage device 204 (step S21), and the three-dimensional shapes of the target object viewed from the directions A1 to A5 are photographed by the photographing devices 121 to 125, respectively. Is calculated and restored from the three-dimensional shape information which is the image set (step S22). The three-dimensional shape restoration algorithm used at this time may use an existing algorithm such as an illuminance difference stereo method for calculating a three-dimensional shape from a plurality of images taken under different illumination conditions.

  When the three-dimensional shape of the target object viewed from each direction is calculated and restored, the texture information is mapped to the coordinates of the three-dimensional shape (step S23) and integrated to calculate the three-dimensional coordinates of the entire circumference (step S24). . Specifically, in the integrated all-round three-dimensional coordinates, a triangular mesh is formed using three-dimensional Delaunay division or the like, and a texture obtained by texture information is attached to each mesh to obtain a three-dimensional model around the entire circumference.

  Finally, the restored all-round three-dimensional model is sent to the display device 30 to display an image viewed from the direction specified by the user. The display device 30 may be a two-dimensional display such as a monitor or a touch panel, or may be displayed on AR (Augmented Reality) and viewed on a display or HMD (Head Mounted Display). Alternatively, it may be displayed on a three-dimensional display.

  In this embodiment, an example of mounting in a cubic housing has been described. However, other polyhedrons such as a triangular pyramid can be applied. In the present embodiment, an example in which illuminance difference stereo is used as the three-dimensional shape restoration algorithm has been described, but an algorithm for obtaining a shape from a shadow such as shape from shading may be applied.

(Second Embodiment)
Next, the second embodiment will be described. Since the basic configuration is the same as that of the first embodiment, the description regarding the configuration is omitted here.
For the imaging devices 121 to 125 in the present embodiment, a camera constituted by a lens and an image sensor is used. In this case, the size of the target object that can be photographed and the size of the housing are determined by the angle of view of the camera and the depth of field. FIG. 6 is a diagram showing the relationship between the size of a target object that can be photographed and the angle of view of the photographing apparatus. That is, when the cameras 121 to 123 having the field angle of view θ are set apart from the region where the three-dimensional measurement is possible by the distance d, the size (width) of the region where the three-dimensional shape can be measured is O ³ . However, O is represented by the following formula.
O = 2dtan (θ / 2)
(Third embodiment)
FIG. 7 is a configuration diagram of the third embodiment. The photographing apparatus according to the first embodiment may be configured by a pinhole array 131 and an image sensor 132 as shown in FIG. Only light rays from the normal direction of the pinhole surface enter each pinhole 131, and this image is formed on the image sensor 132. In this case, even when the distance between the wall surface of the studio apparatus 10 and the target object is short, the front, back, left, and right top of the target object can be captured. Further, since the depth of field is deepened by the pinhole effect, it is possible to obtain a focused image anywhere. This is shown in FIG. Here, the following relational expression is established between the diameter s of the pinhole 131 and the thickness d of the pinhole array surface.
θ = 2tan ^-1 (s / d)
Here, if the distance from the back surface of the pinhole array 131 to the image sensor surface is L in order to prevent the images taken by the adjacent pinholes 131 from overlapping, as shown in FIG. The distance x between the holes may be set so that the following relational expression is satisfied.
x ≧ (2L + d) tan (θ / 2)
By doing so, since the images do not overlap, it is possible to avoid problems such as bleeding.

(Fourth embodiment)
Subsequently, a fourth embodiment will be described. Since the basic configuration is the same as that of the first embodiment, the description regarding the configuration is omitted here.
This embodiment relates to control of the start of the measurement process. That is, the start of the measurement process may be designed so that the user performs an input operation such as a button when starting the measurement, but this embodiment automates the start control of the measurement process. First, the studio apparatus 10 is provided with various sensors such as a brightness sensor that detects internal brightness and an acceleration sensor that detects impact generated during installation. Then, the arithmetic and control unit 20 senses that the studio apparatus 10 is installed on the placement surface of the target object by various sensors equipped in the studio apparatus 10 and automatically starts the measurement process. This form makes it possible to start the three-dimensional shape measurement by an operation that is easy for the user to simply put the studio apparatus 10 on the target object.

(Fifth embodiment)
Further, a fifth embodiment will be described. Since the basic configuration is the same as that of the first embodiment, the description regarding the configuration is omitted here.
In the present embodiment, a lighting group that emits a plurality of different wavelengths is used as the lighting devices 111 to 118. By using the imaging devices 121 to 125 having peak sensitivity in each wavelength band of this illumination group, it becomes possible to perform measurement at different wavelengths at the same time, thereby measuring a three-dimensional shape in a shorter time. Can do. Or after image | photographing with a multiband image sensor, you may divide | segment for every wavelength band by image processing, and may obtain the image group of several illumination conditions.

  Specifically, for example, a target object is illuminated simultaneously from different directions with RGB light sources, photographed with a color image sensor, then divided into RGB images by image processing, and divided into images with illumination conditions from different directions. It is possible. In this case, the same effect is obtained as when three different directions of the light source are blinked and photographed to obtain three images. Moreover, you may obtain the image group of several illumination conditions using two or more imaging devices of a different wavelength band. Specifically, for example, the same effect can be obtained by simultaneously illuminating a target object from different directions with ultraviolet light, visible light, and infrared light sources, and photographing the object with an ultraviolet camera, a color camera, and an infrared camera.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some configurations may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different example embodiments may be combined as appropriate.

  DESCRIPTION OF SYMBOLS 10 ... Studio apparatus, 101-105 ... Panel, 111-118 ... Illumination apparatus, 121-125 ... Imaging | photography apparatus, 131 ... Pinhole array, 132 ... Image sensor, 20 ... Arithmetic control apparatus, 201 ... Bus, 202 ... CPU, 203 ... Program memory, 204 ... Storage device.

Claims (5)

A case having a shape in which a panel is arranged on the side surface and the upper surface and bonded and the bottom surface is opened, a photographing unit arranged on each inner surface of the panel for photographing the inside of the case, and selecting the inside of the case from a plurality of directions And a lighting device for irradiating, a studio apparatus used by covering the target object from the bottom surface side,
The illumination means selectively irradiates the inside of the housing from a plurality of directions, acquires captured images of a plurality of imaging means arranged on each surface of the panel according to the irradiation switching, and from these captured images Calculation control means for calculating three-dimensional information indicating the shape of the target object, and creating an all-round three-dimensional model based on the three-dimensional information ;
Equipped with,
The arithmetic control means includes
Means for capturing three-dimensional shape information of the target object viewed from a plurality of directions by capturing the target object viewed from different directions by the plurality of imaging means under a plurality of different illumination conditions;
Means for acquiring texture information obtained by viewing the target object from a plurality of directions by photographing each of the plurality of photographing means under all illumination lighting conditions;
Means for restoring a three-dimensional shape of the target object viewed from a plurality of directions based on the acquired three-dimensional shape information of the target object;
Means for mapping the acquired texture information to the restored coordinates of the three-dimensional shape and calculating the total three-dimensional coordinates by integrating the texture information to create a full three-dimensional model of the target object;
Three-dimensional shape measurement apparatus comprising: a.   The three-dimensional shape measuring apparatus according to claim 1, wherein the photographing unit includes a lens that is magnified at a predetermined magnification and an image sensor that acquires an image magnified by the lens.   The photographing means is provided on the inner surface side of the panel, and includes a pinhole array in which a plurality of pinholes are formed at predetermined intervals, and an image that is transmitted through the pinholes of the pinhole array and projected onto the light receiving surface. The three-dimensional shape measuring apparatus according to claim 1, further comprising an image sensor to be acquired. The studio apparatus includes detection means for detecting that the target object is placed on a placement surface, and the arithmetic control means starts execution of the processing based on detection by the detection means. The three-dimensional shape measuring apparatus according to claim 1.   The three-dimensional shape measuring apparatus according to claim 1, wherein the imaging unit has sensitivity to different wavelengths, and the illumination unit includes an illumination group that emits light at different wavelengths.

Images