JPH11188183A - Three-dimensional shape data processor and modeling system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prepare a human head model of natural appearance by adding data correction deforming a part corresponding to an extracted area to a waving surface similar to the hair of a human body to the shape model of an object obtained by three-dimensional measurement at a three-dimensional shape data processor for preparing a model. SOLUTION: A stereoscopic model preparing device 1 to be applied to the automatic vending machine of a small article modeled after the face of a customer is provided with a modeling system 1A including a photographic system 30 measuring the shape of an object and a working system 1B working material based on measuring data. Then, coins for the portion of a charge is fed and starting operation is executed, the shape of a person existing within a fixed range in front is measured to display a three-dimensional shape model showing a measuring result. In addition, data correction which automatically deforms the three-dimensional shape model through the use of color photographic information of a head part, is executed by a data processor 40 and the deformation processes a head hair part so as to add stripe and wave pattern by judging the direction of hair.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to three-dimensional shape data processing for creating a model of an existing object.

[0002]

2. Description of the Related Art A portable non-contact type three-dimensional measuring device (three-dimensional camera) has been commercialized and used for data input to a CG system or a CAD system, body measurement, visual recognition of a robot, and the like. As a non-contact measurement method, a slit light projection method (light cutting method) is generally used, but a pattern light projection method, a stereo vision method, an interference fringe method, and the like are also known.

[0003] Also, three-dimensional CG software that can be used in personal computers and small three-dimensional cutting machines for hobbies are commercially available. If these are used, models and creations can be easily manufactured even in ordinary households.

[0004] On the other hand, a kind of vending machine for creating a face photograph sticker of a customer on the spot is gaining in popularity. The user inserts coins for the fee and takes a favorite pose in front of the camera while watching the monitor image. Then, when a predetermined operation is performed, a sheet on which a fixed number of seals are arranged is created and discharged to an outlet. Most models have multiple options for the shape of the facial photograph, the imprint pattern, and the like.

[0005]

According to the above-described three-dimensional measuring apparatus, the shapes of various objects including a human body can be converted into data with the same ease as taking a photograph. Since it is a non-contact type, even when measuring a human body, the person to be measured does not feel troublesome. Therefore, it is conceivable to use this three-dimensional measuring device for creating a facial model that is not a facial photograph. That is, when combined with a three-dimensional processing machine, it is possible to measure a person's face and create a model with an appropriate magnification on the spot.

However, it is difficult to accurately measure the shape of the hair portion of each hair, regardless of the non-contact type and the contact type. Further, even if accurate measurement can be performed, flat processing is performed unless the processing resolution is high. Even if accurate processing can be performed using a processing device with higher resolution, the undulations are very small and appear flat at first glance. In any case, there is a problem that the hair portion of the facial model is too flat to obtain the texture of the hair.

An object of the present invention is to make it possible to create a natural-looking human head model.

[0008]

According to the present invention, a hair portion of a human body shape model is specified by referring to two-dimensional photographing information of an object or by analyzing shape data, and the hair portion is provided with a texture of hair. Is added.

An apparatus according to a first aspect of the present invention is a three-dimensional shape data processing apparatus for creating a model, wherein an area extracted from a shape model of the object obtained by three-dimensional measurement of the object is extracted. Is modified so that the portion corresponding to is deformed to an undulating surface that resembles human hair.

According to a second aspect of the present invention, an area that satisfies a set condition is extracted from a two-dimensional image of an object, and extracted from a shape model of the object obtained by three-dimensional measurement of the object. In this case, data correction is performed to deform a portion corresponding to the specified region into an undulating surface that resembles human hair.

According to a third aspect of the present invention, there is provided an apparatus for classifying line images in an extracted area, dividing the area in accordance with the distribution of each type of line image, and representing a line representative of each section. The pattern most similar to the image is selected from options registered in advance and applied to the deformation of the shape model.

According to a fourth aspect of the present invention, there is provided a modeling system, comprising: a photoelectric conversion device for photographing an object; a three-dimensional measuring device for measuring a shape of the object; and a two-dimensional image of the object obtained by the photoelectric conversion device. And a three-dimensional shape data processing device for performing data correction on the shape model of the object obtained by the three-dimensional measuring device based on the data.

[0013]

FIG. 1 is an external view of a three-dimensional model forming apparatus 1 according to the present invention. The three-dimensional model creation device 1 has a function of measuring the shape of an object and processing the material on the spot based on the measurement data, and is used as a vending machine for small articles in the shape of a customer's face. . The created article is a three-dimensional object in which a model of the face (including the hair) protrudes from a plate of a predetermined shape (for example, a square). Plate surface (background part)
It is also possible to add a specific undulating pattern to the. If an appropriate metal fitting is attached to such an article, it becomes an accessory such as a pendant, a broach, or a key holder. A metal fitting may be attached to the material in advance.

On the front surface of the upper half of the substantially life-size housing 10,
A light emitting window 12 and a light receiving window 14 for optical three-dimensional measurement are provided together with a display 16 for a user to check a pose. The light receiving window 14 is also used for two-dimensional color photography. The lower half of the housing 10 projects forward from the upper half, and the upper surface thereof is an operation panel 18. The product outlet 20 is provided on the front surface of the lower half.

The user stands toward the three-dimensional model forming device 1 and inserts coins for the fee. Thereafter, when the user performs a start operation, the three-dimensional model creation device 1 measures the shape of the object existing within a certain range in front and displays a three-dimensional shape model (for example, a surface model) indicating the measurement result. Then, when the user performs the confirmation operation, the three-dimensional model creation device 1 starts three-dimensional processing according to the measurement result. The product is completed in a matter of minutes. The user takes out the product from the outlet 20.

FIG. 2 is a plan view of the operation panel 18. The operation panel 18 includes a start button 181, a confirmation button 182, a cancel button 183, a joystick 1
84 and a coin slot 185 are provided. The start button 181 is a start operation unit, and the confirmation button 182 is a confirmation operation unit. Joystick 18
Reference numeral 4 is used for changing the composition of the model. Rotation processing of the three-dimensional shape model is performed in response to a pan operation for tilting left and right, a tilt operation for tilting up and down, and a roll operation for rotating a knob, and the processing results are sequentially displayed. The cancel button 183 is an operation unit for instructing re-measurement when the user does not like the displayed three-dimensional shape model. However, the number of valid times is set in the cancel button 183, and it is not possible to instruct re-measurement without any restriction.

FIG. 3 is a functional block diagram of the three-dimensional model creation device 1. The three-dimensional model creation device 1 includes a modeling system 1A that generates a three-dimensional shape model having a model size, and a processing system 1B that makes the three-dimensional shape model visible.

The modeling system 1A includes a photographing system 30 for converting (converting) the appearance information of a user who is an original object into digital data. The imaging system 30 includes a three-dimensional measurement device 34 for converting shape information into data by a slit light projection method, a two-dimensional imaging device 36 for converting color information into data, and a controller 38. Note that other methods may be used for the three-dimensional measurement instead of the slit light projection method. The shape data DS, which is information measured by the three-dimensional measuring device 34, and the color image data DC, which is image information obtained by the two-dimensional image capturing device 36,
Is input to Since the relative relationship between the camera coordinates of the three-dimensional measurement and the two-dimensional imaging is known, it is easy to align the three-dimensional shape model based on the shape data DS with the two-dimensional image. The data processing device 40 includes an image processing circuit (not shown), and performs various data processing including data correction specific to the present invention. The controller 42 of the data processing device 40 also performs overall control of the three-dimensional model creation device 1 and gives appropriate instructions to the controller 38 of the imaging system 30 and the controller 76 of the processing system 1B. The display 16 and the operation input system 80 are connected to the controller 42. The operation input system 80 includes the above-described operation panel 18 and a fee receiving mechanism.

On the other hand, the processing system 1B includes a processing device 72 for cutting a material such as a resin block, a material supply device 74 for supplying a material to a processing position and transporting the processed product to the outlet 20; a controller 76; An outlet sensor 78 is provided. The detection signal of the outlet sensor 78 is input to the controller 42.

It is also possible to adopt a circuit configuration in which the controller 42 is in charge of the control of the photographing system 30 and the processing system 1B, and the controller 38 and the controller 76 are omitted.

FIG. 4 is a perspective view showing an example of the mechanism configuration of the processing system 1B. The material supply device 74 has a stock section 210 for storing a total of eight types of materials. The storage spaces are provided on both sides of the linear transfer path 212, and four elevators 220 are arranged along the transfer path 212 in the storage spaces on each side. A plurality of materials of the same type are stacked on each elevator 220, and the vertical movement of the elevator 220 is controlled so that the uppermost material is located at a predetermined height. When one type of material suitable for the model to be created is designated, the designated material is transferred from the storage space to the transfer path 212 by the pushing rod 218 as the work 216.
Sent out to Then, the work 21 on the transfer path 212
6 is sent to the table 200 of the processing device 72 by the transfer rod 214 with the chuck.

In the table 200, the work 216 is fixed by two stoppers 202 and a clamp jig 204. And it cuts by the drill 208 attached to the rotating shaft 206 which can move up and down, right and left, and back and forth.

When the three-dimensional processing is completed, the work 216 is nipped by the chuck at the tip of the transfer rod 214, carried to the discharge-side end of the transfer path 212, and sent to the discharge port 222. Instead of using the transfer rod 214, the work 216 may be moved from the table 200 to the discharge port 222 in the form of a slide.

The mechanism configuration of the processing system 1B is not limited to the example. For example, the number of elevators can be reduced by arranging the same type of material horizontally on each of the multi-stage shelves, arranging an elevator at one end in the arrangement direction, and extruding the materials from the shelves to the elevators. The work may be carried from the storage position to the processing position to the discharge position by the arm robot. Instead of cutting, it is also possible to create a model by a method such as an additive manufacturing method (including a stereolithography method), a laser processing (thermal processing), and a molding processing (pressing and the like). In addition, regarding the material shape, the user may be able to select a desired external shape, or a material having the shortest processing time is automatically selected from a plurality of types of materials in which a standard face model has been created in advance. You may do so.

In the three-dimensional model creating apparatus 1 having the above-described configuration, in order to create a natural-looking face model in which the texture of the hair is reproduced, the three-dimensional shape model obtained by the three-dimensional measurement is applied to the head. The data processing device 40 performs a data correction for automatically deforming using the color photographing information of the data processing apparatus (including the data processing apparatus 40). More specifically, the deformation is a process of adding a previously registered striped undulating pattern to the hair portion of the three-dimensional shape model, that is, a process of providing a large number of grooves or streaky ridges. The color photographing information is used for specifying the hair part. Further, in the present embodiment, in order to make the model more similar to the real thing, the hair direction (wave line) of each part is determined from the color photographing information, and the hair part is divided according to the result to optimize the hair part for each divided area. Add an undulating pattern. In addition,
The hairstyle may be determined from the shape of the hair part, and an undulating pattern set in advance according to the hairstyle may be uniformly added regardless of the actual waveline.

FIG. 5 is a schematic diagram of extracting hair direction information. The data processing device 40 firstly receives the color image data D
The hair part is extracted from the two-dimensional image represented by C. By performing clustering in a specific color space (for example, L ^* a ^* b ^* color space), the two-dimensional image is divided into regions of the same hue. Labeling is performed on the result to extract a continuous area having the same hue. Matching is performed using a template created in advance based on statistics on the position and color of the face element, and an area corresponding to the hair is selected.

Next, hair direction information is extracted in the following manner. First order differentiation processing is performed on the hair part of the two-dimensional image to detect edges. The edge detection image G21 is binarized at a predetermined threshold. The image G from which the noise of the image is removed by repeating the contraction / expansion processing on the binarized image G22
23 is obtained. An image G24 is obtained by thinning the line while storing the number of connected pixels of the core wire of 4 or less. Each pixel is classified by the number of connected pixels, and only a line segment (line image) consisting of only the pixels of 1 or 2 is left except for the branch point pixel of 3 connected and the intersection pixel of 4 connected. A tracking process is performed on the line segment of the image G25, and a loop-shaped line segment in which all the pixels are connected, that is, the outline of the hair portion is removed, and an image G26 indicating the hair direction is obtained.

FIG. 6 is a schematic diagram of the classification of the hair direction, FIG. 7 is a diagram showing an example of the setting of the striped pattern, and FIG. 8 is a schematic diagram of a partial correction of the shape model. The data processing device 40 converts the two-dimensional image G26 at the stage of extracting the hair direction information in the manner described above,
Scanning is performed while matching the line segment Lg indicating the hair direction with the mask patterns m1 to m8 registered in advance. As a result, the mask pattern having the largest number of matching pixels is determined as the mask pattern of that part. That is,
The hair direction is classified into any one of the mask patterns m1 to m8. Subsequently, as shown in FIG. 7, the two-dimensional image G26 of the hair part is partitioned according to the distribution of the mask pattern, and each of the partitioned areas a1 to a5 is filled with the corresponding mask pattern. That is, a hair pattern is set. Then, as shown in FIG. 8, an image G27 filled with a hair pattern is projected on the shape model U, and the shape model U is partially deformed so as to provide a groove g having a predetermined depth corresponding to the hair pattern hp. .
As a result, a stripe-like undulation resembling hair is added to the hair portion of the shape model U, and the texture of the hair is expressed.
The cross-sectional shape of the groove g can be arbitrarily set, such as a U-shape, a V-shape, and a rectangle. The cross section can be made into a parabolic curve, making it look as if it were cut with a so-called Maruhira chisel.
Further, instead of providing the groove g, it may be raised in a streak shape.
The cross-sectional shape of the ridge can be arbitrarily set similarly to the groove.

The operation of the three-dimensional model forming apparatus 1 will be described below with reference to a flowchart. FIG. 9 is a main flowchart showing a schematic operation. After the power is turned on, the two-dimensional imaging and the display of the imaging result are repeated in a standby period waiting for the operation by the user (# 10, # 12, # 1).
4). In addition, a guidance message is displayed periodically. When a fee is input and the start button 181 is pressed, two-dimensional imaging is performed again and three-dimensional measurement is performed (# 1).
6, # 18). Predetermined data processing is performed (# 20), and the obtained three-dimensional shape model is displayed (# 22). At this time, the appearance is enhanced by applying a known graphic method such as adding a shadow. Then, it waits for an instruction operation. However,
The waiting time is finite, and after the time limit, it is considered that the confirmation operation has been performed.

When the joystick 184 is operated,
As described above, the three-dimensional shape model is rotated and displayed according to the operation (# 24, # 38). Cancel button 18
When 3 is pressed, the operation returns to the operation in the standby period (# 40, # 1).
0). However, in this case, there is no need for the customer to re-enter the fee, and if the start button 181 is pressed, re-measurement is performed.

When the confirm button 182 is pressed (# 2
6) Data for processing control is generated with reference to the processing condition database based on the three-dimensional shape model (# 28).
The material is processed (# 30). When the processing is completed, the product is discharged (# 32), and it is confirmed that the product has been taken out by the takeout port sensor 78, and the process returns to the standby operation (# 34, # 1).
0).

FIG. 10 is a flowchart showing the contents of the data processing of FIG. Here, as described above, the following processes including data correction for expressing the texture of the hair and compression in the depth direction for shortening the processing time and intentionally flattening the design are performed.

A smoothing process is performed to eliminate abnormal data due to noise and to avoid excessive reproduction of fine irregularities (# 200). Perform re-sampling processing (# 21
0). This is a process of converting input data into data aligned with grid points at equal intervals projected in parallel from a certain direction in order to directly face the input data in the processing direction when the face is oblique. For example, if a person's face cannot be measured due to shadows below the ears, the data can be converted to represent a normal face-up face after three-dimensional measurement with the face facing up. If there is no measurement point at the position where the grid point is projected, linear interpolation is performed using the measurement values around the measurement point.
At this time, the projected direction is vertically above when processing, and each grid point has height data. Also,
Even when the input data is perspective projection, the input data can be converted to parallel projection data by this processing.

The missing part without data is complemented (# 22)
0). Various methods such as linear interpolation and weighted completion can be applied as the complementing method. For example, all missing parts of the data are replaced with fixed values (simple interpolation). As the fixed value, a set value, a minimum height, and an average value of the outer peripheral position of the face can be considered. If the missing part is completely surrounded by the valid data part, linear interpolation is performed from the surrounding data. In addition, for parts where accurate data cannot be obtained by optical three-dimensional measurement due to the nature of the target, such as black eyebrows and hair on a human face, the existing three-dimensional shape data is replaced. Is also good. In this case, a standard model of the face (front half of the head) is prepared, and the data of the standard model whose position and size have been adjusted are used for the data missing part. The adjustment of the standard model is performed in the following procedure. 2 in the same manner as in the case of data correction described above.
The eyes and mouth are extracted from the two-dimensional image, and the positions of three reference points are calculated. Then, the standard model is linearly converted so that each reference point of the standard model matches the actually measured shape model. In addition, such composition is not limited to the missing part of the face,
Applicable to any part.

After obtaining a three-dimensional shape model that is faithful to the real shape in each of the above-described processes, data correction specific to the present invention for adding a striped undulation to the hair portion as described above is performed (# 23).
0). At this time, an emphasis may be made to slightly elevate specific parts such as eyes, irises, eyebrows, lips, and cheeks.

A height compression process is performed to reduce the size of the three-dimensional model in the depth direction (# 240). That is,
Reduce the height difference in the depth direction to shorten the processing time.
A flat model is suitable for pendants and medals. Either the uniform compression method or the non-uniform compression method can be applied to the compression, and the compression can be used for each part.

The background portion of the three-dimensional shape model is detected (# 250). This is a pre-process for correcting the background portion. Leave the back of the customer as a blue background,
If the result of the color determination of the two-dimensional image is used, the background portion can be easily and reliably detected.

A background conversion for replacing the background portion with other data is performed (# 260). For example, since the background portion is extremely deep, the data is converted into data having a small depth in order to reduce the processing time. The data to be replaced may be plane data or three-dimensional plane data representing a pattern such as a flower or a tree or a geometric pattern.

The size adjustment is performed so that the full-size three-dimensional shape model is adapted to the product size (# 270). Further, resolution conversion is performed to adapt the data amount to the accuracy of the processing device 72 (# 280). In this processing, a mesh having a predetermined grid width is projected and resampled at grid points, but the projection direction is fixed to the vertical direction during processing. As a point of the resolution conversion (data number conversion), first, the constituent points of the shape model for machining are defined by the point pitch and the vector change amount, and the point pitch range corresponding to the vector change amount is stored in advance. Read from the specified characteristic data table and set. That is, the pitch is increased by thinning out the data, or the pitch is reduced by interpolating the data. When the measurement resolution is sufficiently large, only the thinning may be performed. If the resolution conversion function is provided, the resolution of the three-dimensional measuring device 34 is not limited, so that a usage form in which the measuring means is replaced depending on the application is allowed.

Lastly, positioning is performed by moving the origin of the coordinates in parallel so that the reference position of the three-dimensional shape model matches the reference position of machining (# 290). In the case of using a material in which predetermined irregularities have been formed in advance as described above in the processing, in the processing data generation processing (# 28 in FIG. 9) corresponding to the confirmation operation, 3
The cutting amount is calculated by comparing the three-dimensional shape model with the built-in irregularities.

FIG. 11 is a flowchart showing the contents of the partial data correction process of FIG. As described with reference to FIG. 5, first, a hair part is extracted from the two-dimensional image (# 23).
01). A series of processing of edge detection, noise removal, thinning, line segmentation, and loop removal is performed to extract hair direction information (# 2302 to 2306). The hair direction is classified by matching with the mask pattern (# 230
7) Partition the hair part of the two-dimensional image (# 2308)
Fill each section with an appropriate hair pattern (# 2309).
Thereafter, the hair pattern is projected on the shape model (# 231
0), the shape model is partially deformed to form an undulating surface resembling hair (# 2311).

In the above-described embodiment, an example in which hair direction information is extracted from a two-dimensional image has been described. However, the hair direction can be determined directly from a three-dimensional shape. That is,
A normal vector is obtained at an appropriate sampling density on the model surface, and an area where the direction of the normal vector changes is defined as an undulating portion indicating the hair direction.

In the above-described embodiment, the three-dimensional model creating apparatus 1 is assumed to be used as a vending machine. However, the data processing according to the present invention does not matter whether model creation is paid or free. The size of the model is not limited to the reduced size,
It may be full size or enlarged size.

The detection of the hair region is not limited to the detection based on the two-dimensional image, but may be performed based on three-dimensional shape data. For example, the spatial frequency of the shape data may be analyzed, and the high-frequency component may be detected as hair.

Although it has been described that a stripe pattern is added as a hair pattern, a circular pattern may be used depending on the hair style.

[0046]

According to the first to fourth aspects of the present invention,
This makes it possible to create a natural-looking human head model.

[Brief description of the drawings]

FIG. 1 is an external view of a three-dimensional model forming apparatus according to the present invention.

FIG. 2 is a plan view of an operation panel.

FIG. 3 is a functional block diagram of the three-dimensional model creation device.

FIG. 4 is a perspective view showing an example of a mechanism configuration of the processing system.

FIG. 5 is a schematic diagram of extracting hair direction information.

FIG. 6 is a schematic diagram of classification of a hair direction.

FIG. 7 is a diagram illustrating an example of setting of a stripe pattern.

FIG. 8 is a schematic diagram of a partial correction of a shape model.

FIG. 9 is a main flowchart showing a schematic operation.

FIG. 10 is a flowchart showing the contents of the data processing of FIG. 9;

FIG. 11 is a flowchart showing the contents of a partial data correction process of FIG. 10;

[Explanation of symbols]

 Reference Signs List 40 data processing device (three-dimensional shape data processing device) 1A modeling system 36 two-dimensional photographing device (photoelectric conversion device) 34 three-dimensional measuring device DC color image data (two-dimensional image) DS shape data U shape model m1 to 8 mask pattern (Choice of stripe pattern)

──────────────────────────────────────────────────続 き Continued on front page (51) Int.Cl. ⁶ Identification code FIG09G 5/36 520 G06F 15/62 350A 15/66 470A

Claims (4)

[Claims] 1. A model characterized by adding a data correction to a shape model of an object obtained by three-dimensional measurement on the object so as to deform a portion satisfying a set condition into an undulating surface resembling hair of a human body. A three-dimensional shape data processing device for creation. 2. An area which satisfies a set condition is extracted from a two-dimensional image obtained by capturing an object, and a part corresponding to the extracted area is extracted from a shape model of the object obtained by three-dimensional measurement of the object. A three-dimensional shape data processing apparatus for creating a model, characterized by adding data correction for deforming an undulating surface resembling human hair. 3. A line image in an extracted region is classified, the region is divided according to the distribution of each type of line image, and a pattern most similar to a representative line image is preliminarily classified for each division. 3. The three-dimensional shape data processing device for model creation according to claim 2, wherein the device is selected from registered options and applied to the deformation of the shape model. 4. A photoelectric conversion device for photographing an object, a three-dimensional measurement device for measuring a shape of the object, and a three-dimensional measurement device based on a two-dimensional image of the object obtained by the photoelectric conversion device. 4. A modeling system for creating a model, comprising: the three-dimensional shape data processing device according to claim 1, wherein data is corrected for the shape model of the object obtained by the method.