JP2001209796A - Image processor, area extracting method and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately extract a skin area from the two-dimensional color picture of a human head part. SOLUTION: A sampling area DB1 definitely included in the skin area of the color image DC1 is beforehand extracted on the basis of a distance image DS showing the three-dimensional shape of the human bead part in the color image DC1. A typical skin color is calculated from the pixel value of the area DB1, and a temporary skin area DB2 is calculated from the image DC1 by using the skin color. Further, an illumination state is estimated from the shape shown by the distance image DS and the pixel value in the area DB2 of the image DC1. Then, the pixel value of a pixel having corresponding distance data in the image DC1 is corrected into a pixel value that is not affected by the change of the illumination state due to the shape of the head part. An appropriate skin area DB3 is subsequently extracted by using the corrected color image DC2 and extracting a skin area again.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a technique for extracting a region from a two-dimensional image.

[0002]

2. Description of the Related Art Hitherto, studies have been made on extracting a predetermined area from a two-dimensional color image of an object. As one of the methods, there is a method in which a representative color of a region to be extracted is obtained in advance, and a region having a color close to the representative color is extracted.

[0003]

However, even in an area having the same color on the object, the state of the shadow caused by the illumination is different depending on the three-dimensional shape of the object. And appear as brightness. As a result, it becomes difficult to appropriately divide a color image into regions.

For example, in extracting a skin region from a color image of a person's head, if only a region having a color close to a representative skin color is extracted as a skin region, it may not be appropriate depending on lighting conditions. There are cases where skin regions cannot be extracted. Each part of the skin area on the object (that is, the head) is oriented in various directions when viewed from the camera. Even in the same area of the skin, various colors and brightness are affected by the shadows caused by lighting. This is because In particular, the effect that the skin is not recognized as the skin in the lower part of the chin appears remarkably. as a result,
Even if a region having a color close to a predetermined skin color is simply extracted as a skin region, a phenomenon occurs in which only a part of the actual skin region is extracted.

The present invention has been made in view of the above problems, and has as its object to appropriately perform region extraction from a two-dimensional image by considering the shape of an object.

[0006]

According to a first aspect of the present invention, there is provided an image processing apparatus for extracting a region from a two-dimensional image by using a two-dimensional image and a three-dimensional image which can be associated with each other. The image processing apparatus includes means for correcting a pixel value of a processing area in the two-dimensional image based on a shape of an object indicated by the image, and means for extracting an area from the corrected processing area.

According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, the means for correcting the pixel value includes a means for estimating an illumination state of the object, and the estimated illumination. Means for correcting the pixel value of the processing area to a pixel value independent of the shape of the object based on the state.

According to a third aspect of the present invention, there is provided the image processing apparatus according to the second aspect, wherein the image processing apparatus is based on an area on the object that is substantially perpendicular to a line of sight when the two-dimensional image is acquired. Means for determining a reference area in the two-dimensional image by
Means for determining a reference pixel value from the pixel value of the reference area,
Means for extracting a temporary area from the two-dimensional image based on the reference pixel value, wherein the means for estimating the illumination state corresponds to the pixel value in the temporary area of the two-dimensional image and the temporary area. The illumination state is estimated based on the shape of the region on the object to be processed.

According to a fourth aspect of the present invention, in the image processing apparatus according to any one of the first to third aspects, the object includes a head of a person, and the means for extracting the area includes a head of the head. Extract skin area.

A fifth aspect of the present invention is an area extracting method for extracting an area from the two-dimensional image using a two-dimensional image and a three-dimensional image which can be associated with each other, wherein the method comprises the steps of: The method includes a step of correcting a pixel value of a processing area in the two-dimensional image based on a shape, and a step of extracting an area from the corrected processing area.

According to a sixth aspect of the present invention, there is provided a computer-readable recording medium storing a program for causing a computer to extract a region from a two-dimensional image using a two-dimensional image and a three-dimensional image which can be associated with each other. hand,
A step of correcting the computer to correct a pixel value of a processing area in the two-dimensional image based on a shape of an object indicated by the three-dimensional image; and a step of extracting an area from the corrected processing area. Is executed.

[0012]

DESCRIPTION OF THE PREFERRED EMBODIMENTS <1. First Embodiment><1-1. Configuration of Solid Model Creation System> FIG. 1 is a block diagram showing the configuration of a solid model creation system 1. A three-dimensional model creation system 1 that measures the shape of the head of a person, processes the obtained data, and generates machining data necessary for cutting the workpiece;
A plurality of processing machines 21 to 21 provided for processing a work
24.

The modeling system 10 includes a photographing system 30, a data processing device 40, a display device 16 such as a CRT or a liquid crystal display, a storage device 43 such as a semiconductor memory or a magnetic disk, and an operation input device 44 such as a keyboard and a mouse. It is configured with.

The data processing device 40 has a function as a computer, and receives
The program is read from the recording medium 9 such as a D-ROM or a memory card to the data processing device 40. Then, the CPU in the data processing device 40 performs arithmetic processing according to the read program,
Data processing by the data processing device 40 is executed. Note that all or a part of the data processing device 40 may be configured as a dedicated electric circuit.

The photographing system 30 has a function of measuring the appearance (particularly, the shape of the head including the face) of a user who is an original object, and converting it into digital data (that is, converting it into data). A three-dimensional measuring device 34 that converts the shape information into data by the light projection method and outputs a three-dimensional distance image DS that is so-called three-dimensional shape data, and a two-dimensional color image DC that converts the color information into data. A two-dimensional photographing device 36 for outputting and a controller 38 are provided. As a three-dimensional measurement method, another method may be used instead of the slit light projection method.

Here, the three-dimensional distance image DS is image data given as coordinate values in an XYZ rectangular coordinate system, and is obtained from a measurement reference point in the three-dimensional measuring device 34 to a target object (person's head). It gives the distance information to the measurement point. The three-dimensional measuring device 34 is a device that irradiates an object with illumination light and receives the reflected light to obtain information on the distance to the measurement point. However, when the reflected light cannot be received, measurement data is obtained. I can't. For this reason, the distance image DS also includes information of a valid flag indicating whether measurement data has been obtained.

On the other hand, the pixel value of each pixel of the two-dimensional color image DC is three primary color data, that is, R (red), G
(Green) and B (blue). For example, when the imaging system 30 is configured using the three-dimensional camera disclosed in Japanese Patent Application Laid-Open No. 9-145319, since the three-dimensional measurement and the two-dimensional imaging can be performed from the same viewpoint, the distance image DS The association with the color image DC can be performed very easily.

In this embodiment, the two-dimensional color image DC and the three-dimensional distance image DS are composed of pixel data of the same number of pixels, and pixels having the same index (coordinate value) are associated with each other. It is assumed that For this reason,
When an area or a pixel is specified in one image, it is possible to specify a corresponding area or pixel in the other image.

Even when the three-dimensional measurement and the two-dimensional imaging are performed from different viewpoints, the viewpoint information is added to the distance image DS and the color image DC, respectively.
In addition, since the relative relationship between the coordinates of the three-dimensional measurement result and the coordinates of the two-dimensional imaging result is known, the correspondence between the distance image DS and the color image DC can be performed without any trouble. Such a distance image DS and a color image DC are input to the data processing device 40 as image data.

The processing for an image in the following description is a processing realized by performing an arithmetic processing on image data in the modeling system 10 (particularly, the data processing device 40).

The data processing device 40 captures the distance image DS and the color image DC and performs various data processing such as image processing, so that each of the processing machines 21 to 24 performs processing required for processing a workpiece. The data is generated and stored in the storage device 43 with a predetermined file name.

A controller 42 is provided inside the data processing device 40. The controller 42
It is connected to a controller 38 in the photographing system 30, and receives control signals for photographing a color image and measuring a three-dimensional shape in the photographing system 30. Further, the controller 42 is connected to the operation input device 44 and the display device 16, performs an operation according to the input from the operation input device 44, and changes the display content on the display device 16.

Further, the controller 42 is communicably connected to the plurality of processing machines 21 to 24, and sends commands to the processing machines 21 to 24 to execute processing in each of the processing machines 21 to 24. The operation is instructed, and the state of each of the processing machines 21 to 24 is monitored. That is, the controller 42 of the data processing device 40 is configured to perform overall control and management of the three-dimensional model creation system 1.

Each of the processing machines 21 to 24 has a storage device 171 for storing characteristic data of the processing machine, and cuts a block-shaped or coin-shaped workpiece formed of a material such as resin or metal. A processing device 172 for processing, a display device 173 for displaying a color image corresponding to the processing data being processed by the processing machine and a state of the processing machine, supply of the work from the work mounting position to the processing position, and processing of the processed product. A material supply device 174 that conveys to the outlet, a controller that controls each device inside the processing machine in response to a command from the controller 42 of the modeling system 10, and returns a status to the controller 42 of the modeling system 10. 176 and an external output terminal 177.

The processing device 172 includes a tool such as an end mill used for cutting the work, a sensor for detecting the origin position of the tool, etc., and chips (dust) generated when the work is cut. There is provided a swarf container or the like for accumulating, but these are not shown.

As described above, the three-dimensional model creation system 1 includes a plurality of processing machines 21 to 2 in one modeling system 10.
4 are connected, and the modeling system 1
0 is only for transmitting a command to each of the processing machines 21 to 24 and managing the state of each of the processing machines 21 to 24 by receiving a status. Is the controller 17 in each processing machine.
6 is up to you.

That is, in a series of operations such as measurement of an object, generation of processing data, and processing of a workpiece in the three-dimensional model creation system 1, the modeling system 10 is responsible for measurement of the object and generation of processing data. A plurality of processing machines 21 to 24 are in charge of processing the work. Thus, the processing of the work and the next measurement operation and data processing can be performed in parallel, and the pipeline processing can be performed. As a result, the throughput is improved.

In FIG. 1, a plurality of processing machines 21 to 24 are shown.
Shows a configuration example connected to the modeling system 10. However, even when the number of processing machines is one, measurement processing and data processing in the modeling system 10 and processing in the processing machines can be performed in parallel. So you can
Needless to say, the throughput can be improved as compared with the case where the parallel processing is not performed.

In each of the processing machines 21 to 24, an external output terminal 177 is provided. The external output terminal 177 is turned ON / OFF based on an operation sequence defined in the processing data acquired from the storage device 43. It is configured so that it can be turned off. For example, if a lamp is connected to the external output terminal 177, the lamp can be turned on and off based on the operation sequence defined in the processing data.

Therefore, by adding a control code for turning on / off the external output to the operation sequence defined in the processing data, it is possible to make a user or the like specify a processing machine to start the processing operation. it can. As a result, the user or the like can easily specify which of the plurality of processing machines 21 to 24 the work should be attached to.

<1-2. Operation of Modeling System> Next, the overall operation of the modeling system 10 will be described. FIG. 2 is a flowchart showing the operation of the modeling system 10.

First, as shown in FIG. 2, when the power is turned on to the three-dimensional model creation system 1, the process proceeds to step S1.
The controller 42 of the modeling system 10 performs necessary preparations for appropriately operating each of the processing machines 21 to 24. Specifically, the controller 42 includes the processing device 172
In addition, a command for performing an initial operation such as a home return operation of the material supply device 174 is sequentially transmitted to each of the processing machines 21 to 24, and each of the processing machines 21 to 24 is prepared for operation.

When the controller 42 receives the status indicating that the preparatory operation has not been completed normally, the controller 42 displays the identification name of the processing machine, the operation name in which an error has occurred, and the like.
6 to notify the user or the like of the contents. Also,
The controller 42 excludes the processing machine for which the preparatory operation has not been completed normally from a target to which a command for starting the processing operation is transmitted. Thereby, even if there is a processing machine for which the preparation operation has not been completed normally, the three-dimensional model creation system 1 can be made operable.

Subsequently, proceeding to step S2, the controller 42 of the modeling system 10 diagnoses the state of each of the connected processing machines 21 to 24. This diagnosis is a kind of monitoring operation by the modeling system 10, and is performed by acquiring characteristic data stored in the storage device 171 of each of the processing machines 21 to 24 and analyzing the contents. The characteristic data includes four types of data: tool consumption data, chip volume data, processing machine consumption data, and processing machine model / identification name data.

Next, proceeding to step S3, the modeling system 10 measures the head of the person, and
Input / output processing for transmitting a processing command to one of the devices 1 to 24 is performed. In this input / output processing,
Main processing S31, communication control S32, and state display control S3
3 are performed, and the input / output processing (step S3) proceeds by the three processings mutually requesting a task.

In the main process S31, the measurement of the head, that is, the creation of processing data from the input of data and the selection of a processing machine for processing the work based on the processing data are performed. Note that the main process S31 is a process that is repeatedly performed while the three-dimensional model creation system 1 is constantly operating, and the main process S31 does not end unless a predetermined end operation is performed. Therefore,
For example, when performing continuous processing of a workpiece, the main process S31 is repeatedly performed. This main processing S3
Details of 1 will be described later.

In the communication control S32, a plurality of processing machines 21 to 21
Communication with the H.24, such as sending a command and receiving a status, is performed.

In the state display control S33, each of the processing machines 21 to 24 on the display device 16 of the modeling system 10 is controlled.
The display indicating whether the is being processed or is on standby is updated.

When the end of the repeated main processing (S31) is instructed, the processing exits the input / output processing and proceeds to step S4.
Proceed to.

In step S4, each processing machine 21-2
The diagnosis of No. 4 is performed. The processing performed in step S4 is the same as the diagnosis processing shown in step S2. If there is an abnormality as a result of the diagnosis, the content is displayed on the display device 16 of the modeling system 10 to prompt maintenance.

Thereafter, the process proceeds to step S5, and as a termination process, after preparing for power-off in the data processing device 40, power-off is performed.

<1-3. Details of Main Processing> Next, details of the main processing S31 will be described. FIG. 3 is a flowchart showing the operation of the main processing in the modeling system 10.

As shown in FIG. 3, in the main process S31,
First, a two-dimensional image is displayed (step S10).
1). Here, a real-time two-dimensional captured image is displayed to position the head of the person.
The real-time two-dimensional capture image is displayed with an image frame superimposed so that only the shape of the workpiece to be cut out of the head shape such as a face is visible. FIG. 4 is a diagram illustrating a display screen at this time. As shown in FIG. 4, an image frame 231 corresponding to the outer shape of the work is displayed in the image frame display field RA1 of the main display area RA on the screen of the display device 16.
Is displayed, and a two-dimensional captured image (a profile image in FIG. 4) is displayed inside the circular window.

The flow advances to step S102 to determine whether or not a shooting operation has been performed. If the shooting operation has been performed, the two-dimensional color image D
Photographing of C is performed (step S103). The resulting color image DC is transferred from the imaging system 30 of the modeling system 10 to the internal memory of the data processing device 40. At this time, the image frame display field R shown in FIG.
The image in the image frame 231 displayed in A1 is not a real-time two-dimensional captured image, and the color image DC obtained by the shooting operation is displayed in the image frame 231.

Next, the three-dimensional distance image DS is measured by the three-dimensional measuring device 34 (step S104).
The resulting distance image DS is also a color image D
The data is transferred to the internal memory of the data processing device 40 as in the case of C.

Thereafter, the flow advances to step S105 to perform various editing processes such as image frame position adjustment, image frame size adjustment, background region editing, and skin region editing. FIG. 5 is a flowchart showing details of the editing process.

As shown in FIG. 5, the editing process (step S
In 105), first, a confirmation image is displayed (step S201). In the display of the confirmation image, for example, the pixel for which the three-dimensional measurement of the distance image DS has been normally performed is set to “white”.
Then, a two-dimensional binary image in which the pixels that have not been normally performed are set to “black” is created and displayed on the display device 16.

FIG. 6 shows a display screen at this time. As shown in FIG. 6, a confirmation image 251 indicating whether or not the three-dimensional measurement has been normally performed is displayed in the guidance display field RA2 on the screen of the display device 16, and is displayed at the lower right position of the guidance display field RA2. , A re-photographing button 252 for performing re-photographing and re-measurement is displayed.

By referring to the confirmation image 251, a customer or the like can determine whether or not a part necessary for processing a workpiece can be normally measured three-dimensionally.
When it is determined that the three-dimensional measurement has not been performed normally, the operation input device 44 such as a mouse is operated and the re-photographing button 252 is pressed to display the two-dimensional image shown in FIG. 3 (step S101). Can be returned.

The confirmation image is displayed (step S2).
When performing 01), it may be possible to indicate whether or not a user or the like, which is an object of photography and measurement, has moved during photography. When obtaining the distance image DS with the three-dimensional measuring device 34,
The object needs to be stationary for a certain period of time (for example, about 0.6 seconds) for light scanning or the like. This is because if the target moves during this time, accurate three-dimensional shape measurement cannot be performed.

Next, the process proceeds to step S202, where an outline cutting operation is performed. In the contour cutting operation, the user and the like are displayed in the image frame 2 in the image frame display field RA1 displayed on the display device 16.
31 and the size of the image frame 231 are adjusted.

Thereafter, the flow advances to step S203 to display a background area. The background area can be specified by measuring in advance the color components of a background curtain (blue back) provided on the background side of the human head when a two-dimensional color image is input. That is, the background area can be estimated from the color image DC by comparing the previously measured color component (RGB value) of the blue background with the color component (RGB value) of each pixel of the color image DC. .

It should be noted that the color of the area to be determined as the background area is obtained from the color of the area which is guaranteed to be the background curtain (for example, the area at the upper left and right corners of the color image DC). Is also good.

Then, a two-dimensional color image obtained by performing special processing on the estimated background region is displayed in the image frame 231 in order to make the estimated background region visible to the user or the like. As the special processing, for example, the photographing system 3
In the two-dimensional color image obtained from 0, a predetermined offset RGB value is added to or subtracted from the RGB value of a pixel estimated to be a background area. Such special processing is performed to indicate to the user or the like whether or not the modeling system 10 accurately recognizes the background area. As a result, the area of the color image that is recognized as the background area is displayed, for example, in a reddish color (covered with a red filter), and the user can easily recognize the extracted background area. It becomes possible.

The modeling system 10 can perform either addition or subtraction of the offset RGB values, and displays a radio button on the screen of the display device 16 so that the addition / subtraction can be selected alternatively. And let the customer specify one of them. However, offset RGB
A plurality of types of values may be prepared, and a radio button may be set for each of the values to allow the user to select an offset RGB value, or a user or the like may select an offset RGB value.
You may be comprised so that a value may be directly input.

The reason why a plurality of types of processing are prepared as special processing is that the color distribution of an object (such as the head of a person) varies depending on the person. That is, since the color distribution of the object is indefinite, it may be difficult to distinguish between the area estimated as the background area and other areas depending on the color distribution of the object if only one offset RGB value is set. Because there is. However, if it is configured so that a plurality of types of special processing can be performed, when it is difficult to distinguish the area estimated as the background area from the other areas, the mode of the special processing is changed to another one. There is an advantage that it is possible to distinguish.

When the background area is displayed on the display device 16 as described above, the process proceeds to step S204.

In step S204, a background region editing operation is performed. FIG. 7 is a diagram showing a display screen of the display device 16 at this time. As shown in FIG. 7, step S2
When the display processing of the background area 03 ends, the image frame 2
Inside 31, a color image with a special process applied to the background area is displayed. This background area can be further added by a mouse operation of a user or the like, or can be changed from the background area to another area.

More specifically, the mouse pointer 233 is positioned at a position other than the background area in the image frame 231 and a predetermined click operation and a drag operation are performed to newly add a pixel in the movement locus of the mouse pointer 233 to the background area. Is specified as In addition, by positioning the mouse pointer 233 at a position in the background area in the image frame 231 and performing a predetermined click operation and drag operation, a pixel in the movement locus of the mouse pointer 233 is designated as a part other than the background area. At this time, the area to be edited is image frame 2
There are only two-dimensional color images displayed inside the 31 circular windows, and the image frame 231 itself is not to be edited.

The guidance display field RA2 is provided with a change button 253 for changing the size of the mouse pointer 233, and by selecting one of the change buttons 253, the image frame 23 is displayed.
The size of the mouse pointer 233 displayed in 1 is changed.

In performing the editing operation of the background area, the parts such as the neck and the shoulder which are far away from the face in the depth direction are excluded from the processing target. Therefore, these parts are designated as the background area. Is done.

When the editing process of the background area is completed, the image frame area, the background area,
In addition, three areas other than these areas are distinguished, and are stored in an internal memory in the data processing device 40 with predetermined identification data attached to each area.

Next, the process proceeds to step S205, where a skin region is displayed. In the modeling system 10, extraction of a skin region is automatically performed, and details thereof will be described later. Note that the portion that has already been determined as the background region is excluded from the target for extracting the skin region.

Then, as in the case of step S203, a predetermined offset RGB value is added to or subtracted from a portion of the color image displayed in the image frame 231 which is estimated as a skin region, thereby displaying the skin region. I do. With this display, it is possible to easily grasp which part in the color image is recognized as the skin area.

Next, the process proceeds to step S206, where an editing operation of the skin region is performed. The operation performed here is the same as the editing operation of the background area described in step S204.
That is, by performing a predetermined mouse operation, an area other than the skin area is specified as a skin area, and a portion estimated as the skin area is specified as an area other than the skin area. However, an area already determined as a background area is not to be edited.

As described later, the data processing device 40 of the modeling system 10 is devised to appropriately extract a skin region. Step S206 to obtain an area
Is provided.

When the editing process of the skin area is completed, four areas of the color image DC, namely, the image frame area, the background area, the skin area, and the other areas are distinguished, and a predetermined identification is performed on each area. The data is stored in an internal memory in the data processing device 40 with the data attached.

Next, the process proceeds to step S207, where other areas are displayed. For example, a user or the like performs a predetermined mouse operation to designate a part such as a hair ornament from a color image in the image frame 231. At this time, a part that has already been determined as a background area and a skin area is excluded from the designation target.

The modeling system 10 can estimate the area of the hair ornament and the like by analyzing the color components of the designated hair ornament and the like and extracting all the pixels similar to the color component. At this time, a portion that has already been determined as a background region or a skin region is excluded from estimation targets of other regions.

As in the case of step S203, a predetermined offset R is added to a portion of the color image displayed in the image frame 231 which is estimated to be a region such as a hair ornament.
By adjusting the GB value, an area such as a hair ornament is displayed. With this display, it is possible to easily grasp which part in the color image is recognized as an area such as a hair ornament.

Next, the process proceeds to step S208, where editing operations for other areas such as hair decorations are performed. The operation performed here is the same as the editing operation of the background area described in step S204. In other words, by performing a predetermined mouse operation, an area other than the area such as the hair ornament can be newly designated as the area such as the hair ornament, or a portion estimated as the area such as the hair ornament can be designated as the hair ornament or the like. It is specified as an area other than the area. However, an area that has already been determined as a background area or a skin area is not to be edited. When the editing processing of the other areas such as the hair ornament is completed, in the color image DC, the image frame area, the background area, the skin area,
The five areas of the area such as the hair ornament and the other areas are distinguished, and are stored in the internal memory in the data processing device 40 in a state where predetermined identification data is attached to each area.

When the above processing is completed, the editing processing (step S105) is completed, and the process proceeds to step S106 shown in FIG.

In step S106, two-dimensional hair data is extracted. By the above-described editing process (step S105), the two-dimensional color image DC is divided into five regions: a region such as an image frame region, a background region, a skin region, a hair ornament, and other regions. Therefore, the area recognized as an area other than these can be considered to be equivalent to the hair area. Therefore, in the extraction of the two-dimensional hair data, the pixels included in the hair region are extracted from the pixels constituting the two-dimensional color image, and a filtering process in a predetermined frequency region is performed in order to reproduce the flow shape of the hair. The normalization process is performed to generate a two-dimensional gray image of the hair region.

Next, the flow advances to step S107 to generate a three-dimensional head shape. The three-dimensional head shape is divided into a skin region, a hair region, and a region such as a hair ornament excluding a background region in a two-dimensional color image.

It is considered that most of the skin area and the hair ornament area are included as effective measurement data in the distance image DS, but if they are not included as effective measurement data, the effective area of other parts is not included. Shape interpolation is performed using the part of the measurement data.

On the other hand, most of the hair region is the distance image D
Since it is considered that S is not included as valid measurement data, the shape data of the hair region is restored by calculation. That is, a three-dimensional hair shape that smoothly connects to the skin region is generated from the skin region distance image DS, and a predetermined offset is given to the three-dimensional hair shape in order to reflect the swelling of the hair portion in the skin region.

On the other hand, the gray component of the two-dimensional gray image of the hair region is converted into a component in the height direction (a component in the depth direction). Then, a hair pattern is formed by superimposing the height direction component obtained from the two-dimensional gray image on the three-dimensional hair shape, and three-dimensional shape data of the hair region is obtained. By applying this three-dimensional shape data to the hair region of the distance image DS, a complete three-dimensional shape model can be obtained.

Next, the process proceeds to step S108, where a scaling process is performed. The three-dimensional shape model generated in the three-dimensional hair shape generation process (step S107) has the same size as the head. Therefore, in the scaling process (step S108), the three-dimensional shape model is scaled (enlarged / reduced) so as to be adapted to the size of the work.

First, planar enlargement / reduction will be described. The above-mentioned image frame 231 corresponds to the size of the work to be processed, and since the size of the work is known, the size of the three-dimensional shape model and the size of the two-dimensional color image displayed in the image frame 231. From this, the enlargement / reduction ratio can be determined. Then, by scaling the three-dimensional shape model based on the determined enlargement / reduction ratio, a model suitable for the size of the work can be generated.

On the other hand, in the depth direction, first, the same enlargement / reduction ratio as the planar enlargement / reduction ratio is set. Then, thereafter, a method of compressing / enlarging to a certain work cutting depth (depth dimension) for all the pixels of the three-dimensional shape model, and a set depth (depth dimension) of the three-dimensional shape model And two methods of compressing / expanding to a certain cutting depth of the work after cutting by the cutting can be selected.

When the scaling is performed so that the three-dimensional shape model conforms to the size of the work, the process proceeds to step S109, and processing data is generated from the scaled three-dimensional shape model. When the processing data is generated, the processing data is given a file name and stored in the storage device 43.

Next, the process proceeds to step S110, where a processing machine is selected. In this step S110, as shown in FIG. 2, the main process S31, the communication control S32, and the state display control S33 function, and the main process S31 obtains information of a processing machine that can be processed, and displays it on the display device 16. A message is displayed and the user is prompted to select a processing machine. When a processing machine is selected, communication control S32 is performed so as to send a processing command to the selected processing machine.

Thus, processing is started in the selected processing machine, and a color image indicating the processing state and the person to be processed is displayed in the processing state display area RB shown in FIG. 7 corresponding to the processing machine being processed. Is done.

When the processing up to this point is completed, the main processing S31 determines whether or not there is a termination instruction from the operation input device 44 (step S111). If there is no termination instruction, the processing from step S101 is repeated. By doing so, measurement processing of the head of the next person is started. Therefore, in this case, the modeling system 10 can start the next processing before the processing of the workpieces in the processing machines 21 to 24 is completed, and the modeling system 10 and the plurality of processing machines 21 to 24 can operate in parallel. Processing is realized.

On the other hand, if there is an end instruction in step S111, the main process S31 ends and the process proceeds to step S4 shown in FIG.

<1-4. Details of Display of Skin Area> Next, the operation of the modeling system 10 in the display of the skin area (step S205) shown in FIG. 5 will be described with reference to FIGS.
This will be described with reference to FIG.

FIG. 8 is a flowchart showing the flow of the entire operation of the modeling system 10 in displaying the skin area. FIG. 9 shows the functional configuration of the data processing device 40 for performing the processing of steps S302 to S306 in FIG. It is a block diagram shown with the flow of. Note that the functional configuration shown in FIG. 9 is a function realized by the CPU in the data processing device 40 performing arithmetic processing based on the program read from the recording medium 9 as described above. All or some of the functions may be configured as a dedicated electric circuit. The same applies to the following functional block diagrams.

In the following description, an example of extracting a skin area from an image of a person's profile will be described.

In the display of the skin area (step S205),
First, step S is performed from the entire area of the two-dimensional color image DC.
The background area finally determined in 204 is deleted, and a color image DC1 of only the person area is generated (step S301). The color image DC1 of only the person region (that is, the color image in which the person region is valid) DC1 may be generated in any format. For example, the pixel value of the background region may be a special value. A color image DC may be associated with a binary image indicating a person area.

A two-dimensional color image DC1 showing a person area
When (hereinafter, simply referred to as “color image DC1”) is obtained, next, the sampling region extracting unit 401 performs sampling of the color based on the three-dimensional distance image DS (specifically, sampling region DB1). A binary image showing the sampling area DB1 is obtained (step S3).
02). In the following description, the process of obtaining a predetermined region also refers to a process of obtaining a binary image indicating a region.

Thereafter, the skin area extraction unit 402 obtains a temporary skin area DB2 using the pixel values (that is, colors) of the color image DC1 in the sampling area DB1 (step S).
303). Further, the color image D
Using C1, distance image DS and temporary skin area DB2,
The pixel value of an area where the measurement data (hereinafter, referred to as “distance data”) corresponding to the distance image DS among the pixels of the color image DC1 is corrected (step S304).

The corrected color image DC2 is sent again to the skin region extraction unit 402, and the skin region extraction unit 402 obtains a skin region DB3 using the pixel values in the sampling region DB1 of the corrected color image DC2 ( Step S305).
Thereby, a more appropriate skin region DB than the temporary skin region DB2
3 is extracted.

The obtained skin region DB3 is shaped by the shaping unit 404 (step S306), and the original color image DC is obtained.
The display of the skin region is performed on the display device 16 based on the binary image indicating the skin region DB4 after shaping (step S).
307). That is, the skin area DB of the original color image DC
The RGB value for offset is added or subtracted from the value of the pixel No. 4 to display the color of the skin region DB4 with characteristics.
For example, the R of the pixel value of the skin region DB4 of the color image DC
A predetermined value is uniformly added to the component, and only the skin region DB4 is displayed with a red tint.

Next, steps S302 to S in FIG.
Each process of 306 will be described in detail. FIG. 10 is a flowchart showing the operation of the data processing device 40 in extracting a sampling area (step S302). When the sampling area extraction unit 401 extracts the sampling area DB1, first, a difference (that is, a differential value) between the distance data between each pixel of the distance image DS and surrounding pixels is obtained to obtain a differential image. Then, a binary image showing an area in which the differential value is smaller than the prepared differential value rejection reference value TH1 in the differential image is obtained (step S321).
As a result, the distance image D
A region substantially perpendicular to the direction toward S is extracted.

FIG. 11 is a conceptual diagram illustrating the state of the processing in step S321. In FIG. 11, the state of the distance image DS (there is a part where the distance data is not acquired in the hair region, but the state of the hair is also shown in FIG. 11) and the region to be extracted (the region denoted by reference symbol R21 or R22) ) Are superimposed. By performing such processing on the profile of a person, a skin region is extracted about the cheek portion.

Subsequently, the sampling region extracting section 401 applies a morphological filter to the binary image indicating the extracted regions R21 and R22 to apply the extracted regions R21 and R22.
22 is contracted (step S322).

When an area substantially perpendicular to the line of sight is extracted, the binary image is clustered (also called labeling) to make each cluster identifiable. Then, a binary image showing only the cluster having the largest area is obtained (step S323). As a result, in the example shown in FIG. 11, the region indicated by reference numeral R21 is extracted as a cluster having the largest area (hereinafter, simply referred to as "cluster R21"), and the extraction of a part of the cheek region is realized.

Then, as shown in FIG. 12, the data processing device 40 obtains a minimum rectangle R23 surrounding the cluster R21 (step S324), and reduces this rectangle R23 to half while maintaining the center position. , A rectangle R24 is obtained (step S325). By this processing, a rectangular area which is always included in the actual skin area of the person is obtained.
Then, an area where the corresponding distance data exists in the rectangle R24 is obtained as the sampling area DB1 (step S326).

FIG. 13 shows the operation (steps S303 and S30) of the data processor 40 when extracting a skin region from a two-dimensional color image with reference to the sampling region DB1.
FIG. 14 is a flowchart showing 5), and FIG. 14 is a block diagram showing a functional configuration of the skin region extracting unit 402 shown in FIG. In step S303, the processing is performed on the color image DC1, whereas in step S305, the processing is performed on the corrected color image DC2, except that the processing in both steps is substantially the same. is there. In the following description, first, a process of obtaining the temporary skin region DB2 will be described, and then, a more appropriate skin region DB2 will be described.
3 will be described.

At the time of extracting the temporary skin area DB2 (step S303), first, using the color image DC1 and the sampling area DB1, the skin color calculation unit 421 performs the pixel values of RGB of all the pixels in the sampling area DB1. , A three-dimensional statistical process in the RGB color space is performed (step S331). Thereby, the sampling area DB
The average value of the color at 1 and the color covariance matrix are determined. Further, an inverse matrix of the color covariance matrix is obtained (step S332), and the average value of the color and the inverse matrix of the covariance matrix are passed to the region extraction unit 422 as color statistical data.

The area extracting unit 422 excludes an area having a color whose Mahalanobis distance exceeds 3 in the (effective area of) the color image DC1 by using the color statistical data, and adds a new area (hereinafter referred to as an “intermediate extraction area”). Is obtained (step S333). The binary image indicating the intermediate extraction region is returned to the skin color calculation unit 421, and three-dimensional statistical processing in the RGB color space is performed on the RGB pixel values of all the pixels in the intermediate extraction region (step S334). Thus, the average value of the colors and the covariance matrix of the colors in the intermediate extraction area are obtained. Further, an inverse matrix of the color covariance matrix is obtained (step S335), and the average value of the color and the inverse matrix of the covariance matrix are passed to the region extracting unit 422 as color statistical data from which the influence of noise has been removed.

The area extracting section 422 stores a temporary skin area DB in advance.
The Mahalanobis distance for obtaining 2 is prepared as a threshold value TH2, and the region extraction unit 422 uses the color statistical data obtained again to have a color whose Mahalanobis distance is equal to or smaller than the threshold value TH2 in the color image DC1. The region is extracted as a temporary skin region DB2 (step S33)
6). That is, a binary image is obtained in which the pixel value of a pixel having a Mahalanobis distance equal to or less than the threshold value TH2 is 1 and the pixel value of a pixel having a Mahalanobis distance exceeding the threshold value TH2 is 0.

The extracted temporary skin area DB2 is corrected as described above (FIG. 8: step S304) (details will be described later), and the corrected color image DC2 is returned to the skin area extraction unit 402. And the above steps S331 to S336
Is performed. Thereby, the temporary skin region DB
A skin region DB3 that indicates a skin region more appropriately than 2 is obtained (step S305).

That is, the skin color calculating section 421 obtains color statistical data for the pixels in the sampling area DB1 of the corrected color image DC2 (steps S331 and S33).
2), the region extraction unit obtains an intermediate extraction region having a color with a Mahalanobis distance of 3 or less (step S333), and the skin color calculation unit 421 obtains color statistical data again in the intermediate extraction region (steps S334 and S335).

A Mahalanobis distance for obtaining the skin area DB3 is prepared in advance as a threshold value TH3 in the area extracting unit 422. The Mahalanobis distance is corrected in the color image DC2 using the color statistical data in the area extracting unit 422. An area having a color whose distance is equal to or less than the threshold value TH3 is extracted as the skin area DB2 (step S336).

Next, the correction of the color image DC performed in order to obtain an appropriate skin area DB3 in step S305 will be described. FIG. 15 is a diagram illustrating a temporary skin region DB2 obtained in step S302.

As shown in FIG. 15, the temporary skin area DB2 is obtained as an area smaller than the actual skin area. This is because the sampling area DB1 is obtained based on an area substantially perpendicular to the line of sight, and only an area close to the color of this area is extracted as the temporary skin area DB2.

That is, the peripheral edge of the actual skin region or a portion having large irregularities has a different brightness or color tone of the skin due to the inclination of the surface (in other words, the direction of the normal to the surface is greatly different from the direction of the line of sight). Significantly different from sampling area DB1. As a result, the peripheral portion of the actual skin region or a portion having large irregularities is not extracted as the temporary skin region DB2. Therefore, the correction unit 403 corrects the color (that is, the pixel value) of the color image DC1 in consideration of the actual shape of the skin region and the illumination state using the Lambert illumination model.

[0109] luminance of the diffuse reflection light by the Lambert cosine law (i.e., brightness of the surface) I _d is indicated by the number 1.
In Equation 1, k _d is the diffuse reflection coefficient, I _p is the luminance of the incident light, θ is the incident angle, N is the unit normal vector of the reflecting surface,
L is a unit vector indicating the direction of the incident light, and N _x ,
N _y and N _z are the X, Y and Z components of the vector N, L _x , L _y and L
_z is an X, Y, Z component of the vector L.

[0110]

(Equation 1)

Here, in an actual illumination state, assuming that light (hereinafter, referred to as “environmental light”) incident almost uniformly from all directions around the reflecting surface exists, the brightness of the surface is expressed by the following equation (2). Is shown. In _Equation 2, I _ad is the luminance of the surface due to the ambient light, and _Ia is the luminance of the ambient light.

[0112]

(Equation 2)

In the case where the surface brightness (I _d + I _ad ) for various vectors N can be obtained in the equation (2), a plurality of equations are subjected to multiple regression analysis to obtain the parameter k in the equation (2).
_{d I p L x, k d} I p L y, k d I p L z, the approximate value of k _d I _a (i.e., the solution by least squares method) can be obtained.

Based on the above theory, the correction unit 403 determines that the color of the temporary skin area DB2 is constant, calculates the above parameters, and then uses the distance image DS to obtain the color image DC.
One color is corrected.

It should be noted that such correction is performed for the color image DC1.
Although it is limited to the region where the corresponding distance data exists among the pixels in the middle, it is assumed that the region to be extracted as the skin region DB3 has been adjusted in advance so as to be included in the region having the distance data. That is, it is assumed that the outline of the background area has been appropriately adjusted based on the distance image DS before shifting to the display of the skin area (step S205).

FIG. 16 shows the correction of the color image (step S
FIG. 17 is a flowchart showing the operation of the data processing device 40 in 304), and FIG. 17 is a block diagram showing the functional configuration of the data processing device 40 when correcting a color image.

When correcting a color image, first, a plurality of unit normal vectors N of the surface having the shape indicated by the distance image DS are obtained by the illumination state calculation unit 431 of the correction unit 403 (step S341). That is, a unit vector perpendicular to the actual skin minute area corresponding to each pixel of the temporary skin area DB2 is obtained.

Subsequently, the provisional skin area DB2 is replaced with the actual skin area DB2.
Is assumed to be included in the lighting condition calculation unit 431
Each pixel in the temporary skin area DB2 of the color image DC1
Equations 2 are calculated for the value of the R component of
You. Apply these multiple equations to approximate calculation by multiple regression analysis
A parameter k that indicates the lighting state for the R component
_dI_pL_x, K_dI_pL_y, K_dI_pL_z, K_dI_aAsk for
Step S342). The same applies to the G and B components.
Parameters are required. Below, the parameters related to RGB
Are collectively referred to as a parameter LP.

Next, the pixel value calculation unit 432 calculates the unit normal vector N for all the pixels in the area having the corresponding distance data (hereinafter, referred to as “processing area”) in the color image DC1. The operation according to Equation 1 is performed on all the pixels in the processing area using the vector N and the parameter LP. Thereby, RGB of each pixel in the processing area is obtained.
Brightness I _d of the diffuse reflection light is determined for each component.

Further, for each of the RGB of each pixel in the processing area in the color image DC1, the pixel value ((I _p
+ I _ad ). ) Is subtracted from the luminance I _d of the diffuse reflection light, and the luminance I _{ad of the} surface generated only from the ambient light is _calculated as RGB.
Required for each component. Thus, a corrected color image DC2 having the obtained luminance as a pixel value is obtained (step S343).

As described above, the ambient light is a component of the illumination light that is uniformly incident on the reflecting surface from all directions, and the reflecting surface illuminated by the environmental light is in any direction with respect to the line of sight. Even if there is, brightness and color are constant in the acquired image. Therefore, when the head of the person is illuminated only with the ambient light, it is possible to avoid the influence of the change in the brightness or color of the actual skin region due to the difference in the incident direction of the illumination light.
The corrected color image DC2 is an image acquired in such a virtual illumination state, and has pixel values independent of the shape of the object (ie, not affected by changes in the illumination state due to the shape of the object). Is obtained.

Thus, the corrected color image DC2
Is an image that represents an appropriate skin color even in a peripheral portion of an actual skin region or a skin portion having irregularities. As a result, by inputting the corrected color image DC2 to the skin region extraction unit 402, it becomes possible to obtain a skin region DB3 that indicates a skin region more appropriately than the temporary skin region DB2.

FIG. 18 is a flowchart showing the operation of the data processing device 40 in shaping the skin area (step S306) performed after the skin area DB3 is obtained. In the shaping of the skin region, first, a process of filling a minute hole region existing in the skin region DB3 is performed (step S361). As a result, the color image D
It is possible to remove a region where a hole has been formed due to incorrect correction of C1. In addition, in order to remove the influence of noise included in the distance image DS, a noise may be removed by applying a median filter to the distance image DS in advance before correcting the color image DC1.

Next, the morphological filter Closing is applied (that is, the expansion and contraction processing is repeated), and unnecessary unevenness at the boundary of the skin region DB3 is deleted (step S362). Thereby, the skin region DB3 extracted in step S305 is shaped, and the final skin region DB4
Is required.

Although the extraction of the skin area has been described in detail above, the data processing device 40 first sets the sampling area DB1 to an area substantially perpendicular to the line of sight among the shapes indicated by the distance image DS (that is, the color image DC1). Therefore, the sampling area DB1 can be reliably included in the skin area, and can be set as an area representing a typical skin color.

Then, a temporary skin area DB2 is obtained based on the pixel values of the sampling area DB1, and the temporary skin area DB2 is determined.
Since the lighting state is estimated from, the lighting state can be appropriately estimated. As a result, the correction of the color image DC1 is appropriately performed, and the extraction of the skin region DB3 is properly performed.

<2. Second Embodiment> The extraction of the skin region in the first embodiment is based on the premise that the head, which is the target, has black hair. That is, when the hair is black, the three-dimensional measuring device 34 cannot acquire the distance data of the hair region, so that the processing is performed on the assumption that most of the distance image DS corresponds to the skin. .

However, in the case of gray hair, blonde hair, or dyed hair, the three-dimensional measuring device 34 also acquires the distance data of the hair region. Therefore, extraction of the sampling area in the first embodiment (step S30)
A method of appropriately extracting a skin area even when the hair is not black by changing 2) will be described below as a second embodiment. The configuration of the three-dimensional model creation system 1 according to the second embodiment is the same as that of the first embodiment, and the operation is the same except for the extraction of the sampling area. In the following description, the description will be given with reference numerals used in the first embodiment as appropriate.

FIGS. 19 and 20 show extraction of a sampling area in the second embodiment (step S302).
6 is a flowchart showing a process in.

First, a color image DC showing only the person area
1 is grayed, and the gray image is binarized by a discriminant analysis method (step S411). That is, a histogram of a gray image is obtained, and a threshold value for binarization is obtained from the distribution of pixel values. If the person does not have black hair, there are many pixel values corresponding to the hair color and the skin color in the histogram, so that binarization yields a binary image that roughly shows the hair region and the skin region with different pixel values. Can be

Subsequently, clustering is performed on the area where the pixel value is 0 in the binary image, and a binary image showing only the cluster having the largest area is obtained (step S412). Thereby, the largest cluster indicating either the hair or the skin (hereinafter, referred to as “first cluster”) is obtained.

The obtained first cluster is subjected to a filling process (step S413), and expansion and contraction by a morphological filter (step S41).
4), noise removal and shaping are performed.

After that, the ratio between the area of the first cluster and the area of the area where the corresponding distance data exists in the first cluster is obtained and stored (step S415).

On the other hand, even if the pixel value of the binary image obtained from the gray image is one, steps S412 to S4 are performed.
The same processing as 15 is performed. In other words, the region where the pixel value is 1 is clustered, and a binary image indicating only the cluster having the largest area (hereinafter, referred to as “second cluster”) is obtained (step S416), the second cluster is filled in, and Perform expansion and contraction (step S42)
1, S422). Accordingly, when the first cluster is a cluster corresponding to the hair, the second cluster is obtained as a cluster corresponding to the skin, and when the first cluster is a cluster corresponding to the skin, the second cluster is determined as a cluster corresponding to the hair. Desired.

Thereafter, the ratio of the existence of the distance data in the second cluster is obtained (step S423).

Next, of the first and second clusters, the one in which the ratio of the area where the distance data exists is larger is determined as the temporary area (step S424). In general, the largest cluster representing the head hair is larger than the largest cluster representing the skin, and therefore, according to step S424,
The cluster included in the area corresponding to the skin is obtained as a temporary area.

When the provisional area is obtained, the area in the distance image DS corresponding to the provisional area is differentiated in the same manner as in step S321 of the first embodiment, and the area whose differential value is smaller than a predetermined threshold value is determined. A binary image shown is obtained (step S425). Thus, as in the first embodiment, a part of the skin region approximately perpendicular to the line of sight is reliably extracted.

Thereafter, the same processing as in steps S322 to S326 in the first embodiment is performed, and an appropriate sampling area DB1 is obtained.

As described above, in the second embodiment, even when distance data corresponding to the hair exists, the sampling area D included in the actual skin area is appropriately included.
B1 can be obtained, and the skin region DB3 can be appropriately extracted even when the target person is not black hair.

<3. Modifications> Although the configuration and operation of the three-dimensional model creation system 1 having the data processing device 40 according to the present invention have been described above, the present invention is not limited to the above embodiment, and various modifications are possible. It is.

For example, in the above embodiment, the color image D
Although the background area is deleted from C and the person area is obtained in advance, it is not always necessary to delete the background area in order to extract the skin area. Since the correction of the pixel value in the color image DC is performed only in the area where the distance data exists, the area where the distance data exists may be determined in advance as the processing area to be corrected.

That is, the region where the distance data exists in the color image DC is regarded as a two-dimensional image which can be associated with the distance image DS which is a three-dimensional image. Region extraction involving color correction can be realized using the image and the three-dimensional image.

On the other hand, it is not necessary that all the regions in the two-dimensional color image DC in which the distance data exist are set as the processing regions to be corrected. For example, if a partial area of clothes or hair can be known in advance, such an area may be excluded from the correction target.

In the above embodiment, the color image DC
However, by applying a process for any of RGB to a monochrome image, it is possible to extract a region with a pixel value correction from the monochrome image.

In the above embodiment, the illumination state is estimated by obtaining the parameter of the illumination state of the object, and the pixel value of the color image DC1 is appropriately corrected. If the parameters of the state are known or are pre-determined and prepared by other techniques, applying these parameters to Equation 1 will result in a correction that would be obtained under ambient light only illumination A subsequent color image DC2 can be obtained. That is, if the illumination state is known, the color image DC1 can be appropriately corrected without estimating the illumination state.

In the above embodiment, the head of a person is targeted, but the object is not limited to the head of a person, and various other objects may be used.

In the above embodiment, the temporary skin area D
A set of illumination state parameters is determined for each of RGB, assuming that B2 has a substantially constant color. However, after dividing the temporary skin area DB2, the illumination state parameters are determined for each of the divided areas. May be calculated. In this case, each portion of the processing region to be corrected is corrected using the parameter related to the closest divided region, so that more appropriate correction is realized.

Further, in the above-described embodiment, the three-dimensional model creation system 1 has been described, but the region extraction according to the present invention may be used for another device or a single device. For example, the present invention may be applied to computer graphics that performs modeling using a two-dimensional image and a three-dimensional image that can be associated with each other.

[0149]

According to the first to sixth aspects of the present invention, a processing area in a two-dimensional image is corrected based on the shape of the object, so that appropriate area extraction can be performed.

According to the second aspect of the present invention, an appropriate region can be extracted by estimating an illumination state.

According to the third aspect of the present invention, since the temporary area is extracted from the pixel values of the reference area and the illumination state is estimated from the temporary area, the illumination state can be appropriately estimated.

According to the fourth aspect of the present invention, it is possible to appropriately extract the skin area of the head of a person from a two-dimensional image.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a three-dimensional model creation system 1 having a data processing device according to the present invention.

FIG. 2 is a flowchart showing the operation of the three-dimensional model creation system.

FIG. 3 is a flowchart showing an operation of the modeling system in a main process in FIG. 2;

FIG. 4 is a diagram illustrating a display example of a display device of the modeling system.

FIG. 5 is a flowchart showing an operation of the modeling system in editing in FIG. 3;

FIG. 6 is a diagram showing a display example of the display device after photographing and measurement.

FIG. 7 is a diagram illustrating a display example of the display device during editing.

FIG. 8 is a flowchart mainly showing an operation of the data processing apparatus in displaying a skin area in FIG. 5;

FIG. 9 is a block diagram illustrating a functional configuration of the data processing device when an operation related to display of a skin region is performed.

FIG. 10 is a flowchart showing an operation of the data processing device in extracting a sampling area in FIG. 8;

FIG. 11 is a diagram for explaining a sampling area extraction method.

FIG. 12 is a diagram for explaining a sampling area extraction method.

FIG. 13 is a flowchart showing an operation of the data processing device in extracting a skin region in FIG. 8;

FIG. 14 is a block diagram illustrating a functional configuration of a skin region extraction unit in FIG. 9;

FIG. 15 is a diagram illustrating a temporary skin region.

FIG. 16 is a flowchart illustrating an operation of the data processing device in correcting a color image in FIG. 8;

17 is a block diagram illustrating a functional configuration of a correction unit in FIG.

FIG. 18 is a flowchart showing an operation of the data processing device in shaping the skin region in FIG. 8;

FIG. 19 is a flowchart showing an operation of the data processing device in extracting a sampling area according to the second embodiment.

FIG. 20 is a flowchart showing an operation of the data processing device in extracting a sampling area according to the second embodiment.

[Explanation of symbols]

 Reference Signs List 9 recording medium 40 data processing device 401 sampling region extraction unit 402 skin region extraction unit 403 correction unit 421 skin color calculation unit 422 region extraction unit 431 lighting state calculation unit 432 pixel value calculation unit DB1 sampling region DB2 temporary skin region DC1 color image DS Distance image S304, S305 Step

Claims (6)

[Claims] 1. An image processing apparatus for extracting a region from a two-dimensional image by using a two-dimensional image and a three-dimensional image that can be associated with each other, wherein the image processing device performs a region extraction based on a shape of an object indicated by the three-dimensional image. An image processing apparatus comprising: means for correcting a pixel value of a processing area in a two-dimensional image; and means for extracting an area from the corrected processing area. 2. The image processing apparatus according to claim 1, wherein the means for correcting the pixel value includes: means for estimating an illumination state on the object; and Means for correcting the pixel value of the processing area to a pixel value independent of the shape of the object. 3. The image processing apparatus according to claim 2, wherein the two-dimensional image is based on an area on the object that is substantially perpendicular to a line of sight when the two-dimensional image is acquired. Means for determining a reference area within, means for determining a reference pixel value from the pixel value of the reference area, and means for extracting a temporary area from the two-dimensional image based on the reference pixel value, further comprising: An image for estimating an illumination state based on a pixel value in the temporary area of the two-dimensional image and a shape of an area on the object corresponding to the temporary area. Processing equipment. 4. The image processing apparatus according to claim 1, wherein the object includes a human head, and the area extracting unit extracts a skin area of the head. An image processing apparatus characterized by the above-mentioned. 5. An area extraction method for extracting an area from a two-dimensional image using a two-dimensional image and a three-dimensional image which can be associated with each other, wherein the method is based on a shape of an object indicated by the three-dimensional image. A region extracting method, comprising: correcting a pixel value of a processing region in a two-dimensional image; and extracting a region from the corrected processing region. 6. A computer-readable recording medium in which a program for causing a computer to extract a region from a two-dimensional image using a two-dimensional image and a three-dimensional image that can be associated with each other is recorded, wherein the program is The computer executes a step of correcting a pixel value of a processing area in the two-dimensional image based on a shape of an object indicated by the three-dimensional image, and a step of extracting an area from the corrected processing area. A recording medium characterized by causing a recording medium to be read.