JP4318825B2 - Image processing apparatus and image processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to an image processing apparatus and an image processing method for generating a three-dimensional image from an input image.To the lawIn particular, image conversion processing that requires extraction control of several facial feature points (facial feature portions), such as making facial image data (two-dimensional facial image data) obtained by a CCD camera or the like into a three-dimensional image. About.
[0002]
[Prior art]
2. Description of the Related Art In recent years, in information equipment represented by personal computers, the number of models with a built-in CCD camera is increasing due to advances in miniaturization technology of camera devices and lower prices of camera device components. Therefore, using these functions, a function to generate a 3D human model based on a 2D face image captured by a CCD camera such as yourself, neighbors, acquaintances, etc. The number of PC application software and the like having a function such as performing is increasing.
[0003]
These two-dimensional face image forming technologies have been researched and developed with the theme of facial expression generation, etc., and for 3D model generation (encoding), for example, extraction of shape changes in the eyes and mouth, A paper on composition and coding method “Detection of shape change and coding of face moving image based on 3D shape model” (Science Technical Report IE87-101) has been disclosed. Is a “three-dimensional caricature generation method and apparatus” for automatically generating a caricature viewed from an arbitrary direction using, for example, a front face and a side face caricature drawn manually. (Japanese Patent Laid-Open No. 10-74271) is disclosed, and a single image or a plurality of images are selected from a plurality of types of images having different types such as style and expression based on a single input image data. Generate Aimed "image creating apparatus, an image forming method and an image generating program recording medium" (Japanese Patent Laid-Open No. 10-232950) discloses and.
[0004]
  By the way, in the method of generating a 3D human model based on the face image data (two-dimensional face image data) obtained by a CCD camera or the like as described above, and the method of caricature / animation shaping of a two-dimensional face image. The extraction of facial feature points in the two-dimensional face image data is an essential constituent requirement. A schematic block diagram for extracting the face feature points is shown in FIG. That is, such as CCD cameraoriginalThe face image data input from the image input unit 101 is output to the feature point extraction unit 102. The feature point extraction means 102 extracts the position of each part (eyes, nose, mouth, etc.) of the face, and outputs the extracted coordinate position of each part and the original image to the original image shaping means 103. Here, the extraction of the position of each part of the face by the feature point extraction unit 102 differs depending on what kind of shaping is performed. That is, the data content to be extracted depends on the molding content of the original image shaping unit 103, and is not a point but an area, or the center position and shape of each part.
[0005]
However, in order to facilitate the description here, the following description will be given as “points”. Further, since the subsequent control contents are individually different, the details are omitted. For example, in the case of a 3D human model, the following control procedure is used.
[0006]
That is, (1) a texture is generated based on feature point information and original image data. (2) A three-dimensional human model (a 3D human object whose face portion is a stick) is generated for texture mapping using the feature point information (generated by modifying the standard model). (3) Perform texture mapping. That is, the 3D person object in which the face image obtained from the original image input unit 101 is embedded is encoded. (4) Display a 3D person object. Such a control procedure (1) to (4) is performed. Here, (1) and (2) are generally processed in parallel. That is, there is no order in control.
[0007]
[Problems to be solved by the invention]
As a method for embodying the feature point extraction means 102 described above, all feature points to be extracted are extracted (this is called method 1), and all feature points to be extracted are designated by the user (this is called method 2). There are two types.
[0008]
In the method 1, generally, (a) the outline of a face is extracted from given original image data. (B) Each part is extracted within the range of the contour area of the obtained face. This is realized by two steps. However, in the process of (a), as shown in FIG. 7, original images with extremely different face outline area sizes (left is large, right is small) and positions (left is center, right is left end) are extremely different. Need to be treated equally, which causes a decrease in extraction accuracy, a decrease in speed performance, and the like. In addition, the method for extracting the face contour region is generally realized by extracting the skin color part (currently, the only way to automatically extract is to rely on the skin color part extraction). In particular, the extraction of the nose in the extraction process of each part in the next step (b) is very difficult because the basic color is the same as that of the skin. In addition, there is a fatal problem in the realization by the method 1. As shown in FIG. 8, it occurs when an image in which a plurality of persons are photographed is input as an original image, and the object to be molded cannot be determined.
[0009]
Next, the problem of the method 2 will be described. Before that, the feature point to be extracted by the feature point extraction unit 102 will be described by taking as an example the case of generating a 3D human model.
[0010]
-Feature points to be extracted by the feature point extraction means 102-
The more feature points that are extracted, the more it is possible to generate a 3D human model (a 3D human object whose face part is a sticker) with which the actual texture is cha-mapped. Naturally, the generated 3D human model can also generate a higher level of facial expression and be of high quality. In this case, considering from the following various factors [1) to 4)], the extracted feature points in Method 2 are the nose, eyes (left and right), mouth, ears (left and right), jaws It is reasonable to consider these as 7 points.
[0011]
1) Recognizing facial irregularities
Since the final purpose is to generate and display a three-dimensional image (three-dimensional image), it is necessary to detect the unevenness of the face to some extent, and in that sense, at least, the nose (corresponding to the apex of the face) It is necessary to extract the eyes. The eyes can be replaced by the mouth, but in terms of expressing facial expressions, the eyes are considered to be given priority.
[0012]
2) Face orientation of the original image
There is no guarantee that the original image is necessarily a complete front image. Therefore, in order to generate a stereoscopic image, it is necessary to calculate the orientation of the face of the original image. In order to calculate the orientation of the face, it is necessary to specify at least three points of each part of the face (of which two points must be paired such as the eyes). In this case, It is considered that the eyes (left and right) and the nose located almost in the center of the face are the best. In other words, if it is limited to the left and right orientations, it is possible to replace the nose with the mouth, but considering the calculation of the orientation of the upper and lower faces, it is impossible to calculate with only the eyes (left and right) and the mouth. is there. In addition, the ears (left and right) can be used as an alternative to the eyes (left and right). However, the eye (left and right) is considered to be slightly unsuitable when considering gaze detection when performing more advanced facial expression generation. And the nose are considered the best.
[0013]
3) Extraction of facial contour region from original image
As described above, in the case of automatically detecting facial feature points, in general, a face contour region is extracted from an original image. Even when this process is implemented by a method in which all facial feature points are specified by the user, a detailed processing content is different, but a three-dimensional human model (a 3D human object with a face sticker) is generated. This is processed at the time of generating the face contour of the three-dimensional human model when (in some cases, generating a standard model prepared in advance). In order to facilitate this process (facial contour generation / deformation), at least three feature points constituting the face contour are designated. These three points are the ear (left and right) and the jaw. Is generally.
[0014]
4) Designation as a feature point
In both the case of caricature generation and 3D model generation, the purpose is to output a shaped image resembling a person in the original image, so at least the eyes and mouth are particularly important facial feature points. . The hairstyle, the presence or absence of eyeglasses, the shape, and the like are also important feature points for specifying a person. However, since these are easy-to-part parts and parts to be parted, they are not the main theme in the present invention. In other words, for eyeglasses, although it is possible to extract the eyeglass part, it may be preferable application software to limit to the original image without eyeglasses and prepare some eyeglass parts. The hairstyle is the same as that of glasses. However, in the case of a hairstyle, the original image cannot be made into a shaved head, so some extraction processing is necessary. Here, for ease of explanation, the principle of glasses is described based on the principle that there is no person image with glasses in the original image. The hairstyle is described based on the principle that the hair contour extraction is included in the process of extracting the face contour region.
[0015]
As described above in 1) to 4), which part of the face is designated as the feature point even in the method 2 (that is, “all the feature points to be extracted are designated by the user”). Depending on the final purpose (molded image), it is common to have 7 points or more in the nose, eyes (left and right), mouth, ears (left and right), and jaws.
[0016]
The following two problems also occur in the method 2 that “allows the user to specify all feature points to be extracted”. That is, there is a problem of a user's designation addition and a problem of a designation guidance message for a pair of parts (eyes and ears).
[0017]
The problem of user-specified addition will be explained. In the era when CCD cameras and the like were relatively expensive devices, and information equipment equipped with this device as standard equipment was commercialized for special purposes, the above-described two-dimensional person Use scenes such as making images into three-dimensional images have been commercialized as one function (application software) of a reasonably expensive system (for example, a TV conference system). For this reason, the quality of the three-dimensional image (how to generate a realistic expression, etc.) tends to be more important than the load on the user's generation of the three-dimensional image.
[0018]
However, recently, along with the price reduction of CCD cameras and the high performance of CPU and image processing arithmetic processing, for example, a three-dimensional image of a sender is added to an e-mail and transmitted. For example, 3D images are also being used for entertainment purposes. For this reason, there is a tendency to increase the demand for reducing the user load (user interface) for generating a three-dimensional image and making it easy.
[0019]
Next, the problem of the designation guidance message for the paired parts (eyes and ears) will be explained. Generally, as shown in FIG. 9, the user interface for designating each feature point specifies “XXX”. It becomes an interactive user interface that displays the message "Please." Here, as shown in FIG. 9, when the message “Please specify the right eye” is displayed, the user (hereinafter also referred to as the user) has a problem as to which eye is pointing. That is, the problem is whether the user points to the original right eye of the person image and the right eye on the right side when viewed from the user.
[0020]
The original image of the person is the one facing the user (pointing person), the right eye is on the left side and the left eye is on the right side. Whether it is called the right eye and pointing to the left eye will probably depend on the user.
[0021]
However, on the processing logic, in the “Specify left eye” message that will continue next, you can even specify the reverse eye part that was specified in the “Specify right eye” message. For example, if there is no problem, but the user gets lost on the way, for example, in the message "Please specify your right eye." "Please specify" message, you may notice a mistake in the previous input, and the user may be worried about what to do or specify the right eye, that is, the correct left side again. It is a problem.
[0022]
In addition, at present, as a method for solving the above problem in a pseudo manner, as shown in FIG. 10, by designating each designated part with a number instead of a name, the user is not caused an error or hesitation. There is also a method of taking a user interface in consideration. However, for example, in the case of the mouth part (number 4 part), it is impossible to solve the problem that raises the question of whether the center of the mouth or anywhere in the mouth is acceptable. That is, the user does not know the internal logic. In this case, it is common to use good logic anywhere around the mouth (that is, extraction processing is performed based on the specified point), but the user does not know that itself, so the question of what to do However, having the question itself becomes a problem.
[0023]
  The present invention was devised to solve such a problem, and its purpose is to make it easy to specify the user input accompanying the facial feature point extraction process in 3D conversion or animation conversion of a 2D face image. Image processing apparatus and image processing method capable of realizing high-speed facial feature point extraction processingThe lawIt is to provide.
[0024]
[Means for Solving the Problems]
  In order to solve the above problems, an image processing apparatus according to the present invention provides:originalAn image input means for inputting an image;A color palette prepared in advance to assist in extracting facial parts;Input by image input meansoriginalImage data and thisoriginalOne face part of the face image existing in the image dataThe noseA message prompting the user to specify the display position ofOn the other hand, the original image data and the color palette are displayed side by side.Display means;In accordance with the message prompting specification of the display position of the nose, which is a facial part, the position coordinates of the nose, which is one point on the displayAn input means for inputtingParameter specifying means for specifying the color value of the nose using the color palette displayed on the display means;Entered by the input meansNose positionCoordinateAnd other than the nose using the color value of the nose specified by the parameter specifying meansFace partsBy extracting the skin color or non-skin colorBased on the facial part extracting means to be extracted, the facial part extracted by the facial part extracting means, and the input base point coordinate informationGenerate a texture and map it to a 3D human modelAnd an image generating means for generating a three-dimensional image.Further, the face part extracting means divides the original image into a plurality of search areas based on a relative positional relationship with the nose position coordinates, and gives the nose color value a predetermined allowable range for each search area. By extracting a closed space or a closed figure that is no longer in the range, facial parts in each search area are extracted.
[0025]
Here, the image input means includes not only a camera device such as a CCD camera but also a selection of a target file from a filed image data group. The input means includes a pointing device for inputting any one point on the display such as a mouse.
[0026]
According to the present invention having such a feature, the processing speed is improved and the quality of image data to be generated or shaped is improved as compared with the conventional method 1 “automatic extraction of all face parts”. Can do. Further, even original image data including a plurality of persons can be dealt with by only specifying one feature point (face part). In addition, compared with the conventional method 2 “2 or more face parts specified by the user”, the operational burden on the user can be reduced and the operation error can be reduced without significantly degrading the quality of the generated or formed image data. Can be prevented.
[0027]
In the image processing apparatus of the present invention, one point input by the input means is used as the position coordinate of the nose. That is, the “message that prompts the user to specify the display position of one face part” is a message that prompts the user to input a nose. Considering facial feature point extraction control, it is unlikely that a sideways image or a backward-facing photo will be input as the original image to be input, and a photo that is generally facing the front is normally input. In this case, the nose is positioned approximately at the center of the face. In addition, when the message “Please specify the nose” is displayed, it is considered that few people specify the base of the nose. In addition, there is only one nose per person. Furthermore, the basic logic of face contour and facial part extraction logic is to detect skin color parts or non-skin color parts, in which case nose extraction is compared to other parts such as mouth and eyes, Automatic extraction is difficult. Furthermore, the layout of human face parts is common to all. For these various reasons, in particular, by letting the user specify the nose point, compared to the case where multiple points are specified, the quality of the final output data is not degraded, and the operational burden on the user is reduced or incorrect operations are performed. Prevention can be achieved.
[0028]
  The image processing apparatus of the present invention assists in extracting facial parts.Color palettePreparationAndThe parameter designation means isSpecify the color palette value at the drag position as the nose color value at the drop position by dropping it on the nose position of the original image.It is like that.According to the present invention, a parameter (color palette value) for assisting a feature point is an operation of continuing to press the display position of each parameter value that is converted into a display object (drag operation: a mouse is moved by pointing while continuing to hold a mouse button. The operation of pressing the pen on the tablet and pointing and moving with the pen) is determined by the coordinates when the button is pressed, and the display position of the face part is an operation that stops the continuation of the pressing operation (drag operation) (Drop operation: mouse button up or pen up) Determine the coordinates.
[0029]
  The image processing apparatus of the present invention includes a color palette that assists in extracting facial parts, and the parameter specifying means operates a slider provided in the color palette displayed on the display means, A color palette value (auxiliary parameter) at the slider position is designated as the color value of the nose. The display unit displays a hair color palette and a plurality of patterns of face shapes, and the parameter designating unit is a slider provided in the hair color palette displayed on the display unit. , The color palette value at the slider position is designated as the hair color, and one face shape is designated from among a plurality of patterns of face shapes displayed on the display means.
[0030]
  This auxiliary parameter(Color palette value)Is an alternative means for obtaining high-quality output data even if the number of feature points specified is reduced. However, the amount of auxiliary parameter specified is drastically reduced when no point is specified or when one point is specified. To do. In other words, the quality can be ensured as it is without detailed specification. Also related to usage scene, product concept, extraction logic, etc.The paIn some cases, the purpose can be achieved by specifying parameters, and the drag-and-drop operation is optimal as the man-machine interface in this case.
[0031]
  In addition, a method of face feature point extraction processing that makes it easy for a user to specify input in face feature point extraction processing in three-dimensional conversion or animation conversion of a two-dimensional face image is not necessarily in the form of an apparatus. It is not limited and can be realized as a method. Therefore, the image processing method of the present invention generates a three-dimensional image from an input image.Image generation method using image processing apparatusBecauseA color palette for assisting the extraction of the facial parts is prepared in advance, the step of storing the original image input from the image input means, and the image input meansThe input original image data and one face part of the face image existing in the original image dataThe noseA message prompting the user to specify the display position ofDisplay meansSteps to display;Displaying the original image data input from the image input means and the color palette on the display means, storing the position coordinates of the nose that is one point on the display input from the input means, A step of storing a color value of the nose designated using the color palette displayed on the display means, and a face component extraction means, the position coordinates of the nose inputted from the input means and the color of the designated nose Extracting a facial part other than the nose by extracting a skin color or a part other than the skin color using the value, and an image generating means at the facial part extracting meansBased on the extracted facial parts and the coordinate information of the input base pointGenerate a texture and map it to a 3D human modelGenerating a three-dimensional image.
[0032]
  Furthermore, the technical idea according to the present invention is as follows:Recording medium on which an image processing program for causing a computer to execute each step of the above image processing method is recordedCan also be provided as.
[0033]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
[0034]
FIG. 1 is a block diagram of an image processing apparatus according to the present invention for converting a person's two-dimensional face image data into a three-dimensional face image.
[0035]
The image processing apparatus includes a two-dimensional image input control unit 10, an original image designation unit 11, a nose point designation unit 12, a feature point extraction unit 13, a 3D image encoding unit 14, a 3D image display unit 15, a CCD camera 20, and an input. The apparatus 30, the output device 40, the original image storage device 50, various variable / constant storage devices 51, a standard person model or the like (encoding constant) storage device 52, and a formed image storage device 53.
[0036]
The two-dimensional image input control unit 10 controls the shutter to send the original image data to the connected CCD camera 20 and stores the original image data sent from the CCD camera 20 in the original image storage device 50 ( Control until file creation. In this embodiment, the CCD camera 20 is an image capturing device capable of inputting a color image.
[0037]
The original image designating unit 11 reads an original image file designated by the input device 30 from a plurality of original image files stored in the original image storage device 50, and delivers the original image file to the nose point instruction unit 12. Control up to.
[0038]
The nose point instruction unit 12 outputs the original image data delivered from the original image designating unit 11 to the output device 40, and also inputs the nose point input by the user via the input device 30 and an extraction auxiliary parameter (here, , Face color values described later) are controlled until they are delivered to the feature point extraction unit 13.
[0039]
The feature point extraction unit 13 reads various variables and constant values for extraction from the various variable and constant storage device 51 with reference to the nose point coordinates and face color values (extraction auxiliary parameters) delivered from the nose point instruction unit 12. The feature points of each face are extracted using these values. The extraction method uses the nose point coordinates as the base coordinates, and uses various variables and constant values (extraction auxiliary parameters) read based on the face color values, etc., to divide the original image into several regions, The extraction processing of each part to be extracted such as the mouth portion is processed only in the corresponding region. Further, the feature points extracted here are delivered as feature point information to the 3D image encoding unit 14. A series of control up to this point is performed by the feature point extraction unit 13.
[0040]
The 3D image encoding unit 14 generates a texture based on the delivered feature point information and original image data, and then reads an appropriate 3D human model from the standard human model storage device 52 based on the feature point information. A series of controls for performing texture mapping and storing in the shaped image storage device 53 is performed.
[0041]
The 3D image display unit 15 reads the 3D image from the molded image storage device 53 and performs control to display the 3D image on the output device 40.
[0042]
The above is the control of each part of the image processing apparatus of the present invention. The control content of the nose point instruction unit 12 which is a characteristic part of the present invention will be described in more detail below.
[0043]
FIG. 2 shows an example of the screen layout of the nose point instruction unit 12 when the user specifies an approximate value (color palette value) of the human face color of the original image as an extraction auxiliary parameter. Here, the face color palette specification area [(cX min, cY min) to (cX man, cY man)] is a gradation display of colors (Cmin to Cman) that can be a human face color, and the original image display area [(uX min, uY min) to (uX man, uY man)] displays the image data input from the original image designating unit 11. In addition, a message that guides the user to drag and drop from the position close to the face color of the person in the original image to the nose position of the original image (in this example, "Drag the color palette that matches the person face color, "Please drop at the nose position" is displayed in the message display area at the bottom of the output screen.
[0044]
This guides a series of input operations by dragging a portion close to the face color of the person in the face color palette designated area and dropping it on the nose position of the original image, and the input obtained by the user's input operation Based on the information, the nose point coordinates and the face color palette value in the original image display area are calculated.
[0045]
FIG. 3 is a flowchart showing the control contents of the nose point instruction unit 12, that is, the arithmetic logic (that is, the arithmetic logic of the nose point coordinates and the face color palette value).
[0046]
That is, first, an input screen for guiding a series of input operations for dragging a portion close to the face color of the person in the face color palette designated area shown in FIG. 2) is generated and displayed (step S1). Next, when a “left button down” message is input from the input device 30, for example, a mouse (step S 2), the condition that the coordinate where the mouse is down is within the face color palette designated area (YES in step S 3). The face color palette value is substituted into the variable FaceC from the button-down coordinates (step S4). Subsequent to this processing, if a “left button up” message is input from the mouse while the mouse is down (when YES is determined in step S5), the coordinates where the mouse is up are within the original image display area. The button-up coordinates are substituted into (NosePx, NosePy) as a condition (provided that YES is determined in step S6) (step S7). Finally, using the variables FaceC, NosePx, and NosePy as parameters, control is transferred to the feature point extraction unit 13 (step S8).
[0047]
Note that the feature point extraction unit 13 uses the variables FaceC, NosePx, and NosePy input from the nose point specification unit 12 to determine facial feature points (coordinates, regions, or objects) other than the nose. Is extracted depending on the specifications of the feature point extraction unit 13. Since the basic logic of this extraction process is the same as the conventional logic for extracting facial feature points, a detailed description is omitted here, but the basic idea of the extraction logic is to extract the skin color or a part other than the skin color. This is realized.
[0048]
FIG. 4 shows an outline of how the feature point extraction unit 13 uses the nose point coordinate (facial color palette) values (FaceC, NosePx, NosePy) (the extraction logic using these three variables). Summary) is listed in tabular form.
[0049]
Explaining along FIG. 4, first, the contour extraction can be realized by extracting a closed space that is no longer the face color palette value FaceC ± d (d is an allowable range) around the nose point (NosePx, NosePy). . A method of automatically detecting feature points to be extracted because the processing can be performed centering on a nose point specified in advance (that is, a base point for extracting a region that is not a facial color can be used as a nose point). Compared with the method 1) described in the section, the extraction logic is very easy and enables extraction with extremely high accuracy.
[0050]
Next, in the extraction of the right eye part, the search area is u1 based on the relative positional relationship with the nose point, and the basic extraction logic is a closed figure that is no longer the face color palette value FaceC ± d (d is an allowable range). It can be realized by extracting from the presence of black eyes. Compared with method 1 as described above, in addition to simplification of logic and improvement of accuracy, it is possible to speed up the search time by narrowing down the search area. The extraction of the left eye part and the mouth part is the same except for the search area.
[0051]
Next, the extraction of the ears and jaws is generally not particularly performed in Method 1. In addition, the method of causing the user to specify all the feature points to be extracted (method 2 described in the section of the problem to be solved by the invention) is to specify the ears and jaws by extracting the contour and the orientation of the face. The purpose of this process is to improve accuracy and simplify logic, and it is desirable to substitute for application software that does not require much accuracy. In other words, since the “face orientation” can be calculated from the nose point and the left and right eye points, the orientation is almost the same as the face orientation. In addition, the face contour extraction and the like will not cause any serious problems in terms of speed and accuracy as long as the extraction base point is clear, and other than the method of specifying the facial feature points, It is possible to substitute by a method of specifying a plurality of extraction auxiliary parameters.
[0052]
In addition, although the said embodiment is embodiment when the extraction assistance parameter to which a user designates is 1 parameter of face color in the control content of the nose point instruction | indication part 12, as above-mentioned, there are several to a user. By specifying the extraction assistance parameter, the facial feature point extraction accuracy and processing speed may be shortened.
[0053]
An example of the input screen is shown in FIG. In this example, the face color and hair color are designated by a slider, and the face shape prepared in advance of several patterns is designated by radio buttons. Note that the hair color and face shape parameters that can be newly specified here (in addition to the specification of the face color palette in FIG. 2) can be directly input to the feature point extraction unit 13 in the same manner as the face color parameters. Delivered. In the feature point extraction unit 13, this parameter is mainly used for face contour extraction.
[0054]
Increasing the number of auxiliary extraction parameters not only simplifies the extraction logic, speeds up the extraction, improves the accuracy, but also “has all the feature points to be extracted” specified by many user operations (method 2). Even if you compare, there is no doubt as to which eye will be clicked by "Click on the right eye" and where in the mouth "Click on the mouth" clicks. That is, it is possible to intuitively and easily change the face color and hairstyle color with the slider bar and select the contour.
[0055]
As described above, the image processing apparatus according to the above-described embodiment is realized by the three-dimensional face image processing program stored in a storage unit (not shown). This program is stored in a computer-readable recording medium. In the present invention, this recording medium is not shown, but may be a program medium provided with a program reading device inside the image processing apparatus, and readable by inserting the recording medium therein, or the device. It may be stored in storage means such as an internal program memory. In any case, the stored program may be directly accessed and executed, or in any case, the program is read and the read program is downloaded to a main memory (not shown). Then, the program may be executed. It is assumed that this download program is stored in the apparatus main body in advance.
[0056]
Here, the program medium is a recording medium configured to be separable from the main body, such as a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy disk or a hard disk, CD-ROM, MO, MD, DVD. It may be a medium carrying a fixed program including a disk system such as an optical disk, a card system such as an IC card or an optical card, or a semiconductor memory such as a mask ROM, EPROM, EEPROM, flash ROM or the like.
[0057]
Further, in the present invention, when a means capable of communicating with the outside (wireless communication function or wired communication function via a public line such as the Internet) is provided, it is used to connect to the outside and from there It may be a medium that carries the program in a fluid manner so as to download the program. When the program is downloaded from the communication network in this way, the download program may be stored in the apparatus main body in advance, or may be installed from another recording medium.
[0058]
【The invention's effect】
According to the image processing apparatus of the present invention, the processing speed can be improved and the quality of image data to be generated or shaped can be improved as compared with the case where all conventional facial parts are automatically extracted. Further, even original image data including a plurality of persons can be dealt with by designating only one feature point. In addition, compared with the conventional case where two or more facial parts are specified, it is possible to reduce the operational burden on the user and prevent erroneous operations without significantly degrading the quality of the generated or molded image data. it can.
[0059]
Further, according to the image processing apparatus of the present invention, since one point input by the input means is used as the position coordinate of the nose, the final output is compared with the case where a plurality of points are specified by specifying the nose points by the user. It is possible to reduce the operation burden on the user and prevent erroneous operations without degrading the data quality.
[0060]
Further, according to the image processing apparatus of the present invention, the image processing apparatus further includes parameter specifying means for assisting the extraction of the facial part, and the parameter specifying means drops the auxiliary parameter displayed after moving the displayed auxiliary parameter to the position of the nose by a drag operation. Since the configuration is such that it is specified by performing an operation, high-quality output data can be obtained even if the number of specified feature points is reduced.
[0061]
Furthermore, according to the present invention, there is provided an image processing method capable of improving the processing speed and improving the quality of image data to be generated or shaped as compared with the conventional method of automatically extracting all facial parts. be able to.
[Brief description of the drawings]
FIG. 1 is a block diagram of an image processing apparatus according to the present invention.
FIG. 2 is an explanatory diagram showing an example of a screen layout when a user specifies an approximate value of a human face color of an original image as an extraction auxiliary parameter.
FIG. 3 is a flowchart showing arithmetic logic of nose point coordinates and facial color palette values.
FIG. 4 is an explanatory diagram showing an outline of use of nose point coordinate values in a feature point extraction unit;
FIG. 5 is an explanatory diagram showing an example of an input screen for specifying extraction auxiliary parameters.
FIG. 6 is a schematic block diagram for extracting face feature points.
FIG. 7 is an explanatory diagram illustrating an example of an original image in which the size and position of a face outline region are extremely different.
FIG. 8 is a diagram for explaining a problem that occurs when an image in which a plurality of persons are photographed is input as an original image.
FIG. 9 is an explanatory diagram showing an example of a user interface for designating feature points.
FIG. 10 is an explanatory diagram illustrating an example of a user interface for designating each designated portion by a number instead of a name.
[Explanation of symbols]
10 2D image input controller
11 Original image designation part
12 Nose point designation part
13 Feature point extraction unit
14 3D image encoding unit
15 3D image display
20 CCD camera
30 Input device
40 Output device
50 Original image storage device
51 Various variable and constant storage devices
52 Standard person model (encoding constant) storage device
53 Molded image storage device

Claims (6)

An image input means for inputting an original image;
A color palette prepared in advance to assist in the extraction of facial parts;
While displaying a message prompting designation of the display position of the nose is one facial part of the face image existing in the original image data and within the original image data input by said image input means to the user, the original image data And display means for displaying the color palette side by side ,
In accordance with the message prompting the designation of the display position of the nose that is a facial part, input means for inputting the position coordinates of the nose that is one point on the display;
Parameter specifying means for specifying the color value of the nose using the color palette displayed on the display means;
Using the color values of the designated nose by the position coordinates and the parameter specifying means nasal entered by the entering force means, face parts to be extracted by extracting the locations other than the skin color or skin color facial parts other than the nose Extraction means;
Image generating means for generating a texture based on the face part extracted by the face part extracting means and the input coordinate information of the base point , mapping to a three-dimensional human model, and generating a three-dimensional image; An image processing apparatus characterized by that. The image processing apparatus according to claim 1.
The parameter designating means drags a desired location in the color palette designation area displayed on the display means and drops it on the nose position of the original image, thereby obtaining the color palette value of the drag position at the drop position. An image processing apparatus, which is specified as a color value of a nose . The image processing apparatus according to claim 1.
Image the parameter specifying means, by manipulating the slider provided in the color palette displayed on the display unit, characterized by specifying a color palette value of the slider position as the color values of the nose Image processing device. The image processing apparatus according to claim 3.
The display means displays a hair color palette and a plurality of patterns of face shapes,
The parameter designating means designates a color palette value at the slider position as a hair color by operating a slider provided in the hair color palette displayed on the display means, and is displayed on the display means. images processor characterized by specifying one of the face shape from among a plurality of patterns of the face shape. The image processing apparatus according to any one of claims 1 to 4, wherein:
The face part extracting means divides the original image into a plurality of search areas based on a relative positional relationship with the nose position coordinates, and each nose color range has a predetermined allowable range for the nose color value. An image processing apparatus that extracts facial parts in each search region by extracting a closed space or a closed figure that is no longer a part . An image processing method using an image processing apparatus that generates a three-dimensional image from an input image,
A color palette is prepared in advance to assist in extracting facial parts.
Storing the original image input from the image input means;
Displaying on the display means a message prompting the user to specify the display position of the nose that is one face part of the original image data input from the image input means and the face image existing in the original image data;
Displaying the original image data input from the image input means and the color palette on the display means;
Storing the position coordinates of the nose, which is one point on the display, input from the input means;
Storing the color value of the nose designated using the color palette displayed on the display means;
A step of extracting facial parts other than the nose by extracting a facial color other than the nose using a position coordinate of the nose input from the input means and the color value of the designated nose by extracting the facial color. When,
A step of generating a texture based on the coordinate information of the facial part extracted by the facial part extracting means and the input base point, and generating a three-dimensional image by mapping to a three-dimensional human model ; An image processing method comprising:

Images