JP2005078158A - Image processing device, image processing method, program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device and the like for quickly, precisely and easily recognizing an area, based on an original image. <P>SOLUTION: An image processing device 101, when an operator specifies a square guide line position (step 801), sets a processing area inside a square guide line (step 802). In the processing area, the device 101 recognizes an area (flesh) (step 803). Next, the device 101 sets a processing area outside the area (flesh) (step 805), and recognizes an area (hair) in the processing area (step 806). Then, the device 101 sets a processing area within other than the already recognized areas (step 808) and recognizes area (others) in the processing area (step 809). The device 101 generates mask image data by performing a prescribed process to pixel data in the recognized area to store it to a storage part (step 10). <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus that synthesizes an image with a predetermined region of an original image to create a composite image. More specifically, the present invention relates to an image processing apparatus that performs image composition after recognizing a composition area based on an original image.

  In recent years, there are various colors in cosmetic makeup palettes, and there are many variations when combining various point makeups. It is difficult to try out all patterns, and cosmetic purchasers are demanding a means of searching for a favorite color while easily trying many patterns. In order to realize this requirement, an image composition technique is used.

2. Description of the Related Art Conventionally, an image synthesizing apparatus that synthesizes a texture image for synthesis with a predetermined region of an original image and creates a synthesized image has been proposed (see, for example, [Patent Document 1]).
In addition, an image processing method for performing makeup simulation by creating a composite image with makeup has been proposed (see, for example, “Patent Document 2” and “Patent Document 3”).

The technology shown in [Patent Document 1] facilitates a composite image without a sense of incongruity without special experience or intuition when a texture image such as a product sample image is synthesized with an original image such as a product construction example. To create.
The technique disclosed in [Patent Document 2] extracts parts such as eyes and lips from a face image and synthesizes an image in which makeup is applied to these parts. In this case, the user extracts a part using a rectangular frame for each part.
The technique shown in [Patent Document 3] performs image composition with makeup using a moving image, and confirms the effect of makeup while changing facial expressions.

JP 2000-293667 A JP 2000-151985 JP 2003-44837 A

However, in the technique of [Patent Document 1], it is necessary to prepare mask data indicating a synthesis area in advance (see paragraph [0017] of [Patent Document 1]). When creating mask data based on the original image, it is necessary to manually specify the synthesis area, which increases the labor burden and lacks accuracy. When applied to the synthesis process, it is difficult to specify the contour up to the tip of the hair. Therefore, when performing the image synthesis process, there is a problem that the vicinity of the boundary between the hair and the skin becomes unnatural.

  Further, the technique of [Patent Document 2] has a problem that the processing load increases because the synthesis area is recognized in the rectangular frame area designated by the user each time the image synthesis process is performed. Furthermore, each time the user performs image composition for each part, the user needs to surround and select the part while adjusting the size, shape, etc. of the rectangular frame, which increases the labor burden. There is.

  In the technique of [Patent Document 3], the makeup simulation apparatus recognizes the contours of eyes, eyebrows, lips, and the like based on the position of the eyes. Then, in accordance with changes in the facial expression of the person, it is necessary to track the contours of the eyes, eyebrows, lips, etc., in order to create a moving image. Therefore, there are problems in terms of recognition accuracy, stability, and processing speed.

  The present invention has been made in view of the above-described problems, and provides an image processing apparatus and the like that can perform region recognition and the like quickly, highly accurately, and simply based on an original image. For the purpose.

  In order to achieve the above object, a first invention is an image processing apparatus that performs image processing on an original image, and includes a processing range specifying means for specifying a processing range and an attribute range specifying means for specifying an attribute range. And an area recognition means for recognizing the area by extracting pixels belonging to the attribute range in the processing range of the original image.

In the first invention, the image processing apparatus designates a processing range and an attribute range, extracts pixels belonging to the attribute range in the processing range of the original image, and recognizes the region.
The image processing apparatus can add an additional area to the recognized area and newly recognize it as one area.

The processing range indicates the position range of pixel data to be processed. The image processing apparatus determines and processes whether or not the pixel located within the processing range is a pixel belonging to the region, and determines whether the pixel located outside the processing range is subject to determination or processing. Exclude from
The processing range is specified by surrounding the predetermined range of the original image by using the shape-variable and size-variable rectangular guidelines for the original image displayed on the display device by the operator of the image processing apparatus. be able to. The image processing apparatus can also automatically set the outside of the recognized area as the processing range.
Thus, by specifying the processing range, it is possible to reduce the processing load and the processing time.

The attribute range indicates a range of attributes (color gradation, luminance, hue, brightness, saturation, etc.) of pixels belonging to the region. The image processing apparatus recognizes the position / range of the pixel of the original image corresponding to the attribute range as a region.
The attribute range can be designated by designating a reference color and a distance from the reference color in a color range, for example, three primary color (RGB) color spaces. The reference color may be specified by the operator of the image processing apparatus specifying any pixel of the original image displayed on the display device with a pointing device such as a mouse.

The image processing apparatus recognizes a skin area by specifying a processing area by surrounding a predetermined area of the original image showing the face, and then sets the outside of the recognized skin area as the processing area Recognize
In this case, the image processing apparatus performs the recognition process of the skin region by limiting the processing range, and then recognizes the hair region using the outside of the recognized skin region as the processing range. The region such as the pupil is not misrecognized from each other, and the processing range is specified only once by surrounding the predetermined range, and the speed of processing is not impaired.

The image processing apparatus performs predetermined processing on the recognized area and holds it in the storage unit (for example, a mask image with the color set to white ((R component, G component, B component) = (255, 255, 255)). Create.)
Further, the image processing apparatus holds an image for composition (for example, an image with makeup or a texture image) in the storage unit.

When the region and the image for synthesis are designated, the image processing apparatus calls the data related to the region and the data related to the image for synthesis from the storage unit, synthesizes the image for synthesis in the region of the original image, Create and display. For example, the image processing apparatus synthesizes and displays an image in which makeup is applied to the skin region.
In this case, simple, high-definition, and realistic makeup can be virtually realized in real time on the basis of image data of the model and the purchaser's own face.

A second invention is an image processing method for performing image processing on an original image, the processing range specifying step for specifying a processing range, the attribute range specifying step for specifying an attribute range, and the processing of the original image A region recognition step of recognizing a region by extracting pixels belonging to the attribute range in the range.
The second invention relates to an image processing method executed by the image processing apparatus according to the first invention.

A third invention is a program for causing a computer to function as the image processing apparatus of the first invention.
A fourth invention is a computer-readable recording medium on which a program for causing a computer to function as the image processing apparatus of the first invention is recorded.
The above-mentioned program may be held and distributed on a recording medium such as a CD-ROM, or the program can be transmitted and received via a communication line.

  According to the present invention, it is possible to provide an image processing apparatus or the like that enables quick, high-precision, and simple area recognition based on an original image.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of an image processing apparatus and the like according to the invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, the same reference numerals are given to components having substantially the same functional configuration, and redundant description will be omitted.

First, the configuration of the image processing apparatus 101 according to the embodiment of the present invention will be described with reference to FIG.
FIG. 1 is a hardware configuration diagram of an image processing apparatus 101 according to an embodiment of the present invention.
As shown in FIG. 1, in the image processing apparatus 101, a control unit 102, an input unit 103, a display unit 104, an input / output unit 105, a storage unit 106, a communication unit 107 and the like are connected to each other via a system bus 108. Note that the hardware configuration in FIG. 1 is an example, and various configurations can be adopted depending on the application and purpose.

  The control unit 102 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

  The CPU calls and executes an execution program stored in a storage unit, ROM, recording medium, etc., an OS (operation system) program, etc., in a work memory area on the RAM, and performs arithmetic processing (four arithmetic operations, comparison operations, etc.) It performs operation control of hardware and software, and implements various functions described later (see FIGS. 7 to 9, FIG. 14, etc.).

The ROM is a non-volatile memory, and permanently stores programs, data, and the like, and stores, for example, a BIOS (Basic Input Output System) that controls input / output to a hard disk or the like in cooperation with the OS.
The RAM is a volatile memory, and temporarily stores programs, data, and the like.

The input unit 103 inputs data and includes an input device such as a pointing device such as a mouse, a keyboard, and a numeric keypad.
An operator can perform operation instructions, operation instructions, data input, maintenance management, and the like for the image processing apparatus 101 via the input unit 103.

The display unit 104 includes a display device (display device) such as a CRT monitor and a liquid crystal panel, and a logic circuit (video adapter or the like) for realizing a video function of a computer in cooperation with the display device. The video adapter generates an image signal to be output to the display device.
The display device may be configured integrally with the image processing apparatus 101 or may be configured independently and connected to the image processing apparatus 101.

The input / output unit 105 performs data input / output, for example, a floppy disk drive, a PD drive, a CD drive (-ROM, -R, -RW, etc.), a DVD drive (-ROM, -R, -RW, -RAM). Etc.) and media input / output devices such as MO drives.
The operator can input a program executed by the CPU, data necessary for program execution, an OS (Operating System), and the like to the image processing apparatus 101 via the input / output unit 105. And install application programs on the hard disk.

  The storage unit 106 is a device that stores data, and includes a hard disk, a memory, and the like. The storage unit 106 stores a program executed by the CPU, data necessary for program execution, an OS (Operating System), and the like.

The communication unit 107 is a communication interface that mediates communication with the image capturing apparatus 109 (such as a digital camera) connected to the image processing apparatus 101 via the connection line 110, and includes a communication control device such as a communication adapter, a communication port, and the like. Have.
Note that when the image processing apparatus 101 and the terminal apparatus are connected via a network such as the Internet or a LAN to exchange various data, the communication unit 107 is provided with a network adapter, a LAN adapter, and the like. The network adapter and the LAN adapter communicate with the outside in cooperation with the OS communication program controlled by the CPU.
The system bus 108 is a path that mediates transmission and reception of control signals, data signals, and the like between each unit and each device.

Further, an output unit (not shown) having a printer device or the like may be provided and used for maintenance and management of the image processing apparatus 101.
Note that a computer or the like having the same configuration can be used as the image processing apparatus 101.

  The imaging device 109 is a device that captures an image of the subject 111 and acquires the image data, and is, for example, a digital camera. Image data acquired by the imaging device 109 is input to the image processing apparatus 101 via the connection line 110 and the communication unit 107, or via the recording medium 112 and the input / output unit 105.

Next, information stored in the storage unit 106 of the image processing apparatus 101 will be described with reference to FIGS. 1 and 2 to 6. As shown in FIG. Image data 121, mask image data 122, texture image data 123, composite image data 124, execution program 125, OS 126, and the like are stored.

The original image data 121 holds display data of the original image for each pixel. The original image is an image that is a source of image synthesis, and is, for example, an image obtained by imaging the subject 111 by the imaging device 109 or an image input by the recording medium 112.
The mask image data 122 holds display data of the mask image for each pixel. The mask image is an image indicating a region where image synthesis is performed, and a texture image is synthesized in the region. As will be described in detail later, the mask image is created based on the original image.

The texture image data 123 holds texture image display data for each pixel. The texture image is image data to be synthesized with the original image, for example, data relating to colors, patterns, and the like. It is not always necessary to specify the shape, position, etc. for the texture image.
The composite image data 124 holds display data of the composite image for each pixel. The synthesized image is an image obtained by synthesizing the texture image with the original image. For example, the synthesized image is an image obtained by replacing the color and pattern of a specific area of the original image with the color and pattern of the texture image.

FIG. 2 is a diagram illustrating an aspect of pixel data.
The pixel data 201 illustrated in FIG. 2 includes fields such as a position 211, an R component 212, a G component 213, and a B component 214. The pixel data can have various configurations.
The original image data 121, the mask image data 122, the texture image data 123, and the composite image data 124 hold display data (pixel data) for each pixel.

A position 211 indicates a pixel position, for example, a coordinate position (0, 0) on the XY plane.
The R component 212, the G component 213, and the B component 214 indicate the values of the R component (red), G component (green), and B component (blue) of the three primary colors in the color space, respectively. 128 ”,“ 10 ”, and“ 50 ”.

  Note that the mask image data 122 is obtained by extracting pixels in a region where image composition is performed. The values of the R component 212, the G component 213, and the B component 214 of the mask image data 122 are held based on the mode of the synthesis process in the image processing apparatus 101. For example, the mask image obtained by painting the image synthesis area white In the case of the data 122, the values of the R component 212, the G component 213, and the B component 214 are “255”, “255”, and “255”, respectively.

FIG. 3 is a diagram illustrating an aspect of the original image 301 that is developed and displayed based on the original image data 121. The data (original image data) of the original image 301 holds display data (pixel data) for each pixel 302.
FIG. 4 is a diagram illustrating an aspect of the mask image 401 developed and displayed based on the mask image data 122. The data of the mask image 401 (mask image data) holds display data (pixel data) for each pixel 402.
FIG. 5 is a diagram showing an aspect of a texture image 501 developed and displayed based on the texture image data 123. The texture image 501 data (texture image data) holds display data (pixel data) for each pixel 502.
FIG. 6 is a diagram showing an aspect of a composite image 601 developed and displayed based on the composite image data 124. The data of the composite image 601 (composite image data) holds display data (pixel data) for each pixel 602.

Next, the overall operation of the image processing apparatus 101 will be described with reference to FIG.
FIG. 7 is a flowchart showing the overall operation of the image processing apparatus 101.

The image processing apparatus 101 (control unit 102) holds the input original image data in the storage unit 106 (step 701). The input of the original image data may be the input of the original image data captured by the imaging device 109 via the connection line 110, or the input of the original image data recorded on the recording medium 112 via the input / output unit 105. You may make it do.
The image processing apparatus 101 (control unit 102) creates mask image data indicating a region based on the original image data, and holds it in the storage unit 106 (step 702).

  When a region to be synthesized with the original image (synthesized region) is designated, the image processing apparatus 101 (control unit 102) extracts mask image data indicating the region from the storage unit 106 (step 703), and a texture for synthesis. When an image is designated, the designated texture image data is extracted from the storage unit 106 (step 704), and the original image data and the texture image data are synthesized in the synthesis area indicated by the mask image data to create synthesized image data. The image is held in the storage unit 106 and a composite image is displayed (step 705).

FIG. 17 is a diagram illustrating an aspect of an operation screen 1700 displayed by the image processing apparatus 101 in image composition processing or the like (FIG. 7: Steps 703 to 705).
When the operator designates an area (skin) 1701 as a synthesis area and designates a texture image 1702 as a texture image to be synthesized via the screen 1700, the image processing apparatus 101 assigns the texture image 1702 to the area (skin) 1701. The synthesized image 1703 is displayed.

Next, the operation of the image processing apparatus 101 in the mask image data creation process (FIG. 7: step 702) will be described with reference to FIG.
FIG. 8 is a flowchart showing the operation of the image processing apparatus 101 in the mask image data creation process.

First, the image processing apparatus 101 creates image data (mask image data (skin)) indicating a skin region based on the original image data.
When the operator designates the rectangular guideline position (step 801), the image processing apparatus 101 (control unit 102) sets the processing range inside the rectangular guideline (step 802).

  The processing range indicates the position range of the pixel data to be processed. The image processing apparatus 101 determines and processes whether or not the pixel data located within the processing range is a pixel belonging to the region, and the pixel data located outside the processing range is subject to determination. Excluded from processing.

The image processing apparatus 101 (control unit 102) recognizes a region (skin) in the processing range (step 803), extracts pixel data belonging to the region (skin) from the original image data, and performs predetermined processing ( For example, the color is set to white ((R component, G component, B component) = (255, 255, 255))) to create mask image data (skin) and hold it in the storage unit 106 (step 804).
The area recognition process (step 803) will be described later.

Subsequently, the image processing apparatus 101 performs processing for creating image data (mask image data (hair)) indicating a hair region based on the original image data.
The image processing apparatus 101 sets the processing range outside the recognized area (skin) (step 805).

The image processing apparatus 101 (control unit 102) recognizes a region (hair) within the processing range (step 806), extracts pixel data belonging to the region (hair) from the original image data, and performs predetermined processing ( For example, the color is set to white ((R component, G component, B component) = (255, 255, 255))) to create mask image data (hair) and hold it in the storage unit 106 (step 807).
The area recognition process (step 806) will be described later.

Subsequently, the image processing apparatus 101 creates image data (mask image data (others)) indicating other regions (eye region, lip region, eyebrow region, etc.) based on the original image data.
The image processing apparatus 101 sets the processing range other than the recognized area (step 808).

The image processing apparatus 101 (control unit 102) recognizes a region (other) in the processing range (step 809), extracts pixel data belonging to the region (other) from the original image data, and performs predetermined processing ( For example, the color is set to white ((R component, G component, B component) = (255, 255, 255))) to create mask image data (others), which is held in the storage unit 106 (step 810).
The area recognition process (step 809) will be described later.

Next, the operation of the image processing apparatus 101 in the region recognition process (FIG. 8: step 803, step 806, step 809) will be described with reference to FIG.
FIG. 9 is a flowchart showing the operation of the image processing apparatus 101 in the area recognition process.

  When the color range of a pixel to be extracted as a region is designated (step 901), the image processing apparatus 101 (control unit 102) extracts pixel data in the designated color range from the original image data within the processing range. Thus, an area is acquired (step 902).

When adding an area (Yes in step 903), the image processing apparatus 101 (control unit 102) repeats the processes in steps 901 to 902.
When an area is not added (No in step 903), the image processing apparatus 101 (control unit 102) corrects the missing area (step 904) and determines the area (step 905).

FIG. 10 is an explanatory diagram of the missing area correction process (FIG. 9: step 904).
In the region acquisition process (FIG. 9: Step 901 to Step 903, etc.), a portion that should be recognized as a region may not be recognized as a region due to the influence of illumination at the time of photographing the original image data, resulting in a missing region. is there.

As shown in FIG. 10, the acquisition region (hair) 1001 before the missing region correction processing has a missing region 1002 inside thereof. In this case, the image processing apparatus 101 corrects the missing region 1002 as a region (hair) to obtain a region (hair) 1003.
For example, the image processing apparatus 101 calculates the area ratio between the region (hair) 1001 and the missing region 1002, and corrects the missing region 1002 as a region (hair) when the proportion of the missing region 1002 is less than a predetermined rate. .

  Next, the operation of the image processing apparatus 101 and the flow of processing will be described with reference to FIGS.

FIG. 11 is a diagram showing an aspect of a screen 1100 displayed by the image processing apparatus 101 in the original image data input process (FIG. 7: step 701).
An operator (subject 111 or the like) photographs the subject 111 with the imaging device 109 via the screen 1100 and inputs original image data to the image processing device 101.

  The operator confirms whether the face portion of the subject 111 is within the image display frame 1101 and then presses the button 1102 (“shoot”). The imaging device 109 captures the face portion of the subject 111, and the image processing device 101 holds the image data sent from the imaging device 109 in the storage unit 106 as original image data.

  FIG. 12 shows one aspect of a screen 1200 displayed by the image processing apparatus 101 in the mask image data (skin) creation processing (FIG. 7: Step 702, FIG. 8 Steps 801 to 804, FIG. 9: Steps 901 to 905). FIG.

The operator adjusts the position, size, shape, and the like of the rectangular guideline 1201 on the screen 1200 so that the skin area can be accommodated (step 801), and the color of the pixel data extracted as the area (skin) from the original image data A range is designated (step 901), and a button 1204 ("Acquire region (skin)") is pressed.
The image processing apparatus 101 acquires a region (skin) 1205 (step 902).

  The color range can be specified using the reference color specifying pointer 1202 and the range length specifying bar 1203. The reference color designation pointer 1202 designates a color that becomes the reference of the color range, and the color of the pixel that has been pointed or clicked becomes the reference color. The range length designation bar 1203 designates the size of the color range and the like, and indicates, for example, the limit of the distance from the reference color in the color space.

FIG. 16 is a diagram illustrating an aspect of the color range 1601.
The color range 1601 is defined by a sphere whose center is a reference color 1602 (R ₀ , G ₀ , B ₀ ) and whose radius is a range length 1603 (L) in the RGB color space.
If the color of the pixel pointed or clicked by the reference color designation pointer 1202 is the reference color (R ₀ , G ₀ , B ₀ ) and the value designated by the range length designation bar 1203 is the range length (L), the color range Is a set of pixels having color values in the RGB space that satisfy [Equation 1].
[(R−R ₀ ) ² + (G−G ₀ ) ² + (B−B ₀ ) ² ] ^(1/2) ≦ L ⁽¹⁾

For example, in a color space represented by 8-bit values (0 to 255) for each of R, G, and B, a reference color ((R ₀ , G ₀ , B ₀ ) = (214, 165, 144)), a range length (L = 30), R: 184 to 244, G: 135 to 195, B: 114 to 174, and pixels having values of R, G, and B that satisfy the above [Expression 1] Extracted as a pixel.

  FIG. 13 is a diagram showing an aspect of a screen 1300 displayed by the image processing apparatus 101 in the region addition processing (FIG. 9: Step 903).

When adding an area of skin (Yes in Step 903), the operator designates a color range of pixel data to be extracted as an additional area (skin) by using a reference color designation pointer 1302 and a range length designation bar 1303 (Step 901). ), And a button 1304 ("Add region (skin)") is pressed.
The image processing apparatus 101 acquires an additional region (skin) 1305 (step 902).
Note that the operator may designate the processing range related to the region addition processing anew on the screen 1300 using the rectangular guideline 1301.

  When determining the skin region (No in step 903), the operator presses a button 1306 ("region (skin) determination"). The image processing apparatus 101 finally determines a region (skin) through processing such as region recognition and region addition.

The operation for acquiring the region (hair) is the same as the operation for acquiring the region (skin), but the image processing apparatus 101 sets the processing range outside the region (skin) (FIG. 8). : Step 805), the operator does not need to perform an operation on the rectangular guideline (designation of the processing range).
The operation for acquiring the region (other) is the same as the operation for acquiring the region (skin), but the image processing apparatus 101 sets the processing range to a range other than the recognized region (see FIG. 8: Step 808), the operator does not need to perform an operation on the rectangular guideline (designation of the processing range).

  Through the above process, the image processing apparatus 101 recognizes the region (skin) using the inside of the rectangular guideline as the processing range, and then recognizes the region (hair) using the outside of the region (skin) as the processing range, Then, the area (others) is recognized. The image processing apparatus 101 performs predetermined processing on the pixel data in the recognized area to create mask image data, and stores the mask image data in the storage unit.

  The color of the hair and eyebrows, pupils, etc. may be similar, and region recognition may be difficult. However, the image processing apparatus 101 performs skin region recognition processing with a limited processing range. Then, since the hair region is recognized using the outside of the recognized skin region as the processing range, the hair region and the eyebrow and pupil regions are not erroneously recognized. In addition, the processing range specified by the rectangular guideline is only once, and the speed of processing is not impaired.

Next, the operation of the image processing apparatus 101 in the area recognition process different from the procedure shown in FIG. 9 will be described with reference to FIGS.
FIG. 14 is a flowchart showing the operation of the image processing apparatus 101 in a region recognition process different from the procedure shown in FIG.
FIG. 15 is a diagram showing a flow of region recognition processing different from the procedure shown in FIG.

  In the area recognition process shown in FIG. 9, the image processing apparatus 101 has been described as determining whether or not all pixel data in the processing range belong to a predetermined color range. However, the area recognition process shown in FIG. Then, after detecting at least one boundary pixel (a pixel on the boundary of the region), the image processing apparatus 101 sets a vicinity of the boundary pixel as a processing range and detects a new boundary pixel.

When the color range is designated (step 1401), the image processing apparatus 101 (control unit 102) detects at least one boundary pixel (a pixel on the boundary of the region) (step 1402).
The image processing apparatus 101 (control unit 102) sets the vicinity of the boundary pixel as a new processing range (step 1403), and detects a new boundary pixel in the pixel data within the processing range (step 1404).

  If the detected boundary pixel does not form a closed region (No in step 1405), the image processing apparatus 101 (control unit 102) repeats the processing in steps 1403 to 1404, and the detected boundary pixel is closed. (Yes in step 1405), the closed area is determined as a recognition area (step 1406).

  As shown in FIG. 15, when performing region (hair) recognition processing, the image processing apparatus 101 searches for boundary pixels from point A to point B in the original image 1501 and detects one boundary pixel 1502. Then, the vicinity of the boundary pixel 1502 (for example, a range equal to or less than a predetermined distance from the boundary pixel) is set as a processing range 1503, and a new boundary pixel 1504 is detected in the processing range 1503, and the boundary pixel is closed. When the region is formed, this is determined as a region (hair) 1505.

  As described above, in the area recognition process shown in FIG. 14, the processing range is limited to the vicinity of the boundary pixel, so that the processing load on the image processing apparatus 101 is reduced. In addition, if an area where the boundary pixels are closed is formed, it is determined as a recognition area, so there is no possibility that a missing area will occur.

As described above, according to the embodiment of the present invention, the image processing apparatus limits the processing range to perform the skin area recognition process, and then sets the outside of the recognized skin area as the processing range. Since the hair region is recognized, the hair region and the eyebrow and pupil regions are not erroneously recognized. Further, the processing range can be designated by the operator's operation only once, and the speed of processing is not impaired.
Further, by limiting the processing range, it is possible to reduce the processing load and the processing time in the image processing apparatus.
In the above-described embodiment, the processing range specified by the operator is described as using the rectangular guideline. However, not only the rectangular guideline but also various procedures can be used. For example, an elliptical guideline having a variable shape and size, a guideline based on a free curve, and the like can be used.

In addition, since the image processing apparatus recognizes the synthesis area in units of pixels based on the original image, it is possible to perform natural image display even near the boundary of the area when image synthesis is performed. For example, by recognizing the region of the hair tip, the vicinity of the boundary between the hair and the skin can be displayed in a more natural manner.
Therefore, simple, high-definition, and realistic makeup can be virtually realized in real time based on the model captured on the apparatus (personal computer or the like) and the face of the cosmetic purchaser himself.

In addition, the image processing apparatus recognizes all necessary synthesis areas based on the original image data before performing the image synthesis process, and stores them in the storage unit as mask image data. It is not necessary to perform region recognition processing, and image composition processing can be performed quickly.
Furthermore, the operator only has to select a recognized area when designating the composite area, and the operation burden is reduced. For example, it is not necessary to surround and select the rectangular frame while adjusting the size, shape, etc. of each part for which image synthesis is to be performed and each time image synthesis is performed.

The composite area, the mask image, etc. can be designated by designating one point in a desired area of the original image with a mouse or the like. For example, when designating a region (skin), the operator clicks the cheek part of the face of the original image with the mouse, and the image processing apparatus calls the mask image data (skin) from the storage unit. In this case, the image processing apparatus can call up the mask image data (skin) by searching for the mask image data including the pixel position at the click position in the area in the storage unit.
Furthermore, since the image processing apparatus recognizes all necessary areas based on the original image data and stores them in the storage unit as mask image data before performing the image composition process, a list of recognition areas and mask image data (illustrated) May not be displayed. In this case, the operator can select a composite area, a mask image, and the like from this list.

  Regarding the area recognition processing, various procedures can be used according to the characteristics of each area. For example, the upper three pixels of the eye area may be set as the eyeline area. In the lip area recognition process, the processing range of the image processing apparatus is reduced by setting the lower third of the face as the processing range. You may make it aim at reduction.

  In addition, the image processing apparatus has been described as performing processing relating to area recognition, processing relating to image synthesis, and the like. (Image synthesizing apparatus or the like) may be configured independently of each other, or may be configured as an integrated apparatus (image processing apparatus or the like) as shown in the above-described embodiment.

In addition, when capturing and inputting an original image, the background of the subject may be set to a blue background to improve the accuracy and speed of area recognition.
In the above-described embodiment, the color range is specified for each of the three primary color components (R, G, B), but is not limited thereto. For example, brightness, hue, brightness, saturation, etc. may be taken into consideration.

Next, another embodiment of the present invention will be described with reference to FIG.
FIG. 18 is a schematic configuration diagram of an image processing system 1800 according to another embodiment of the present invention.

As shown in FIG. 18, the image processing system 1800 is configured by connecting an image processing device 1801, a terminal device 1802, and the like via a network 1803. Usually, a plurality of terminal devices 1802 are connected to the network 1803.
The image processing apparatus 1801 to the terminal apparatus 1802 can communicate with each other via the network 1803.

The image processing apparatus 1801 is the same as the image processing apparatus 101 in the above-described embodiment.
The terminal device 1802 is a device that exchanges various data with the image processing device 1801 via the network 1803, and is, for example, a personal computer, an information portable terminal, a cellular phone, or the like.

  The network 1803 is a network such as the Internet, and may be wired or wireless. Moreover, in order to ensure safe and smooth communication, a dedicated line, OBN (Open Business Network), Internet VPN (Virtual Private Network), etc. may be used instead of a network that can be used by the public such as the Internet. .

  In the above-described embodiment, the original image data is input from the imaging device to the image processing device via a connection line or a recording medium, and the composite image is input to a display device provided in the image processing device. In the other embodiment shown in FIG. 18, the original image data is transmitted from the terminal device 1802 to the image processing device 1801 via the network 1803 such as the Internet to display the image. The composite image data is returned from the device 1801 to the terminal device 1802, and the terminal device 1802 displays the composite image.

  In this case, the user of the image processing system 1800 (the operator of the terminal device 1802) can view the composite image on the terminal device 1802 without going to a store or the like where the image processing device 1801 is installed. Further, since the image processing system 1800 can handle a plurality of users at the same time, it is possible to efficiently promote purchase willingness for a larger number of system users in a short time. Can also reduce the burden.

  The preferred embodiments of the image processing apparatus and the like according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea disclosed in the present application, and these are naturally within the technical scope of the present invention. Understood.

1 is a schematic configuration diagram of an image processing apparatus 101 according to an embodiment of the present invention. The figure which shows the one aspect | mode of pixel data The figure which shows the one aspect | mode of the original image 301 The figure which shows the one aspect | mode of the mask image 401 The figure which shows the one aspect | mode of the texture image 501 The figure which shows the one aspect | mode of the synthesized image 601. A flowchart showing the overall operation of the image processing apparatus 101 A flowchart showing the operation of the image processing apparatus 101 in the mask image data creation process A flowchart showing the operation of the image processing apparatus 101 in the area recognition process. Explanatory drawing about missing area correction processing The figure which shows the one aspect | mode of the screen 1100 which the image processing apparatus 101 displays in original image data input processing The figure which shows the one aspect | mode of the screen 1200 which the image processing apparatus 101 displays in mask image data (skin) creation processing. The figure which shows the one aspect | mode of the screen 1300 which the image processing apparatus 101 displays in an area | region addition process. The flowchart which shows operation | movement of the image processing apparatus 101 in the area | region recognition process different from the procedure shown in FIG. The figure which shows the flow of the area | region recognition process different from the procedure shown in FIG. The figure which shows the one aspect | mode of the color range 1601 The figure which shows the one aspect | mode of the operation screen 1700 which the image processing apparatus 101 displays in an image compositing process. Schematic configuration diagram of an image processing system 1800 according to another embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ......... Image processing apparatus 102 ......... Control part 103 ......... Input part 104 ......... Display part 105 ......... Input / output part 106 ......... Storage part 107 ......... Communication part 109 ......... Imaging device 111... Subject 201. ......... Range length designation bar 1601 ......... Color range 1602 ......... Standard colors (R ₀ , G ₀ , B ₀ )
1603 ......... Range length (L)

Claims (14)

An image processing apparatus that performs image processing on an original image,
A processing range specification means for specifying a processing range;
An attribute range specifying means for specifying an attribute range;
Region recognition means for recognizing a region by extracting pixels belonging to the attribute range in the processing range of the original image;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the attribute range specifying unit specifies a reference color and a distance from the reference color in a color space.   The image processing apparatus according to claim 1, wherein the processing range specifying unit specifies the processing range by surrounding a predetermined range of the original image.   The image processing apparatus according to claim 1, wherein the processing range designating unit sets the outside of the recognized region as a processing range.   Specify the processing range by surrounding the predetermined range of the original image showing the face to recognize the skin region, and then set the processing region outside the recognized skin region to recognize the hair region The image processing apparatus according to claim 1, wherein: Holding means for holding the region and the image for synthesis;
Image combining means for combining the image for combining with the region of the original image;
The image processing apparatus according to claim 1, further comprising: An image processing method for performing image processing on an original image,
A process range specification process for specifying a process range;
An attribute range specification process for specifying an attribute range;
A region recognition step of recognizing a region by extracting pixels belonging to the attribute range in the processing range of the original image;
An image processing method comprising:   The image processing method according to claim 7, wherein the attribute range specifying step specifies a reference color and a distance from the reference color in a color space.   The image processing method according to claim 7, wherein the processing range specifying step specifies the processing range by surrounding a predetermined range of the original image.   The image processing method according to claim 7, wherein in the processing range specifying step, a region outside the recognized region is set as a processing range.   Specify the processing range by surrounding the predetermined range of the original image showing the face to recognize the skin region, and then set the processing region outside the recognized skin region to recognize the hair region The image processing method according to claim 7. Holding step for holding the region and the image for synthesis;
An image synthesis step of synthesizing the image for synthesis with the region of the original image;
The image processing method according to claim 7, further comprising:   A program that causes a computer to function as the image processing apparatus according to any one of claims 1 to 6. A computer-readable recording medium on which a program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 6 is recorded.

Images