JPH09106419A - Clothes fitting simulation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To display the picture of the person dressing clothes desired to fit without a sense of incongruity by deforming a picture of a first person, based on memorized first and second feature point. SOLUTION: Picture data of a manikin in which the feature point is written and clothes picture data for fitting are written in an external storage device 18. A picture input device fetches the picture of the person desiring fitting. This person is arranged at the same position and in the same pose as the manikin to write the piece of picture data in a video memory 20. After various kinds of data is written in the external storage device 18 and the video memory 20 like this, CPU 14 changes the body shape and the color of the manikin to be close to the body shape and the skin color of the person and changes the shapes of the clothes to match with the changed manikin in addition to synthesize with the manikin. Then synthesized picture data is temporarily written in the video memory 20 and after then converted to an analog video signal by a D/A converter 28 to be outputted from CRT 30.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a fitting simulation method, and more particularly to a fitting simulation method for displaying desired clothes by combining them with an image of a person who wants to try them on.

[0002]

2. Description of the Related Art An example of a conventional fitting simulation method of this type is disclosed in Japanese Patent Application Laid-Open No. 6-187398 (G06F 15/60) filed on July 8, 1994. In this conventional technique, a contour line is extracted from an image of a person who wants to be tried on by the image pickup unit, two points are attached at predetermined positions on the contour line, and try-on is made based on the interval. It is intended to enlarge or reduce the image of clothes and synthesize it with the image of the person.

[0003]

However, in such a conventional technique, when a person who wants to try on wears long-sleeved clothes, and the short-sleeved clothing is selected as the clothes to be tried on by the person, the display section displays. May display an image with missing arms,
There was a problem that the image of the sleeve of the clothes actually worn by the person was displayed. In order to solve such a problem, it is conceivable that an image of another person's arm is formed in the image of this person, but this causes a feeling of strangeness in the image.

Therefore, a main object of the present invention is to provide a try-on simulation method capable of displaying an image of the person wearing the clothes desired to be try-on without any discomfort.

[0005]

The first invention is (a) first
Storing the first person image with the feature points, (b) storing the second feature point of the second person image corresponding to the first feature point, and (c) based on the first feature point and the second feature point. This is a fitting simulation method for deforming the first person image.

A second invention is (a) extracting the second color component of a pixel included in the second predetermined portion of the second person image, and (b) the second
The first color component of the pixel included in the first predetermined portion of the first person image corresponding to the predetermined portion is extracted, (c) the conversion coefficient is calculated based on the first color component and the second color component, and ( d) A fitting simulation method in which the color components of the pixels included in the first person image are changed based on the conversion coefficient.

According to a third aspect of the present invention, (a) the clothing image is stored together with the third feature point, and (b) the clothing image is transformed based on the first feature point and the third feature point of the first person image, (c) A fitting simulation method in which the deformed clothing image is combined with the first person image.

[0008]

According to the first aspect of the invention, the first person image such as the mannequin image having the first feature points is stored, and the second person image such as the image of the person who wants to try on the second feature image is stored. Memorized with dots. Then, the mannequin image is deformed so that the second feature points match the first feature points.

In the second invention, for example, the skin color component of the second person image such as the person who wants to try on and the color component of the corresponding portion of the first person image such as the image of the mannequin are extracted to extract the two. The conversion factor is determined, which changes the color of the entire mannequin. As a result, the skin color of the mannequin image approaches the skin color of the person who wants to try on. In the third invention, the third feature point of the clothing image is transformed so as to match the first feature point of the first person image such as a mannequin image transformed according to the person image, and the transformed clothing is obtained. The image is combined with the first person image.

[0010]

According to the first aspect of the invention, since the first person image is deformed based on the first contour information and the second contour information, desired clothes are combined with the deformed first person image. By doing so, the image of the person wearing the clothes can be displayed without a sense of discomfort. According to the second invention, the color of the first person image is changed to the color of the desired portion of the second person image. Therefore, if desired clothes are combined with the first person image, the clothes can be changed. It is possible to display the image of the wearing person without any discomfort.

According to the third aspect of the invention, since the clothes image is transformed according to the first person image and then combined, the person image wearing the clothes can be displayed without a feeling of strangeness. The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Referring to FIG. 1, a simulation apparatus 10 of this embodiment includes a bus 12 by which a CPU 1
4, the working memory 16, the external storage device 18, the video memory 20, the A / D converter 22, the keyboard 24 and the mouse 26 are connected to each other. A CRT 30 is connected to the video memory 20 via a D / A converter 28,
An image input device 32 is connected to the / D converter 22.

In the external storage device 18, as shown in FIG. 2, image data of the mannequin 34 with characteristic points (marks for indicating the features of the outer shape) and clothes for try-on as shown in FIG. 4 are shown. Image data is written. More specifically, in order to obtain image data of the mannequin 34, the mannequin 34 is arranged at a position separated from the image input device 32 by a predetermined distance, and the image is captured by the image input device 32.
Then, the image is A / D converted. Further, in order to extract the clothes image data, clothes are put on the mannequin 34 arranged at the above-described position as shown in FIG. 3, the image is displayed on the CRT 30, and the outline of the clothes of the image is displayed by the mouse 26. Traced. The image input device 32 also captures an image of a person 36 who wants to try on as shown in FIG. This person 36 is also arranged at the same position as the above-mentioned mannequin 34 in the same pose as the mannequin 34, and the image data thereof is written in the video memory 20.

After the various data are written in the external storage device 18 and the video memory 20 in this manner, the CPU
14 changes the body shape and color of the mannequin 34 so as to be closer to the body shape and skin color of the person, and further transforms the shape of the clothes so as to match the changed mannequin 34.
To synthesize. The combined image data is once written in the video memory 20, converted into an analog video signal by the D / A converter 28, and output from the CRT 30.

The CPU 14 writes the clothing image data in the external storage device 18 according to the flow shown in FIG. That is, first, in step S1, the image of the mannequin 34 with clothes as shown in FIG. 3 is written in the video memory 20.
Next, in step S3, feature points are extracted from the mannequin image data written in the external storage device 18, the feature points are attached to the corresponding positions of the mannequin image written in the video memory 20, and the mannequin image is output from the CRT 30. To do.
Then, if it is determined in step S7 that the outline of the clothes worn by the mannequin 34 is traced by the mouse 26, the image of the clothes is extracted in step S9, and the external storage device 1
Write to 8. By performing the above-described processing on the mannequin 34 wearing the respective clothes shown in FIG. 3, a plurality of clothes image data is written in the external storage device 18.

The CPU 14 also carries out a simulation process of putting a desired clothes on a person who wants to try on the clothes according to the flow shown in FIGS. That is, first, step S
The image input device 32 captures the person image shown in FIG. 5 (A), which has been placed in a predetermined position at 11, and writes it in the video memory 20, and outputs this person image to the CRT 30 at step S13. Next, in step S15, it is determined whether the mouse 26 has been added to the outline of the person image as shown in FIG. 5B. It should be noted here that the feature points should be placed on the contour line of the body while avoiding the contour of the clothes. When the characteristic points are added, that is, when the characteristic points are written in the video memory 20, the image data of the mannequin 34 shown in FIG. 2 is read from the external storage device 18 and written in the working memory 16 in step S17. continue,
In step S19, the feature points of the mannequin image corresponding to the feature points attached to the person image are searched, and in step S21, the mannequin image is deformed so that the feature points of both images match, and the feature of the mannequin image is adjusted accordingly Also move the points. When transforming the mannequin image, a technique such as affine transformation is used.

When transforming the mannequin image,
As shown in FIG. 13, the area of the mannequin image is {(1,2,5,6), (2,
3,6,7), (3,4,7,8), (5,6,9,10)…} etc. are divided into areas surrounded by 4 points, and each area is height, weight and chest measurement.・
It may be changed based on body shape information such as hip circumference and sitting height. As the body type data used at this time,
It is possible that the body shape data, etc. measured by the Human Life Engineering Center, an incorporated organization of the Ministry of International Trade and Industry.

When the deformation of the mannequin image is completed, it is determined in step S23 whether or not the contour of the skin portion of the human image displayed on the CRT 30 is traced by the mouse 26. If "YES", the color components (R, G, B) of the pixels within the contour are calculated in step S25, and the color components (R, G, B) of the corresponding pixels in the mannequin image are calculated in step S27. To calculate. Next, in step S29, the conversion coefficient A _i of the color component is calculated according to equation 1.

[0019]

[Formula 1] N _i = A _i · M _i (1 ≦ i ≦ n) N _i : Color component of person image M _i : Color component of mannequin image Detailed calculation of conversion coefficient A _i using foot part then, first, in step S25 for each pixel shown in FIG. 10 (a) calculates the color component n ₁ to n _n of the person 34, then the color component n ₁ as shown in FIG. 10 (B) in steps S27 ~
The color components M _{1 to} M _n of the mannequin 34 corresponding to N _n are calculated, and then the conversion coefficients A _{1 to} A _n are calculated for each pixel.
Is calculated. Each color component N _{1 to} N _n and M ₁ to
Of course, M _n is obtained for each color component (R, G, B), and the conversion coefficient A _i is also obtained for each color component (R, G, B). When the conversion coefficient is calculated in this way,
In step S31, it is determined whether or not the skin portion of the person image is newly designated. If "YES", step S25
However, if “NO”, in step S29, the average value of all the calculated conversion coefficients is calculated for each color component (R, G, B).
It is calculated every time.

Then, in step S35, the color components (R, G, B) of the pixels of the mannequin image are calculated, and in step S37.
Then, the conversion coefficient averaged in step S33 is added to this color component. Succeedingly, in a step S39, the pixel calculated in the step S35 is replaced with the integrated value, and in a step S41, it is determined whether or not the replacing process is performed for all the pixels of the mannequin image. If “NO” here, the process returns to step S35, but if “YES”, as shown in FIG. 14, the face joint part of the mannequin image is matched so that the feature points of the face joint part indicated by the diagonal lines match. Is transformed to synthesize the face part. At the face joint, the color components (R ₁ , G ₁ , B of the mannequin image are
₁ ) and the color components (R ₂ , G ₂ , B ₂ ) of the human image,
As shown in FIG.

[0021]

R = a × R ₁ + (1-a) × R ₂ G = a × G ₁ + (1-a) × G ₂ B = a × B ₁ + (1-a) × B ₂ ( However, 0
≦ a ≦ 1) Then, as shown in FIG. 15, the value a of the composition ratio is gradually increased from 0 to 1, and the mixing ratio of the color components of both images is changed so that the face connecting portion is naturally formed. Smooth it so that you can see it. Through the above processing, the simulation data of the person 36 who is not wearing clothes is created.

Subsequently, in step S49, the external storage device 1
The clothing image data written in 8 is read out and multi-screen displayed on the CRT 30 as shown in FIG. However, before the screen is displayed, the type of clothes is narrowed down by a normal keyword search or the preference the customer has selected in the past.
Then, in step S51, it is determined whether any clothes are selected by the keyboard 24 or the mouse 26. If "YES", the selected clothes image data is written in the work memory 16 in step S53. next,
In step S55, the characteristic points of the clothes image corresponding to the characteristic points of the mannequin image are searched, and in step S57, the clothes are deformed by affine transformation or the like so that the characteristic points of both images match. Then, in step S59, the clothes are combined with the mannequin image and output from the CRT 30. by this,
The CRT 30 outputs simulation data in which desired clothes are put on the person 36.

In operation, when the person 36 who wants to try on stands at a position separated from the image input device 32 by a predetermined distance, the CPU 14 takes in the image of the person by the image input device 32 and writes the body shape of the mannequin 34 written in advance. And the color are processed to approximate the figure and skin color of this person. Further, the clothing desired to be tried on by the person 36 is displayed on the CRT 30 in a multi-screen manner, and the clothing selected from the clothing is deformed and combined so as to fit the mannequin 34.

In the above description, the ratio of the skin color is calculated from the exposed parts of the skin at a plurality of places of the person and the mannequin, but it may be represented by one place. Also, this conversion ratio may be multiplied to the entire image of the mannequin. In this case, it is not necessary to specify or extract the outline of the person image. According to this embodiment, the mannequin 34 is deformed so as to approach the body shape and skin color of the person 36 and the clothes are combined with the mannequin 34. Therefore, the state in which the person 36 actually wears the clothes is simulated. Can be displayed.

In this embodiment, the average value of the conversion coefficients calculated in step S33 is simply added to the color components of each pixel of the mannequin image in step S37. The skin color component may be weighted according to the distance from the pixel where the skin is exposed, and the weighted value may be multiplied by the average value of the conversion coefficient to reproduce the color of the pixel to be converted. . That is, for example, in the case of converting the color component of the point A of the person 36 shown in FIG. 11, the conversion coefficients of the R components of the representative points selected from the neck, the hand and the foot are α1, α2 and α3, respectively, and the respective representative points are It is assumed that the ratio of the distances from A to A is a: b: c. In this case, the converted R component R _m ′ at the point A is obtained according to the equation (3).

[0026]

## EQU3 ## R _m ′ = {(1 / a) / d} · α1 · R _m +
{(1 / b) / d} · α2 · R _m + {(1 / c) / d}
Α3 · R _m R _m : R component at point A of the mannequin image d = (1 / a) + (1 / b) + (1 / c) Further, the conversion coefficients of the G component at the representative point are β1, β2 and β
3 and the conversion coefficients of the B component are γ1, γ2, and γ3, the G component G _m ′ and the B component B _m ′ after conversion at the point A can be obtained by Equations 4 and 5.

[0027]

## EQU4 ## G _m ′ = {(1 / a) / d} · β1 · G _m +
{(1 / b) / d} · β2 · G _m + {(1 / c) / d}
Β3 · G _m G _m : G component at point A of the mannequin image d = (1 / a) + (1 / b) + (1 / c)

[0028]

[Formula 5] B _m ′ = {(1 / a) / d} · γ1 · B _m +
{(1 / b) / d} · γ2 · B _m + {(1 / c) / d}
.Gamma.3.B _m B _m : B component at the point A of the mannequin image d = (1 / a) + (1 / b) + (1 / c) As a result, lighting when the human image is captured by the image input device 32 Even if there is a difference in color components between parts of the body due to (Lighting), the color of the simulated image can be naturally changed and represented.

More specifically, step S in FIG.
The flow shown in FIG. 12 is processed instead of 33 to S41.
That is, if “NO” in the step S31, the step
In S61, specify one pixel included in the mannequin image and
Color component R_m, G_mAnd B _mIs calculated. Next,
At step S63, an arbitrary contour within the contour traced by the mouse 26 is selected.
The pixel is designated as the representative point, and the representative point is selected in step S65.
The ratio of the distance from the point to the pixel specified in step S61
calculate. Then, in step S67, follow the formulas 3-5.
Color component R_m′, G_m'And B_m′ Is calculated and
In step S69, the color component R of the pixel specified in step S61
_m, G_mAnd B_mThe color component R_m′, G_m'and
B_m’ Then, in step S71, all images
Judge whether the element has been processed, and if “NO”, step
Return to step S61, but if "YES", the step S43.
Move to.

In this embodiment, all the characteristic points are added when the characteristic points are added to the person image displayed on the CRT 30. However, some representative points are designated, and a part of the body of the person is specified between them. Automatically interpolating based on statistical data or the like, and further extracting the contour line of the person 34 including clothes to automatically specify all the feature points based on the statistical data of the body of the person. May be. Also, the mannequin 34
Image data of the work memory 1 in step S17 of FIG.
6 is written in the simulation device 10
May be written in the work memory in advance when is started. Further, the characteristic points indicating the contour of the mannequin 34 are attached to the mannequin 34 itself, but may be attached manually by operating the mouse 26 after inputting the mannequin image.

Furthermore, in this embodiment, only the face portion is combined with the mannequin image. However, the exposed portion of the human image such as the neck, hands and feet is cut out from the human image and cut into the mannequin. Combine it into an image,
You may make it perform a smoothing process. Further, in this embodiment, the conversion coefficient is calculated for the color components (R, G, B), but it may be calculated for the luminance component and the hue component. Further, in this embodiment, the conversion coefficient is obtained for each pixel between the person image and the mannequin image. However, in calculating the conversion coefficient of each pixel, first, the pixels having the same color component are detected in the mannequin image. , The color component values of the pixels of the person image corresponding to this pixel may be averaged, and the conversion coefficient may be obtained from this average value. Moreover, you may change a color component like Claim 8.

[Brief description of the drawings]

FIG. 1 is a block diagram showing one embodiment of the present invention.

FIG. 2 is an illustrative view showing a mannequin image.

FIG. 3 is an illustrative view showing a clothing image worn on a mannequin image.

FIG. 4 is an external view showing a mannequin wearing clothes.

5A and 5B are external views showing a person image.

FIG. 6 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

FIG. 7 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

FIG. 8 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

FIG. 9 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

FIG. 10 is an illustrative view showing corresponding pixels of a foot portion of a human image and a mannequin image.

FIG. 11 is an external view showing a person image.

FIG. 12 is a block diagram showing a part of the operation of another embodiment of the present invention.

FIG. 13 is an illustrative view showing a part of the operation of the embodiment in FIG. 1;

FIG. 14 is an illustrative view showing a part of the operation of the embodiment in FIG.

FIG. 15 is an illustrative view showing a part of the operation of the embodiment in FIG. 1;

[Explanation of symbols]

 10 ... Simulation device 14 ... CPU 16 ... Working memory 18 ... External storage device 20 ... Video memory 24 ... Keyboard 26 ... Mouse 30 ... CRT 32 ... Image input device

Claims (10)

[Claims] 1. A memory for storing a first person image together with a first feature point, a memory for storing a second feature point of a second person image corresponding to the first feature point, and a memory for storing the second feature point. A fitting simulation method for deforming the first person image based on the first feature point and the second feature point. 2. The step (c) comprises (c-1) searching for a second feature point corresponding to the first feature point, and (c-2) corresponding to the first feature point.
2. The fitting simulation method according to claim 1, wherein the first person image is deformed so that the feature points and the second feature points match, and (c-3) the first feature points are moved in accordance with the deformation. 3. (d) replacing the face portion of the first person image transformed in step (c) with the face portion of the second person image,
The fitting simulation method according to claim 1. 4. The step (d) comprises: (d-1) substituting a part of the face portion of the second person image so as to overlap with the first person image, and (d-2) performing the replacement in the overlapping portion. 4. The fitting simulation method according to claim 3, wherein the color components of the first human image and the color components of the second human image are weighted with a predetermined weight to synthesize the color components. 5. (a) extracting a second color component of a pixel included in a second predetermined portion of the second human image, and (b) a first predetermined portion of the first human image corresponding to the second predetermined portion. Extracting a first color component of a pixel included in, a (c) calculating a conversion coefficient based on the first color component and the second color component, and (d) a pixel included in the first human image. A fitting simulation method for changing a color component based on the conversion coefficient. 6. The first predetermined portion and the second predetermined portion include a plurality of pixels, and the step (c) calculates the transformation coefficient of the average of the plurality of pixels, and the step (d)
The fitting simulation according to claim 5, wherein (d-1) the color component of the pixel included in the first person image is multiplied by the conversion coefficient, and (d-2) the multiplication result is the color component of the pixel. Method. 7. The first person image and the second person image are at least two locations of the first predetermined portion and the second person image.
Including a predetermined portion, the step (c) calculates the conversion coefficient by dividing the second color component by the first color component, and the step (d) includes the conversion coefficient, the step ( Based on the ratio of the distance from the pixel for which the first color component was calculated in b) to the pixel included in the first person image and the first color component of the pixel included in the first person image,
The fitting simulation method according to claim 5, wherein the first color component of the pixels included in the person image is changed. 8. The first person image and the second person image include at least first and second predetermined portions and a second predetermined portion, and the step (c) includes (c -1) The second in each place
The conversion coefficient indicating the relationship between the color component and the first color component is calculated, and the step (d) includes (d-1) determining the color component of the pixel included in the first human image from the pixel. The color component of the pixel included in the changed first person image is changed based on the distance to the first place and the conversion coefficient calculated at the first place, and (d-2) The fitting simulation method according to claim 5, wherein the change is performed again based on the distance from the pixel to the second location and the conversion coefficient calculated at the second location. 9. A clothing image is stored together with a third feature point, and a clothing image is transformed based on the first feature point and the third feature point of a first person image. ) Combining the deformed clothes image with the first person image,
Try-on simulation method. 10. The step (b) includes (b-1) searching the first feature point corresponding to the third feature point, and (b-2) finding the corresponding first feature point and third feature point. The fitting simulation method according to claim 9, wherein the clothing images are transformed so as to match.