JPH09102033A - Picture synthesis method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To output a natural synthesis picture without projection from CRT by detecting the projection of an outline and deforming the picture of a dress. SOLUTION: The pictures of a shirt and a jacket and the picture of a person who desires trying-on are stored in an external storage device 16. A microcomputer 14 overlaps and writes the outlines of the shirt and the jacket into a work memory 20. A part projected from the jacket in the outline of the shirt is detected. The shirt is deformed based on the result. The microcomputer 14 overwrites the outline of the person to the outlines of the shirt and the jacket, and deforms the jacket so that the outline of the person does not project from the outline of the jacket. The pictures of the shirt and the jacket are deformed based on the outlines, and the jacket, the shirt and the person are written into a layer 22 in the order. The respective pictures are synthesized and outputted from CRT 18.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizing method,
Especially, for example, you can combine multiple clothes images that are layered,
The present invention relates to an image synthesizing method for synthesizing a clothing image with a person image.

[0002]

2. Description of the Related Art An example of a conventional image synthesizing method of this type is
Japanese Patent Application Laid-Open No. 6-1873 filed on July 8, 1994
No. 98 [G06F 15/60]. According to this conventional technique, a contour line is extracted from an image of a person who wants to be tried on, two points are added at predetermined positions on the contour line, and an image of clothes to be tried on is enlarged or enlarged based on the intervals. It is intended to reduce the size and synthesize it with the image of the person.

[0003]

However, using such a conventional technique, for example, two points are added to the outline of the image of the shirt shown in FIG. 26 (A), and FIG. When the image of the jacket shown in) is scaled up and down to be combined with the shirt, the sleeve of the shirt protrudes from the jacket as shown in FIG. 27, and an unnatural combined image is output. When images of clothes such as shirts and skirts that are worn at different positions are displayed in a composite manner, simply combining those clothes with a person image causes a gap between the clothes as shown in FIG. There was a problem that an unnatural synthetic image was output.

Therefore, a main object of the present invention is to provide an image synthesizing method capable of outputting a natural synthetic image.

[0005]

The first invention is (a) first
The image and the second image are stored, and (b) the first contour when the first contour of the first image and the second contour of the second image are superposed on each other.
One of the contour and the second contour detects a protruding portion protruding from the other, and (c) deforms at least one of the first image and the second image so that the protrusion does not occur, and (d ) An image synthesizing method of synthesizing the first image and the second image.

A second invention is (a) storing a first clothing image and a plurality of second clothing images, and (b) a first clothing image.
It is determined whether or not a predetermined portion of the second contour protrudes from the first contour when the contour and the second contour of the first second clothing image are overlapped, and (c) the second clothing image without protrusion based on the determination result. Is selected, and (d) the first clothes image and the selected second clothes image are combined.

A third invention is (a) preparing a memory, and (b)
Writing the first clothes image to be worn on the upper half of the body and the second clothes image to be worn on the lower half of the body at a predetermined position in the memory, (c)
Transforming at least one of the first clothes image and the second clothes image such that the first clothes image and the second clothes image partially overlap each other, and (d) combine the first clothes image and the second clothes image, This is an image composition method.

[0008]

According to the first aspect of the present invention, the first clothes image and the second clothes image are written in the memory by, for example, separately wearing the first clothes and the second clothes on the mannequin and photographing each with a video camera. Next, for example, the contours of the first clothing image and the second clothing image are overlapped with each other, and, for example, a portion of the predetermined portion of the second contour that is out of the first contour is detected. Then, the second clothes image is deformed based on the detection result, and the deformed second clothes image and the first clothes image are combined.

In the second invention, the first clothes image and the second clothes image are written in the memory in the same manner as in the first invention,
For example, the first contour of the first clothing image and the second contour of the first second clothing image are overlapped, and the predetermined portion of the second contour is the first portion.
It is determined whether or not it extends from the contour. Then, based on the determination result, the second clothes image having no protrusion is selected, and the first clothes image and the selected second clothes image are combined.

In the third invention, the first clothes image and the second clothes image are written in a predetermined position of the memory in the same manner as in the first invention so that the first clothes image and the second clothes image partially overlap each other. At least one of the first clothes image and the second clothes image is transformed to. For example, a first contour below the first clothing image and a second contour above the second clothing image are designated, and the first clothing image and the second contour are arranged so that the two points that are most distant from each other coincide with each other. At least one of the second clothing images is transformed. Then, the first clothes image and the second clothes image are combined.

[0011]

According to the first aspect of the present invention, the protrusion between the first contour and the second contour is detected, the second image is deformed, and the deformed second image and first image are combined. Therefore, it is possible to output a natural image that does not overflow. According to the second aspect of the present invention, it is possible to determine whether the second contour extends beyond the first contour, select a second clothing image that does not cause the second contour, and combine the second clothing image with the first clothing image. A natural image can be output.

According to the third invention, at least one of the first clothes image and the second clothes image is deformed so that the first clothes image and the second clothes image partially overlap each other. Therefore, the first clothes image is deformed. It is possible to output a natural composite image with no gap between the image and the second clothing image. The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Referring to FIG. 1, an image synthesizing apparatus 10 of this embodiment includes a video camera 12. The video camera 12 first extracts a clothing image from the image of the mannequin 26 shown in FIG. 3 (B) and stores it in the external storage device 16. Next, the video camera 12 captures the image of the person 28 who wants to try on and is shown in FIG. 9B, and stores it in the external storage device 16. After that, the microcomputer 14 synthesizes desired clothes with the image of the person 28 according to the input signal from the mouse 24, and outputs the synthesized image from the CRT 18.

The microcomputer 14 extracts a clothing image according to the flow shown in FIG. 2 and writes it in the external storage device 16. That is, first, in step S1, the image of the upper bodyless mannequin 26 shown in FIG. 3A is written in the work memory 20, and then in step S3, as shown in FIG. The image is written in the work memory 20. Next, in step S5, the difference between the two images is detected, and the difference value corresponds to the pixel having a value other than 0, and FIG.
Pixels included in the image are extracted. As a result, the clothes image of the upper body is extracted. Subsequently, the clothes image data is converted into the RGBA format in step S7, and the clothes image is displayed on the CRT 18 in step S9. Therefore, the clothing image shown in FIG. 4 is displayed on the CRT 18.

"A" in the RGBA format means an alpha value which represents a combination ratio of an image to be combined with an image to be combined in a range of "0" to "255". That is, in the case of transparent clothes, the alpha value matches the transparency. In this embodiment, this alpha value is used to extract the contour of the image. That is, the alpha values of the pixels included in the extracted image are set to "1" to "255",
If the alpha values of the other pixels are set to "0", the alpha values near the contour of the clothing image are expressed as shown in FIG. 6, and the contour line is detected with the changing point of this alpha value as a clue.

When the processing of step S9 is completed, next, in step S11, the operator operates the mouse 24 to determine whether or not a feature point is added to the contour line of the clothes image as shown in FIG. If “YES”, step S1
At 3, the clothing image data and the feature points are stored in the external storage device 16 and the process is terminated. By performing such processing on a plurality of types of clothes, the external storage device 16 stores the clothes images 30a to 30d shown in FIGS. The external storage device 16 stores clothes images 30a to 30d of a plurality of sizes.

Next, the microcomputer 14 stores in the external storage device 16 the image data of the person 28 who has feature points on the contour line and who wants to try on as shown in FIG. 11 according to the flow shown in FIG. That is, first, in step S15, the background image shown in FIG. 9A is written in the work memory 20, and then in step S17, as shown in FIG. 9B, the background image shown in FIG. The image of the person 28 not wearing is written in the work memory 20. Subsequently, in step S19, the difference between the two images is calculated, and the pixel corresponding to the pixel whose difference value is not 0 and included in the image captured in step S17 is extracted. As a result, only the person image with the background removed can be obtained. After that, this person image data is converted into the RGBA format in step S21 and displayed on the CRT 18 in step S23. Therefore, the person image shown in FIG. 10 is displayed on the CRT 18. Also in this case, all the alpha values of the pixels included in the person image are “255”, and the alpha values of the pixels other than the person image are “0”. Thereafter, in step S25, the operator operates the mouse 24 to determine whether or not a feature point is added to the contour line of the person image as shown in FIG.
If "ES", the person image data and the characteristic points are written in the external storage device 16 in step S29, and the process ends.

In this way, a plurality of types of clothes images 30a
When the data up to 30d and the image data of the person 28 who wants to be tried on are stored in the external storage device 16, the microcomputer 14 performs a process of combining the desired clothes image with the person image according to the flow shown in FIGS. . That is, first, the plurality of clothing images 30a to 30d read from the external storage device 16 in step S29 are multi-displayed on the CRT 18. Next, in step S31, it is determined whether or not a clothing image to be combined is selected by the operator. If “YES”, the clothing image and the person image are recorded in the layer 22 in step S33. If the clothing image selected in step S31 is the clothing images 30b and 30c shown in FIGS. 7B and 7C, the clothing image 30c is the first layer of the layer 22 and the clothing image is the layer 22. 30b
Is recorded in the second layer, and the image of the person 28 is recorded in the third layer. Then, in step S35, the contour line 32c of the clothing image 30c written in the first layer is detected by the alpha value and written in the work memory 20, and the feature points "1" to "12" are written therein as shown in FIG. And specify multiple areas enclosed by dotted lines. Then, the contour line 3 of the clothing image 30b written in the second layer of the layer 22 in step S37.
2b is detected and is overwritten on the contour line 32c as shown in FIG. 17 in accordance with the characteristic points "1" to "12" attached to the clothes images 30b and 30c.

Now, the positions of the points a to t which partition the area will be described. The points a, b, t, and n are positioned so as to surround the contour lines 32b and 32c, the point a is on the extension line of the characteristic points "1" and "7", and the point b is the characteristic point "4". It is an extension of "10". The point c is located at the midpoint between the feature points “1” and “7”, the point e is located at the midpoint between the feature points “4” and “10”, and the point d is the midpoint between the points c and e. , And the point f is located at the midpoint between the feature points “7” and “10”. The point p is a point that passes through the midpoint of the characteristic points “6” and “8” and the characteristic point “7” and intersects the line segment np, and the point r is the characteristic point “10” and the characteristic points “9” and “11”. Is a point that passes through the midpoint of and intersects the line segment nt. A point g is a point where a straight line passing through the characteristic points “5” and “6” intersects the line segment an, a point h is an intermediate point between the characteristic points “5” and “6”, and a point i is a characteristic point “5”. "And" 6 "
A straight line passing through is a point intersecting with the line segment 7p.

A point m is a point where a straight line passing through the characteristic points "11" and "12" intersects the line segment bt, and a point l is a characteristic point "1".
It is an intermediate point between 1 "and" 12 ", and a point k is a point where a straight line passing through the characteristic points" 11 "and" 12 "intersects with the line segment 10r. A point j is characteristic points" 8 "and" 9 ". Is an intersection of a line segment nt of a straight line passing through the points f and j, a point o is an intersection of a line segment nt passing through the points c and h, and a point s is a point e and It is the intersection of the straight line passing through 1 and the line segment nt.The regions abec, cdf7, de10f, achg, c7i are defined by the points defined above.
h, 7fqp, f10rq, 10elk, eblm, g
hon, hippo, klsr and lmts are formed.

In step S39, the characteristic point "1" →
"2", "3" → "4", "4" → "12", "11"
→ "10", "10" → "9", "8" → "7",
In "7" → "6" and "5" → "1", it is checked whether or not the contour line 32b protrudes from the contour line 32c, and in step S41, it is determined whether or not the contour line protrudes. If "NO" here, the process proceeds to a step S49, but if "YES", it is determined at a step S43 whether or not there is an image of the same type of clothing image in which the sleeve or the trunk portion does not protrude. If "YES" here, the process proceeds to step S47.
If "NO", the protruding portion is deformed in step S45, and the process proceeds to step S47. In step S47, it is determined whether there is a clothing image to be processed next, and "YES".
If so, the process returns to step S37, but if “NO”, the person 2 recorded in the third layer of the layer 22 in step S49
The contour line 28a of the image No. 8 is detected and written in the work memory 20, and the feature points "1 '" to "1" as shown in FIG.
2 '"and area are designated.

Then, in step S51, step S45
The contour line deformed in accordance with the characteristic points "1" to "12" and the characteristic points "1 '" to "12'" of the person 28 in FIG.
As shown in 9, the person image is overwritten. After that, in step S53, the characteristic points “1 ′” → “2 ′”, “3 ′” →
"4 '", "4'" → "12 '", "11'" → "1"
0 '","10'"→" 9 '","8'"→" 7 '",
In "7 '" → "6'" and "5 '" → "1'", it is checked whether the contour line 28a is out of the contour line 32c, and the contour line 32c is deformed in step S55. Then, in step S57, the clothes images 30c and 30b written in the first and second layers of the layer 22 are deformed according to the deformed contour lines 32c and 32b, and in step S59, the clothes images 30c and 30b and the third layer. The image of the person 28 written in is projected in parallel to create a composite image. At this time, with respect to the area where the respective images overlap, the upper layers of the layers are preferentially combined. Then, in step S61, the composite image is displayed on the CRT 18, and the process ends.

The microcomputer 14 executes step S45 in FIG.
Process according to the subroutine shown in. That is, first, in step S65, it is determined whether or not the contour line 32b is protruding at the portion connected by the characteristic points "1" and "2" of the contour line 32c. If "NO", the process proceeds to step S71. If "YES", in step S67, two points having the maximum mutual distance are detected. Then, in step S69, the condition that the two points overlap and the points (here, points c, d, and e) located on the dotted line of the area surrounding the protruding contour line and included in the contour line 32c are left unchanged. The contour line 32b
Is affine transformed. Similarly, steps S71 to S8
1, the portion of the contour line 32d connected by the characteristic points "3" and "4", the portion connected by the characteristic points "4" and "12", the portion connected by the characteristic points "11" and "10",
Portions connected by "10" and "9", portions connected by feature points "8" and "7", feature points "7" and "6"
Affine transformation is applied to the portion connected by and the portion surrounded by the characteristic points “5” and “1”.

The microcomputer 14 also processes step S55 in accordance with the subroutine shown in FIG. That is,
First, at step S81, the characteristic point "1 '" shown in FIG.
In "2 '", it is determined whether the contour line 28a extends beyond the contour line 32c, and if "NO", step S
The process proceeds to step 89, but if "YES", step S8
In step 5, two points having the maximum mutual distance are detected, and the contour line 32c is affine-transformed as in step S69. Similarly, in steps S89 to S99, the feature point “3 ′” →
"4 '", feature point "4'" → "12 '", feature point "1"
1 ′ ”→“ 10 ′ ”, characteristic point“ 10 ′ ”→“ 9 ′ ”, characteristic point“ 8 ′ ”→“ 7 ′ ”, characteristic point“ 7 ′ ”→“ 6 ′ ”and characteristic point“ 5 ′ ” "-""1'" is detected and the affine transformation is performed. The contour line deformed by the affine transformation is basically the contour line 32c, but the portion where the contour line 32b protrudes from the contour line 32c ( For the sleeve portion), the contour line 32b is affine-transformed, so that the contour line 32b is changed to the contour line 2 as shown in FIG.
When 8a is protruding, the contour line 32b is affine-transformed as shown in FIG.

According to this embodiment, the image of the clothes to be worn underneath is detected by detecting a portion protruding from the clothes image to be worn on the clothes, and the clothes image to be worn below is transformed, and further, the clothes image and the person image are combined. Since the clothes image is deformed by detecting the protrusion between the clothes, a natural fitting image without protrusion can be output when the clothes image is combined with the person image.

Although the clothing images 30b and 30c have been described in this embodiment, the present invention can of course be applied to clothing images such as skirts and sweaters. Further, in this embodiment, the contours of the clothing image are overlapped to detect the protruding portion, but the coordinate data of the respective contours may be compared to detect the protruding portion.

Since the image synthesizing apparatus 10 of the other embodiment is almost the same as the image synthesizing apparatus 10 shown in FIG. 1, FIG. 1 is referred to in describing this embodiment, and only different points will be described. The upper-body and lower-body clothing images captured by the video camera 12 are written in the external storage device 16 in the external storage device 16, and the microcomputer 14 processes the flow charts shown in FIGS. The image is displayed on the CRT 18.

First, referring to FIG. 22, the microcomputer 14 first writes the clothing image for the upper half of the body in the first layer of the layer 22 in step S1, and then writes the clothing image for the lower half of the body in the second layer of the layer 22 in step S3. Write in. Then step S
In step 5, the contours of the respective clothing images are written in the work memory 20, and in steps S7 and S9, the contours 1 → 2 of the clothing image for the upper body are designated, and the contours 3 → 4 of the clothing image for the lower body are designated. Judge whether it was done.
Here, the contours 1 → 2 and 3 → 4 are the contours of the hem portion of the clothing image for the upper body and the contours of the waist portion of the clothing image for the lower body, as can be seen from FIG. If both are "YES" here, the contour 1 is determined in step S11.
The contour of the upper body clothing image is affine-transformed so that the two most distant points of → 2 and 3 → 4 match. As a result, the contour 1 → 2 of the upper body clothing image moves to the position of contour 1 ′ → 2 ′ as shown in FIG.

Then, in step S13, it is judged whether or not there are points 3 and 4 on the contour of the upper body clothing image,
If “YES”, the process directly proceeds to step S17,
If "NO", the contour is affine-transformed in step S15 so that the points 3 and 4 are on the contour of the upper body clothing image, and the process proceeds to step S17. Step S1
In 7, the clothes images in the first layer and the second layer are transformed according to the contours thus converted. After that, in step S19, the upper half-body and lower half-body clothes images are projected in parallel to form a composite image, and in step S21 the composite image is displayed on the CRT 18, and the process ends.

As a result of the processing performed by the microcomputer 14 in this way, when the shirt and skirt are actually worn on the CRT 18, there is no gap between the shirt and the skirt, and the composite image is displayed naturally. can do. Although the clothes image and the like are stored in the external storage device in these embodiments, it is conceivable to carry out the following in actual commercialization.
That is, a product catalog CD-ROM in which clothing images and model images are stored in advance is issued, and the user photographs his or her image in a specific photographing service department and stores it in the FD. Then, the user performs a fitting simulation for synthesizing clothes with himself or synthesizing clothes for the model image.

[Brief description of the drawings]

FIG. 1 is a block diagram showing one embodiment of the present invention.

FIG. 2 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

3 (A) and (B) are images of a manikin standing in front of a background captured by a video camera.

FIG. 4 is an image showing clothes and displayed on a CRT.

FIG. 5 shows a garment with feature points on the contour line and CR
It is an image displayed on T.

FIG. 6 is an illustrative view showing an alpha value of a clothes image.

FIG. 7 shows a garment and is displayed on a CRT.

FIG. 8 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

9A is an image in which a background is captured by a video camera, and FIG. 9B is an image in which a person standing in front of the background is captured by a video camera.

FIG. 10 is an image showing a person and displayed on a CRT.

FIG. 11 shows a person with a feature point on the contour line and C
It is an image displayed on the RT.

FIG. 12 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

FIG. 13 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

FIG. 14 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

FIG. 15 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

FIG. 16 is an illustrative view showing a contour line of a clothes image, a feature point attached to the contour line, and a region dividing the contour line.

FIG. 17 is an illustrative view showing a contour line of a clothing image, a feature point attached to the contour line, and a region dividing the contour line.

FIG. 18 is an illustrative view showing a contour line of a person, a feature point attached to the contour line, and a region dividing the contour line.

FIG. 19 is an illustrative view in which a contour line of clothes is superimposed on a contour line of a person.

FIG. 20 is an illustrative view showing a contour line near a sleeve.

FIG. 21 is an illustrative view showing a contour line near a sleeve.

FIG. 22 is a flowchart showing one portion of behavior of another embodiment of the present invention.

FIG. 23 is a flowchart showing one portion of behavior of another embodiment of the present invention.

FIG. 24 is an illustrative view showing one portion of behavior of another embodiment.

FIG. 25 is an illustrative view showing one portion of behavior of another embodiment.

FIG. 26 is an image showing clothes and displayed on a CRT.

FIG. 27 is an image showing clothes synthesized by the conventional technique and displayed on a CRT.

[Explanation of symbols]

 10 ... Image synthesizer 14 ... Microcomputer 20 ... Work memory 22 ... Layer

Claims (8)

[Claims] 1. (a) A first image and a second image are stored in the memory, and (b) A first contour of the first image and a second outline of the second image.
When the contours are overlapped, the first contour and the second contour
One of the contours detects a protruding portion protruding from the other, and (c) at least one of the first image and the second image is deformed so that the protrusion does not occur based on the detection result, and (d) An image synthesizing method for synthesizing a first image and a second image. 2. The step (b) comprises: (b-1) superimposing the first contour and the second contour, and (b-2) detecting the protruding portion in a predetermined portion of the superposed contour image. The image synthesizing method according to claim 1. 3. The first image is a first clothing image, the second image is a second clothing image to be worn under the first clothing image, and step (b-2) is the second clothing image. The image synthesizing method according to claim 2, wherein a portion of the contour that is out of the first contour is detected. 4. The first image is a person image, and the second image is a human image.
The image is a clothing image, and the step (b-2) is the first image.
The image synthesizing method according to claim 1, wherein a portion of the contour that is out of the second contour is detected. 5. The step (c) comprises: (c-1) deforming the second contour based on the detection result, and (c-2) deforming the second contour.
The image synthesizing method according to claim 1, wherein the second image is transformed according to a contour. 6. (a) a first clothing image and a plurality of second clothing images are stored in the memory, and (b) a first contour of the first clothing image and a second contour of one second clothing image are superimposed. In this case, it is determined whether or not a predetermined portion of the second contour extends beyond the first contour, (c) a second clothing image having no protrusion is selected based on the determination result, and (d) the first clothing image and An image synthesizing method for synthesizing the selected second clothing image. 7. (a) Preparing a memory, (b) writing a first clothing image to be worn on the upper half of the body and a second clothing image to be worn on the lower half of the body to a predetermined position in the memory, and (c) the first clothing. Transforming at least one of the first clothes image and the second clothes image such that the clothes image and the second clothes image partially overlap each other; and
(d) An image synthesizing method for synthesizing the first clothes image and the second clothes image. 8. (a) The step (c) includes (c-1) the first
Designating a first contour below the clothes image and a second contour above the second clothes image, and (c-2) so that the two furthest points of the first contour and the second contour coincide. The image combining method according to claim 7, wherein at least one of the first clothes image and the second clothes image is transformed.