WO2020171237A1 - Dispositif de traitement d'image, modèle appris, dispositif de collecte d'image, procédé de traitement d'image et programme de traitement d'image - Google Patents

Dispositif de traitement d'image, modèle appris, dispositif de collecte d'image, procédé de traitement d'image et programme de traitement d'image Download PDF

Info

Publication number
WO2020171237A1
WO2020171237A1 PCT/JP2020/007392 JP2020007392W WO2020171237A1 WO 2020171237 A1 WO2020171237 A1 WO 2020171237A1 JP 2020007392 W JP2020007392 W JP 2020007392W WO 2020171237 A1 WO2020171237 A1 WO 2020171237A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
posture
try
clothing
user
Prior art date
Application number
PCT/JP2020/007392
Other languages
English (en)
Japanese (ja)
Inventor
五十嵐　健夫
信行 梅谷
Original Assignee
国立大学法人東京大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 国立大学法人東京大学 filed Critical 国立大学法人東京大学
Priority to JP2021502253A priority Critical patent/JP7497059B2/ja
Publication of WO2020171237A1 publication Critical patent/WO2020171237A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/80—Creating or modifying a manually drawn or painted image using a manual input device, e.g. mouse, light pen, direction keys on keyboard

Definitions

  • the present invention relates to an image processing device, a learned model, an image collecting device, an image processing method, and an image processing program.
  • the present invention provides an image processing device, a learned model, an image collecting device, an image processing method, and an image processing program that can reduce the calculation load when trying on a virtual wearing product.
  • An image processing apparatus uses, as input data, an image of a reference wear product worn by a wearer in a predetermined posture, and an image of a try-on wear product worn by a wearer in a posture common to the predetermined posture.
  • the image of the user wearing the reference fitting is input to the learned model learned by machine learning using the learning data associated with the output as the output data, and the image of the user trying on the fitting is displayed.
  • An image generation unit for generating is provided.
  • the learned model is learned using the learning data in which the image of the reference wear product and the image of the try-on wear product are associated with each other, and the image of the user wearing the reference wear product is input to this learning model.
  • the image generation unit extracts the image of the reference wear product and the image of the body part of the user from the image of the user wearing the reference wear product, and extracts the image of the extracted reference wear product from the image of the reference wear product.
  • An image of the user who tried on the fitting-on article by combining the image of the fitting-on article output from the learned model and the image of the extracted body part of the user by inputting to the learned model It may be generated.
  • the image generation unit extracts depth information of an image of the reference worn product and depth information of an image of the body part of the user from the image of the user wearing the reference worn product, and the extracted depth.
  • the image of the fitting product and the image of the body part of the user may be combined based on the information.
  • the learning data is a first variation in which the wearer wears the reference wearable image worn by the wearer in a predetermined posture as the input data, and the wearer wears it in a posture common to the predetermined posture.
  • An image of the fitting-on product is associated as the output data
  • an image of the fitting-on product of a second variation which the wearing subject wears in a posture common to the predetermined posture, is output to the input data.
  • the image generation unit accepts selection of a variation of the try-on fitting product, inputs the image of the user wearing the reference wear product to the learned model, and attaches the try-on fitting of the selected variation.
  • An image of the user who tried on the item may be generated.
  • the worn article is clothing
  • the learning data uses the image of the reference clothing worn by the wearer in a predetermined posture as the input data, and wears it in a posture common to the predetermined posture. It may include data in which the image of the try-on garment worn by is associated as the output data.
  • the calculation load in the virtual clothing fitting is reduced. be able to.
  • An image acquisition apparatus is a robot mannequin, a camera, a robot control unit that controls the posture of the robot mannequin, and the robot mannequin to which a wearing article is attached based on the control of the robot control unit.
  • An image capturing control unit that captures an image of the attached article with the camera in a state in which the orientation is controlled to a predetermined orientation and stores the captured image in a storage unit in association with the orientation.
  • the image of the reference wear product and the image of the try-on wear product can be accurately associated with each other, and the reliability of the learning data used for learning the learned model can be increased.
  • An image processing method uses, as input data, an image of a reference wear product worn by a wearer in a predetermined posture, and an image of a try-on wear product worn by a wearer in a posture common to the predetermined posture.
  • the image of the user wearing the reference fitting is input to the learned model learned by machine learning using the learning data associated with the output as the output data, and the image of the user trying on the fitting is displayed.
  • An image generation step of generating is included.
  • An image processing program uses, as input data, an image of a reference wear product worn by a wearer in a predetermined posture in a computer, and try-on wear worn by a wearer in a posture common to the predetermined posture.
  • An image of a user wearing the reference wear product is input to a learned model learned by machine learning using learning data in which an image of a product is associated as output data, and a user trying on the try-on wear product.
  • the process for generating the image is executed.
  • FIG. 1 is a block diagram showing a schematic configuration of a first embodiment of an image processing device.
  • FIG. 6 is a diagram for explaining an example of an image generation process.
  • the flowchart which shows the processing content of an image generation process.
  • FIG. 6 is a diagram for explaining an example of an image generation process.
  • FIG. 6 is a diagram for explaining an example of an image generation process. It is a figure which shows an example of the hardware constitutions of an image processing apparatus.
  • the image collection device 1 includes, for example, a camera 10, a robot mannequin 20, a display 30, and an image processing device 100. These devices and devices are connected to each other by a communication line, a wireless communication network, or the like.
  • the configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted, or another configuration may be added.
  • the camera 10 is, for example, a digital camera using a solid-state image sensor such as CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor).
  • the camera 10 is, for example, fixedly arranged in front of the robot mannequin 20, and captures an image of the robot mannequin 20 on which clothes, which is an example of a wearing article, is attached.
  • the camera 10 may be, for example, a stereo camera or a depth camera.
  • the robot mannequin 20 is, for example, a humanoid robot.
  • the robot mannequin 20 is configured so that it can be driven with, for example, four degrees of freedom for both shoulders, four degrees of freedom for both elbows, and two degrees of freedom for rotation of the waist, and can take various postures while wearing clothes. It is possible.
  • the display 30 is, for example, a liquid crystal display, and displays an image generated by the image processing device 100.
  • the image processing apparatus 100 includes, for example, a control unit 110 and a storage unit 120.
  • the control unit 110 is realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software).
  • a hardware processor such as a CPU (Central Processing Unit) executing a program (software).
  • some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), etc. Part: including circuitry), or may be realized by cooperation of software and hardware.
  • the program may be stored in advance in a storage device such as a HDD or a flash memory of the image processing apparatus 100, or in a removable storage medium such as a DVD or a CD-ROM, and the storage medium is used as a drive device. It may be installed in the HDD or flash memory of the image processing apparatus 100 by being attached.
  • the control unit 110 includes, for example, a robot control unit 112, a shooting control unit 114, a learning unit 116, and an image generation unit 118.
  • the robot controller 112 controls the operation of the robot mannequin 20.
  • the robot control unit 112 controls the posture of the robot mannequin 20, for example, by controlling the amount of movement of the robot mannequin 20 in each degree of freedom.
  • Each degree of freedom of the robot mannequin 20 is centered on a central axis extending in the vertical direction in addition to the degree of freedom of pose of the robot mannequin 20, such as the degree of freedom of both shoulders, the degree of freedom of both elbows, and the degree of rotation of the waist.
  • the degree of freedom of the rotational position of the robot manikin 20 is included.
  • the degree of freedom of the robot mannequin 20 may include, for example, the angle of the robot mannequin 20 with respect to the shooting position/direction of the camera 10.
  • the above description is merely one example of the posture of the robot manikin 20, and the present invention is not limited to this.
  • the image capturing control unit 114 controls the image capturing operation of the camera 10.
  • the imaging control unit 114 captures an image of the robot mannequin 20 in a state where the posture of the robot mannequin 20 is controlled to a predetermined posture based on the control of the robot control unit 112, for example.
  • the imaging control unit 114 captures images of the robot mannequin 20 while changing the rotational position of the robot mannequin 20 for various poses of the robot mannequin 20 under the control of the robot control unit 112, for example.
  • the imaging control unit 114 stores the captured image in the storage unit 120 in association with the posture of the robot mannequin 20.
  • the image capturing control unit 114 captures, for example, an image of the robot mannequin 20 wearing measurement clothing and an image of the robot mannequin 20 wearing try-on clothing.
  • the image capturing control unit 114 divides the image of the measurement clothing from the image of the robot mannequin 20 on which the measurement clothing is worn, and associates the divided image with the posture of the robot mannequin 20 at the time of capturing the image to obtain image data 122. It is stored in the storage unit 120 as.
  • the imaging control unit 114 divides the image of the try-on garment into regions from the image of the robot mannequin 20 wearing the try-on garment, and associates the region-divided image with the posture of the robot mannequin 20 at the time of capturing the image.
  • the data 122 is stored in the storage unit 120.
  • FIG. 2 is a diagram showing an example of the data content of the image data 122.
  • the data attributes of the image data 122 include, for example, the type of image, image information, and the posture of the robot mannequin 20.
  • the type of image is the type of clothing worn by the robot mannequin 20, and includes, for example, measurement clothing and try-on clothing.
  • the measurement garment is an example of a reference garment referred to when synthesizing the try-on garment.
  • the measurement garment is, for example, a garment for acquiring information about the body of the user, and it is preferable that the measurement garment be easily distinguished from the skin color of the human body and easily associated with various try-on garments.
  • the try-on garment is a garment to be virtually tried on, and includes, for example, a plurality of types of garments having different designs.
  • the image information includes information about the brightness value (R, G, B) of each image position of the image captured by the camera 10.
  • the image information may further include depth information for each image position of the image captured by the camera 10.
  • the posture of the robot manikin 20 is a parameter defined by a combination of motion amounts of the robot manikin 20 in each degree of freedom.
  • the learning unit 116 stores, in the storage unit 120, learning data 124 in which an image of measurement clothing is used as input data and an image of measurement clothing and an image of try-on clothing in which the posture of the robot mannequin 20 is common are associated as output data. To do. That is, the learning unit 116 associates the image of the measurement clothing and the image of the try-on clothing captured in the same pose from the same direction and stores them in the storage unit 120 as the learning data 124.
  • the learning unit 116 refers to the image data 122 stored in the storage unit 120, for example, and stores the learning data 124 in the storage unit 120 in association with the image of the measurement clothing and the image of the try-on clothing.
  • the learning unit 116 learns a learning model by machine learning using the learning data 124 and generates a learned model 126.
  • the learned model 126 is composed of, for example, a neural network which is a type of machine learning model.
  • the learning unit 116 increases the number of data of the learning data 124 by adding processes such as parallel movement, enlargement/reduction, rotation, and noise addition to the image of the measurement clothing and the image of the try-on clothing. Data augmentation may be performed.
  • FIG. 3 is a diagram showing an example of data contents of the learning data 124.
  • the learning data 124 includes a plurality (N in the illustrated example) of image information “A1” to “AN” in which the postures of the robot manikin 20 are different from each other as the image information of the measurement clothing. There is. Further, the learning data 124 is associated with the image information “A1” to “AN” of the measurement clothes and the image information of the try-on clothes in which the posture of the robot mannequin 20 is common.
  • the learning data 124 is for each clothing (“Clothing 1”, “Clothing 2”, etc.) in which the image information “A1” to “AN” of the measurement clothing and the robot mannequin 20 have the same posture. It is associated with image information.
  • FIG. 4 is a diagram showing an example of the correspondence between the image of the measurement clothing and the image of the try-on clothing.
  • the image information of the measurement clothing and the image information of the try-on clothing are associated with each other for each posture of the robot mannequin 20.
  • the learning data 124 corresponds to, for example, the image information “A1” of the measurement clothing and the image information “B11” of the trial clothing when the posture of the robot mannequin 20 is “posture 1”.
  • the image information “A2” of the measurement clothing when the posture of the robot mannequin 20 is “posture 2” and the image information “B12” of the try-on clothing are associated with each other.
  • the image generation unit 118 inputs the image of the user wearing the measurement clothing as the learned model 126 and generates the image of the user who tried on the try-on clothing.
  • the image generation unit 118 extracts, for example, the image of the measurement clothing and the image of the body part of the user from the image of the user wearing the measurement clothing, and inputs the extracted image of the measurement clothing to the learned model 126.
  • the image of the try-on garment output from the learned model 126 and the extracted image of the body part of the user are combined.
  • FIG. 5 is a diagram for explaining an example of image generation processing by the image processing apparatus 100.
  • the image processing apparatus 100 first divides the image of the measurement clothing from the image of the robot mannequin 20 wearing the measurement clothing and divides the area into the robot wearing the try-on clothing.
  • the image of the try-on garment is divided into regions from the image of the mannequin 20.
  • the image processing apparatus 100 associates the image of the measurement clothing and the image of the try-on clothing, which are divided into regions, as learning data 124, and causes the learning model to be learned by machine learning using the learning data 124. As a result, the learned model 126 is generated.
  • the image processing apparatus 100 divides the image of the measurement clothing and the image of the user's body part into regions based on the image of the user wearing the measurement clothing.
  • the image processing apparatus 100 may acquire the depth information of the image of the measurement clothing and the depth information of the image of the body part of the user. Thereby, the area of the image of the measurement clothing and the image of the body part of the user are accurately divided.
  • the image processing apparatus 100 inputs the image of the measurement clothing, which has been divided into regions, to the learned model 126. As a result, the image of the try-on clothing corresponding to the posture of the user is output from the learned model 126.
  • the image processing apparatus 100 puts on the try-on garment by synthesizing the image of the try-on garment output from the learned model 126 and the image of the user's body part divided into regions as described above. Generate a user image.
  • the image processing apparatus 100 synthesizes the image of the user wearing the try-on garment so that, for example, the image of the body part of the user is located in front of the image of the measurement garment.
  • the image processing apparatus 100 may generate the image of the user wearing the try-on clothing, for example, based on the depth information of the image of the measurement clothing and the depth information of the image of the body part of the user.
  • the image processing apparatus 100 may, for example, combine the user's face or neck with the neck of the clothing, combine the user's arm with the sleeve of the clothing, or the like. It is possible to accurately combine the images.
  • FIG. 6 is a flowchart showing an example of the learning process of the learned model 126. Prior to the processing of the flowchart shown in FIG. 6, it is assumed that the image of the measurement clothing for each posture of the robot manikin 20 has been captured. Further, the process of the flowchart shown in FIG. 6 is executed by a predetermined operation as a trigger, for example, when the try-on garment is attached to the robot mannequin 20.
  • the robot control unit 112 controls the robot mannequin 20 wearing the trial clothing to a predetermined posture (step S10).
  • the image capturing control unit 114 captures an image of the robot mannequin 20 using the camera 10 (step S12).
  • the imaging control unit 114 divides the image of the try-on clothing from the image of the robot mannequin 20 into regions (step S14).
  • the imaging control unit 114 stores the image of the measurement clothing and the image of the trial clothing in which the posture of the robot mannequin 20 is common in association with each other in the storage unit 120 as learning data 124 (step S16).
  • the learning unit 116 uses, in addition to the learning data 124 stored in the storage unit 120, the data padded by performing data augmentation on the learning data 124, by using the learning model by machine learning. Is learned (step S18).
  • the imaging control unit 114 determines whether the learning is finished (step S20). Then, when the imaging control unit 114 determines that the learning has not ended, the robot control unit 112 changes the posture of the robot mannequin 20 (step S22), and changes the posture of the trial clothing to the image of the measurement clothing. The processes of steps S12 to S20 are repeated until the image association is completed. On the other hand, when the imaging control unit 114 determines that the learning has ended, the processing of this flowchart ends.
  • FIG. 7 is a flowchart showing an example of image generation processing. Prior to the processing of the flowchart shown in FIG. 7, it is assumed that the try-on garment has been selected by the user in advance. Further, the process of the flowchart shown in FIG. 7 is executed by a predetermined operation as a trigger, for example, when an image of the user wearing the measurement clothing is captured.
  • the image generation unit 118 divides the image of the user wearing the measurement clothing into the image of the body part of the user and the image of the measurement clothing (step S30).
  • the image generation unit 118 inputs the image of the measurement clothing that has been divided into regions in the previous step S30 into the learned model 126 (step S32).
  • the image generation unit 118 synthesizes the image of the try-on garment output from the learned model 126 and the image of the user's body part that has been divided into regions in step S30 (step S34).
  • the image generation unit 118 outputs the combined image to the display 30 (step S36). This completes the processing of this flowchart.
  • the image processing device 100 uses the image of the measurement clothing worn by the robot mannequin 20 in a predetermined posture as input data, and outputs the image of the try-on clothing worn by the robot mannequin 20 in a posture common to the predetermined posture.
  • the learned model 126 is learned by machine learning using the learning data 124 associated as data. Further, the image processing apparatus 100 inputs the image of the user wearing the measurement clothing to the learned model 126 to generate the image of the user wearing the try-on clothing. That is, the image of the measurement clothing worn by the user is converted into the image of the try-on clothing using the learned model 126, and the image of the user wearing the try-on clothing is generated using the converted image.
  • the image processing apparatus 100 extracts the image of the measurement clothing and the image of the body part of the user from the image of the user wearing the measurement clothing, and inputs the extracted image of the measurement clothing to the learned model 126. To do. Further, the image processing apparatus 100 synthesizes the image of the try-on garment output from the learned model 126 and the image of the body part of the user extracted previously to obtain the image of the user trying on the try-on garment. To generate. That is, by aligning and synthesizing the image of the try-on clothing converted using the learned model 126 and the image of the user's body part, it is possible to accurately synthesize the image of the user trying on the try-on clothing. it can.
  • the image processing apparatus 100 extracts the depth information of the image of the measurement clothing and the depth information of the image of the body part of the user from the image of the user wearing the measurement clothing, and based on the extracted depth information, The image of the try-on clothes and the image of the body part of the user are combined. Thereby, the image of the user who tried on the try-on clothes can be more accurately combined.
  • the image processing apparatus 100 captures an image of clothing with the camera 10 in a state where the posture of the robot mannequin 20 on which the clothing is attached is controlled to a predetermined orientation, and the captured clothing image is displayed on the robot mannequin 20. It is stored in the storage unit 120 in association with the posture. Thereby, the image of the measurement clothing and the image of the try-on clothing can be accurately associated, and the reliability of the learning data 124 can be improved.
  • the image processing apparatus 100 learns to include images of the robot mannequin 20 viewed from various rotation positions, such as images of the robot mannequin 20 viewed from the front, in addition to images viewed from the front.
  • the data for use 124 is configured. As a result, even when the user takes various postures, it is possible to accurately synthesize the image of the user who tried on the try-on clothing.
  • the image processing apparatus 100 configures the learning data 124 by associating the image of the measurement clothing and the image of the try-on clothing. Accordingly, it is possible to accurately predict the deformation of the try-on garment due to the change in the posture of the user, and it is possible to more accurately synthesize the image of the user wearing the try-on garment.
  • the learning unit 116 uses the image of the measurement clothing as input data, and the images of the trial clothing of a plurality of variations in which the image of the measurement clothing and the posture of the robot manikin 20 are common as output data.
  • the associated learning data 124 is stored in the storage unit 120.
  • the plurality of variations include, for example, the color and size of the try-on garment.
  • the learning data 124 uses, for example, the image of the measurement clothing as input data, associates the image of the first variation try-on clothing in which the posture of the measurement clothing and the robot mannequin 20 are common as output data, and An image of clothing is used as input data, and an image of a second variation of try-on clothing in which the postures of the measurement clothing and the robot mannequin 20 are common is associated as output data.
  • FIG. 8 is a diagram showing an example of the correspondence between the image of the measurement clothing and the image of the try-on clothing.
  • the image information of the measurement clothing and the image information of the try-on clothing are associated with each other for each posture of the robot mannequin 20.
  • the learning data 124 corresponds to, for example, the image information “A1” of the measurement clothing in which the posture of the robot mannequin 20 is “posture 1” and the image information “B11 ⁇ ” of the S-size trial clothing. It is attached.
  • the image information “A1” of the measurement clothing in which the posture of the robot manikin 20 is “posture 1” and the image information “B11 ⁇ ” of the M-size try-on clothing are associated with each other. Further, the image information “A1” of the measurement clothing in which the posture of the robot mannequin 20 is “posture 1” and the image information “B11 ⁇ ” of the L-size try-on clothing are associated with each other.
  • FIG. 9 is a diagram for explaining an example of image generation processing by the image processing apparatus 100.
  • the image processing apparatus 100 divides the image of the user wearing the measuring clothes into the image of the measuring clothes and the image of the body part of the user. Then, the image processing apparatus 100 inputs the image of the measurement clothing, which has been divided into regions, to the learned model 126.
  • the size of the try-on garment output from the learned model 126 (“M size” in the illustrated example) is selected in advance. As a result, an image of the try-on garment corresponding to the size selected in advance is output from the learned model 126.
  • the image processing apparatus 100 synthesizes the image of the try-on garment output from the learned model 126 and the image of the user's body part, which is divided into regions as described above, so that the user wearing the try-on garment. Generate an image of.
  • the image processing apparatus 100 accepts the selection of the variation of the try-on clothing, inputs the image of the user wearing the measurement clothing to the learned model 126, and tries on the try-on clothing of the previously selected variation. Generate an image of the user who made the request. Accordingly, it is possible to generate an image of the user who has tried on the try-on garment by distinguishing it for each variation of the try-on garment selected by the user.
  • the third embodiment is different from the first embodiment in the method of associating the image of the measurement clothing and the image of the try-on clothing. Therefore, in the following description, a configuration different from that of the first embodiment will be mainly described, and a duplicate description of the same or corresponding configuration as that of the first embodiment will be omitted.
  • the learning unit 116 uses the image of measurement clothing as input data, and associates the image of measurement clothing with the image of try-on clothing in which the body shape and posture of the robot manikin 20 are common as output data.
  • the learning data 124 is stored in the storage unit 120.
  • the body shape of the robot mannequin 20 is defined by parameters such as arm circumference, shoulder width, and abdominal circumference.
  • the learning data 124 includes, for example, an image of the measurement clothing worn by the robot mannequin 20 corresponding to a normal body shape, and an image of the try-on clothing in which the measurement clothing and the robot mannequin 20 have the same body shape and posture.
  • the data 124 is configured. That is, the learning data 124 associates the image of the measurement clothing with the image of the try-on clothing so as to correspond to users of a plurality of body types, and the number of data of the learning data 124 is increased.
  • FIG. 10 is a diagram showing an example of the correspondence between the image of the measurement clothing and the image of the try-on clothing.
  • the image information of the measurement clothing and the image information of the try-on clothing are associated with each other for each body type and posture of the robot mannequin 20.
  • the learning data 124 is, for example, for the robot mannequin 20 corresponding to a normal body type, the image information “A1” of the measurement clothing in which the posture of the robot mannequin 20 is “posture 1” and the trial clothing.
  • the image information “B11” is associated.
  • the image information “A1X” of the measurement clothing in which the posture of the robot mannequin 20 is “posture 1” and the image information “B11X” of the try-on clothing are associated with each other. ..
  • FIG. 11 is a diagram for explaining an example of image generation processing by the image processing apparatus 100.
  • the image processing apparatus 100 extracts the image of the measurement clothing and the image of the body part of the user from the image of the user who has the normal figure wearing the measurement clothing. Divide into areas. Then, the image processing apparatus 100 inputs the image of the measurement clothing, which has been divided into regions, to the learned model 126. As a result, an image of the try-on garment close to a normal body shape is output from the learned model 126. After that, the image processing apparatus 100 synthesizes the image of the try-on garment output from the learned model 126 and the image of the user's body part, which is divided into regions as described above, so that the user wearing the try-on garment. Generate an image of.
  • the image processing apparatus 100 includes an image of the obesity-type user wearing measurement clothing, an image of the measurement clothing, and an image of the body part of the user as an area. To divide. Then, the image processing apparatus 100 inputs the image of the measurement clothing, which has been divided into regions, to the learned model 126. As a result, an image of the try-on garment close to the obesity type is output from the learned model 126. After that, the image processing apparatus 100 synthesizes the image of the try-on garment output from the learned model 126 and the image of the user's body part, which is divided into regions as described above, so that the user wearing the try-on garment. Generate an image of.
  • the image processing apparatus 100 learns the learned model 126 using the learning data 124 in which the image of the measurement clothing and the image of the try-on clothing are associated with each other so as to correspond to the users of a plurality of body types. There is. In addition, the image processing apparatus 100 inputs the image of the user wearing the measurement clothing into the learned model 126 to generate images in which the users of various body types wear the try-on clothing. That is, by configuring the learning data 124 so as to correspond to users of various body types, it is possible to generate a highly realistic image of try-on clothing while suppressing the calculation load.
  • the image processing apparatus 100 controls the robot mannequin 20 to collect an image of the measurement clothing and an image of the try-on clothing as the learning data 124.
  • the image of the measurement clothing and the image of the try-on clothing may be collected as the learning data 124 by changing the posture of the subject wearing the clothing.
  • the learned model 126 is learned using the image of the measurement clothing worn by the user.
  • the learned model 126 may be learned using an image of normal clothing worn by the user.
  • the learning data 124 may be configured by estimating the posture of the user based on the image of the ordinary clothing worn by the user and associating the estimated posture of the user with the image of the ordinary clothing.
  • the image in which the ordinary clothes are attached to the robot mannequin 20 may be used, or the subject is caused to wear the ordinary clothes. Images may be used.
  • FIG. 12 is a diagram illustrating an example of the hardware configuration of the image processing apparatus 100 according to the embodiment.
  • the image processing apparatus 100 includes a communication controller 100-1, a CPU 100-2, a RAM (Randome Access Memory) 100-3 used as a working memory, and a ROM (Read Only Memory) 100 for storing a boot program and the like.
  • a storage device 100-5 such as a flash memory or an HDD (Hard Disk Drive), a drive device 100-6, etc. are connected to each other by an internal bus or a dedicated communication line.
  • the communication controller 100-1 communicates with components other than the image processing apparatus 100.
  • a program 100-5a executed by the CPU 100-2 is stored in the storage device 100-5.
  • This program is expanded in the RAM 100-3 by a DMA (Direct Memory Access) controller (not shown) or the like and executed by the CPU 100-2.
  • a DMA Direct Memory Access
  • the robot control unit 112, the shooting control unit 114, the learning unit 116, and the image generation unit 118 are realized.
  • a storage device that stores a program
  • a hardware processor executes a program stored in the storage device
  • the learning data in which the image of the reference wear product worn by the wearer in a predetermined posture is used as input data and the image of the try-on wear product worn by the wearer in a posture common to the predetermined posture is used as output data is used.
  • An image processing device configured to input an image of a user wearing the reference wearing product to a learned model learned by machine learning and generate an image of a user trying on the fitting wear product. ..
  • Image generation system 10... Camera, 20... Robot mannequin, 30... Display, 100... Image processing device, 110... Control part, 112... Robot control part, 114... Shooting control part, 116... Learning part, 118... Image Generation unit, 120... Storage unit, 122... Image data, 124... Learning data, 126... Learned model.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Processing Or Creating Images (AREA)

Abstract

L'invention concerne un dispositif de traitement d'image, un modèle appris, un dispositif de collecte d'image, un procédé de traitement d'image et un programme de traitement d'image avec lesquels une charge de calcul peut être réduite lors de la réalisation d'essayage virtuel d'un article vestimentaire. Un dispositif de traitement d'image (100) comprend une unité de génération d'image (118) qui génère une image d'un utilisateur essayant un article vestimentaire d'essayage par fourniture en entrée d'une image de l'utilisateur portant un article vestimentaire de référence dans un modèle appris (126) qui a appris par apprentissage automatique à l'aide de données d'apprentissage (124) dans lesquelles sont associés : des images, servant de données d'entrée, des articles vestimentaires de référence portés par un agent d'habillement dans une posture prescrite ; et des images, servant de données de sortie, des articles vestimentaires d'essayage portés par l'agent d'habillement dans une posture commune à la posture prescrite.
PCT/JP2020/007392 2019-02-22 2020-02-25 Dispositif de traitement d'image, modèle appris, dispositif de collecte d'image, procédé de traitement d'image et programme de traitement d'image WO2020171237A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2021502253A JP7497059B2 (ja) 2019-02-22 2020-02-25 画像処理装置、学習済みモデル、画像収集装置、画像処理方法、および、画像処理プログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201962809088P 2019-02-22 2019-02-22
US62/809,088 2019-02-22

Publications (1)

Publication Number Publication Date
WO2020171237A1 true WO2020171237A1 (fr) 2020-08-27

Family

ID=72144680

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/007392 WO2020171237A1 (fr) 2019-02-22 2020-02-25 Dispositif de traitement d'image, modèle appris, dispositif de collecte d'image, procédé de traitement d'image et programme de traitement d'image

Country Status (2)

Country Link
JP (1) JP7497059B2 (fr)
WO (1) WO2020171237A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200380594A1 (en) * 2018-02-21 2020-12-03 Kabushiki Kaisha Toshiba Virtual try-on system, virtual try-on method, computer program product, and information processing device
WO2022161301A1 (fr) * 2021-01-28 2022-08-04 腾讯科技（深圳）有限公司 Procédé et appareil de génération d'images, dispositif informatique et support de stockage lisible par ordinateur

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019163218A1 (fr) * 2018-02-21 2019-08-29 株式会社東芝 Système d'essayage virtuel, procédé d'essayage virtuel, programme d'essayage virtuel, dispositif de traitement d'informations et données d'apprentissage

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019163218A1 (fr) * 2018-02-21 2019-08-29 株式会社東芝 Système d'essayage virtuel, procédé d'essayage virtuel, programme d'essayage virtuel, dispositif de traitement d'informations et données d'apprentissage

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
EHARA,JUN ET AL.: "Texture Overlay onto Deformable Surface for Virtual Clothing", IEICE TECHNICAL REPORT, vol. 105, no. 536, 2005, pages 129 - 134, XP058167000, DOI: 10.1145/1152399.1152431 *
YASUDA,TOMOMI ET AL.: "A Study on a Virtual Dressing Simulation System toward both Simpleness and Cloth Deformation", IEICE TECHNICAL REPORT, vol. 109, no. 471, 2010, pages 91 - 96 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200380594A1 (en) * 2018-02-21 2020-12-03 Kabushiki Kaisha Toshiba Virtual try-on system, virtual try-on method, computer program product, and information processing device
WO2022161301A1 (fr) * 2021-01-28 2022-08-04 腾讯科技（深圳）有限公司 Procédé et appareil de génération d'images, dispositif informatique et support de stockage lisible par ordinateur

Also Published As

Publication number Publication date
JPWO2020171237A1 (ja) 2021-12-16
JP7497059B2 (ja) 2024-06-10

Similar Documents

Publication Publication Date Title
US20210074005A1 (en) Image processing method and apparatus, image device, and storage medium
US20220182598A1 (en) Systems, methods and apparatuses for stereo vision and tracking
EP3479296B1 (fr) Système d'habillage virtuel utilisant un traitement d'image, un apprentissage automatique et une vision artificielle
TWI554951B (zh) 實現虛擬試戴的方法和裝置
TWI488071B (zh) 非接觸式三度空間人體資料擷取系統及方法
CN110609617B (zh) 虚拟镜子的装置、系统和方法
US8976230B1 (en) User interface and methods to adapt images for approximating torso dimensions to simulate the appearance of various states of dress
EP3745352B1 (fr) Procédés et systèmes pour déterminer des mesures corporelles et fournir des recommandations de tailles de vêtements
US20150248583A1 (en) Image processing apparatus, image processing system, image processing method, and computer program product
WO2023109753A1 (fr) Procédé et appareil de génération d'animation de personnage virtuel, et support de stockage et terminal
Gültepe et al. Real-time virtual fitting with body measurement and motion smoothing
WO2020171237A1 (fr) Dispositif de traitement d'image, modèle appris, dispositif de collecte d'image, procédé de traitement d'image et programme de traitement d'image
CN106210504A (zh) 图像处理装置、图像处理系统及图像处理方法
WO2020203656A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations, et programme
CN109671141B (zh) 图像的渲染方法和装置、存储介质、电子装置
CN107481280B (zh) 一种骨骼点的矫正方法及计算设备
CN104966284A (zh) 基于深度数据获得对象尺寸信息的方法及设备
Xu et al. 3d virtual garment modeling from rgb images
JP2022036963A (ja) サイズ測定システム
CN109766782A (zh) 基于svm的实时肢体动作识别方法
Zong et al. An exploratory study of integrative approach between 3D body scanning technology and motion capture systems in the apparel industry
Yolcu et al. Real time virtual mirror using kinect
WO2020156627A1 (fr) Étrier virtuel : création rapide d'avatars métriquement précis à partir de mesures 3d
Ram et al. A review on virtual reality for 3D virtual trial room
Cha et al. Mobile. Egocentric human body motion reconstruction using only eyeglasses-mounted cameras and a few body-worn inertial sensors

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20760298

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021502253

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20760298

Country of ref document: EP

Kind code of ref document: A1