WO2022006683A1 - Tension-map based virtual fitting room systems and methods - Google Patents

Tension-map based virtual fitting room systems and methods Download PDF

Info

Publication number
WO2022006683A1
WO2022006683A1 PCT/CA2021/050950 CA2021050950W WO2022006683A1 WO 2022006683 A1 WO2022006683 A1 WO 2022006683A1 CA 2021050950 W CA2021050950 W CA 2021050950W WO 2022006683 A1 WO2022006683 A1 WO 2022006683A1
Authority
WO
WIPO (PCT)
Prior art keywords
clothing
fit
customer
digital human
clothing item
Prior art date
Application number
PCT/CA2021/050950
Other languages
French (fr)
Inventor
Daya Karunita WIMALASURIYA
Original Assignee
Wimalasuriya Daya Karunita
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wimalasuriya Daya Karunita filed Critical Wimalasuriya Daya Karunita
Publication of WO2022006683A1 publication Critical patent/WO2022006683A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0641Shopping interfaces
    • AHUMAN NECESSITIES
    • A41WEARING APPAREL
    • A41HAPPLIANCES OR METHODS FOR MAKING CLOTHES, e.g. FOR DRESS-MAKING OR FOR TAILORING, NOT OTHERWISE PROVIDED FOR
    • A41H1/00Measuring aids or methods
    • AHUMAN NECESSITIES
    • A41WEARING APPAREL
    • A41HAPPLIANCES OR METHODS FOR MAKING CLOTHES, e.g. FOR DRESS-MAKING OR FOR TAILORING, NOT OTHERWISE PROVIDED FOR
    • A41H3/00Patterns for cutting-out; Methods of drafting or marking-out such patterns, e.g. on the cloth
    • A41H3/007Methods of drafting or marking-out patterns using computers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0621Item configuration or customization

Definitions

  • This invention relates in general to methods, apparatuses, and systems for providing a virtual fitting room to meet customer expectations of a fit, connecting an online clothing retail site with a user-friendly operation for brand owners, retailers, and end customers.
  • the present invention discloses methods, apparatuses, and systems for Tension-Map based Virtual Fitting Room Systems and Methods.
  • the Tension Map Based Virtual Fitting Room comprises of an integrated set of 7 modules designed to address the online clothing fit-on functionality.
  • the Access Control module allows a customer to connect to TMBVFR to create a personal profile that may connect to a digital human representative of the customer's body di mensions and keep a track of the customer's try-on history. For example, a unique customer name or identifier paired with a secret password or passphrase may provide this access con trol functionality.
  • the Web portal Integration module enables the rest of TMBVFR to connect with eCommerce web portals so that the online customers for those sites can access TMBVFR functionality. It further enables the rest of TMBVFR to connect with clothing manufacturers and brand owners so that they may add their clothing products to TMBVFR. A customer who gains access to TMBVFR through the Access Control module may virtually fit-on such clothing products.
  • the Digital Human module enables the cre ation of a specific 3D model of the customer based on the specific body dimensions and the ethnic profile of that customer.
  • a customer who gains access to TMBVFR through the Access Control module may cause the customer's Digital Human to virtually fit-on clothing products made available through the Web portal Integration module.
  • the Clothing Simulator module enables the creation of 3D models of each clothing product made available through the Web portal Inte gration module.
  • the 3D clothing image of each clothing item built using the 3D models may be virtually put on and matched with a customer's Digital Human obtained from the Digital Human module to achieve a virtual fit-on.
  • the Virtual Fit-on module provides for the virtual fitting be tween a customer’s Digital Human from the Digital Human module and a clothing product selected by that customer through the Web portal integration module generating a 3D view of how the clothing item is fit-on the Digital Human.
  • the Tension-Map module provides the customer with a 3D visualization of the distribution of tension when the customer's Digital Human from the Digital Human module wears the clothing item made available through the Web portal inte - gration module, indicating which areas of the clothing item may feel tight and loose leading to an enhanced perception of the clothing item's fit-on.
  • the Proactive Recommendations module will pro-actively rec ommend to the customer, who obtained access to TMBVFR through the Access Control mod ule, alternative clothing items based on current and past fit-on history maintained by the TM BVFR.
  • FIG. 1 is a diagram of the modules and components of the Tension Map Based Vir tual Fitting Room (TMBVFR) according to some embodiments;
  • Figure 2 is a diagram of the TMBVFR Web Portal Integration process according to some embodiments.
  • Figure 3 is a diagram of the TMBVFR Clothing Item Creation process according to some embodiments.
  • Figure 4 is a diagram of the TMBVFR End To End Virtual Fit-on process according to some embodiments.
  • Figure 5 is a diagram of the TMBVFR Digital Human process according to some em bodiments
  • Figure 6 is a diagram of the TMBVFR Virtual Fit-on process according to some em bodiments
  • Figure 7 is a diagram of the TMBVFR Tension Map process according to some em bodiments
  • Figure 8 is a diagram of the TMBVFR Fabric Elasticity and Compressibility process according to some embodiments.
  • Figure 9 is a diagram showing the body measurements for clothing items for a Tech Pack according to some embodiments.
  • Figure 10 is a diagram of a Tech Pack according to some embodiments.
  • Figure 11 is an illustration showing the customer measurements input screen of the TMBVFR Digital Human module according to some embodiments.
  • Figure 12 is an illustration showing the generated Digital Human based on customer measurements input to the TMBVFR according to some embodiments
  • Figure 13 is an illustration showing the input of customer measurements and the out put generated for a Digital Human via TMBVFR for the purpose of performing a virtual fit- on;
  • Figure 14 is an illustration showing the input and output of the virtual fit-on process via the TMBVFR Virtual Fit-on module according to some embodiments;
  • Figure 15 is an illustration showing how the output changes when a virtual fit-on is performed by TMBVFR with the input of different clothing sizes according to some embodi ments.
  • Figure 16 is an illustration showing the output of the TMBVFR Tension Map module according to some embodiments. DETAIFED DESCRIPTION OF EMBODIMENTS OF THE INVENTION
  • the Tension-Map Based Virtual Fitting Room provides a virtual fitting room to meet customer expectations of a fit, connects an online clothing retail site with a user-friendly operation for brand owners, retailers and end customers.
  • the TMBVFR comprises of an integrated set of 7 modules designed to address the online clothing fit-on func - tionality as described below.
  • the Access Control module allows the customer who connect TMBVFR to create a profile for them that may connect their digital human and keep a track of their try-on history. It can allow customer faster access to fitting via the existing digital human and may also al - low customers to change their digital human as they wish, based on how their body dimensions change.
  • TMBVFR When accessing the TMBVFR, it may prompt the use to sign-in via existing profile name and password, create a new profile or continue to virtually fit-on as a guest.
  • a “Virtual Fit-On” is a concept which may enable a customer shopping online to se - lect a clothing item in an online store and validate with a high degree of certainty that the selected clothing item may be a good fit for his/her body at the time of shopping. It can be realized via software tools added to eCommerce web portals and a method to capture the cus - tomer’s body dimensions and match it with the selected clothing item and to show the result for the customer to see.
  • the Web portal Integration module enables the rest of TMBVFR to connect with eCommerce web portals so that the online customers for those sites can access TMBVFR functionality. It may also connect clothing manufacturers and brand owners to add their clothing products to TMBVFR so that customers can virtually fit-on those products when they do online shopping.
  • the Digital Human module enables the creation of a Digital Human (a specific 3D model of the customer based on the specific body dimensions and the ethnic profile of that customer) of any online customer who wish to virtually fit-on a clothing product via a Web portal subscribed to the TMBVFR.
  • the Digital Human may be created by prompting the user to enter a set of dimensions and parameters.
  • the TMBVFR may accept measurements used for custom clothes taken by a tailor.
  • users may use a depth camera and take pictures and let a tool use depth camera picture data to generate the Digital Human.
  • the Clothing Simulator module enables the creation of 3D models of each clothing product that the online customers need to virtually fit-on.
  • the clothing design dimensions are fed into this module as an input from a relevant source via clothing brand owners or manufac turers in 3D design details and it is processed to come up with a 3D clothing image of each clothing items, which is ready to put on and matched with a Digital Human.
  • the generated 3D clothing images are stored in the TMBVFR database in cloud storage and can be re-used as needed for virtual fit-on.
  • the 3D clothing images may be revised as per subsequent changes in the design of the clothing items.
  • the interface may work with the most commonly used clothing design tools in the market to feed in clothing designs to TMBVFR and in alternate versions, TMBVFR may be compatible with every clothing design tool.
  • the Virtual Fit-on module does the virtual fitting between a customer’s Digital Hu man and a clothing product selected by that customer.
  • the module may use a 2-step process to generate a 3D view of how the clothing item is fit-on the Digital Human. Firstly, the clothing item may be aligned with the Digital Human in a 3D space, to enable to proceed to fit on. Secondly, via an iteration of pro - Des may be executed to match surface of the Digital Human and the clothing item.
  • the tool may generate a 3D image of how the dress may look fitted on the Digital Human, displaying any matching and wrinkles areas and the 3D image can be rotated view from various angles. Also, the Digital Human with cloth fitted on can be rotated to see how the fit-on is visible from all angles (a feature that is not even possible with a physical fitting room).
  • the fit-on with rotating view may be made avail able and in another embodiment, other technology and capabilities encompassing additional features such as the ability to see how the clothing fit-on change when the Digital Human is put on static postures (bending, sitting, reclining, sitting), ability to see how the clothing fit- on change when the Digital Human is put on dynamic body movements (walking, running, jumping), factor in the fabric elasticity and compressibility of the clothing items, factor in the presence of undergarments and multiple layers of clothing, factor in the translucency of the fabrics and generate a 3D view of how the dress fitted on the Digital Human may look like under different lighting conditions (natural sunlight, incandescent light, fluorescent light)
  • the Tension-Map module provides the customer with a 3D visualization of the distri bution of tension when the customer wears the clothing item, indicating which areas of the clothing item may feel tight and loose.
  • the TMBVFR tool captures the surface measurements and fabric properties of the clothing item and (input from the virtual fitting functionality) and uses an algorithm to calculate the tension values as it is fitted on to the Digital Human. These tension values are visually represented on the Digital Human color-coded based on tension values and enable the customer to take an informed decision based on how comfortable the clothing items may feel when it is used by the customer.
  • the Tension-Map (a color coded map of the digital human which displays different pre-defined colors applied based on the calculated surface tension values simulating how tight or loose the clothing item feels when it is fit-on that body) with the color coded tension distribution display on Digital Human may be made available and in alternate versions, the Tension-Map may work with the new capabilities for Virtual Fit-on where applicable.
  • the Proactive Recommendations module is able to pro-actively recommend the cus tomer alternative clothing items based on current and past (if a registered user) fit-on history.
  • the past fit-on and Tension-Map details are input for this module and based on that and any Tension-Map preferences of the customer and may scan through other items available in the store and may propose listed/rated recommended alternatives to fit-on and purchase.
  • TMB- VFR Tension-Map Based Virtual Fitting Room
  • TMBVFR The detailed functionality of TMBVFR may be via a series of interconnected work flows and processes integrating a number of modules of an AR/VR (Augmented Reality /Vir tual Reality) based software application.
  • AR/VR Augmented Reality /Vir tual Reality
  • the workflows and processes themselves are tool agnostic, the articulation refers to the R&D work that has been done so already with a view of expanding on that (op - tional features) for the practical delivery of the functionality or to incorporate newer technol ogy as in alternate versions.
  • the descriptions refer to the word “clothing”, this tech nology is applicable to perform virtual fit-ons for an entire range of fashion and clothing re - lated products including Clothing, Clothing Accessories, Shoes, Jewelry, Wearables or the like.
  • the word “Clothing” may be representative of that entire product range for all in tended purposes with respect to the TMBVFR.
  • FIG. 1 shows the high level architectural overview of the TMBVFR modules and components.
  • the Web portal Integration module 101 connects the rest of the TMBVFR with eCommerce web portals via the external internet 102, through a secure authentication process.
  • the Access Control module 104 creates the user accounts for users from eCommerce Web portals who wish to register their profdes to use for clothing fit-ons.
  • the Digital Human module 105 may create and manage the 3D body profdes of customers. Both the Access Con trol module and Digital Human modules may be connected with the Customer Database 110.
  • Clothing Simulator module 106 enables the creation of 3D images of all clothing items for customers to perform a virtual fit-on with their Digital Human.
  • Clothing Simulator module is connected with the 3D Clothing Assets Storage 111 and another Database 112 that can store all clothing related parameters as well as clothing material related parameters.
  • the Self-Serve Product Upload module 103 can make it possible for Clothing manufacturers and Clothing brand owners to add any products they produce and/or sell by invoking the functionality of Clothing Simulator module with minimum or no technical support/intervention.
  • the Virtual Fit-on module 107 may use the information from the Customer Database 110 and Clothing parameters Database 112 for customers to fit on clothes and can also be connected to the Vir tual Fit-on History Database 113.
  • the Tension-Map module 108 may be a functionality on top of Virtual Fit-on where the customers may be able to specifically see where a clothing item may feel too tight or too loose when they wear it.
  • Proactive Recommendations module 109 may be able to scan through the history of previous virtual fit-ons done by a registered cus tomer and review the changes of customer’s body profile and recommend alternative clothing items that could be a good fit for a customer.
  • Proactive Recommendations module is con nected with Virtual Fit-on History Database shown 113, Customer Fit-on History Database 116, Customer Database 110 and Clothing parameters Database 112. The purging the virtual fit-on history 115 may happen subject to an affirmative response to a customer choice de noted by Decision Block 114.
  • the Web portal Integration process 200 in Figure 2 covers how the TMBVFR module is integrated with other Web portals (eCommerce retailers and Brand Owners) so that the cus - tomers of eCommerce web portals can access and use the TMBVFR to fit-on products.
  • Web portals eCommerce retailers and Brand Owners
  • TMBVFR may be a common virtual fitting room that can be used by any eCommerce web portal in the world. Also it may come in with an easy plugin process for any subscribed web portals to connect to TMBVFR and can make all their products are added to the fitting room and any customer shopping at their web portal can have the ability to use the virtual fitting room.
  • the TMBFR may be a product that can offer a new software service for eCom merce web portals which sell Clothing related a products - “Fit-on AS A Service” (FAAS), where the eCommerce web portals can subscribe and obtain the service which can then be used by the online customers free of charge to virtually fit-on clothing products online.
  • FAS Fit-on AS A Service
  • the customers who register their profile with TMBVFR may also able to use the same registered customer profile with any eCommerce web portal they do online shopping if it is subscribed to the TMBVFR.
  • the process 200 starts with the eCommerce Portal or Brand Owner becoming a sub scriber of the TMBVFR web portal as per Block 201. As an authorized subscriber, the eCom merce Portal or Brand Owner may be onboarded to TMBVFR to have the connectivity.
  • the TMBVFR may provide a software plug-in to the subscribed eCommerce web portal as per Block 202. Then as denoted in Block 203, the eCommerce web portal may save the TMBVFR plug-in as per the instructions in it to proceed to the next steps of connectivity.
  • the TMBVFR may register the eCommerce web portal in the TMBVFR Admin portal. Once it is registered as a trusted web portal, TMBFR can then make configuration changes to whitelist the eCommerce web portal and all traffic from it to provide connectivity, as shown in Block 205.
  • the eCommerce web portal may do any relevant changes on their front end to leverage the TMBVFR functionality to its online customers. This may include changes such as adding a button “VFR” that may be displayed along with any item they sell online that can be virtually fit-on. Once the button is pressed the online user may be con nected with TMBVFR to go through the End To End Virtual Fit-on process described later in this document.
  • the TMBVFR may enable the traffic connectivity, monitoring and analysis from that site to TMBVFR. This may facilitate the con nectivity for online customers of the eCommerce web portal to use the TMBVFR.
  • the TMBVFR may be launched with compatibility with any web portal that uses any web portal hosting platform (such as Magento, Shopiiy or the like). However, with the likelihood that TMBVFR may be used by web portals using new platforms that may come up in future, the TMBVFR may be enhanced continually as so that it can be integrated with any eCommerce platform.
  • any web portal hosting platform such as Magento, Shopiiy or the like.
  • the Clothing Item Creation process 300 shown in Figure 3 starts with the subscribed eCommerce Web portal or a Clothing Brand Owner proceeding to upload the products they may offer for virtual fit-on by online customers who shop online for that particular clothing item.
  • the availability of the clothing item in TMBVFR as a 3D Clothing Item and eCom merce Web portal having a subscription to TMBVFR are prerequisites for the online customer to be able to a virtual fit-on that particular clothing item at that particular eCommerce web portal.
  • the Clothing Item Creation process 300 described under Figure 3 here applies to the uploading one clothing product to TMBVFR and when an eCommerce web portal or a Cloth ing Brand Owner subscribes to TMBVFR, there may be multiple uploads to have all products ready to be virtually fit-on.
  • the first step, as per Block 301, is to capture clothing details via user interface provided to a Subscriber Vendor (eCommerce Web portal or Clothing Brand Owner). Once the clothing details are in, a validation of data denoted in Block 302 is done to see if the data is sufficient to create a Tech Pack of the clothing item, which may be the main input to get the 3D Clothing Item created. If the data is insufficient, the process denoted in Block 303 may revert back to the Subscriber Vendor to capture missing data.
  • the clothing design details can be obtained as a sketch, photograph, pattern or the like. The body measurements may be relevant to international dress sizes as per Figure 9. As per 900, the dress design details may be collected via multiple sources and consolidated so that it may be used as an input to generate a Tech Pack.
  • the Tech Pack for the clothing item is created as per Block 304.
  • the created Tech Pack may adhere to industry standards to make it complete to be applicable for clothing design and applicable clothing item manufacturing process as shown in Figure 10.
  • the Tech Pack data is then used to generate the 3D Clothing Item for the clothing product.
  • any size (such as largest or smallest size) of the clothing item may be selected as denoted in Block 305.
  • the pre-defined digital human body types are scanned (via an existing database of pre-defined human body types in the TMBVFR backend) and an applicable digital human body type may be located to fit the clothing item.
  • morph targets are generated (A “morph target” is a deformed version of a shape, which can be represented as a snapshot of vortex locations for a specific mesh that have been deformed in some way) of applicable digital humans based on the digital human used to determine the ranges of Digital Human types who could likely fit- on this clothing item.
  • a 3D Clothing Item is created as in Block 308 by using a digital clothing item creation technology platform (such as Marvelous Designer, Clothes3D or the like).
  • a digital clothing item creation technology platform such as Marvelous Designer, Clothes3D or the like.
  • the process in Block 309 is then used to generate surface geometry expressed as a mesh of triangles (presentation done via a graphics Application Programming Interface (API) created for use in web applications, based on an open graphics language) which expands to have different surface tensions when morph targets are applied and determine tension metrics ranges applicable on the surface of the clothing item, using the 3D Clothing item and its surface measurements as the basis.
  • API Application Programming Interface
  • This data may enable determination of Tension-Map calculations when the clothing items is fit-on by a Digital Human within the morph targets.
  • TMBVFR Admin Portal may be used to finalize the 3D Clothing Item.
  • the fabric compressibility and elasticity data is then applied as per subroutine 800 described later, to the 3D Clothing Item data and the morph targets tension metrics ranges are regenerated after factoring in the fabric compressibility and elasticity. If fabric compressibility and elasticity data is not available for a clothing item, this step can be skipped and the 3D Clothing Item created in previous process may be used.
  • the subroutine 800 may be repeated for each size of the clothing item to have fabric compressibility and elasticity details 3D Clothing Item for each size available.
  • the 3D Clothing Item and all supporting data (including tension metrics) for all the clothing item sizes are stored at the cloud storage for TMBVFR. If com - pressibility and elasticity were factored in, then the data in the temporary fde mentioned in Block 807 may also be transferred to the cloud storage.
  • the Tension-Map color coding information is captured as denoted in Block 313 and added to the data related to the 3D Clothing Item in the TMBVFR cloud storage.
  • the clothing item may be ready to be virtually fit-on by online cus tomers who are shopping for clothing on an online eCommerce portal that has subscribed to TMBVFR.
  • the Clothing Simulator comprises of newly developed capabilities as well as the use of well-established digital clothing item creation technology platform for 3D clothing item creation.
  • the TMBVFR may be evolved to be compatible with any clothing design tools used in the garment manufacturing industry.
  • the 3D Clothing Item creation (and virtual fit-on) can be done without fabric compressibility and elasticity, it may be added in alternate versions as the capability exists as validated in R&D work and since it may provide the functionality and the customer experience.
  • the TMBVFR may also have a Self-Serve module, which can automate the process to generate 3D Clothing Items in bulk with minimum of now technical team intervention to add more products to be available via TMBVFR for online shoppers to perform virtual fit- ons.
  • a Self-Serve module which can automate the process to generate 3D Clothing Items in bulk with minimum of now technical team intervention to add more products to be available via TMBVFR for online shoppers to perform virtual fit- ons.
  • the online customers of those eCommerce web portals can use the TMBVFR by in voking the End-To-End Virtual Fit-on process in Figure 4.
  • the End-To-End Virtual Fit-on process in Figure 4 is a combination of processes which aggregates elements from Digital Human process(subroutine 500), Virtual Fit-on process (subroutine 600) and Tension-Map process(subroutine 700), which can simulate the end-to-end process on how the TMBVFR functionality may work when a customer is using the TMBVFR when doing online shopping.
  • End-To-End Virtual Fit-on process commences when the customer selects a clothing item as per Block 401 from an eCommerce web portal with subscription to TMB VFR and clicks the VFR button as per Block 402 to try a virtual fit-on.
  • the connectivity may be established with TMBVFR via the API Managers and TMB VFR may check if the customer has a registered profde as denoted in the Decision Block 403.
  • the TMBVFR may run the digital human creation process as per subroutine 500 described later.
  • the Digital Human process in Figure 5 starts with the customer (an end user who is shopping for clothing on an online eCommerce portal that has subscribed to TMBVFR) as per Block 501, being prompted to provide a set of attributes to create the Digital Human similar to the customer’s physical body.
  • the attributes taken as input in this process include a set of body measurements (height, neck, chest, waist, sleeve, upper arm girth, bust, hip, thigh, and calf), gender, ethnicity, skin color and hair color, which can be used to create a unique Digital Human for that particular customer only.
  • the entered gender, ethnicity, height and age may be used in conjunction with an existing database of pre-defmed human body types to determine the closest pre-defmed body type to match those parameters.
  • the application is initiated and an average body for that height may appear.
  • the model may allow adjusting the measurements of that model based on the height as well as ethnicity.
  • the skin color and hair color of these pre-defmed bodies can also be changed.
  • the pre-defmed human body details are stored in the hosted NoSQL Database of the TMBVFR backend.
  • an iterative algorithm is used to determine the morph targets for the body attributes of this pre-defmed Digital Human via iterative interpolation of values.
  • the morph target technique is created by utilizing out of the box features available in 3D modelling animation tools which offer the capability to produce and change modelling objects.
  • the Digital Human then may be converted as per Block 504 to make it a pre-defmed Digital Human subject to the morph targets. This may result in a specific Digital Human with identical body dimensions as the customer.
  • a technology platform with a digital image building algorithm of a 3D modelling animation tool (such as what is available in Maya tool) may be used to kick-in and compare the specific body dimensions captured for the customer with each body attribute).
  • a 3D modelling animation tool such as what is available in Maya tool
  • the tool can interpolate between maximum and minimum morph targets to get the respective specific body attribute for that Digital Human, and this may be done for each body attribute.
  • the skin color and hair color may also be applied to the Digital Human (which may be useful in some processes).
  • the Decision Block 507 denotes the option for the customer to save his profile and if the customer responds in affirmative, the customer profile may be saved in the TMBVFR cloud storage as shown in Block 508. If the response under Block 507 is negative from the customer, the attributes and the Digital Human may be retained in a temporary storage for the session and may be purged after the customer session is closed.
  • the created Digital Human is made viewable to the customer as per Block 506 by utilizing a web portal digital object presentation technology platform (such as WebGL) that uses JavaScript API for rendering interactive 2D and 3D graphics within any compatible web browser without using plug-ins as the underpinning delivery technology.
  • a web portal digital object presentation technology platform such as WebGL
  • JavaScript API for rendering interactive 2D and 3D graphics within any compatible web browser without using plug-ins as the underpinning delivery technology.
  • FIG. 11 shows the customer measurements input screen.
  • Figure 12 shows the input data in the input screen (1201) and the corresponding Digital Human (1202) created by TMBVFR are illustrated.
  • the Digital Human process comprises of newly developed capabilities as well as the use of well-established tools and technology platforms in AR/VR space at the moment as well as unique new processes and algorithms developed as part of TMBVFR. As technology evolves, the appropriate decision to use any other advanced or emerging technology domains may be made based on a needs assessment.
  • the capabilities in alternate versions include providing the customer the ability to capture images from a depth camera and using depth camera images to create a Digital Human instead of user manually entering body attributes.
  • the list of captured body attributes can also be revised based on the specific needs pertaining to the measurement used for custom clothes fit-on.
  • the virtual fit-on functionality as per subroutine 600 described later covers the steps from the point an online customer selects a particular clothing item from an eCommerce portal to try out, until the customer sees the results of the virtual fit-on.
  • the customer may be given an option to view the Tension-Map denoted by Decision Block 404 and if the customer chose to view it, TMBVFR may run the tension-map functionality as per subroutine 700 described later and once it is done, the customer satisfaction may be tested as denoted by Block 411.
  • the Tension-Map process (subroutine 700) denoted by Figure 7 covers the generation of the Tension-Map for the customer to take a more informed decision on how the clothing item is likely to feel on the body once it is put on. While the virtual fit-on may pro - vide customer an idea if a selected clothing item is possible to be put on and be used, the Tension-Map may give more insight on which parts of the clothing item may feel tight and loose and good fit and how much in a graphical way so that the customer can make a more informed decision to purchase a clothing item based on the fit.
  • the first step in the process as denoted by Block 701 may be a completed virtual fit- on match between and Digital Human and a 3D Clothing item corresponding to the customer and the selected clothing item respectively.
  • the Decision Block 702 points out an option for the customer to manually specify Tension-Map color coding intervals or use the TMBVFR system default values. If the selection is affirmative, as per Block 703, the customer can define the intervals and color codes which may be applicable for the fitting of the clothing item with the digital human. If the selection is affirmative or when the selection is negative in Block 703, the process moves to the next step.
  • the mesh of triangles and the average tension metrics as well as average triangle edges in the surface mesh for the 3D Clothing Item is selected. (As it may fit-on to a pre-determined Digital Human type - the fit).
  • TMBVFR has average clothing size of the relevant 3D Clothing Item in the backend.
  • the delta of each edge/triangle with re - spect to baselines is calculated under Block 706. If the increment is more than 40%, then the edges/triangles are loosely tight and if it exceeds by 70% the edge/triangle is considered highly tight.
  • These hardcoded values may be replaced by values defined by the customer if the customer has opted to do so under Block 704. These values can determine the thresholds when loosely tight and high tight matching may occur.
  • the Tension-Map color coding is determined by referencing to the tolerance intervals for Tension- Map color coding. Then under Block 708, the color coding is applied to the Digital Human. As shown by Block 709, then the digital human with Tension-Map color coding is ready to be displayed via a web portal digital object presentation technology platform.
  • FIG. 16 Screenshots of the TMBVFR tool screens for Tension-Map is shown in Figure 16. It first illustrates how the tension map may look like for a customer with a waist size 90cm putting on a large sized clothing item(1601), demonstrated via a virtual fit-on image between the corresponding 3D Clothing Image and the Digital Human for the clothing item and customer respectively. Then it also shows how the tension map may change when the same clothing item is put on a customer with a waist size of 109cm, demonstrated by changing the Digital Human to resemble a customer body with that size (1602). The comparison between the two tension maps may show how different customers with different body sizes may feel tension when trying to fit-on a clothing item of same size. [0105] In alternate versions, more functionality may be added to include undergarments and revise the calculations to create a revised Tension-Map, even if undergarments were not con sidered during the first part of the virtual fit-on.
  • the process may en compass the enhanced functional features for improved customer experience.
  • the Static Postures and Dynamic Movement for Virtual Fit-on is an improvement to the Virtual Fit-on process where the algorithms which use the 3D Clothing and Digital Human may generate a virtual fit of the clothing item when the customer is in different postures and dynamic movements. This may be done via changes in Digital Hu man, Clothing Item Creation, Virtual Fit-on, and Tension-Map processes.
  • the Virtual Fit-on with Undergarments and multiple lay ers of clothing is an improvement to the Virtual Fit-on process where the algorithms which use the 3D Clothing and Digital Human can apply the thickness of undergarments or more layers of clothing (defined by the customer or taking average values) and make a more realis tic fit-on of clothing items. This may be done via changes in Digital Human, Clothing Item Creation, Virtual Fit-on, and Tension-Map processes.
  • the Block 407 shows the Virtual Fit-on with different lighting conditions is a new process which follows the basic virtual fit-on and Tension-Map processes that a customer can opt in to go into, before checkout. Under this process, the color properties and translucency of fabrics of clothing items are captured, and the customer can have the opportunity to view how the color and appearance of the clothing may look after fit-on under different lighting condi tions such a sun light, moon light, incandescent light, fluorescent light, etc. This may be done via changes in Clothing Item Creation, Virtual Fit-on and Tension-Map processes.
  • the customer may be asked if satisfied with the virtual fit-on. If the response is affirmative, the process may move on to the next step and if the response is negative, the process as shown under Block 408 may allow the selection of another product to match and then the process can loop back to the start of a new virtual fit-on cycle starting as if was an affirmative response to Block 403 (which may move to subroutine 500 and onwards described above).
  • the saved values for the Digital Human may be re-used for this step also as it can still be a continuous active session.
  • the customer may be asked if the TMBVFR tool can propose a matching product to virtually fit-on. If response is affirmative, as per Block 409, the Proactive Recommendations tools can run an algorithm to determine a product that is likely a good fit for the available Digital Human and then move to Block 408. If the response is negative, the process may directly move to Block 408.
  • the Proactive Recommendations is a new process to enable the TMBVFR to pro-ac tively make recommendations to the customer based on the most recent fit-on and the past history of fit-on.
  • This process may work with the creation of a new module and Database, where the history of registered customers’ fit-On and the likelihood of returns and purchasing are stored and used to predict the likely behavior and once a fit-on is done, TMBVFR can propose customer some similar products to do fit-on next, especially when the most recent fit- on is not a good fit. This may encompass the data on customer fit-on purchase/retums history and an Artificial Intelligence engine.
  • a simplified version of the same process may be used to suggest the most likely clothing item size for a customer who is most recent virtual fit-on was not selected after fit-on. This may be done via changes in Clothing Item Creation, Virtual Fit- on and Tension-Map processes.
  • the process can proceed to checkout.
  • the TMBVFR may relay the details back to the eCommerce web portal and the session may be closed. If it was a customer with a registered profde, then the virtual fit-on details may be saved in the cloud storage (under the Customer Fit-on History Database) and if it was a guest who opted not to register pro - file, the details of the session may be purged.
  • the Digital Human subroutine 500 in Figure 5 starts with the customer (an end user who is shopping for clothing on an online eCommerce portal that has subscribed to TMB- VFR) as per Block 501, being prompted to provide a set of attributes to create the Digital Human similar to the customer’s physical body.
  • the attributes taken as input in this process include a set of body measurements (height, neck, chest, waist, sleeve, upper arm girth, bust, hip, thigh, and calf), gender, ethnicity, skin color and hair color, which can be used to create a unique Digital Human for that particular customer only.
  • the entered gender, ethnicity, height and age may be used in conjunction with an existing database of pre-defmed human body types to determine the closest pre-defmed body type to match those parameters.
  • the application is initiated and an average body for that height may appear.
  • the model may allow adjusting the measurements of that model based on the height as well as ethnicity.
  • the skin color and hair color of these pre-defmed bodies can also be changed.
  • the pre-defmed human body details are stored in the hosted NoSQL Database of the TMBVFR backend.
  • an iterative algorithm is used to determine the morph targets for the body attributes of this pre-defmed Digital Human via iterative interpolation of values.
  • the morph target technique is created by utilizing out of the box features available in 3D modelling animation tools which offer the capability to produce and change modelling objects.
  • the Digital Human then may be converted as per Block 504 to make it a pre-defmed Digital Human subject to the morph targets. This may result in a specific Digital Human with identical body dimensions as the customer.
  • a technology platform with a digital image building algorithm of a 3D modelling animation tool (such as what is available in Maya tool) may be used to kick-in and compare the specific body dimensions captured for the customer with each body attribute).
  • a 3D modelling animation tool such as what is available in Maya tool
  • the tool can interpolate between maximum and minimum morph targets to get the respective specific body attribute for that Digital Human, and this may be done for each body attribute.
  • the skin color and hair color may also be applied to the Digital Human (which may be useful in some processes).
  • the Decision Block 507 denotes the option for the customer to save his profile and if the customer responds in affirmative, the customer profile may be saved in the TMBVFR cloud storage as shown in Block 508. If the response under Block 507 is negative from the customer, the attributes and the Digital Human may be retained in a temporary storage for the session and may be purged after the customer session is closed.
  • the created Digital Human is made viewable to the customer as per Block 506 by utilizing a web portal digital object presentation technology platform (such as WebGL) that uses JavaScript API for rendering interactive 2D and 3D graphics within any compatible web browser without using plug-ins as the underpinning delivery technology.
  • a web portal digital object presentation technology platform such as WebGL
  • JavaScript API for rendering interactive 2D and 3D graphics within any compatible web browser without using plug-ins as the underpinning delivery technology.
  • FIG. 11 shows the customer measurements input screen.
  • Figure 12 shows the input data in the input screen (1201) and the corresponding Digital Human (1202) created by TMBVFR are illustrated.
  • the Digital Human process comprises of newly developed capabilities as well as the use of well-established tools and technology platforms in AR/VR space at the moment as well as unique new processes and algorithms developed as part of TMBVFR. As technology evolves, the appropriate decision to use any other advanced or emerging technology domains may be made based on a needs assessment.
  • the capabilities in alternate versions include providing the customer the ability to capture images from a depth camera and using depth camera images to create a Digital Human instead of user manually entering body attributes.
  • the list of captured body attributes can also be revised based on the specific needs pertaining to the measurement used for custom clothes fit-on.
  • the first step in this Virtual Fit-on subroutine 600 in Figure 6 is to input the 3D Clothing item and the Digital Human to kick off the process as per Block 601.
  • the morph targets for the clothing item is determined by extracting the morph target values for the clothing item via pre-determined the digital human types.
  • the process may proceed to consider the applicability of the undergarments, depending on the customer choice as per Decision Block 603 to do so and applicable undergarment data may be applied as denoted by Block 604 for the clothing item dimensions if the customer choose to do the virtual fit-on with undergarments on and may then move on to morph target comparison as per Block 605. If the customer opted not to use undergarments, the process can directly move on to the next step as per Block 605. Under Block 605, the morph targets may be recalculated via an iterative algorithm to simulate the customer trying on the clothing item with undergarments. If no undergarments selected, then the original morph targets may be used for virtual fit-on.
  • the backend of TMBVFR has predefined undergarments with various morph targets for each body attribute.
  • the backend calculates the morph interpolated values for each body attributes based on the measurements for the Digital Human. Based on these interpolated values, TMBVFR backend can morph (between maximum and minimum morph targets) the digital clothing undergarments based on customer selection of the undergarments to get the respective clothing fit for that specific Digital Human.
  • a second algorithm is used (same as Block 609 described later) to remove collisions and make the fit-on of the undergarment to be a realistic representation of the actual body putting on an undergarment.
  • the morph targets may be compared with the Digital Human as denoted by Decision Block 606 to see if the 3D Clothing Item is within fit-on applicable region (within morph targets) for the Digital Human or if it is an outlier (outside morph targets). If it is an outlier, the message may be displayed for the customer and an option may be given to try out a different size of the clothing item.
  • an algorithm may iteratively match between morph targets for the Digital Human and the 3D Clothing Item to see the fit between the two. Since the TMBVFR has morph targets for each body attribute, when the specific Digital Human details are available, the backend can calculate the morph interpolated value for each body attribute of the Digital Human. Based on those interpolated values, the TMBVFR back- end may then morph the 3D Clothing Item between the maximum and minimum morph targets to get the respective clothing fit applicable for that specific Digital Human.
  • TMBVFR backend a ray is cast from each vertex from the body outwards which is used to identify the first object the ray intersects with. If the first intersecting object is the cloth, there is an overlap between the Digital Human and the 3D Clothing Item has occurred and a different fitting adjustment has to be tried until there is no overlap (no collision).
  • TMBVFR uses Bounding Volume Hierarchies (BVHs) to accelerate the ray intersections.
  • the output of the Virtual Fit-on process can be a display of measurements, 3D clothing item and the an illustration of how the dress once fit-on the customers body would look like via a simulation by the 3D Clothing Item and the Digital Human.
  • Screenshots of the TMBVFR tool screens for Virtual Fit-on is shown in Figure 13, Figure 14 and Figure 15.
  • Figure 13 shows the measurements captured (1301) and the generated Digital Human (1302) for a virtual fit-on.
  • Figure 14 illustration encompasses the selection of a clothing item (1401) for a virtual fit-on and the selected clothing item (1402) which has a 3D Clothing Image in TMBVFR.
  • Figure 14 shows how the selected clothing item may likely look like after the fit- on with the customer’s body demonstrated via the virtual fit-on(1403) between the corresponding Digital Human and 3D Clothing Image of the customer and clothing item respectively.
  • Figure 15 it illustrates how three different sizes of the same clothing item may appear if they were put on the same customer, demonstrated by selecting 3D Clothing Images for three sizes of the clothing item and performing a virtual fit-on with the Digital Human for the customer’s body - for small size (1501), medium size (1502) and large size (1503).
  • the Tension-Map process subroutine 700 in Figure 7 covers the generation of the Tension-Map for the customer to take a more informed decision on how the clothing item is likely to feel on the body once it is put on. While the virtual fit-on may provide customer an idea if a selected clothing item is possible to be put on and be used, the Tension-Map may give more insight on which parts of the clothing item may feel tight and loose and good fit and how much in a graphical way so that the customer can make a more informed decision to purchase a clothing item based on the fit.
  • the first step in the process as denoted by Block 701 may be a completed virtual fit- on match between and Digital Human and a 3D Clothing item corresponding to the customer and the selected clothing item respectively.
  • the Decision Block 702 points out an option for the customer to manually specify Tension-Map color coding intervals or use the TMBVFR system default values. If the selection is affirmative, as per Block 703, the customer can define the intervals and color codes which may be applicable for the fitting of the clothing item with the digital human. If the selection is affirmative or when the selection is negative in Block 703, the process moves to the next step.
  • the mesh of triangles and the average tension metrics as well as average triangle edges in the surface mesh for the 3D Clothing Item is selected. (As it may fit-on to a pre-determined Digital Human type - the fit).
  • TMBVFR has average clothing size of the relevant 3D Clothing Item in the backend.
  • the delta of each edge/triangle with re - spect to baselines is calculated under Block 706. If the increment is more than 40%, then the edges/triangles are loosely tight and if it exceeds by 70% the edge/triangle is considered highly tight.
  • These hardcoded values may be replaced by values defined by the customer if the customer has opted to do so under Block 704. These values can determine the thresholds when loosely tight and high tight matching may occur.
  • the Tension-Map color coding is determined by referencing to the tolerance intervals for Tension- Map color coding. Then under Block 708, the color coding is applied to the Digital Human. As shown by Block 709, then the digital human with Tension-Map color coding is ready to be displayed via a web portal digital object presentation technology platform.
  • FIG. 16 Screenshots of the TMBVFR tool screens for Tension-Map is shown in Figure 16. It first illustrates how the tension map 1615 may look like for the customer 1605 with a waist size 90cm putting on a large sized clothing item 1610, demonstrated via a virtual fit-on image between the corresponding 3D Clothing Image and the Digital Human for the clothing item and customer respectively. Then it also shows the changed tension map 1625 when the same clothing item 1610 is put on a customer 1620 with a waist size of 109cm, demonstrated by changing the Digital Human to resemble a customer body with that size. The comparison between the two tension maps may show how different customers with different body sizes may feel tension when trying to fit-on a clothing item of same size.
  • the first step as per Block 801 is to obtain fabric compressibility and elasticity details for the clothing item.
  • the overlap of changes when the fabric compressibility and elasticity are factored in is established to figure out the new range of possible pre-defined digital humans that the clothing items may be applicable.
  • tension metrics for regular fit and stretch fit are established.
  • the Block 804 denotes the algorithm which can determine the range of digital bodies that the particular clothing item size may be virtually fit- on.
  • the tension metrics may be calculated.
  • the morph targets for regular fit and stretch fit may be calculated for clothing item.
  • the clothing item details may be updated in a temporary file to reflect the impact of compressibility and elasticity data.

Abstract

A system and methods for providing a virtual fitting room to meet customer expectations of a fit, connecting an online clothing retail site with a user-friendly operation for brand owners, retailers, and end customers. The Tension Map Based Virtual Fitting Room (TMBVFR) comprises an integrated set of 7 modules designed to address the online clothing fit-on functionality.Starting from the Access Control module that manages customers, the Web portal Integration module enables the rest of TMBVFR to connect with eCommerce web portals. The Digital Human module, Clothing Simulator module, Virtual Fit-on module, and Tension-Map module together provides the customer with a 3D visualization of the close fit-on of a selected clothes item on the customer's digital human model with the visual representation of distribution of tension when the virtually fit-on when taking specific postures or making specific movements. The Proactive Recommendations module is designed to pro-actively recommend to the customer clothing items based on historical data on fit-on episodes.

Description

TENSION-MAP BASED VIRTUAL FITTING ROOM SYSTEMS AND METHODS
SPECIFICATION
FIELD OF THE INVENTION
[0001] This invention relates in general to methods, apparatuses, and systems for providing a virtual fitting room to meet customer expectations of a fit, connecting an online clothing retail site with a user-friendly operation for brand owners, retailers, and end customers.
BACKGROUND OF THE INVENTION
[0002] Online clothing is one of the fastest growing sectors in the global eCommerce busi - ness with $600 Billion clothing sales online in 2019 and expected to be over $1 Trillion by year 2025. While the industry has consistently shown double-digit annual growth percentage and may continue to grow exponentially, one area of concern is the sales return.
[0003] Approximately 30%-40% of all online clothing sales is returned. Since the online clothing customers check on the sizes of clothes and order, since there is no equivalent to a fit-on room found in a brick and mortar store, the clothing fit-on happen after delivery to the customer and thus, the high rate of returns. While many other factors of return such as color, style exist, the online retailers have figured out way to provide more information, reasons re lated to the lack of clothing product’s good fit (to match the online customer’s body) remains the main contributing factor for online clothing sales return accounting to approximately 80% of returns.
[0004] When an online clothing item returns, it impacts multiple times in the global supply chain of online clothing. Direct costs of the return such as shipping charges, rework/repack age /disposal cost, mark-downs/re-sale cost adds up to equal or higher than the original mar ket price of the item in concern.
[0005] Given the above, the global value of online clothing sales return attributed to fitting issues as of 2019 was estimated at $ 144 Billion and may grow proportionate to or at higher rate than the double-digit annual growth percentage online clothing. CNBC termed the online clothing sales return may become a “Trillion dollar problem” in the next few years.
[0006] While the online sales returns due to fitting may never become zero, it is expected that given the right toolset provided to customers, at least 60% of the returns can be avoided, which has converted into a business opportunity of saving $ 84 Billion based on 2019 figures. This a loss currently shared by clothing brand owners and some online retailers. In addition to this $ 84 Billion, there can also be savings on warehouse retail space and an environmental impact too, by reducing returns, making it a business of reducing wastage in online retailing supply chain.
[0007] During the last decade, a number of tech companies and online retailers have been de veloping several variants on virtual fitting rooms, where a customer can choose a clothing product and virtually fit on the dress to avoid the returns. Though these virtual fitting rooms have helped a few niche sectors as an efficiency gain, there has not been a marked reduction of online clothing returns or a tangible saving for many brand owners and online retailers. SUMMARY OF THE INVENTION
[0008] To overcome limitations in the prior art described above, and to overcome other limi tations that will become apparent upon reading and understanding the present specification, the present invention discloses methods, apparatuses, and systems for Tension-Map based Virtual Fitting Room Systems and Methods.
[0009] In accordance with one embodiment of the invention, the Tension Map Based Virtual Fitting Room (TMBVFR) comprises of an integrated set of 7 modules designed to address the online clothing fit-on functionality.
[0010] The Access Control module allows a customer to connect to TMBVFR to create a personal profile that may connect to a digital human representative of the customer's body di mensions and keep a track of the customer's try-on history. For example, a unique customer name or identifier paired with a secret password or passphrase may provide this access con trol functionality.
[0011] In more particular embodiments, the Web portal Integration module enables the rest of TMBVFR to connect with eCommerce web portals so that the online customers for those sites can access TMBVFR functionality. It further enables the rest of TMBVFR to connect with clothing manufacturers and brand owners so that they may add their clothing products to TMBVFR. A customer who gains access to TMBVFR through the Access Control module may virtually fit-on such clothing products.
[0012] In other more particular embodiments, the Digital Human module enables the cre ation of a specific 3D model of the customer based on the specific body dimensions and the ethnic profile of that customer. A customer who gains access to TMBVFR through the Access Control module may cause the customer's Digital Human to virtually fit-on clothing products made available through the Web portal Integration module.
[0013] In another more particular embodiment, the Clothing Simulator module enables the creation of 3D models of each clothing product made available through the Web portal Inte gration module. The 3D clothing image of each clothing item built using the 3D models may be virtually put on and matched with a customer's Digital Human obtained from the Digital Human module to achieve a virtual fit-on.
[0014] In another embodiment, the Virtual Fit-on module provides for the virtual fitting be tween a customer’s Digital Human from the Digital Human module and a clothing product selected by that customer through the Web portal integration module generating a 3D view of how the clothing item is fit-on the Digital Human.
[0015] In a particular embodiment, the Tension-Map module provides the customer with a 3D visualization of the distribution of tension when the customer's Digital Human from the Digital Human module wears the clothing item made available through the Web portal inte - gration module, indicating which areas of the clothing item may feel tight and loose leading to an enhanced perception of the clothing item's fit-on.
[0016] In some embodiments, the Proactive Recommendations module will pro-actively rec ommend to the customer, who obtained access to TMBVFR through the Access Control mod ule, alternative clothing items based on current and past fit-on history maintained by the TM BVFR.
[0017] These and various other advantages and features of novelty which characterize the in vention will be apparent from the following description, drawings and the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 is a diagram of the modules and components of the Tension Map Based Vir tual Fitting Room (TMBVFR) according to some embodiments;
[0019] Figure 2 is a diagram of the TMBVFR Web Portal Integration process according to some embodiments;
[0020] Figure 3 is a diagram of the TMBVFR Clothing Item Creation process according to some embodiments;
[0021] Figure 4 is a diagram of the TMBVFR End To End Virtual Fit-on process according to some embodiments;
[0022] Figure 5 is a diagram of the TMBVFR Digital Human process according to some em bodiments;
[0023] Figure 6 is a diagram of the TMBVFR Virtual Fit-on process according to some em bodiments;
[0024] Figure 7 is a diagram of the TMBVFR Tension Map process according to some em bodiments;
[0025] Figure 8 is a diagram of the TMBVFR Fabric Elasticity and Compressibility process according to some embodiments;
[0026] Figure 9 is a diagram showing the body measurements for clothing items for a Tech Pack according to some embodiments;
[0027] Figure 10 is a diagram of a Tech Pack according to some embodiments;
[0028] Figure 11 is an illustration showing the customer measurements input screen of the TMBVFR Digital Human module according to some embodiments;
[0029] Figure 12 is an illustration showing the generated Digital Human based on customer measurements input to the TMBVFR according to some embodiments;
[0030] Figure 13 is an illustration showing the input of customer measurements and the out put generated for a Digital Human via TMBVFR for the purpose of performing a virtual fit- on;
[0031] Figure 14 is an illustration showing the input and output of the virtual fit-on process via the TMBVFR Virtual Fit-on module according to some embodiments;
[0032] Figure 15 is an illustration showing how the output changes when a virtual fit-on is performed by TMBVFR with the input of different clothing sizes according to some embodi ments; and
[0033] Figure 16 is an illustration showing the output of the TMBVFR Tension Map module according to some embodiments. DETAIFED DESCRIPTION OF EMBODIMENTS OF THE INVENTION
[0034] Please see concluding remarks, in this Detailed Description of Embodiments of the Invention, which contain defined terms and which describe how to read and interpret this De - tailed Description. Elements labeled with a label number including a trailing letter, as in, “127A” or “127B” represent one or more similar such elements, potentially with variations; singular references herein to one such element or to any object or noun, regardless whether drawn or whether labeled with a trailing letter, shall be understood to refer to one or more, unless the context makes clear otherwise. Where multiple variations are illustrated with a trailing letter, when referred to without the trailing letter, all such variations are referred to.
[0035] The phrases “in one embodiment”, “in various embodiments”, “in some embodiments”, and the like are used repeatedly. Such phrases may not refer to the same embodiment. The terms “comprising”, “having”, and “including” are synonymous, unless the context dictates otherwise.
[0036] Reference is now made in detail to the description of the embodiments as illustrated in the drawings. While embodiments are described in connection with the drawings and related descriptions, there is no intent to limit the scope to the embodiments disclosed herein. On the contrary, the intent is to cover all alternatives, modifications and equivalents. In alternate embodiments, additional devices, or combinations of illustrated devices, may be added to, or combined, without limiting the scope to the embodiments disclosed herein.
[0037] The Tension-Map Based Virtual Fitting Room (TMBVFR) provides a virtual fitting room to meet customer expectations of a fit, connects an online clothing retail site with a user-friendly operation for brand owners, retailers and end customers. The TMBVFR comprises of an integrated set of 7 modules designed to address the online clothing fit-on func - tionality as described below.
[0038] The Access Control module allows the customer who connect TMBVFR to create a profile for them that may connect their digital human and keep a track of their try-on history. It can allow customer faster access to fitting via the existing digital human and may also al - low customers to change their digital human as they wish, based on how their body dimensions change.
[0039] When accessing the TMBVFR, it may prompt the use to sign-in via existing profile name and password, create a new profile or continue to virtually fit-on as a guest.
[0040] A “Virtual Fit-On” is a concept which may enable a customer shopping online to se - lect a clothing item in an online store and validate with a high degree of certainty that the selected clothing item may be a good fit for his/her body at the time of shopping. It can be realized via software tools added to eCommerce web portals and a method to capture the cus - tomer’s body dimensions and match it with the selected clothing item and to show the result for the customer to see.
[0041] The Web portal Integration module enables the rest of TMBVFR to connect with eCommerce web portals so that the online customers for those sites can access TMBVFR functionality. It may also connect clothing manufacturers and brand owners to add their clothing products to TMBVFR so that customers can virtually fit-on those products when they do online shopping.
[0042] The Digital Human module enables the creation of a Digital Human (a specific 3D model of the customer based on the specific body dimensions and the ethnic profile of that customer) of any online customer who wish to virtually fit-on a clothing product via a Web portal subscribed to the TMBVFR. In the MVP for commercial launch, the Digital Human may be created by prompting the user to enter a set of dimensions and parameters. In an alter nate embodiment the TMBVFR may accept measurements used for custom clothes taken by a tailor. In still another embodiment, users may use a depth camera and take pictures and let a tool use depth camera picture data to generate the Digital Human.
[0043] The Clothing Simulator module enables the creation of 3D models of each clothing product that the online customers need to virtually fit-on. The clothing design dimensions are fed into this module as an input from a relevant source via clothing brand owners or manufac turers in 3D design details and it is processed to come up with a 3D clothing image of each clothing items, which is ready to put on and matched with a Digital Human. The generated 3D clothing images are stored in the TMBVFR database in cloud storage and can be re-used as needed for virtual fit-on. The 3D clothing images may be revised as per subsequent changes in the design of the clothing items. In one embodiment the interface may work with the most commonly used clothing design tools in the market to feed in clothing designs to TMBVFR and in alternate versions, TMBVFR may be compatible with every clothing design tool.
[0044] The Virtual Fit-on module does the virtual fitting between a customer’s Digital Hu man and a clothing product selected by that customer. Once the customer Digital Human and clothing item is input, the module may use a 2-step process to generate a 3D view of how the clothing item is fit-on the Digital Human. Firstly, the clothing item may be aligned with the Digital Human in a 3D space, to enable to proceed to fit on. Secondly, via an iteration of pro - cesses may be executed to match surface of the Digital Human and the clothing item.
[0045] Once matching is done, the tool may generate a 3D image of how the dress may look fitted on the Digital Human, displaying any matching and wrinkles areas and the 3D image can be rotated view from various angles. Also, the Digital Human with cloth fitted on can be rotated to see how the fit-on is visible from all angles (a feature that is not even possible with a physical fitting room). In one embodiment, the fit-on with rotating view may be made avail able and in another embodiment, other technology and capabilities encompassing additional features such as the ability to see how the clothing fit-on change when the Digital Human is put on static postures (bending, sitting, reclining, sitting), ability to see how the clothing fit- on change when the Digital Human is put on dynamic body movements (walking, running, jumping), factor in the fabric elasticity and compressibility of the clothing items, factor in the presence of undergarments and multiple layers of clothing, factor in the translucency of the fabrics and generate a 3D view of how the dress fitted on the Digital Human may look like under different lighting conditions (natural sunlight, incandescent light, fluorescent light)
[0046] The Tension-Map module provides the customer with a 3D visualization of the distri bution of tension when the customer wears the clothing item, indicating which areas of the clothing item may feel tight and loose. The TMBVFR tool captures the surface measurements and fabric properties of the clothing item and (input from the virtual fitting functionality) and uses an algorithm to calculate the tension values as it is fitted on to the Digital Human. These tension values are visually represented on the Digital Human color-coded based on tension values and enable the customer to take an informed decision based on how comfortable the clothing items may feel when it is used by the customer. In the MVP, the Tension-Map (a color coded map of the digital human which displays different pre-defined colors applied based on the calculated surface tension values simulating how tight or loose the clothing item feels when it is fit-on that body) with the color coded tension distribution display on Digital Human may be made available and in alternate versions, the Tension-Map may work with the new capabilities for Virtual Fit-on where applicable.
[0047] The Proactive Recommendations module is able to pro-actively recommend the cus tomer alternative clothing items based on current and past (if a registered user) fit-on history. The past fit-on and Tension-Map details are input for this module and based on that and any Tension-Map preferences of the customer and may scan through other items available in the store and may propose listed/rated recommended alternatives to fit-on and purchase.
[0048] The detailed mechanism of how the Tension-Map Based Virtual Fitting Room (TMB- VFR) work to provide an enhanced perception of a virtual fit-on is described below with ref erence to the embodiments as illustrated in the drawings.
[0049] The detailed functionality of TMBVFR may be via a series of interconnected work flows and processes integrating a number of modules of an AR/VR (Augmented Reality /Vir tual Reality) based software application.
[0050] While the workflows and processes themselves are tool agnostic, the articulation refers to the R&D work that has been done so already with a view of expanding on that (op - tional features) for the practical delivery of the functionality or to incorporate newer technol ogy as in alternate versions. Although the descriptions refer to the word “clothing”, this tech nology is applicable to perform virtual fit-ons for an entire range of fashion and clothing re - lated products including Clothing, Clothing Accessories, Shoes, Jewelry, Wearables or the like. Hence, the word “Clothing” may be representative of that entire product range for all in tended purposes with respect to the TMBVFR.
[0051] Figure 1 shows the high level architectural overview of the TMBVFR modules and components. The Web portal Integration module 101 connects the rest of the TMBVFR with eCommerce web portals via the external internet 102, through a secure authentication process. The Access Control module 104 creates the user accounts for users from eCommerce Web portals who wish to register their profdes to use for clothing fit-ons. The Digital Human module 105 may create and manage the 3D body profdes of customers. Both the Access Con trol module and Digital Human modules may be connected with the Customer Database 110. Clothing Simulator module 106 enables the creation of 3D images of all clothing items for customers to perform a virtual fit-on with their Digital Human. Clothing Simulator module is connected with the 3D Clothing Assets Storage 111 and another Database 112 that can store all clothing related parameters as well as clothing material related parameters. The Self-Serve Product Upload module 103 can make it possible for Clothing manufacturers and Clothing brand owners to add any products they produce and/or sell by invoking the functionality of Clothing Simulator module with minimum or no technical support/intervention. The Virtual Fit-on module 107 may use the information from the Customer Database 110 and Clothing parameters Database 112 for customers to fit on clothes and can also be connected to the Vir tual Fit-on History Database 113. The Tension-Map module 108 may be a functionality on top of Virtual Fit-on where the customers may be able to specifically see where a clothing item may feel too tight or too loose when they wear it. Proactive Recommendations module 109 may be able to scan through the history of previous virtual fit-ons done by a registered cus tomer and review the changes of customer’s body profile and recommend alternative clothing items that could be a good fit for a customer. Proactive Recommendations module is con nected with Virtual Fit-on History Database shown 113, Customer Fit-on History Database 116, Customer Database 110 and Clothing parameters Database 112. The purging the virtual fit-on history 115 may happen subject to an affirmative response to a customer choice de noted by Decision Block 114.
[0052] The detailed articulation of the processes in the TMBVFR workflow is as follows:
[0053] The Web portal Integration process 200 in Figure 2 covers how the TMBVFR module is integrated with other Web portals (eCommerce retailers and Brand Owners) so that the cus - tomers of eCommerce web portals can access and use the TMBVFR to fit-on products.
[0054] Unlike most other virtual fitting rooms which are proprietary tools of a few brands or owned by one particular eCommerce web portal, TMBVFR may be a common virtual fitting room that can be used by any eCommerce web portal in the world. Also it may come in with an easy plugin process for any subscribed web portals to connect to TMBVFR and can make all their products are added to the fitting room and any customer shopping at their web portal can have the ability to use the virtual fitting room.
[0055] Hence, the TMBFR may be a product that can offer a new software service for eCom merce web portals which sell Clothing related a products - “Fit-on AS A Service” (FAAS), where the eCommerce web portals can subscribe and obtain the service which can then be used by the online customers free of charge to virtually fit-on clothing products online. The customers who register their profile with TMBVFR may also able to use the same registered customer profile with any eCommerce web portal they do online shopping if it is subscribed to the TMBVFR.
[0056] The process 200 starts with the eCommerce Portal or Brand Owner becoming a sub scriber of the TMBVFR web portal as per Block 201. As an authorized subscriber, the eCom merce Portal or Brand Owner may be onboarded to TMBVFR to have the connectivity.
[0057] The TMBVFR may provide a software plug-in to the subscribed eCommerce web portal as per Block 202. Then as denoted in Block 203, the eCommerce web portal may save the TMBVFR plug-in as per the instructions in it to proceed to the next steps of connectivity.
[0058] Then, as per Block 204, the TMBVFR may register the eCommerce web portal in the TMBVFR Admin portal. Once it is registered as a trusted web portal, TMBFR can then make configuration changes to whitelist the eCommerce web portal and all traffic from it to provide connectivity, as shown in Block 205.
[0059] As per Block 206, the eCommerce web portal may do any relevant changes on their front end to leverage the TMBVFR functionality to its online customers. This may include changes such as adding a button “VFR” that may be displayed along with any item they sell online that can be virtually fit-on. Once the button is pressed the online user may be con nected with TMBVFR to go through the End To End Virtual Fit-on process described later in this document.
[0060] Once the eCommerce web portal is whitelisted, the TMBVFR may enable the traffic connectivity, monitoring and analysis from that site to TMBVFR. This may facilitate the con nectivity for online customers of the eCommerce web portal to use the TMBVFR.
[0061] As the majority of the world’s eCommerce web portals are designed and running on hosting platforms, the TMBVFR may be launched with compatibility with any web portal that uses any web portal hosting platform (such as Magento, Shopiiy or the like). However, with the likelihood that TMBVFR may be used by web portals using new platforms that may come up in future, the TMBVFR may be enhanced continually as so that it can be integrated with any eCommerce platform.
[0062] The Clothing Item Creation process 300 shown in Figure 3 starts with the subscribed eCommerce Web portal or a Clothing Brand Owner proceeding to upload the products they may offer for virtual fit-on by online customers who shop online for that particular clothing item. The availability of the clothing item in TMBVFR as a 3D Clothing Item and eCom merce Web portal having a subscription to TMBVFR are prerequisites for the online customer to be able to a virtual fit-on that particular clothing item at that particular eCommerce web portal.
[0063] The Clothing Item Creation process 300 described under Figure 3 here applies to the uploading one clothing product to TMBVFR and when an eCommerce web portal or a Cloth ing Brand Owner subscribes to TMBVFR, there may be multiple uploads to have all products ready to be virtually fit-on.
[0064] The first step, as per Block 301, is to capture clothing details via user interface provided to a Subscriber Vendor (eCommerce Web portal or Clothing Brand Owner). Once the clothing details are in, a validation of data denoted in Block 302 is done to see if the data is sufficient to create a Tech Pack of the clothing item, which may be the main input to get the 3D Clothing Item created. If the data is insufficient, the process denoted in Block 303 may revert back to the Subscriber Vendor to capture missing data. The clothing design details can be obtained as a sketch, photograph, pattern or the like. The body measurements may be relevant to international dress sizes as per Figure 9. As per 900, the dress design details may be collected via multiple sources and consolidated so that it may be used as an input to generate a Tech Pack.
[0065] Once the data is complete, the Tech Pack for the clothing item is created as per Block 304. The created Tech Pack may adhere to industry standards to make it complete to be applicable for clothing design and applicable clothing item manufacturing process as shown in Figure 10.
[0066] The Tech Pack data is then used to generate the 3D Clothing Item for the clothing product.
[0067] After the Tech Pack is created, any size (such as largest or smallest size) of the clothing item may be selected as denoted in Block 305.
[0068] Then as shown in Block 306, the pre-defined digital human body types are scanned (via an existing database of pre-defined human body types in the TMBVFR backend) and an applicable digital human body type may be located to fit the clothing item.
[0069] As denoted in Block 307, morph targets are generated (A “morph target” is a deformed version of a shape, which can be represented as a snapshot of vortex locations for a specific mesh that have been deformed in some way) of applicable digital humans based on the digital human used to determine the ranges of Digital Human types who could likely fit- on this clothing item.
[0070] Then a 3D Clothing Item is created as in Block 308 by using a digital clothing item creation technology platform (such as Marvelous Designer, Clothes3D or the like).
[0071] The process in Block 309 is then used to generate surface geometry expressed as a mesh of triangles (presentation done via a graphics Application Programming Interface (API) created for use in web applications, based on an open graphics language) which expands to have different surface tensions when morph targets are applied and determine tension metrics ranges applicable on the surface of the clothing item, using the 3D Clothing item and its surface measurements as the basis. This data may enable determination of Tension-Map calculations when the clothing items is fit-on by a Digital Human within the morph targets.
[0072] Once the 3D clothing item is created for one size, all other clothing sizes are selected and then the processes denoted by Blocks 306 through 310 are carried out until all sizes of the clothing item is created as 3D items. Then as per Block 311, TMBVFR Admin Portal may be used to finalize the 3D Clothing Item.
[0073] The fabric compressibility and elasticity data is then applied as per subroutine 800 described later, to the 3D Clothing Item data and the morph targets tension metrics ranges are regenerated after factoring in the fabric compressibility and elasticity. If fabric compressibility and elasticity data is not available for a clothing item, this step can be skipped and the 3D Clothing Item created in previous process may be used.
[0074] The subroutine 800 may be repeated for each size of the clothing item to have fabric compressibility and elasticity details 3D Clothing Item for each size available. [0075] As per Block 312, the 3D Clothing Item and all supporting data (including tension metrics) for all the clothing item sizes are stored at the cloud storage for TMBVFR. If com - pressibility and elasticity were factored in, then the data in the temporary fde mentioned in Block 807 may also be transferred to the cloud storage.
[0076] The Tension-Map color coding information is captured as denoted in Block 313 and added to the data related to the 3D Clothing Item in the TMBVFR cloud storage.
[0077] Once that is done, the clothing item may be ready to be virtually fit-on by online cus tomers who are shopping for clothing on an online eCommerce portal that has subscribed to TMBVFR.
[0078] The Clothing Simulator comprises of newly developed capabilities as well as the use of well-established digital clothing item creation technology platform for 3D clothing item creation. As technology evolves, the TMBVFR may be evolved to be compatible with any clothing design tools used in the garment manufacturing industry. While the 3D Clothing Item creation (and virtual fit-on) can be done without fabric compressibility and elasticity, it may be added in alternate versions as the capability exists as validated in R&D work and since it may provide the functionality and the customer experience.
[0079] The TMBVFR may also have a Self-Serve module, which can automate the process to generate 3D Clothing Items in bulk with minimum of now technical team intervention to add more products to be available via TMBVFR for online shoppers to perform virtual fit- ons.
[0080] Once the TMBVFR has subscribed eCommerce web portals and 3D clothing items uploaded, the online customers of those eCommerce web portals can use the TMBVFR by in voking the End-To-End Virtual Fit-on process in Figure 4.
[0081] The End-To-End Virtual Fit-on process in Figure 4 is a combination of processes which aggregates elements from Digital Human process(subroutine 500), Virtual Fit-on process (subroutine 600) and Tension-Map process(subroutine 700), which can simulate the end-to-end process on how the TMBVFR functionality may work when a customer is using the TMBVFR when doing online shopping.
[0082] The End-To-End Virtual Fit-on process commences when the customer selects a clothing item as per Block 401 from an eCommerce web portal with subscription to TMB VFR and clicks the VFR button as per Block 402 to try a virtual fit-on.
[0083] The connectivity may be established with TMBVFR via the API Managers and TMB VFR may check if the customer has a registered profde as denoted in the Decision Block 403.
[0084] If the customer does not have a registered profile, the TMBVFR may run the digital human creation process as per subroutine 500 described later.
[0085] The Digital Human process in Figure 5 starts with the customer (an end user who is shopping for clothing on an online eCommerce portal that has subscribed to TMBVFR) as per Block 501, being prompted to provide a set of attributes to create the Digital Human similar to the customer’s physical body. The attributes taken as input in this process include a set of body measurements (height, neck, chest, waist, sleeve, upper arm girth, bust, hip, thigh, and calf), gender, ethnicity, skin color and hair color, which can be used to create a unique Digital Human for that particular customer only.
[0086] In the next step under Block 502, the entered gender, ethnicity, height and age may be used in conjunction with an existing database of pre-defmed human body types to determine the closest pre-defmed body type to match those parameters. Once the height is entered, the application is initiated and an average body for that height may appear. The model may allow adjusting the measurements of that model based on the height as well as ethnicity. The skin color and hair color of these pre-defmed bodies can also be changed. The pre-defmed human body details are stored in the hosted NoSQL Database of the TMBVFR backend.
[0087] Then as shown in Block 503, an iterative algorithm is used to determine the morph targets for the body attributes of this pre-defmed Digital Human via iterative interpolation of values. The morph target technique is created by utilizing out of the box features available in 3D modelling animation tools which offer the capability to produce and change modelling objects. The Digital Human then may be converted as per Block 504 to make it a pre-defmed Digital Human subject to the morph targets. This may result in a specific Digital Human with identical body dimensions as the customer.
[0088] Then as per Block 505, a technology platform with a digital image building algorithm of a 3D modelling animation tool (such as what is available in Maya tool) may be used to kick-in and compare the specific body dimensions captured for the customer with each body attribute). As the pre-defmed Digital Human has various morph targets for each body attribute, when a value for a body attribute is entered, the tool can interpolate between maximum and minimum morph targets to get the respective specific body attribute for that Digital Human, and this may be done for each body attribute.
[0089] Then, the skin color and hair color may also be applied to the Digital Human (which may be useful in some processes).
[0090] Once the above is done, the Digital Human is ready for use for Fit-on work as well as to be saved in the profile. The Decision Block 507 denotes the option for the customer to save his profile and if the customer responds in affirmative, the customer profile may be saved in the TMBVFR cloud storage as shown in Block 508. If the response under Block 507 is negative from the customer, the attributes and the Digital Human may be retained in a temporary storage for the session and may be purged after the customer session is closed.
[0091] The created Digital Human is made viewable to the customer as per Block 506 by utilizing a web portal digital object presentation technology platform (such as WebGL) that uses JavaScript API for rendering interactive 2D and 3D graphics within any compatible web browser without using plug-ins as the underpinning delivery technology.
[0092] Screenshots of the TMBVFR tool screens for Digital Human is shown in Figure 11 and Figure 12. Figure 11 shows the customer measurements input screen. In Figure 12, the input data in the input screen (1201) and the corresponding Digital Human (1202) created by TMBVFR are illustrated.
[0093] The Digital Human process comprises of newly developed capabilities as well as the use of well-established tools and technology platforms in AR/VR space at the moment as well as unique new processes and algorithms developed as part of TMBVFR. As technology evolves, the appropriate decision to use any other advanced or emerging technology domains may be made based on a needs assessment.
[0094] On the functional side, the capabilities in alternate versions include providing the customer the ability to capture images from a depth camera and using depth camera images to create a Digital Human instead of user manually entering body attributes. The list of captured body attributes can also be revised based on the specific needs pertaining to the measurement used for custom clothes fit-on.
[0095] If the customer already has a registered profile at the Decision Block 403 or if the customer has gone through subroutine 500, then the virtual fit-on process can be kicked off.
[0096] The virtual fit-on functionality as per subroutine 600 described later, covers the steps from the point an online customer selects a particular clothing item from an eCommerce portal to try out, until the customer sees the results of the virtual fit-on. [0097] Once the virtual fit-on is done, the customer may be given an option to view the Tension-Map denoted by Decision Block 404 and if the customer chose to view it, TMBVFR may run the tension-map functionality as per subroutine 700 described later and once it is done, the customer satisfaction may be tested as denoted by Block 411. If customer choose not to use a Tension-Map, the process can directly go to test customer satisfaction as denoted by Block 411.The Tension-Map process (subroutine 700) denoted by Figure 7 covers the generation of the Tension-Map for the customer to take a more informed decision on how the clothing item is likely to feel on the body once it is put on. While the virtual fit-on may pro - vide customer an idea if a selected clothing item is possible to be put on and be used, the Tension-Map may give more insight on which parts of the clothing item may feel tight and loose and good fit and how much in a graphical way so that the customer can make a more informed decision to purchase a clothing item based on the fit.
[0098] The first step in the process as denoted by Block 701, may be a completed virtual fit- on match between and Digital Human and a 3D Clothing item corresponding to the customer and the selected clothing item respectively.
[0099] As the next step, the Decision Block 702 points out an option for the customer to manually specify Tension-Map color coding intervals or use the TMBVFR system default values. If the selection is affirmative, as per Block 703, the customer can define the intervals and color codes which may be applicable for the fitting of the clothing item with the digital human. If the selection is affirmative or when the selection is negative in Block 703, the process moves to the next step.
[0100] For the next step, shown by Block 704, the mesh of triangles and the average tension metrics as well as average triangle edges in the surface mesh for the 3D Clothing Item is selected. (As it may fit-on to a pre-determined Digital Human type - the fit).
[0101] Then as per Block 705, the triangle edges of the mesh of triangles are revised based on how the 3D Clothing is stretched across the surface of the Digital Human when virtual fit- on is done
[0102] TMBVFR has average clothing size of the relevant 3D Clothing Item in the backend. When a 3D Clothing Item is fit-on a Digital Human, the delta of each edge/triangle with re - spect to baselines is calculated under Block 706. If the increment is more than 40%, then the edges/triangles are loosely tight and if it exceeds by 70% the edge/triangle is considered highly tight. These hardcoded values may be replaced by values defined by the customer if the customer has opted to do so under Block 704. These values can determine the thresholds when loosely tight and high tight matching may occur.
[0103] Once the recalculated tension values are available, as shown in Block 707, the Tension-Map color coding is determined by referencing to the tolerance intervals for Tension- Map color coding. Then under Block 708, the color coding is applied to the Digital Human. As shown by Block 709, then the digital human with Tension-Map color coding is ready to be displayed via a web portal digital object presentation technology platform.
[0104] Screenshots of the TMBVFR tool screens for Tension-Map is shown in Figure 16. It first illustrates how the tension map may look like for a customer with a waist size 90cm putting on a large sized clothing item(1601), demonstrated via a virtual fit-on image between the corresponding 3D Clothing Image and the Digital Human for the clothing item and customer respectively. Then it also shows how the tension map may change when the same clothing item is put on a customer with a waist size of 109cm, demonstrated by changing the Digital Human to resemble a customer body with that size (1602). The comparison between the two tension maps may show how different customers with different body sizes may feel tension when trying to fit-on a clothing item of same size. [0105] In alternate versions, more functionality may be added to include undergarments and revise the calculations to create a revised Tension-Map, even if undergarments were not con sidered during the first part of the virtual fit-on.
[0106] Once the Tension-Map is generated, as per the alternate versions, the process may en compass the enhanced functional features for improved customer experience.
[0107] As shown under Block 405, The Static Postures and Dynamic Movement for Virtual Fit-on is an improvement to the Virtual Fit-on process where the algorithms which use the 3D Clothing and Digital Human may generate a virtual fit of the clothing item when the customer is in different postures and dynamic movements. This may be done via changes in Digital Hu man, Clothing Item Creation, Virtual Fit-on, and Tension-Map processes.
[0108] As denoted under Block 406, the Virtual Fit-on with Undergarments and multiple lay ers of clothing is an improvement to the Virtual Fit-on process where the algorithms which use the 3D Clothing and Digital Human can apply the thickness of undergarments or more layers of clothing (defined by the customer or taking average values) and make a more realis tic fit-on of clothing items. This may be done via changes in Digital Human, Clothing Item Creation, Virtual Fit-on, and Tension-Map processes.
[0109] The Block 407 shows the Virtual Fit-on with different lighting conditions is a new process which follows the basic virtual fit-on and Tension-Map processes that a customer can opt in to go into, before checkout. Under this process, the color properties and translucency of fabrics of clothing items are captured, and the customer can have the opportunity to view how the color and appearance of the clothing may look after fit-on under different lighting condi tions such a sun light, moon light, incandescent light, fluorescent light, etc. This may be done via changes in Clothing Item Creation, Virtual Fit-on and Tension-Map processes.
[0110] Once the virtual fit-on is completed, as per Decision Block 411, the customer may be asked if satisfied with the virtual fit-on. If the response is affirmative, the process may move on to the next step and if the response is negative, the process as shown under Block 408 may allow the selection of another product to match and then the process can loop back to the start of a new virtual fit-on cycle starting as if was an affirmative response to Block 403 (which may move to subroutine 500 and onwards described above). The saved values for the Digital Human may be re-used for this step also as it can still be a continuous active session.
[0111] In alternate versions, as per Decision Block 410, the customer may be asked if the TMBVFR tool can propose a matching product to virtually fit-on. If response is affirmative, as per Block 409, the Proactive Recommendations tools can run an algorithm to determine a product that is likely a good fit for the available Digital Human and then move to Block 408. If the response is negative, the process may directly move to Block 408.
[0112] The Proactive Recommendations is a new process to enable the TMBVFR to pro-ac tively make recommendations to the customer based on the most recent fit-on and the past history of fit-on. This process may work with the creation of a new module and Database, where the history of registered customers’ fit-On and the likelihood of returns and purchasing are stored and used to predict the likely behavior and once a fit-on is done, TMBVFR can propose customer some similar products to do fit-on next, especially when the most recent fit- on is not a good fit. This may encompass the data on customer fit-on purchase/retums history and an Artificial Intelligence engine. A simplified version of the same process may be used to suggest the most likely clothing item size for a customer who is most recent virtual fit-on was not selected after fit-on. This may be done via changes in Clothing Item Creation, Virtual Fit- on and Tension-Map processes.
[0113] Once virtual fit-on done and response at Decision Block 411 is affirmative, as per Block 412, the process can proceed to checkout. At this point, the TMBVFR may relay the details back to the eCommerce web portal and the session may be closed. If it was a customer with a registered profde, then the virtual fit-on details may be saved in the cloud storage (under the Customer Fit-on History Database) and if it was a guest who opted not to register pro - file, the details of the session may be purged.
[0114] The Digital Human subroutine 500 in Figure 5 starts with the customer (an end user who is shopping for clothing on an online eCommerce portal that has subscribed to TMB- VFR) as per Block 501, being prompted to provide a set of attributes to create the Digital Human similar to the customer’s physical body. The attributes taken as input in this process include a set of body measurements (height, neck, chest, waist, sleeve, upper arm girth, bust, hip, thigh, and calf), gender, ethnicity, skin color and hair color, which can be used to create a unique Digital Human for that particular customer only.
[0115] In the next step under Block 502, the entered gender, ethnicity, height and age may be used in conjunction with an existing database of pre-defmed human body types to determine the closest pre-defmed body type to match those parameters. Once the height is entered, the application is initiated and an average body for that height may appear. The model may allow adjusting the measurements of that model based on the height as well as ethnicity. The skin color and hair color of these pre-defmed bodies can also be changed. The pre-defmed human body details are stored in the hosted NoSQL Database of the TMBVFR backend.
[0116] Then as shown in Block 503, an iterative algorithm is used to determine the morph targets for the body attributes of this pre-defmed Digital Human via iterative interpolation of values. The morph target technique is created by utilizing out of the box features available in 3D modelling animation tools which offer the capability to produce and change modelling objects. The Digital Human then may be converted as per Block 504 to make it a pre-defmed Digital Human subject to the morph targets. This may result in a specific Digital Human with identical body dimensions as the customer.
[0117] Then as per Block 505, a technology platform with a digital image building algorithm of a 3D modelling animation tool (such as what is available in Maya tool) may be used to kick-in and compare the specific body dimensions captured for the customer with each body attribute). As the pre-defmed Digital Human has various morph targets for each body attribute, when a value for a body attribute is entered, the tool can interpolate between maximum and minimum morph targets to get the respective specific body attribute for that Digital Human, and this may be done for each body attribute.
[0118] Then, the skin color and hair color may also be applied to the Digital Human (which may be useful in some processes).
[0119] Once the above is done, the Digital Human is ready for use for Fit-on work as well as to be saved in the profile. The Decision Block 507 denotes the option for the customer to save his profile and if the customer responds in affirmative, the customer profile may be saved in the TMBVFR cloud storage as shown in Block 508. If the response under Block 507 is negative from the customer, the attributes and the Digital Human may be retained in a temporary storage for the session and may be purged after the customer session is closed.
[0120] The created Digital Human is made viewable to the customer as per Block 506 by utilizing a web portal digital object presentation technology platform (such as WebGL) that uses JavaScript API for rendering interactive 2D and 3D graphics within any compatible web browser without using plug-ins as the underpinning delivery technology.
[0121] Screenshots of the TMBVFR tool screens for Digital Human is shown in Figure 11 and Figure 12. Figure 11 shows the customer measurements input screen. In Figure 12, the input data in the input screen (1201) and the corresponding Digital Human (1202) created by TMBVFR are illustrated. [0122] The Digital Human process comprises of newly developed capabilities as well as the use of well-established tools and technology platforms in AR/VR space at the moment as well as unique new processes and algorithms developed as part of TMBVFR. As technology evolves, the appropriate decision to use any other advanced or emerging technology domains may be made based on a needs assessment.
[0123] On the functional side, the capabilities in alternate versions include providing the customer the ability to capture images from a depth camera and using depth camera images to create a Digital Human instead of user manually entering body attributes. The list of captured body attributes can also be revised based on the specific needs pertaining to the measurement used for custom clothes fit-on.
[0124] The first step in this Virtual Fit-on subroutine 600 in Figure 6 is to input the 3D Clothing item and the Digital Human to kick off the process as per Block 601.
[0125] Then as per Block 602, the morph targets for the clothing item is determined by extracting the morph target values for the clothing item via pre-determined the digital human types.
[0126] Once morph targets are available, the process may proceed to consider the applicability of the undergarments, depending on the customer choice as per Decision Block 603 to do so and applicable undergarment data may be applied as denoted by Block 604 for the clothing item dimensions if the customer choose to do the virtual fit-on with undergarments on and may then move on to morph target comparison as per Block 605. If the customer opted not to use undergarments, the process can directly move on to the next step as per Block 605. Under Block 605, the morph targets may be recalculated via an iterative algorithm to simulate the customer trying on the clothing item with undergarments. If no undergarments selected, then the original morph targets may be used for virtual fit-on. The backend of TMBVFR has predefined undergarments with various morph targets for each body attribute. The backend calculates the morph interpolated values for each body attributes based on the measurements for the Digital Human. Based on these interpolated values, TMBVFR backend can morph (between maximum and minimum morph targets) the digital clothing undergarments based on customer selection of the undergarments to get the respective clothing fit for that specific Digital Human. As collisions between the Digital Human and the 3D Clothing can occur (i.e. fitting with some parts of the undergarment settles inside Digital Human) a second algorithm is used (same as Block 609 described later) to remove collisions and make the fit-on of the undergarment to be a realistic representation of the actual body putting on an undergarment.
[0127] Once the 3D Clothing Item morph targets are established, the morph targets (with or without undergarments as applicable) may be compared with the Digital Human as denoted by Decision Block 606 to see if the 3D Clothing Item is within fit-on applicable region (within morph targets) for the Digital Human or if it is an outlier (outside morph targets). If it is an outlier, the message may be displayed for the customer and an option may be given to try out a different size of the clothing item.
[0128] If the Digital Human and the 3D Clothing Item are within fit-on applicable region, then the 3D Clothing Item and Digital Human (with or without undergarments based on earlier selections) are placed on a 3D space for alignment to proceed to the virtual fit-on as per Block 607.
[0129] Once aligned, as denoted in Block 608, an algorithm may iteratively match between morph targets for the Digital Human and the 3D Clothing Item to see the fit between the two. Since the TMBVFR has morph targets for each body attribute, when the specific Digital Human details are available, the backend can calculate the morph interpolated value for each body attribute of the Digital Human. Based on those interpolated values, the TMBVFR back- end may then morph the 3D Clothing Item between the maximum and minimum morph targets to get the respective clothing fit applicable for that specific Digital Human.
[0130] As this matching has the tendency to disregard the collisions between the Digital Human and the 3D Clothing (i.e. fitting with some parts of 3D Clothing settling inside Digital Human), a second algorithm shown in Block 609 is used to detect such collisions and go back and re-do the matching as per Block 610 until the virtual fit-on can be finalized with zero collisions. In the TMBVFR backend, a ray is cast from each vertex from the body outwards which is used to identify the first object the ray intersects with. If the first intersecting object is the cloth, there is an overlap between the Digital Human and the 3D Clothing Item has occurred and a different fitting adjustment has to be tried until there is no overlap (no collision). TMBVFR uses Bounding Volume Hierarchies (BVHs) to accelerate the ray intersections.
[0131] The output of the Virtual Fit-on process can be a display of measurements, 3D clothing item and the an illustration of how the dress once fit-on the customers body would look like via a simulation by the 3D Clothing Item and the Digital Human. Screenshots of the TMBVFR tool screens for Virtual Fit-on is shown in Figure 13, Figure 14 and Figure 15. Figure 13 shows the measurements captured (1301) and the generated Digital Human (1302) for a virtual fit-on. Figure 14 illustration encompasses the selection of a clothing item (1401) for a virtual fit-on and the selected clothing item (1402) which has a 3D Clothing Image in TMBVFR. Figure 14 then shows how the selected clothing item may likely look like after the fit- on with the customer’s body demonstrated via the virtual fit-on(1403) between the corresponding Digital Human and 3D Clothing Image of the customer and clothing item respectively. In Figure 15, it illustrates how three different sizes of the same clothing item may appear if they were put on the same customer, demonstrated by selecting 3D Clothing Images for three sizes of the clothing item and performing a virtual fit-on with the Digital Human for the customer’s body - for small size (1501), medium size (1502) and large size (1503).
[0132] The Tension-Map process subroutine 700 in Figure 7 covers the generation of the Tension-Map for the customer to take a more informed decision on how the clothing item is likely to feel on the body once it is put on. While the virtual fit-on may provide customer an idea if a selected clothing item is possible to be put on and be used, the Tension-Map may give more insight on which parts of the clothing item may feel tight and loose and good fit and how much in a graphical way so that the customer can make a more informed decision to purchase a clothing item based on the fit.
[0133] The first step in the process as denoted by Block 701, may be a completed virtual fit- on match between and Digital Human and a 3D Clothing item corresponding to the customer and the selected clothing item respectively.
[0134] As the next step, the Decision Block 702 points out an option for the customer to manually specify Tension-Map color coding intervals or use the TMBVFR system default values. If the selection is affirmative, as per Block 703, the customer can define the intervals and color codes which may be applicable for the fitting of the clothing item with the digital human. If the selection is affirmative or when the selection is negative in Block 703, the process moves to the next step.
[0135] For the next step, shown by Block 704, the mesh of triangles and the average tension metrics as well as average triangle edges in the surface mesh for the 3D Clothing Item is selected. (As it may fit-on to a pre-determined Digital Human type - the fit).
[0136] Then as per Block 705, the triangle edges of the mesh of triangles are revised based on how the 3D Clothing is stretched across the surface of the Digital Human when virtual fit- on is done
[0137] TMBVFR has average clothing size of the relevant 3D Clothing Item in the backend. When a 3D Clothing Item is fit-on a Digital Human, the delta of each edge/triangle with re - spect to baselines is calculated under Block 706. If the increment is more than 40%, then the edges/triangles are loosely tight and if it exceeds by 70% the edge/triangle is considered highly tight. These hardcoded values may be replaced by values defined by the customer if the customer has opted to do so under Block 704. These values can determine the thresholds when loosely tight and high tight matching may occur.
[0138] Once the recalculated tension values are available, as shown in Block 707, the Tension-Map color coding is determined by referencing to the tolerance intervals for Tension- Map color coding. Then under Block 708, the color coding is applied to the Digital Human. As shown by Block 709, then the digital human with Tension-Map color coding is ready to be displayed via a web portal digital object presentation technology platform.
[0139] Screenshots of the TMBVFR tool screens for Tension-Map is shown in Figure 16. It first illustrates how the tension map 1615 may look like for the customer 1605 with a waist size 90cm putting on a large sized clothing item 1610, demonstrated via a virtual fit-on image between the corresponding 3D Clothing Image and the Digital Human for the clothing item and customer respectively. Then it also shows the changed tension map 1625 when the same clothing item 1610 is put on a customer 1620 with a waist size of 109cm, demonstrated by changing the Digital Human to resemble a customer body with that size. The comparison between the two tension maps may show how different customers with different body sizes may feel tension when trying to fit-on a clothing item of same size.
[0140] In alternate versions, more functionality may be added to include undergarments and revise the calculations to create a revised Tension-Map, even if undergarments were not considered during the first part of the virtual fit-on.
[0141] Under the Fabric Elasticity and Compressibility subroutine 800 in Figure 8, the first step as per Block 801 is to obtain fabric compressibility and elasticity details for the clothing item. Under Block 802, the overlap of changes when the fabric compressibility and elasticity are factored in is established to figure out the new range of possible pre-defined digital humans that the clothing items may be applicable. Then as per Block 803, tension metrics for regular fit and stretch fit are established. The Block 804 denotes the algorithm which can determine the range of digital bodies that the particular clothing item size may be virtually fit- on. Once that is done, as per Block 805, the tension metrics may be calculated. Then, as per Block 806, the morph targets for regular fit and stretch fit may be calculated for clothing item. As per Block 807, the clothing item details may be updated in a temporary file to reflect the impact of compressibility and elasticity data.
[0142] The above disclosed description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these em - bodiments will be readily apparent to those skilled in the art, and the generic methodology described herein can be applied to other embodiments without departing from the spirit or scope of the invention. Thus, it is to be understood that the description and drawings presented herein represent presently preferred embodiments of the invention and are therefore representative of the subject matter broadly contemplated by the present invention. It is further understood that the scope of the present invention fully encompasses other embodiments that may become obvious to those skilled in the art and that the scope of the present invention is ac - cordingly limited by nothing other than the appended claims. y y y y y

Claims

TENSION-MAP BASED VIRTUAL FITTING ROOM SYSTEMS AND METHODSCLAIMS What is claimed is:
1. A method comprising: causing, at least in part, a reception of an identification code from a user and a set of clothing item codes from a database; in response to the received identification code, causing, at least in part, association of a 3D virtual model of the user as a digital human with a set of specific body measurements and characteristics and a general ethinic profile of body characteristics and; in response to a user selected clothing item code from the received set of clothing item codes, at least in part, the generation of 3D clothing images; and causing, at least in part the virtual draping of the 3D clothing images of the selected clothing item on the 3D virtual model of the user allowing for a virtual fit-on view of the cloth item to the body viewable from all angles and in a selected set of static postures and in a selected set of dynamic body movements.
2. A method of claim 1, further comprising: the virtual fit-on view obtained through an initial alignment of the clothing images with the virtual model in 3D space followed by an iterative process of matching the surfaces of the digital human and the clothing item.
3. A method of claim 1, further comprising: the generation of a 3D image showing the virtual fit-on of clothing item to digital human fac toring in the fabric elasticity and compressibility of the clothing item, factoring in the pres ence of undergarments and multiple layers of clothing, factoring in the translucency of the fabric under a selected set of lighting conditions.
4. A method of claim 1, further comprising: the generation of a tension-map overlayed on the digital human that visually represents the fitness characteristic of the clothing item on the digital human indicating a range of tight fit ness and loose fitness computed through surface measurements and fabric properties of the clothing item.
5. A method of claim 1, further comprising: the computation of a close fit-on between the clothing item and the digital human obtained through the casting of a set of rays from each vertex in the digital human body outward to de termine collisions with other digital surfaces signifying overlap between clothing item and digital human, which is removed through an iterative process of fitness adjustment.
PCT/CA2021/050950 2020-07-10 2021-07-12 Tension-map based virtual fitting room systems and methods WO2022006683A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202063050685P 2020-07-10 2020-07-10
US63/050,685 2020-07-10

Publications (1)

Publication Number Publication Date
WO2022006683A1 true WO2022006683A1 (en) 2022-01-13

Family

ID=79553380

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2021/050950 WO2022006683A1 (en) 2020-07-10 2021-07-12 Tension-map based virtual fitting room systems and methods

Country Status (1)

Country Link
WO (1) WO2022006683A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023147712A1 (en) * 2022-02-07 2023-08-10 苏州大学 Method for evaluating dynamic fit of garment
WO2024036943A1 (en) * 2022-08-17 2024-02-22 华为云计算技术有限公司 Virtual garment management system and related method

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050154487A1 (en) * 2002-03-22 2005-07-14 Wang Kenneth K. Method and device for viewing, archiving and transmitting a garment model over a computer network
US20110298897A1 (en) * 2010-06-08 2011-12-08 Iva Sareen System and method for 3d virtual try-on of apparel on an avatar
US20140010449A1 (en) * 2012-07-09 2014-01-09 Stylewhile Oy System and method for generating image data for on-line shopping
US20150154691A1 (en) * 2013-12-02 2015-06-04 Scott William Curry System and Method For Online Virtual Fitting Room
US20170156430A1 (en) * 2014-07-02 2017-06-08 Konstantin Aleksandrovich KARAVAEV Method for virtually selecting clothing
US20180092416A1 (en) * 1999-06-11 2018-04-05 Zenimax Media, Inc. Method and system for a computer-rendered three-dimensional mannequin
US20190026954A1 (en) * 2016-01-27 2019-01-24 Nitin Vats Virtually trying cloths on realistic body model of user
US20200126316A1 (en) * 2018-10-19 2020-04-23 Perfitly, Llc. Method for animating clothes fitting
WO2020104990A1 (en) * 2018-11-21 2020-05-28 Vats Nitin Virtually trying cloths & accessories on body model

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180092416A1 (en) * 1999-06-11 2018-04-05 Zenimax Media, Inc. Method and system for a computer-rendered three-dimensional mannequin
US20050154487A1 (en) * 2002-03-22 2005-07-14 Wang Kenneth K. Method and device for viewing, archiving and transmitting a garment model over a computer network
US20110298897A1 (en) * 2010-06-08 2011-12-08 Iva Sareen System and method for 3d virtual try-on of apparel on an avatar
US20140010449A1 (en) * 2012-07-09 2014-01-09 Stylewhile Oy System and method for generating image data for on-line shopping
US20150154691A1 (en) * 2013-12-02 2015-06-04 Scott William Curry System and Method For Online Virtual Fitting Room
US20170156430A1 (en) * 2014-07-02 2017-06-08 Konstantin Aleksandrovich KARAVAEV Method for virtually selecting clothing
US20190026954A1 (en) * 2016-01-27 2019-01-24 Nitin Vats Virtually trying cloths on realistic body model of user
US20200126316A1 (en) * 2018-10-19 2020-04-23 Perfitly, Llc. Method for animating clothes fitting
WO2020104990A1 (en) * 2018-11-21 2020-05-28 Vats Nitin Virtually trying cloths & accessories on body model

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023147712A1 (en) * 2022-02-07 2023-08-10 苏州大学 Method for evaluating dynamic fit of garment
WO2024036943A1 (en) * 2022-08-17 2024-02-22 华为云计算技术有限公司 Virtual garment management system and related method

Similar Documents

Publication Publication Date Title
US11244223B2 (en) Online garment design and collaboration system and method
US10049500B2 (en) Augmented reality e-commerce for home improvement
US10777020B2 (en) Virtual representation creation of user for fit and style of apparel and accessories
KR100511210B1 (en) Method for converting 2d image into pseudo 3d image and user-adapted total coordination method in use artificial intelligence, and service besiness method thereof
US9420253B2 (en) Presenting realistic designs of spaces and objects
US11640672B2 (en) Method and system for wireless ultra-low footprint body scanning
US10497053B2 (en) Augmented reality E-commerce
US20170132842A1 (en) Augmented reality e-commerce for in store retail
KR101707707B1 (en) Method for fiiting virtual items using human body model and system for providing fitting service of virtual items
US9607419B2 (en) Method of fitting virtual item using human body model and system for providing fitting service of virtual item
US8700477B2 (en) Garment fit portrayal system and method
US20190188784A1 (en) System, platform, device and method for personalized shopping
US20150134495A1 (en) Omni-channel simulated digital apparel content display
US20160078663A1 (en) Cloud server body scan data system
KR102130709B1 (en) Method for providing digitial fashion based custom clothing making service using product preview
US20110298897A1 (en) System and method for 3d virtual try-on of apparel on an avatar
WO2022006683A1 (en) Tension-map based virtual fitting room systems and methods
KR102202843B1 (en) System for providing online clothing fitting service using three dimentional avatar
WO2003081536A1 (en) System and method for 3-dimension simulation of glasses
CN110490710B (en) Self-service decoration system based on internet
KR101869927B1 (en) Method of manufacturing design product and server performing the same
CA2735680A1 (en) Online shopping system and method using 3d reconstruction
Grimal et al. Mass customization in apparel industry-implication of consumer as co-creator
US11948057B2 (en) Online garment design and collaboration system and method
WO2018182938A1 (en) Method and system for wireless ultra-low footprint body scanning

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21836985

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21836985

Country of ref document: EP

Kind code of ref document: A1