WO2023226454A1 - 商品信息处理方法、装置、终端设备及存储介质 - Google Patents

商品信息处理方法、装置、终端设备及存储介质 Download PDF

Info

Publication number
WO2023226454A1
WO2023226454A1 PCT/CN2023/071992 CN2023071992W WO2023226454A1 WO 2023226454 A1 WO2023226454 A1 WO 2023226454A1 CN 2023071992 W CN2023071992 W CN 2023071992W WO 2023226454 A1 WO2023226454 A1 WO 2023226454A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
human body
model
product
candidate
Prior art date
Application number
PCT/CN2023/071992
Other languages
English (en)
French (fr)
Inventor
吕承飞
王炳琦
Original Assignee
阿里巴巴（中国）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴（中国）有限公司 filed Critical 阿里巴巴（中国）有限公司
Publication of WO2023226454A1 publication Critical patent/WO2023226454A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0641—Shopping interfaces
    • G06Q30/0643—Graphical representation of items or shoppers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • This application relates to the field of Internet technology, and in particular to a product information processing method, device, terminal equipment and storage medium.
  • the user When purchasing a product, the user opens the e-commerce application and learns various information such as the product's category, pictures or reviews through the product details page, and accordingly understands whether the product is suitable for him or her; when it is determined that the product is suitable for him or her, through pre-registration Use your electronic account to place orders and pay, and finally complete the purchase of goods.
  • Various aspects of this application provide a product information processing method, device, terminal equipment and storage medium to solve the technical problems that the current shopping model is relatively monotonous, not flexible enough, and sometimes cannot meet the special shopping needs of users, and expands the The functional model of e-commerce applications is conducive to bringing new shopping experiences to users.
  • Embodiments of the present application provide a product information processing method, which is suitable for a first user's terminal device, including: displaying a three-dimensional 3D human body model of the second user, where the 3D human body model is three-dimensionally reconstructed based on multiple images of the second user. obtained; select a candidate product and render the candidate product onto the 3D human body model to obtain the fusion effect of the candidate product and the 3D human body model; based on the fusion of the candidate product and the 3D human body model The effect is to select a target product adapted to the second user from the candidate products.
  • Another embodiment of the present application also provides a product information processing method, including: obtaining an implicit three-dimensional 3D representation model of the user.
  • the implicit 3D representation model is a three-dimensional reconstruction based on a neural network based on multiple images of the user. can be constructed; render a 3D human body model of the user based on the implicit 3D representation model, and display the 3D human body model; select candidate products, and render the candidate products onto the 3D human body model to obtain The fusion effect of the candidate product and the 3D human body model.
  • a product information processing device which can be applied to the terminal equipment of the first user, including: a display module for displaying a three-dimensional 3D human body model of the second user, the 3D human body model is based on the second user's terminal device. Multiple images of the user are obtained through three-dimensional reconstruction; a selection module is used to select candidate products; a rendering module is used to render the candidate products onto the 3D human body model to obtain the candidate products and the 3D human body The fusion effect of the model; the selection module is also used to: select a target product adapted to the second user from the candidate products according to the fusion effect of the candidate product and the 3D human body model.
  • Another embodiment of the present application also provides a product information processing device, including: an acquisition module, used to acquire an implicit three-dimensional 3D representation model of the user.
  • the implicit 3D representation model is based on a neural network based on multiple images of the user. obtained by three-dimensional reconstruction; a rendering module for rendering a 3D human body model of the user based on the implicit 3D representation model, and displaying the 3D human body model; a selection module for selecting candidate products; the rendering module It is also used to: render the candidate product onto the 3D human body model to obtain the fusion effect of the candidate product and the 3D human body model.
  • An embodiment of the present application also provides a terminal device, including: a memory and a processor; the memory is used to store a computer program, and the processor is coupled to the memory to execute the computer program to implement the steps in the above method.
  • Embodiments of the present application also provide a computer-readable storage medium storing a computer program.
  • the computer program When executed by a processor, the processor can implement the steps in the above method.
  • the first user is allowed to obtain the 3D human body model of the second user, and render the fusion effect of the candidate product and the second user's 3D human body model, and then based on the fusion effect of the candidate product and the second user's 3D human body model
  • the fusion effect is used to select suitable target products for the second user.
  • the second user selects the product, which is equivalent to seeing the trial effect of the candidate product on the second user in advance through VR, and can accurately select the product for the second user.
  • the first user accurately selects products for the second user based on the second user's 3D human body model, which is a new e-commerce application model that expands the e-commerce application model and improves e-commerce applications.
  • the flexibility of use can meet the special needs of one user to purchase goods for another user, which is conducive to bringing users a new shopping experience.
  • Figure 1a is a schematic flowchart of a product information processing method provided by an exemplary embodiment of the present application
  • Figures 1b-1c are schematic structural diagrams of a product information processing system provided by an exemplary embodiment of the present application.
  • Figure 2a is a schematic flowchart of a product information processing method provided by yet another exemplary embodiment of the present application.
  • Figure 2b is a makeup trial rendering of a 3D human body model provided by another exemplary embodiment of the present application.
  • Figure 3 is a schematic structural diagram of a product information processing device provided by another exemplary embodiment of the present application.
  • Figure 4 is a schematic structural diagram of a product information processing device provided by another exemplary embodiment of the present application.
  • Figure 5 is a schematic structural diagram of a terminal device provided by another exemplary embodiment of the present application.
  • a new e-commerce application model is provided, allowing the first user to use the The e-commerce application installed on the terminal device purchases goods for the second user.
  • the first user is allowed to obtain the 3D human body model of the second user, and the fusion effect of the candidate product and the second user's 3D human body model is rendered, and then based on the fusion effect of the candidate product and the second user's 3D human body model, The second user selects the target product that matches the second user.
  • the second user selects the product, which is equivalent to seeing the trial effect of the candidate product on the second user in advance through VR, and can accurately select the product for the second user.
  • the first user accurately selects products for the second user based on the second user's 3D human body model, which is a new e-commerce application model that expands the e-commerce application model and improves e-commerce applications.
  • the flexibility of use can meet the special needs of one user to purchase goods for another user, which is conducive to bringing users a new shopping experience.
  • Figure 1a is a schematic flowchart of a product information processing method provided by an exemplary embodiment of the present application, which is applicable to the first user terminal device. As shown in Figure 1a, the method includes:
  • the 3D human body model is obtained by three-dimensional reconstruction based on multiple images of the second user;
  • a user who directly purchases goods through a terminal device can be called a first user.
  • the first user's terminal device can be a smart handheld device, such as a smartphone or a tablet, or a desktop device, such as a laptop computer. Or desktop computers, etc., or smart wearable devices, such as smart watches, smart bracelets, etc., or various smart home appliances with display screens, such as smart TVs, smart large screens, or smart robots that can realize network communication and A smart device that can install applications.
  • the application can be an independent APP or a small program that relies on the independent APP to run.
  • the terminal device used by the first user can be called the first terminal device.
  • the first terminal device used by the first user has an e-commerce application installed on it.
  • the first user can use the first terminal to The e-commerce application on the device purchases goods, and the first user can not only purchase goods for himself, but also help the second user purchase goods through the e-commerce application installed on the first terminal device.
  • the second user can be a person with People with whom the first user has an associated relationship, such as family, friends, classmates or colleagues of the first user.
  • the second user can obtain the second product through the first user's terminal device.
  • the user's 3D human body model is based on the second user's 3D human body model, and the target product adapted to the second user is selected from the products provided by the e-commerce application.
  • the 3D human body model is based on multiple images of the second user. Obtained from three-dimensional reconstruction.
  • the 3D reconstruction based on multiple images of the second user may be performed by implicit 3D reconstruction or by explicit 3D reconstruction.
  • the 3D human body model of the second user may be performed based on implicit 3D reconstruction.
  • the obtained implicit 3D representation model is rendered, or it can be rendered based on the human body mesh (mesh) model obtained by explicit 3D reconstruction.
  • this new e-commerce model in which the first user selects products for the second user through the e-commerce application on the first terminal device based on the second user's 3D human body model expands the e-commerce application model and improves the It improves the flexibility of e-commerce applications and can meet the special needs of one user to purchase goods for another user, which is conducive to bringing a new shopping experience to users.
  • the multiple images of the second user refer to images including at least part of the human body structure of the second user, and the multiple images refer to images including the same human body structure taken from different viewing angles.
  • the 3D human body model constructed based on the multiple images may be a partial human body model of the second user, or may be an overall human body model of the second user.
  • multiple images contain the upper body information of the second user, and a 3D model corresponding to the upper body of the second user can be constructed based on these images.
  • multiple images contain head information of the second user, and a 3D model corresponding to the second user's head can be constructed based on these images.
  • multiple images contain the second user's whole body information, and a 3D model of the second user's entire human body can be constructed based on these images.
  • the first terminal device after the first terminal device obtains the three-dimensional 3D human body model of the second user, it can display the second user's three-dimensional human body model through the graphical user interface of the first terminal device, specifically on a page provided by the e-commerce application.
  • 3D human body model and in response to the first user's operation of selecting candidate products for the second user in the e-commerce application, Select the candidate product; and then render the candidate product onto the 3D human body model to obtain the fusion effect of the candidate product and the 3D human body model.
  • the fusion effect of the candidate product and the 3D human body model will be different.
  • the fusion effect of the candidate product and the 3D human body model can be understood as the trial effect of the candidate product on the second user. , which can reflect the matching degree between the candidate product and the second user.
  • a target product adapted to the second user is selected from the candidate products.
  • a candidate product that has the best fusion effect with the 3D human model or meets the requirements can be selected from the candidate products as the target product.
  • the second user selects the product, which is equivalent to seeing the trial effect of the candidate product on the second user in advance through VR, and can accurately select the product for the second user. Able to achieve accurate matching.
  • the 3D human body model may be obtained by implicit three-dimensional reconstruction based on multiple images of the second user.
  • optional ways to display the second user's 3D human body model include: obtaining the second user's implicit 3D representation model.
  • the implicit 3D representation model is obtained by performing three-dimensional reconstruction based on neural networks based on multiple images of the second user. is a three-dimensional implicit representation of the second user's human body; according to the second user's implicit 3D representation model, a 3D human body model of the second user is rendered.
  • the three-dimensional reconstruction method based on neural networks that is, the implicit three-dimensional reconstruction method, can better retain the texture information of the reconstructed object (in this embodiment, the human body structure to be reconstructed by the second user), which is conducive to improving the three-dimensional reconstruction method. Quality of reconstructed human models.
  • the detailed process of performing three-dimensional reconstruction based on the neural network based on multiple images of the second user please refer to subsequent embodiments and will not be described again here.
  • the 3D human body model may be obtained by explicit three-dimensional reconstruction based on multiple images of the second user.
  • optional ways to display the second user's 3D human body model include: obtaining the second user's human body mesh model, and rendering the second user's 3D human body model based on the human body mesh model.
  • the human mesh model is obtained by explicit three-dimensional reconstruction based on multiple images of the second user.
  • the explicit 3D reconstruction method can also be called the traditional 3D reconstruction method.
  • the human mesh model refers to a mesh model that can reflect the surface characteristics of the second user's human body and can explicitly represent the second user in three dimensions.
  • the human mesh model includes the second user's human body surface points and the spatial coordinates of each human body surface point. Color information.
  • the human body surface points can form triangular surfaces and vertices in the human mesh model.
  • the human mesh model specifically includes multiple triangular surfaces and vertices.
  • the attribute information of the vertices includes the spatial coordinates of the vertices, color information, material information, and other texture information.
  • the vertices are human body surface points, and each triangular surface also includes multiple human body surface points.
  • the spatial coordinates and color information of other human body surface points on the triangular surface except the human body surface point as the vertex can be determined by the points on the triangular surface to which they belong.
  • the spatial coordinates and color information of the three vertices are calculated by interpolation.
  • the first terminal device obtains the implicit 3D representation model or human mesh model of the second user, which can be done in the following ways, but is not limited to:
  • Method 1 The first terminal device sends a model acquisition request to other devices to request the second user's hidden data.
  • formula 3D representation model or human body mesh model receiving the implicit 3D representation model or human body mesh model of the second user returned by other devices based on the model acquisition request.
  • Method 2 The first terminal device receives the implicit 3D representation model or human body mesh model of the second user actively sent by other devices. Specifically, when other devices acquire or generate the second user's implicit 3D representation model or human body mesh model, they actively send the second user's implicit 3D representation model or human body mesh model to the first terminal device.
  • Method 3 The first terminal device sends an image acquisition request to other devices to request to acquire multiple images of the second user; the other devices return multiple images of the second user to the first terminal device according to the image acquisition request; the first terminal After acquiring multiple images of the second user, the device performs neural network-based 3D reconstruction or explicit 3D reconstruction based on the multiple images of the second user to obtain an implicit 3D representation model or human mesh model of the second user.
  • Method 4 The first terminal device receives multiple images of the second user actively sent by other devices. Specifically, when the other device obtains or collects multiple images of the second user, it actively sends the multiple images of the second user to the first terminal device. After acquiring multiple images of the second user, the first terminal device performs neural network-based 3D reconstruction or explicit 3D reconstruction based on the multiple images of the second user to obtain the second user's implicit 3D representation model or human body mesh. Model.
  • the other device may be a second terminal device used by the second user.
  • the commodity information processing system corresponding to the e-commerce application scenario may include the first terminal device 11 used by the first user and the third terminal device 11 used by the first user.
  • the second terminal device 12 used by two users, the first terminal device 11 and the second terminal device 12 are communicatively connected.
  • the first terminal device 11 interacts with the second terminal device 12 to obtain the second user's implicit 3D representation model or human mesh model or multiple images from the second terminal device 12 .
  • the product information processing system includes, in addition to the first terminal device 11 and the second terminal device 12, a server device 13.
  • the first terminal device 11 and the second terminal device 12 respectively Communicate with the server device 13.
  • the first terminal device 11 , the second terminal device 12 and the server device 13 interact with each other, so that the first terminal device 11 obtains the second user's implicit 3D representation model or human mesh model from the server device 13 or Multiple images.
  • the second terminal device 12 uploads multiple images of the second user to the server device 13; or, after the second terminal device 12 collects multiple images of the second user, based on the multiple images of the second user Perform neural network-based three-dimensional reconstruction or perform explicit three-dimensional reconstruction to obtain the second user's implicit 3D representation model or human body mesh model, and upload the second user's implicit 3D representation model or human body mesh model to the server device 13 .
  • the second terminal device can collect multiple images of the second user, perform neural network-based three-dimensional reconstruction or explicit three-dimensional reconstruction based on the multiple images of the second user, and obtain the hidden image of the second user.
  • Mode 3D representation model or human body mesh model upload the second user's implicit 3D representation model or human body mesh model to the server device, and maintain the model identification of the implicit 3D representation model or human body mesh model.
  • the second terminal device actively provides the second user's implicit 3D representation model or the model identifier of the human mesh model to the first terminal device, or the second terminal device provides the first terminal device with the model identification based on the model acquisition request sent by the first terminal device.
  • the second user's implicit 3D representation model or the model identification of the human mesh model is gathered from the second user, perform neural network-based three-dimensional reconstruction or explicit three-dimensional reconstruction based on the multiple images of the second user, and obtain the hidden image of the second user.
  • Mode 3D representation model or human body mesh model upload the second user's implicit 3D representation model or human body
  • the first terminal device obtains the second user's implicit 3D representation model or human body mesh model from the server device according to the model identifier.
  • the implementation method of model identification is not limited. Any implementation method of an implicit 3D representation model or human mesh model that can uniquely identify the second user is applicable to the embodiment of this application.
  • the second user's account information or the identification information of the second terminal device can be used as the model identification of the second user's implicit 3D representation model or human mesh model.
  • the second terminal device can collect multiple images of the second user and upload the multiple images of the second user to the server device.
  • the server device maintains the first user and the second user in advance. based on the binding relationship, actively send multiple images of the second user to the first terminal device used by the first user, or alternatively, based on the image acquisition request sent by the first terminal device, Multiple images of the second user are sent to the first terminal device used by the first user.
  • the first terminal device performs neural network-based three-dimensional reconstruction or explicit three-dimensional reconstruction based on multiple images of the second user provided by the server device to obtain an implicit 3D representation model or a human mesh model of the second user.
  • the second user's 3D human body model can be rendered according to the second user's implicit 3D representation model.
  • a specific implementation method of rendering a 3D human body model of the second user based on the second user's implicit 3D representation model is as follows: based on the image features of multiple images of the second user, determine the second user's corresponding The human body space range.
  • the human body space range corresponding to the second user is the shape and volume of the second user's implicit 3D representation model; then based on the human body space range and the implicit 3D representation model, an initial three-dimensional model corresponding to the second user is generated.
  • the initial three-dimensional model is The model includes the second user's human body surface points.
  • the second user's human body surface points may be points that reflect appearance features, such as eyebrows, eyes, noses, mouths, ears and joints of the body.
  • the initial three-dimensional model is then put on the The average value of the viewing angle information of the first line of sight corresponding to each human body surface point is converted into the color information of each surface point to obtain the 3D human body model of the second user.
  • the first line of sight refers to the angle of view captured in each image. Pixel sight.
  • the detailed implementation process of this embodiment is related to the generation process of the implicit 3D representation model. For details, please refer to the description in subsequent embodiments, and will not be described again here.
  • the product can be selected from various products provided by the e-commerce application.
  • a specific implementation method of rendering the candidate product onto the 3D human body model to obtain the fusion effect of the candidate product and the 3D human body model is as follows: first, perform feature estimation on the 3D human body model to obtain multiple human body feature points and their positions.
  • the multiple human body feature points may include but are not limited to: facial features feature points, human skeleton feature points, and hand feature points, which may be determined by the human body structure of the second user corresponding to the 3D human body model.
  • the trial locations on the human body are different. For example, for beauty products, the trial location is usually on the facial features of the human body; for clothing products, the trial location is usually on the upper body of the human body, which is related to the human skeletal characteristics; for wearable products, the trial location is usually on the wrist and other parts of the human body.
  • the trial model corresponding to the candidate product is rendered to the target feature point position to obtain the fusion effect of the candidate product and the 3D human body model.
  • Optional specific implementation methods include at least one of the following operations:
  • the target feature point position is the position corresponding to the facial feature points.
  • the 2D trial model corresponding to the beauty product is rendered to the position corresponding to the facial feature points to obtain the beauty product and 3D Fusion effect of human body model.
  • the positions corresponding to the facial features feature points can be the positions corresponding to the eyebrows, eyes, nose, mouth and chin.
  • the target feature point position is the lip position. Since lipstick only needs to be applied to the surface of the lips, the trial model corresponding to the lipstick is a two-dimensional model. During the fitting, the two-dimensional trial model corresponding to the lipstick is rendered to the third The second user's lip position is used to obtain the fusion effect of the lipstick and the second user's lips.
  • the target feature point position is the position corresponding to the human skeleton feature point.
  • the 3D trial model corresponding to the clothing product is rendered to the position corresponding to the human skeleton feature point to obtain the clothing product and 3D human body model. fusion effect.
  • the positions corresponding to the human skeleton feature points can be the position points corresponding to the shoulder, elbow, elbow, hip, knee and ankle.
  • the target feature points are the shoulder circumference, elbow, and elbow.
  • the three-dimensional trial model corresponding to the coat is rendered to the position corresponding to the above-mentioned human skeleton feature points to obtain the fusion effect of the coat and the 3D human body model.
  • the target feature point position is the position corresponding to the hand feature point
  • the 3D trial model corresponding to the wearable product is rendered to the position corresponding to the hand feature point to obtain the wearable product and 3D human body model.
  • the hand feature points can be the corresponding positions of the wrist and finger joints.
  • the location of the feature points can also be the locations corresponding to the feature points on the neck and ears.
  • the target feature point is the position corresponding to the wrist.
  • the 3D trial model corresponding to the bracelet is rendered to the position corresponding to the wrist to obtain the fusion effect of the bracelet and the 3D human body model.
  • the implementation of selecting candidate products is not limited.
  • the first user can search for the desired product as a candidate product in the e-commerce application according to his or her own preferences or needs. example For example, the first user can enter the keyword "A brand lipstick" in the search box, and then click the "Search" control to issue a search request; the first terminal device responds to the search request issued by the first user and searches out all A brand lipstick products. and displayed on the search results page. The first user can select the lipstick product on the search result page as a candidate product.
  • the first terminal device may generate a product recommendation list based on the first user's historical operation data, and use the products in the product recommendation list as candidate products.
  • the methods provided by the following embodiments can also be used to select candidate products.
  • the 3D human body model of the second user can be displayed on the first interface
  • the first interface can be any page, floating layer or pop-up window provided by the e-commerce application.
  • the first interface may also display other controls to facilitate the first user to select target products for the second user based on the second user's 3D human body model.
  • the first interface can be divided into multiple areas, for example, into a first area, a second area, and a third area. Among them, the positional relationship between the first area, the second area and the third area is not limited.
  • the three areas are respectively distributed on the left, middle, right or upper, middle and bottom of the first interface, or the three areas can also be The disorder is distributed in different positions on the first interface.
  • the 3D human body model of the second user can be displayed in the first area of the first interface; correspondingly, the 3D human body model of the second user can be displayed in the second area of the first interface.
  • At least one product selection control, and further optionally, at least one of a sharing control, a collection control, an additional purchase control and an order control is displayed in the third area of the first interface.
  • the second area, the first area and the third area are distributed on the left, center and right of the first interface as an example for illustration, but it is not limited to this.
  • different product selection controls can correspond to different product types.
  • a makeup try-on selection control a clothes try-on selection control, a glasses try-on selection control and a watch try-on selection control are shown. , but is not limited to this.
  • another implementation of selecting candidate products is as follows: in response to the first user's triggering operation on any product selection control, determine to display at least one product under the product type corresponding to the triggered product selection control; continue to respond Based on the first user's product selection operation, the selected product is determined as a candidate product.
  • at least one product under the product type corresponding to the triggered product selection control may be recommended by the first terminal device based on historical operation data generated by the first user for the product type.
  • the historical operation data here may include: product information of this product type added to the shopping cart by the first user, product information of this product type previously purchased, product information of this product type that has been collected, and multiple browsing The product information of this product type that has been passed.
  • each product selection control displayed in the second area can be a total control for a type of product, and each product selection control can also include at least one product selection sub-control, and can also be based on at least one product selection sub-control. Attribute information of the product corresponding to the control. The adaptability is to add at least one product selection sub-control respectively. Add next-level sub-controls, in which the attribute information can be the brand, color and material of the product, etc.
  • the product selection control and the corresponding product selection sub-control can form a hierarchical relationship, and the hierarchical relationship corresponds to the hierarchical relationship between product categories, that is, the product selection control corresponds to the first-level product category.
  • One layer of product selection sub-controls corresponds to the first-level sub-category until it reaches the leaf category. Accordingly, select a candidate product, and the specific implementation method is as follows: in response to the first user's selection operation on any product selection control displayed in the second area, display the first-level product selection sub-control corresponding to the product selection control; continue In response to the first user's selection operation on any product selection sub-control in each level of product selection sub-control until reaching the product selection sub-control corresponding to the leaf category; in response to the product selection sub-control corresponding to the leaf category The selection operation of any product selection sub-control in the product selection sub-control displays a list of products that can be selected under the leaf category; in response to the first user's selection operation in the product list, the selected product is determined as a candidate product.
  • multiple product selection controls at least include: clothing product selection controls, wearable product selection controls, and beauty product selection controls.
  • the next-level product selection sub-control corresponding to the clothing product selection control at least includes : Jacket product selection sub-control, T-shirt product selection sub-control, Pants product selection sub-control and Skirt product selection sub-control;
  • the next-level product selection sub-control corresponding to the wearing product selection control includes: Hat product selection sub-control Control, glasses product selection sub-control, earrings product selection sub-control, necklace product selection sub-control, bracelet product selection sub-control, ring product selection sub-control and watch product selection sub-control; the next-level product corresponding to the beauty product selection control
  • the selection sub-controls include: eyebrow pencil product selection sub-control, eyeliner product selection sub-control, eye shadow product selection sub-control, foundation product selection sub-control, lipstick product selection sub-control, etc.
  • the eyebrow pencil product selection sub-control, the eyeliner product selection sub-control, and the eye shadow product selection sub-control are displayed.
  • foundation product selection sub-control and lipstick product selection sub-control etc.; continue to respond to the first user's selection operation of the lipstick product selection sub-control, continue to display the next-level selection sub-control, assuming that the next-level selection sub-control corresponds to different Lipstick brand selection sub-control; continue to respond to the first user's selection operation on the lipstick brand sub-control, continue to display the next-level selection sub-control, assuming that the next-level selection sub-control is a selection sub-control corresponding to different lipstick colors; respond to The selection operation of the lipstick color sub-control displays a list of lipstick products corresponding to the selected lipstick color under the selected lipstick brand; in response to the first user's selection operation of a certain lipstick product in the lipstick product list, it is determined that the selected lipstick product is Select lipstick as a candidate product.
  • At least one of a sharing control, a collection control, an additional purchase control, and an order control may be displayed in the third area of the first interface; accordingly, when selecting products suitable for the second user from the candidate products, After matching the target product, you can also perform at least one of the following operations: in response to the first user's triggering operation on the sharing control, send the link information of the target product to the second user's terminal device, so that the second user purchases the target product.
  • the first user In response to the first user's triggering operation on the collection control, add the link information of the target product to the favorites; In response to the first user's triggering operation on the collection control, A user triggers the purchase control to add the target product to the shopping cart; in response to the first user triggering the order control, an order is placed for the target product, where the account corresponding to the order operation can be the first
  • the delivery address corresponding to the order operation can be the delivery address of the second user, or the delivery address corresponding to the order operation can also be the delivery address of the first user. Further, when the delivery address is that of the first user, the first user can transfer the target merchandise to the second user after receiving the merchandise.
  • the 3D human body model of the second user may be rendered based on the implicit 3D representation model of the second user.
  • the process of performing three-dimensional reconstruction based on neural networks based on multiple images of the second user to obtain the implicit 3D representation model of the second user may be executed by the first terminal device, or may be executed by the second terminal device, or it may be The server device executes, and there are no restrictions on this.
  • At least part of the human body structure of the second user in the real world can be photographed from different shooting angles to obtain multiple images containing at least part of the human body structure of the second user (such as the head or upper body or the entire human body).
  • video extract multiple images containing the second user from the video.
  • a 360-degree surrounding shooting method around the second user can be used to obtain multiple images of the second user.
  • different images correspond to different camera poses, and the camera pose includes the position and attitude of the shooting device when capturing the image. Among them, this embodiment does not limit the shooting device.
  • the shooting device may be, for example, but is not limited to: the second user's terminal device, and the terminal device has a camera. After acquiring multiple images, calculate the camera pose corresponding to each image respectively, and determine the multiple first lines of sight emitted by the camera when shooting each image based on the camera pose and camera internal parameters corresponding to each image. Perspective information of each first line of sight. Spatial point sampling is performed on each first line of sight to obtain multiple spatial points. It should be understood that the perspective information of spatial points sampled from the same first line of sight is the perspective information of the first line of sight. After obtaining multiple spatial points, the spatial coordinates of the multiple spatial points and their viewing angle information are used to perform three-dimensional reconstruction based on the neural network.
  • This process can be a process of continuous model training, but is not limited to this.
  • the second user can eventually be obtained Implicit 3D representation model.
  • a human mesh model corresponding to the second user can be constructed based on multiple images.
  • the human mesh model includes the second user's human body surface points and their color information.
  • the following embodiment can also be used to obtain the implicit 3D representation model of the second user.
  • This implementation includes the following steps: first, perform three-dimensional reconstruction based on neural networks based on multiple images of the second user to obtain an initial implicit three-dimensional representation model for implicit 3D expression of the second user, and the human body surface points of the second user versus Corresponds to the pixel in the image and the first line of sight that captured the pixel.
  • the 3D reconstruction process to obtain the initial implicit 3D representation model is a traditional 3D reconstruction process based on neural networks.
  • an explicit three-dimensional model corresponding to the second user is constructed.
  • the explicit three-dimensional model includes the color information of the second user's human body surface points.
  • the color information of each surface point is based on The average viewing angle information of the first line of sight corresponding to the surface point is determined.
  • the second line of sight corresponding to the surface point on the explicit three-dimensional model is randomly generated, and the average viewing angle information corresponding to the second line of sight corresponding to each surface point is generated based on the color information of each surface point.
  • a neural network-based 3D reconstruction is performed based on the initial implicit 3D representation model to obtain the target implicit 3D expression of the target object.
  • the target implicit 3D representation model obtained in this embodiment can be used as the implicit 3D representation model of the second user in the above embodiments.
  • each pixel on an image corresponds to a first line of sight.
  • the pixels in the sample image are imaged by the first line of sight hitting a point on the human body surface of the second user.
  • the first line of sight is the line of sight that captures the pixel. It can be seen from this that there is a corresponding relationship between each human body surface point of the second user, a pixel point, and the first line of sight that captured the pixel point. Different pixels in each image correspond to different human body surface points of the second user, and different human body surface points correspond to different first sight lines.
  • the initial implicit three-dimensional representation model or the target implicit three-dimensional representation model can implicitly express the second user in three dimensions. For example, it can express the second user's human body shape, contour, skin texture, color and other multiple dimensions. Human body information.
  • the initial implicit three-dimensional representation model or the target implicit three-dimensional representation model is a fully connected neural network.
  • the fully connected neural network is also called a multi-layer perceptron (MLP).
  • MLP multi-layer perceptron
  • the initial implicit three-dimensional representation model is a fully connected neural network.
  • the three-dimensional representation model or the target implicit three-dimensional representation model predicts the volume density and color information of the spatial point respectively based on the spatial coordinates and perspective information of the input spatial point.
  • the initial implicit three-dimensional representation model or the target implicit three-dimensional representation model can express for:
  • x (x, y, z)
  • x is recorded as the spatial coordinate (x, y, z) of the spatial point
  • d ( ⁇ , ⁇ )
  • ⁇ is the azimuth angle
  • ⁇ is the elevation angle
  • c (r, g, b)
  • c is recorded as the color information of the space point (r, g, b)
  • r refers to red (Red, R)
  • g refers to green (Green, G)
  • b refers to blue Color (Blue, B).
  • ⁇ is recorded as the volume density of a spatial point.
  • the initial implicit three-dimensional representation model or the target implicit three-dimensional representation model includes the F ⁇ network used to predict ⁇ volume density and the Fc network used to predict c color information. Therefore, the initial implicit three-dimensional representation model or the target implicit three-dimensional representation model can be further expressed as:
  • the input of the F ⁇ network is the spatial coordinate x of the spatial point
  • the output is the volume density of the spatial point and the intermediate feature f.
  • the input of the Fc network is the intermediate feature f and the perspective information d of the spatial point, and the input is the color information RGB value of the spatial point.
  • the volume density is only related to the spatial coordinate x
  • the color information RGB value is related to the spatial coordinate and viewing angle information.
  • the camera pose corresponding to each image is calculated respectively, and the camera pose and camera internal parameters corresponding to each image are used to determine whether the camera is taking each image.
  • Multiple first lines of sight and the viewing angle information of each first line of sight are emitted simultaneously. Sampling is performed on each first line of sight to obtain multiple spatial points.
  • the spatial coordinates of the multiple spatial points and their perspective information are used to perform 3D reconstruction based on the neural network. This process can be performed multiple times in batches, and finally the initial implicit 3D representation model can be obtained.
  • three-dimensional reconstruction based on neural networks can be carried out in a continuous iterative manner. For example, k images can be randomly selected each time, image blocks of size m*n can be randomly selected from the k images, and each of the k image blocks can be used to The spatial coordinates and perspective information of the spatial point on the first line of sight corresponding to the pixel point are subjected to three-dimensional reconstruction (or model training) based on the neural network, until the three-dimensional reconstruction process is terminated when the loss function of the three-dimensional reconstruction process meets the set requirements.
  • k is a natural number greater than or equal to 1, and k is less than or equal to the total number of images; m and n are natural numbers greater than or equal to 1, m and n respectively represent the number of pixels of the image block in the horizontal and vertical dimensions, m is less than Or equal to the width of the original image (the width dimension corresponds to the horizontal direction), n is less than or equal to the length of the image (the length dimension corresponds to the vertical direction), m and n can be the same or different.
  • multiple spatial points can be sampled on each first line of sight at equal intervals, that is, the sampling interval between any two adjacent spatial points is the same. Different sampling intervals can also be used to sample multiple spatial points on each first line of sight, and the size of the sampling interval is not limited.
  • an explicit three-dimensional model corresponding to the second user can be constructed based on the initial implicit three-dimensional representation model and multiple images.
  • the explicit three-dimensional model may refer to a Mesh (grid) model that can reflect the surface characteristics of the second user and can perform an explicit three-dimensional representation of the second user.
  • the explicit three-dimensional model includes the second user's Human body surface points and the spatial coordinates and color information of each human body surface point.
  • the color information of each human body surface point on the explicit three-dimensional model is determined based on the average viewing angle information of the first line of sight corresponding to the human body surface point, indicating the average viewing angle corresponding to any line of sight corresponding to the human body surface point. information.
  • the color information of each human body surface point on the explicit three-dimensional model is not the real color information generated by the second user under light illumination, but the average viewing angle of each first line of sight corresponding to the human body surface point.
  • the information has color information with a mapping relationship.
  • constructing an explicit 3D model corresponding to the second user based on the initial implicit 3D representation model and multiple images includes: determining the spatial range corresponding to the second user based on the image characteristics of the multiple images; An initial three-dimensional model corresponding to the second user is generated based on the spatial range and the initial implicit 3D representation model.
  • the initial three-dimensional model includes the human body surface points on the second user; for any human body surface point, at least one third of the human body surface point corresponding to the human body surface point is generated. a glance The average value of the viewing angle information is converted into the color information of the human body surface points to obtain an explicit three-dimensional model.
  • an algorithm such as Structure from Motion (SfM) can be used to process the image features of multiple images to estimate the sparse 3D point position corresponding to the second user, and the sparse 3D point position corresponding to the second user.
  • SfM Structure from Motion
  • the spatial range may be a spatial range having length, width and height, for example, it may be a cube space or a cuboid space, but is not limited thereto.
  • one implementation method of generating the initial three-dimensional model corresponding to the second user based on the spatial range and the initial implicit three-dimensional representation model is: generating the scalar field corresponding to the second user based on the spatial range and the initial implicit three-dimensional representation model.
  • Data, scalar field data includes multiple volume elements (Volume Pixels), which can be referred to as voxels; perform triangular surface analysis on multiple volume elements to obtain multiple triangular surfaces contained in the initial three-dimensional model, and multiple triangular surfaces on multiple triangular surfaces. Vertices and their spatial coordinates, multiple triangles and multiple vertices are used to define each human body surface point contained in the initial three-dimensional model.
  • the color information of the human body surface point can be determined in the following way: for any human body surface point, at least one first line of sight corresponding to the human body surface point is determined from the first line of sight corresponding to different camera poses. It needs to be explained What is interesting is that the same human body surface point will only have one first line of sight corresponding to the human body surface point in the same camera pose.
  • the same human body surface point will usually be covered by two Or captured by more than two camera poses, that is to say, there are usually two or more first lines of sight from different camera poses corresponding to the same human body surface point, but there will also be special situations, that is, a certain human body
  • the surface point is only captured in one camera pose, that is, there is only one first line of sight corresponding to the human surface point.
  • the average value of the viewing angle information of at least one first line of sight corresponding to the human body surface point is calculated, and the average value is converted into the color information of the human body surface point and saved.
  • a viewing angle pre-stored map corresponding to each image can also be generated.
  • the viewing angle pre-stored map stores the third view angle corresponding to each pixel point in the image.
  • Perspective information for one line of sight It is worth noting that based on the camera pose and camera internal parameters of the captured image, it is not difficult to determine the linear equation information of the first line of sight that emerges from the optical center position when the image was captured and passes through the surface point corresponding to the pixel point of the image. Based on the The linear equation information of a line of sight can quickly obtain the viewing angle information of the first line of sight based on geometric principles.
  • any human body surface point convert the average value of the viewing angle information of at least one first line of sight corresponding to the human body surface point into the color information of the human body surface point to obtain an explicit three-dimensional model, including: for any human body surface point Surface points, based on the camera poses corresponding to multiple images, combined with the initial three-dimensional model, determine at least one target image containing the target pixel points corresponding to the human body surface point from multiple images; the perspective corresponding to at least one target image The average value of the viewing angle information of the first line of sight corresponding to the target pixel point stored in the pre-stored image is converted into the color information of the human body surface point.
  • a virtual line of sight corresponding to each human body surface point on the explicit 3D model that is different from the first line of sight can also be randomly generated.
  • the randomly generated virtual line of sight is called the second line of sight.
  • the second line of sight is the virtual line of sight emitted by the hypothetical virtual camera.
  • target any of the explicit 3D models For body surface points the second line of sight corresponding to the human body surface point can be randomly generated, and the average viewing angle information corresponding to the second line of sight corresponding to the human body surface point can be generated based on the color information of the human body surface point.
  • the first line of sight corresponding to the human body surface point can be used as the reference line of sight
  • the second line of sight corresponding to the human body surface point can be randomly generated within a certain range of the reference line of sight.
  • randomly generating a second line of sight corresponding to the human body surface point based on the first line of sight corresponding to the human body surface point includes: based on the spatial coordinates of the human body surface point and the perspective information of the first line of sight corresponding to the human body surface point, A line of sight that passes through the human body surface point and is different from the first line of sight corresponding to the human body surface point is randomly generated as the second line of sight.
  • a candidate space range is determined based on the spatial coordinates of the human body surface point and the perspective information of the first line of sight corresponding to the target pixel point; in the candidate space range, a randomly generated line passes through the human body surface point and is different from the The line of sight of the first line of sight corresponding to the target pixel is used as the second line of sight.
  • the candidate spatial range can be a spatial range of any shape.
  • the candidate space range is a cone space range with the spatial coordinates of the human body surface points as circles and the first line of sight corresponding to the target pixel point as the center line.
  • the angle range between the second line of sight and the first line of sight passing through the human body surface point may be [- ⁇ , ⁇ ] degrees. Where, eta is, for example, 30 degrees.
  • the second line of sight can be randomly generated for the human body surface points corresponding to each pixel point in multiple images, so that multiple second lines of sight can be randomly generated, and multiple second line of sight can be obtained.
  • the average viewing angle information corresponding to the two sight lines can further be used to further utilize the average viewing angle information corresponding to the multiple second sight lines and the spatial coordinates of the spatial points on the multiple second sight lines to continue to perform 3D reconstruction based on the neural network based on the initial implicit 3D representation model ( or model training) to obtain the target implicit 3D representation model.
  • the average viewing angle information corresponding to each second line of sight and the spatial coordinates of the spatial points on the second line of sight are sequentially used to continue the three-dimensional reconstruction based on the initial implicit 3D representation model.
  • stereoscopic rendering technology is used to use the predicted last batch of The volume density of each spatial point on each second line of sight is integrated by integrating the RGB color information of each spatial point on each second line of sight to obtain the predicted RGB color of the pixel corresponding to each second line of sight in the previous batch.
  • the color information of the pixel calculates the loss function. If the loss function converges, the three-dimensional reconstruction (or model training) process is completed. If the loss function does not converge, the model parameters are adjusted and the average value corresponding to the second line of sight of the next batch is used. The visual angle information and the spatial coordinates of the spatial points on the next batch of second sight lines continue to be iteratively trained until the loss function converges.
  • neural network-based three-dimensional reconstruction and traditional three-dimensional reconstruction are respectively performed based on multiple images of the second user to obtain an initial implicit three-dimensional representation model and an explicit three-dimensional model; based on the explicit three-dimensional model Generate random sight lines and average viewing angles based on the initial implicit 3D representation model.
  • the target implicit 3D representation model is obtained.
  • the initial implicit 3D representation model and the target implicit 3D representation model are both neural network models that implicitly represent the second user in three dimensions.
  • the random line of sight and its corresponding average visual angle information are used to enhance the line of sight data, and continue based on the enhanced line of sight data.
  • the three-dimensional reconstruction of the neural network can obtain an implicit 3D representation model that is highly robust to line of sight.
  • the first user can select an adapted target product for the second user based on the second user's 3D human body model through the e-commerce application of the first terminal device. It should be noted that in addition to this method, each user can also check whether the candidate product is suitable for him based on the fusion effect of the candidate product and his or her own 3D human body model, and then decide whether to purchase the corresponding product. Based on this, embodiments of the present application also provide another product information processing method.
  • Figure 2a is a schematic flowchart of a product information processing method provided by another exemplary embodiment of the present application. As shown in Figure 2a, the method includes:
  • the implicit 3D representation model is obtained by performing three-dimensional reconstruction based on neural networks based on multiple images of the user;
  • the method provided in this embodiment is applicable to any user's terminal device, for example, it may be the first terminal device of the first user in the above embodiment, or it may be the second terminal device of the second user. If it is the first terminal device of the first user, the implicit 3D representation model refers to the implicit 3D representation model of the first user; if it is the second terminal device of the second user, the implicit 3D representation model refers to the second terminal device of the second user. Implicit 3D representation model of the user.
  • the method further includes: selecting a target product whose fusion effect meets the requirements from the candidate products.
  • the method further includes: performing at least one of the following operations on the target product: adding the link information of the target product to favorites; adding the target product to the shopping cart; placing an order for the target product;
  • the link information of the target product is shared with other users to enable other users to purchase the target product.
  • the implementation of sharing the link information of the target product to other users is not limited.
  • the link information of the target product can be shared with other users through in-application messages, or through Taobao passwords. Share the link information of the target product to other users, etc.
  • obtaining the user's implicit 3D representation model includes: obtaining multiple images of the user, and performing three-dimensional reconstruction based on the neural network based on the multiple images of the user to obtain the user's implicit 3D representation model; A 3D representation model is used to render a 3D human body model of the user.
  • each user can render the fusion effect of the candidate product and the user's 3D human body model based on the user's own 3D human body model. Based on this, the user can understand whether the candidate product is suitable for him or her, and then the user can use the candidate product according to the user's 3D human body model.
  • the fusion effect of the 3D human body model can be used to select suitable target products for yourself.
  • users choose products for themselves based on the fusion effect of their own 3D human body model and candidate products, which is equivalent to seeing the trial effect of candidate products on themselves through VR, and can accurately select products for themselves and achieve accurate matching. Bring new shopping experience to users.
  • the user's implicit 3D representation model can also be saved.
  • Product selection or shopping For users, they only need to turn on the camera once and take relevant images for three-dimensional reconstruction of the human body model. This can be used directly during product selection without having to turn on the camera again.
  • the embodiments of this application can solve the problem of privacy leakage caused by opening the camera multiple times, can also solve the problem of large memory and computing resources occupied by real-time image processing, and can also solve the superposition of virtual and reality caused by the complex shooting environment. It can also solve some problems such as the inability to conduct AR trials because there is no camera or the shooting environment or conditions are not available.
  • a three-dimensional reconstruction method based on neural networks can be used to construct a three-dimensional human body model of the user. Compared with the traditional three-dimensional reconstruction method, the reconstruction effect is more effective and reliable, and the applicable scope is wider.
  • An e-commerce APP is installed on the user's terminal device, such as a mobile phone, and the function of purchasing goods based on a 3D human body model is added to the e-commerce APP.
  • users they can open the e-commerce APP, find the new function of purchasing goods based on 3D human models on the relevant pages of the e-commerce APP, and then enable this function.
  • shooting controls can be displayed on the page to guide the user to complete the creation of the 3D human model.
  • the user can click the shooting control, and the terminal device responds to the user's triggering operation of the shooting control on the page, calls the camera of the terminal device, and prompts the user to use the camera to surround the user to collect a video or multiple images containing at least part of the human body appearance characteristics. Images, these images can reflect the appearance characteristics of the same parts of the human body from different perspectives.
  • a generation control can be displayed on the page.
  • three-dimensional reconstruction based on the neural network is performed based on the multiple collected images to generate the user's implicit 3D Representation model, and save the user's implicit 3D representation model locally or upload it to the server device.
  • the implicit 3D representation model may be a NERF model.
  • a 3D human body model that is basically similar to the user's real person can be output, or is called a 3D virtual human.
  • the makeup trial page of the e-commerce APP which is divided into three areas. They are the first area, the second area and the third area respectively.
  • the first area is used to display the user's 3D human body model obtained by inference and neural rendering by the NERF model.
  • the second area displays at least one product selection control.
  • the third area The sharing control, collection control, purchase control and order control are displayed in the area.
  • the user's NERF model can be loaded and the user's 3D human body model obtained by inference and neural rendering based on the NERF model can be displayed in the first area. After that, the user selects candidate products through the product control displayed in the second area according to his or her own needs.
  • a product list under the product category will be displayed.
  • the product list can be displayed in the form of a sub-control.
  • users can select the products they need from the product list.
  • the makeup trial model of the product can be rendered to the user's 3D human body model to display the A picture of the makeup trial effect allows users to decide whether to purchase the product based on their satisfaction with the makeup trial effect. It should be noted that users can also perform operations such as zooming in and viewing details on the 3D human model.
  • Users can decide whether to purchase the product based on their satisfaction with the makeup trial effect. If the user is not satisfied with the makeup trial effect, they can directly ignore the product; if the user is satisfied with the makeup trial effect, but still needs When considering whether to purchase, the user can trigger the add shopping cart or collection control in the third area to add the product to the shopping cart or collection for subsequent viewing; if the user is satisfied with the makeup trial effect and decides to purchase, the third area can be triggered. Area's order control to purchase the product.
  • users in addition to purchasing products for themselves through their own NERF models, users can also authorize their own NERF models to other users, so that other users can render the user's products based on the user's NERF model.
  • 3D virtual person based on the 3D virtual person, purchases goods for the user and achieves social purposes such as sending gifts.
  • the user can also obtain other users' NERF models and purchase products for other users based on other users' NERF models to achieve social purposes such as sending gifts.
  • Users obtain other users' NERF models through their own terminal devices and save them locally. When they need to select products for other users, they can enter the makeup trial page and perform inference and neural rendering based on other users' NERF models.
  • 3D human body model and display the 3D human body models of other users in the first area of the makeup trial page; after that, select candidate products for makeup trial through the product controls displayed in the second area, and determine whether the candidate products meet the requirements based on the makeup trial effect.
  • Other users can then select target products that other users are satisfied with. After selecting the target product, the user can add the target product to the shopping cart or favorites, or share the link of the target product with other users, or directly place an order for help.
  • Other users purchase the target product and change the delivery address to the other user's address so that the product can be mailed directly to other users.
  • the target feature point position of the glasses is the position corresponding to the bridge of the nose and the ears.
  • the three-dimensional trial model of the glasses is rendered to the position corresponding to the bridge of the nose and ears to obtain the fusion effect of the glasses and the 3D human body model.
  • the target feature points of the T-shirt are the shoulders, arms, chest and waist.
  • the three-dimensional trial model of the T-shirt is rendered to the position corresponding to the above-mentioned human bone feature points, and the T-shirt and The fusion effect of the 3D human body model and the rendering effect are shown in Figure 2b.
  • the makeup trial method based on the 3D human body model belongs to the category of VR makeup trial.
  • the real person reconstruction technology based on NERF can obtain a 3D virtual human. In this process, you only need to turn on the camera once and take relevant photos for three-dimensional reconstruction. Can. After completing the 3D virtual human, it can be used for trial use without turning on the camera, which can effectively solve various problems faced by traditional AR product trials.
  • NERF's real-person reconstruction technology by taking multi-view photos of the human body as input, neural rendering is performed through the NERF model to render a realistic 3D human body model, which is more effective and reliable than traditional solutions. It uses the latest neural rendering technology to reconstruct The three-dimensional human body model has a wider scope of application and better universality.
  • Figure 3 is a schematic structural diagram of a product information processing device provided by an exemplary embodiment of the present application, which can be applied to a first user terminal device. As shown in FIG. 3 , the device includes: a display module 31 , a selection module 32 and a rendering module 33 .
  • the display module 31 is used to display the three-dimensional 3D human body model of the second user.
  • the 3D human body model is obtained by three-dimensional reconstruction based on multiple images of the second user.
  • the selection module 32 is used to select candidate products.
  • the rendering module 33 is used to render the candidate product onto the 3D human body model to obtain the fusion effect of the candidate product and the 3D human body model.
  • the selection module 32 is also configured to: select a target product adapted to the second user from the candidate products based on the fusion effect of the candidate product and the 3D human body model.
  • the display module 31 when used to display the three-dimensional 3D human body model of the second user, it is specifically used to: obtain the implicit 3D representation model of the second user.
  • the implicit 3D representation model is based on multiple images of the second user.
  • the three-dimensional reconstruction based on the neural network is a three-dimensional implicit representation of the second user's human body; rendering the second user's 3D human body model based on the second user's implicit 3D representation model; or obtaining the second user's human body model
  • the human body mesh model renders the second user's 3D human body model based on the human body mesh model.
  • the display module 31 when used to obtain the second user's implicit 3D representation model or human body mesh model, the display module 31 is specifically used to: obtain a model identifier used to identify the second user's implicit 3D representation model or human body mesh model; According to the model identifier, the second user's implicit 3D representation model or human body mesh model is obtained from the server.
  • the server maintains the implicit 3D representation model or human body mesh model of each user; or, multiple images of the second user are obtained, Perform neural network-based three-dimensional reconstruction or traditional three-dimensional reconstruction based on multiple images of the second user to obtain an implicit 3D representation model or human mesh model of the second user.
  • the rendering module 33 is configured to render the second user according to the implicit 3D representation model of the second user.
  • the user's 3D human body model is specifically used to: determine the spatial range corresponding to the second user based on the image features of multiple images of the second user; generate an initial 3D model corresponding to the second user based on the spatial range and the implicit 3D representation model.
  • the initial three-dimensional model includes surface points on the second user; the average value of the viewing angle information of the first line of sight corresponding to each surface point on the initial three-dimensional model is converted into the color information of each surface point, respectively, to obtain the second user
  • the first line of sight refers to the line of sight of each pixel captured in each image.
  • the rendering module 33 when used to render the candidate product onto the 3D human body model to obtain the fusion effect of the candidate product and the 3D human body model, it is specifically used to: perform feature estimation on the 3D human body model to obtain multiple human body feature points. and its position information; based on multiple human body feature points and their position information, determine the target feature point position on the 3D human body model that is suitable for the candidate product; render the trial model corresponding to the candidate product to the target feature point position to obtain the candidate The fusion effect of products and 3D human models.
  • the multiple human body feature points include: facial features feature points, human skeleton feature points and hand feature points;
  • the rendering module 33 When the rendering module 33 is used to render the trial model corresponding to the candidate product to the target feature point position to obtain the fusion effect of the candidate product and the 3D human body model, it is specifically used for at least one of the following operations: If the candidate product is a beauty product, For products, the target feature point position is the position corresponding to the facial feature point, and the 2D trial model corresponding to the beauty product is rendered to the position corresponding to the facial feature point to obtain the fusion effect of the beauty product and the 3D human body model; if The candidate product is a clothing product, and the target feature point position is the position corresponding to the human skeleton feature point.
  • the three-dimensional trial model corresponding to the clothing product is rendered to the position corresponding to the human skeleton feature point to obtain the relationship between the clothing product and the 3D human body model. Fusion effect; if the candidate product is a wearable product, the target feature point position is the position corresponding to the hand feature point, and the three-dimensional trial model corresponding to the wearable product is rendered to the position corresponding to the hand feature point to obtain the wearable product and Fusion effect of 3D human body model.
  • the display module 31 when used to display the three-dimensional 3D human body model of the second user, it is specifically used to: display the 3D human body model of the second user in the first area of the first interface, and in the second area of the first interface.
  • At least one product selection control is displayed, and different product selection controls correspond to different product types; accordingly, when the selection module 32 is used to select candidate products, it is specifically used to: in response to the triggering operation of any product selection control, determine the display At least one product under the product type corresponding to the triggered product selection control; in response to the product selection operation, the selected product is determined as a candidate product.
  • a sharing control, a collection control, an additional purchase control and an order control is displayed in the third area of the first interface; after selecting a target product adapted to the second user from the candidate products, the product information
  • the processing device is also used for at least one of the following operations: in response to the triggering operation of the sharing control, sending the link information of the target product to the terminal device of the second user, so that the second user purchases the target product; in response to the triggering of the collection control Operation, add the link information of the target product to your favorites; respond to the trigger operation of the add-purchase control, add the target product to the shopping cart; respond to the trigger operation of the order control, place an order for the target product,
  • the delivery address corresponding to the order operation is the delivery address of the second user.
  • Figure 4 is a schematic structural diagram of another product information processing device provided by an exemplary embodiment of the present application. As shown in Figure 4, the device includes: an acquisition module 41, a rendering module 42 and a selection module 43.
  • the acquisition module 41 is used to obtain the user's implicit three-dimensional 3D representation model.
  • the implicit 3D representation model is obtained by performing three-dimensional reconstruction based on neural networks based on multiple images of the user.
  • the rendering module 42 is used to render the user's 3D human body model based on the implicit 3D representation model, and display the 3D human body model.
  • the selection module 43 is used to select candidate products.
  • the rendering module 42 is also used to render the candidate product onto the 3D human body model to obtain the fusion effect of the candidate product and the 3D human body model.
  • the product information processing device is also configured to select a target product whose fusion effect meets the requirements from the candidate products.
  • the product information processing device is also configured to perform at least one of the following operations on the target product: add the link information of the target product to favorites; add the target product to the shopping cart; place an order for the target product Operation: Share the link information of the target product to other users so that other users can purchase the target product.
  • Figure 5 is a schematic structural diagram of a terminal device provided by an exemplary embodiment of the present application.
  • the terminal device includes: a memory 50a and a processor 50b; the memory 50a is used to store a computer program, and the processor 50b is coupled with the memory 50a to execute the computer program to implement the following steps:
  • Display the second user's three-dimensional human body model which is obtained by three-dimensional reconstruction based on multiple images of the second user; select candidate products; render the candidate products onto the 3D human body model to obtain the candidate products and the 3D human body model
  • the fusion effect according to the fusion effect of the candidate product and the 3D human body model, select the target product that is suitable for the second user from the candidate products.
  • the processor 50b when used to display the three-dimensional 3D human body model of the second user, it is specifically used to: obtain the implicit 3D representation model of the second user.
  • the implicit 3D representation model is based on multiple images of the second user.
  • the 3D reconstruction based on the neural network is a 3D implicit representation of the second user's human body; rendering the second user's 3D human body model based on the second user's implicit 3D representation model; or obtaining the second user's 3D human body model.
  • the human body mesh model renders the second user's 3D human body model based on the human body mesh model.
  • the processor 50b is used to obtain the implicit 3D representation model or human mesh model of the second user.
  • the processor 50b is specifically used to: obtain the model identifier used to identify the second user's implicit 3D representation model or human body mesh model; obtain the second user's implicit 3D representation model or human body mesh model from the server according to the model identifier, and serve
  • the client maintains an implicit 3D representation model or human mesh model of each user; or, obtains multiple images of the second user, and performs neural network-based 3D reconstruction or traditional 3D reconstruction based on the multiple images of the second user to obtain The second user’s implicit 3D representation model or human mesh model.
  • the processor 50b when the processor 50b is used to render the 3D human body model of the second user according to the implicit 3D representation model of the second user, the processor 50b is specifically used to: determine the second user according to the image characteristics of multiple images of the second user.
  • the spatial range corresponding to the user ; generate an initial three-dimensional model corresponding to the second user based on the spatial range and the implicit 3D representation model.
  • the initial three-dimensional model includes the surface points on the second user; convert the first three-dimensional model corresponding to each surface point on the initial three-dimensional model
  • the average value of the visual angle information of the line of sight is converted into the color information of each surface point to obtain the 3D human body model of the second user.
  • the first line of sight refers to the line of sight of each pixel captured in each image.
  • the processor 50b when the processor 50b is used to render the candidate product onto the 3D human body model to obtain the fusion effect of the candidate product and the 3D human body model, the processor 50b is specifically used to: perform feature estimation on the 3D human body model to obtain multiple human body feature points. and its position information; based on multiple human body feature points and their position information, determine the target feature point position on the 3D human body model that is suitable for the candidate product; render the trial model corresponding to the candidate product to the target feature point position to obtain the candidate The fusion effect of products and 3D human models.
  • the plurality of human body feature points include: facial features feature points, human skeleton feature points and hand feature points; the processor 50b is used to render the trial model corresponding to the candidate product to the target feature point position to obtain the candidate product and
  • the fusion effect of the 3D human body model is specifically used for at least one of the following operations: If the candidate product is a beauty product, the target feature point position is the position corresponding to the facial feature point, and the two-dimensional trial model corresponding to the beauty product is rendered.
  • the target feature point position is the position corresponding to the human skeleton feature point, and the corresponding position of the clothing product
  • the three-dimensional trial model is rendered to the position corresponding to the human skeleton feature point to obtain the fusion effect of the clothing product and the 3D human body model
  • the target feature point position is the position corresponding to the hand feature point, and the wearer will The three-dimensional trial model corresponding to the product is rendered to the position corresponding to the hand feature point to obtain the fusion effect of the wearable product and the 3D human body model.
  • the processor 50b when used to display the three-dimensional 3D human body model of the second user, is specifically configured to: display the 3D human body model of the second user in the first area of the first interface, and in the second area of the first interface. At least one product selection control is displayed, and different product selection controls correspond to different product types; accordingly, when the processor 50b is used to select candidate products, it is specifically used to: in response to the triggering operation of any product selection control, determine the display At least one product under the product type corresponding to the triggered product selection control; in response to the product selection operation, the selected product is determined as a candidate product.
  • a sharing control, a collection control, an additional purchase control, and an order control is displayed in the third area of the first interface; after selecting a target product adapted to the second user from the candidate products, the processor 50b is also used for at least one of the following operations: in response to the triggering operation of the sharing control, sending the link information of the target product to the second user's terminal device, so that the second user purchases the target product; in response to the triggering operation of the collection control , add the link information of the target product to your favorites; respond to the triggering operation of the purchase control, add the target product to the shopping cart; respond to the triggering operation of the order control, place an order for the target product, and place the order
  • the shipping address corresponding to the operation is the shipping address of the second user.
  • the terminal device also includes: a communication component 50c, a display 50d, a power supply component 50e, an audio component 50f and other components. Only some components are schematically shown in Figure 5, which does not mean that the terminal device only includes the components shown in Figure 5.
  • terminal equipment provided by this embodiment can implement the technical solution described in the method embodiment of Figure 1a.
  • the specific implementation principles of each of the above modules or units can be found in the corresponding content in the above method embodiments. Here No longer.
  • An exemplary embodiment of the present application provides a computer-readable storage medium storing computer programs or instructions.
  • the processor can implement the steps in the above method, which will not be described again here.
  • embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions
  • the device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.
  • These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device.
  • Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.
  • a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • processors CPUs
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • Memory may include non-permanent storage in computer-readable media, random access memory (RAM) and/or non-volatile memory in the form of read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
  • RAM random access memory
  • ROM read-only memory
  • flash RAM flash random access memory
  • Computer-readable media includes both persistent and non-volatile, removable and non-removable media that can be implemented by any method or technology for storage of information.
  • Information may be computer-readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, tape disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device.
  • computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Finance (AREA)
  • Software Systems (AREA)
  • Accounting & Taxation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Geometry (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

一种商品信息处理方法、装置、终端设备及存储介质。其中方法包括，显示第二用户的三维3D人体模型（S101）；选择候选商品，将候选商品渲染至3D人体模型上，以得到候选商品与3D人体模型的融合效果（S102）；根据候选商品与3D人体模型的融合效果，从候选商品中选择与第二用户适配的目标商品（S103）。

Description

商品信息处理方法、装置、终端设备及存储介质
本申请要求于2022年05月27日提交中国专利局、申请号为202210593701.5、申请名称为“商品信息处理方法、装置、终端设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及互联网技术领域，尤其涉及一种商品信息处理方法、装置、终端设备及存储介质。
背景技术
随着互联网技术的发展，基于互联网的应用越来越多。基于电商应用，用户可以足不出户，购买自己所需的各种商品。在使用电商应用购买商品之前，用户需要预先注册电子账号，该电子账号用于标识用户，并关联用户的收货地址、支付账号等信息。
在购买商品时，用户打开电商应用，通过商品详情页了解商品的类目、图片或评价等各种信息，并据此了解该商品是否适合自己；当确定该商品适合自己时，通过预先注册的电子账号进行下单和支付，最终完成商品的购买操作。
虽然，基于电商应用的在线购物可以极大地方便用户，但是目前的购物模式较为单调，灵活性不够，有时无法满足用户特殊的购物需求，用户的购物体验有待进一步提升。
发明内容
本申请的多个方面提供一种商品信息处理方法、装置、终端设备及存储介质，用以解决目前的购物模式较为单调，灵活性不够，有时无法满足用户特殊的购物需求的技术问题，拓展了电商应用功能模式，有利于给用户带来新的购物体验。
本申请实施例提供一种商品信息处理方法，适用于第一用户的终端设备，包括：显示第二用户的三维3D人体模型，所述3D人体模型是根据第二用户的多张图像进行三维重建得到的；选择候选商品，将所述候选商品渲染至所述3D人体模型上，以得到所述候选商品与所述3D人体模型的融合效果；根据所述候选商品与所述3D人体模型的融合效果，从所述候选商品中选择与所述第二用户适配的目标商品。
本申请另一实施例还提供一种商品信息处理方法，包括：获取用户的隐式三维3D表征模型，所述隐式3D表征模型是根据用户的多张图像进行基于神经网络的三维重 建得到的；基于所述隐式3D表征模型渲染出所述用户的3D人体模型，并显示所述3D人体模型；选择候选商品，将所述候选商品渲染至所述3D人体模型上，以得到所述候选商品与所述3D人体模型的融合效果。
本申请又实施例还提供一种商品信息处理装置，可应用于第一用户的终端设备，包括：显示模块，用于显示第二用户的三维3D人体模型，所述3D人体模型是根据第二用户的多张图像进行三维重建得到的；选择模块，用于选择候选商品；渲染模块，用于将所述候选商品渲染至所述3D人体模型上，以得到所述候选商品与所述3D人体模型的融合效果；所述选择模块还用于：根据所述候选商品与所述3D人体模型的融合效果，从所述候选商品中选择与所述第二用户适配的目标商品。
本申请又一实施例还提供一种商品信息处理装置，包括：获取模块，用于获取用户的隐式三维3D表征模型，所述隐式3D表征模型是根据用户的多张图像进行基于神经网络的三维重建得到的；渲染模块，用于基于所述隐式3D表征模型渲染出所述用户的3D人体模型，并显示所述3D人体模型；选择模块，用于选择候选商品；所述渲染模块还用于：将所述候选商品渲染至所述3D人体模型上，以得到所述候选商品与所述3D人体模型的融合效果。
本申请实施例还提供一种终端设备，包括：存储器和处理器；存储器，用于存储计算机程序，处理器与存储器耦合，用于执行计算机程序，以用于实现以上所述方法中的步骤。
本申请实施例还提供一种存储有计算机程序的计算机可读存储介质，当计算机程序被处理器执行时，致使处理器能够实现以上所述方法中的步骤。
在本申请实施例中，允许第一用户获取第二用户的3D人体模型，并渲染出候选商品与第二用户的3D人体模型的融合效果，进而根据候选商品与第二用户的3D人体模型的融合效果，为第二用户选择与之适配的目标商品。其中，基于第二用户的3D人体模型与候选商品的融合效果为第二用户选择商品，相当于通过VR预先看到了候选商品在第二用户上的试用效果，可以准确地为第二用户选择商品，能够做到精准匹配；另外，第一用户基于第二用户的3D人体模型准确地为第二用户选择商品，是一种新的电商应用模式，拓展了电商应用模式，提高电商应用的使用灵活性，可以满足一个用户为另一个用户选购商品的特殊需求，有利于给用户带来新的购物体验。
附图说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1a为本申请一示例性实施例提供的商品信息处理方法的流程示意图；
图1b-1c为本申请一示例性实施例提供的商品信息处理系统的结构示意图；
图2a为本申请又一示例性实施例提供的商品信息处理方法的流程示意图；
图2b为本申请又一示例性实施例提供的3D人体模型的试妆效果图；
图3为本申请又一示例性实施例提供的商品信息处理装置的结构示意图；
图4为本申请又一示例性实施例提供的商品信息处理装置的结构示意图；
图5为本申请又一示例性实施例提供的终端设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请具体实施例及相应的附图对本申请技术方案进行清楚、完整地描述。显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
针对现有电商购物模式较为单调，灵活性不够，无法满足用户特殊的购物需求等问题，在本申请实施例中，提供了一种新的电商应用模式，允许第一用户通过自己使用的终端设备上的安装的电商应用为第二用户选购商品。具体地，允许第一用户获取第二用户的3D人体模型，并渲染出候选商品与第二用户的3D人体模型的融合效果，进而根据候选商品与第二用户的3D人体模型的融合效果，为第二用户选择与之适配的目标商品。
其中，基于第二用户的3D人体模型与候选商品的融合效果为第二用户选择商品，相当于通过VR预先看到了候选商品在第二用户上的试用效果，可以准确地为第二用户选择商品，能够做到精准匹配；另外，第一用户基于第二用户的3D人体模型准确地为第二用户选择商品，是一种新的电商应用模式，拓展了电商应用模式，提高电商应用的使用灵活性，可以满足一个用户为另一个用户选购商品的特殊需求，有利于给用户带来新的购物体验。
以下结合附图，详细说明本申请各实施例提供的技术方案。
图1a为本申请一示例性实施例提供的商品信息处理方法的流程示意图，适用于第一用户终端设备。如图1a所示，该方法包括：
101、显示第二用户的三维3D人体模型，3D人体模型是根据第二用户的多张图像进行三维重建得到的；
102、选择候选商品，将候选商品渲染至3D人体模型上，以得到候选商品与3D人体模型的融合效果；
103、根据候选商品与3D人体模型的融合效果，从候选商品中选择与第二用户适配的目标商品。
在本实施例中，可以将直接通过终端设备选购商品的用户称为第一用户，第一用户的终端设备可以是智能手持设备，例如智能手机、平板电脑，可以是台式设备，例如笔记本电脑或台式电脑等，也可以是智能穿戴设备，例如智能手表、智能手环等，还可以是各种带有显示屏幕的智能家电，例如智能电视、智能大屏或智能机器人等可以实现网络通信且能安装应用程序的智能设备。应用程序可以是独立的APP，也可以是依赖于独立APP运行的小程序。
在本实施例中，为了便于区分，可以将第一用户使用的终端设备称为第一终端设备，第一用户使用的第一终端设备上安装有电商应用，第一用户可以通过第一终端设备上的电商应用选购商品，并且第一用户除了可以给自己选购商品外，也可以通过第一终端设备上安装的电商应用帮助第二用户选购商品，第二用户可以是与第一用户具有关联关系的人群，例如第一用户的家人、朋友、同学或同事等。在第一用户通过第一终端设备上的电商应用为第二用户选购商品的过程中，为了保证能够选择到与第二用户精准匹配的商品，可以通过第一用户的终端设备获取第二用户的3D人体模型，基于第二用户的3D人体模型，从电商应用提供的商品中选购与第二用户适配的目标商品，其中，3D人体模型是根据第二用户的多张图像进行三维重建得到的。可选地，根据第二用户的多张图像进行三维重建可以是隐式三维重建的方式，也可以是显示三维重建的方式，相应地，第二用户的3D人体模型可以是根据隐式三维重建得到的隐式3D表征模型渲染得到的，也可以是根据显式三维重建得到的人体网格(mesh)模型渲染得到的。本实施例中，这种由第一用户根据第二用户的3D人体模型通过第一终端设备上的电商应用为第二用户选择商品的新的电商模式，拓展了电商应用模式，提高了电商应用的使用灵活性，可以满足一个用户为另一个用户选购商品的特殊需求，有利于给用户带来新的购物体验。
在本实施例中，第二用户的多张图像是指至少包含第二用户的部分人体结构在内的图像，且多张图像是指从不同视角拍摄的包含相同人体结构的图像。其中，根据多张图像中包含的人体结构的不同，基于多张图像构建出的3D人体模型可以是第二用户的局部人体模型，也可以是第二用户的整体人体模型。在一可选实施例中，多张图像中包含第二用户的上半身信息，则基于这些图像可以构建出第二用户上半身对应的3D模型。在另一可选实施例中，多张图像中包含第二用户的头部信息，则基于这些图像可以构建出第二用户头部对应的3D模型。在又一可选实施例中，多张图像中包含第二用户的全身信息，则基于这些图像可以构建出第二用户的整个人体3D模型。
在本实施例中，第一终端设备获取到第二用户的三维3D人体模型后，可以通过第一终端设备的图形用户界面，具体是指在电商应用提供的页面上显示第二用户的三维3D人体模型，以及响应于第一用户在电商应用中为第二用户选择候选商品的操作， 选择出候选商品；进而将候选商品渲染至3D人体模型上，以得到候选商品与3D人体模型的融合效果。根据候选商品的不同，候选商品与3D人体模型的融合效果会有所不同，无论是哪种候选商品，候选商品与3D人体模型的融合效果可以理解为是候选商品在第二用户上的试用效果，能够反映候选商品与第二用户的匹配度。之后，根据候选商品与3D人体模型的融合效果，从候选商品中选择与第二用户适配的目标商品。可选地，可以从候选商品中，选择与3D人体模型的融合效果最优或符合要求的候选商品作为目标商品。其中，基于第二用户的3D人体模型与候选商品的融合效果为第二用户选择商品，相当于通过VR预先看到了候选商品在第二用户上的试用效果，可以准确地为第二用户选择商品能够做到精准匹配。
在一可选实施例中，3D人体模型可以是根据第二用户的多张图像进行隐式三维重建得到的。基于此，显示第二用户的3D人体模型的可选方式包括：获取第二用户的隐式3D表征模型，隐式3D表征模型是根据第二用户的多张图像进行基于神经网络的三维重建得到的，是对第二用户的人体的三维隐式表征；根据第二用户的隐式3D表征模型，渲染出第二用户的3D人体模型。其中，基于神经网络的三维重建方式，即隐式三维重建方式，可以更好地保留被重建对象(在本实施例中是指第二用户待重建的人体结构)的纹理信息，有利于提高三维重建的人体模型的质量。关于根据第二用户的多张图像进行基于神经网络的三维重建得到的详细过程可参见后续实施例，在此暂不赘述。
在另一可选实施例中，3D人体模型可以是根据第二用户的多张图像进行显式三维重建得到的。基于此，显示第二用户的3D人体模型的可选方式包括：获取第二用户的人体mesh模型，根据人体mesh模型渲染出第二用户的3D人体模型。其中，人体mesh模型是根据第二用户的多张图像进行显式三维重建得到的。显式三维重建方式也可以称为传统的三维重建方式。人体mesh模型是指能够反映第二用户的人体表面特征且能够对第二用户进行显式三维表示的mesh模型，人体mesh模型包括第二用户的人体表面点及每个人体表面点的空间坐标和颜色信息。这些人体表面点可形成人体mesh模型中的三角面和顶点，人体mesh模型具体包括多个三角面和顶点，顶点的属性信息包括顶点的空间坐标、颜色信息、材质信息以及其它纹理信息等。顶点是人体表面点，每个三角面也包括多个人体表面点，其中，三角面上除作为顶点的人体表面点之外的其它人体表面点的空间坐标和颜色信息可由其所属三角面上的三个顶点的空间坐标和颜色信息进行插值计算得到。
在本实施例中，第一终端设备获取第二用户的隐式3D表征模型或人体mesh模型，可以采用但不限于以下几种方式：
方式一：第一终端设备向其他设备发送模型获取请求，以请求获取第二用户的隐 式3D表征模型或人体mesh模型；接收其它设备根据该模型获取请求返回的第二用户的隐式3D表征模型或人体mesh模型。
方式二：第一终端设备接收其他设备主动发送的第二用户的隐式3D表征模型或人体mesh模型。具体地，其他设备在获取或生成第二用户的隐式3D表征模型或人体mesh模型的情况下，主动向第一终端设备发送第二用户的隐式3D表征模型或人体mesh模型。
方式三：第一终端设备向其它设备发送图像获取请求，以请求获取第二用户的多张图像；其它设备根据该图像获取请求向第一终端设备返回第二用户的多张图像；第一终端设备在获取第二用户的多张图像之后，基于第二用户的多张图像进行基于神经网络的三维重建或进行显式三维重建，得到第二用户的隐式3D表征模型或人体mesh模型。
方式四：第一终端设备接收其他设备主动发送的第二用户的多张图像。具体地，其他设备在获取或采集到第二用户的多张图像的情况下，主动向第一终端设备发送第二用户的多张图像。第一终端设备在获取第二用户的多张图像之后，基于第二用户的多张图像进行基于神经网络的三维重建或进行显式三维重建，得到第二用户的隐式3D表征模型或人体mesh模型。
上述实施例中，其他设备可以是第二用户使用的第二终端设备，如图1b所示，电商应用场景对应的商品信息处理系统中可以包括第一用户使用的第一终端设备11和第二用户使用的第二终端设备12，第一终端设备11和第二终端设备12通信连接。可选地，第一终端设备11与第二终端设备12进行交互，从第二终端设备12处获取第二用户的隐式3D表征模型或人体mesh模型或多张图像。
进一步可选地，如图1c所示，商品信息处理系统除了包括第一终端设备11和第二终端设备12之外，还包括服务端设备13，第一终端设备11和第二终端设备12分别与服务端设备13通信连接。相应的，第一终端设备11、第二终端设备12及服务端设备13之间进行交互，使得第一终端设备11从服务端设备13获取第二用户的隐式3D表征模型或人体mesh模型或多张图像。可选地，第二终端设备12将第二用户的多张图像上传至服务端设备13；或者，第二终端设备12采集到第二用户的多张图像之后，基于第二用户的多张图像进行基于神经网络的三维重建或进行显式三维重建，得到第二用户的隐式3D表征模型或人体mesh模型，并将第二用户的隐式3D表征模型或人体mesh模型上传至服务端设备13。
在一可选实施例中，第二终端设备可以采集第二用户的多张图像，根据第二用户的多张图像进行基于神经网络的三维重建或进行显式三维重建，得到第二用户的隐式 3D表征模型或人体mesh模型，并将第二用户的隐式3D表征模型或人体mesh模型上传至服务端设备，并维护隐式3D表征模型或人体mesh模型的模型标识。第二终端设备主动向第一终端设备提供第二用户的隐式3D表征模型或人体mesh模型的模型标识，或者，第二终端根据第一终端设备发送的模型获取请求，向第一终端设备提供第二用户的隐式3D表征模型或人体mesh模型的模型标识。第一终端设备根据该模型标识，从服务端设备获取第二用户的隐式3D表征模型或人体mesh模型。在本实施例中，并不限定模型标识的实现方式，凡是能够唯一标识第二用户的隐式3D表征模型或人体mesh模型的实现方式均适用于本申请实施例。例如，可以使用第二用户的账号信息，或者是第二终端设备的标识信息，作为第二用户的隐式3D表征模型或人体mesh模型的模型标识。
在另一可选实施例中，第二终端设备可以采集第二用户的多张图像，将第二用户的多张图像上传至服务端设备，服务端设备预先维护有第一用户和第二用户之间的绑定关系，基于该绑定关系，主动将第二用户的多张图像发送至第一用户使用的第一终端设备，或者，也可以根据第一终端设备发送的图像获取请求，将第二用户的多张图像发送至第一用户使用的第一终端设备。第一终端设备根据服务端设备提供的第二用户的多张图像进行基于神经网络的三维重建或进行显式三维重建，得到第二用户的隐式3D表征模型或人体mesh模型。
接续于上述一些可选实施例，在获取到第二用户的隐式3D表征模型后，可以根据第二用户的隐式3D表征模型，渲染出第二用户的3D人体模型。可选地，根据第二用户的隐式3D表征模型，渲染出第二用户的3D人体模型的一种具体实施方式如下：根据第二用户的多张图像的图像特征，确定第二用户对应的人体空间范围，第二用户对应的人体空间范围为第二用户的隐式3D表征模型的形状和体积；然后基于人体空间范围和隐式3D表征模型生成第二用户对应的初始三维模型，初始三维模型包括第二用户的人体表面点，第二用户的人体表面点可以是具有体现外貌特征的点，例如眉毛、眼睛、鼻子、嘴巴、耳朵及身体的关节等特征点；之后将初始三维模型上每个人体表面点对应的第一视线的视角信息的平均值，分别转换为每个表面点的颜色信息，以得到第二用户的3D人体模型，第一视线是指每张图像中拍摄到各像素点的视线。该实施例的详细实施过程与隐式3D表征模型的生成过程相关，具体可参见后续实施例中的描述，在此暂不赘述。
进一步，无论采用上述哪种方式得到第二用户的3D人体模型，在得到第二用户的3D人体模型后，为了给第二用户选择到精准匹配的商品，可以从电商应用提供的各商品中选择出候选商品，将候选商品渲染至3D人体模型上，以得到候选商品与3D 人体模型的融合效果。可选地，将候选商品渲染至3D人体模型上，得到候选商品与3D人体模型的融合效果的一种具体实施方式如下：首先对3D人体模型进行特征估计，得到多个人体特征点及其位置信息；然后根据多个人体特征点及其位置信息，确定3D人体模型上与候选商品适配的目标特征点位置；之后将候选商品对应的试用模型渲染至目标特征点位置上，以得到候选商品与3D人体模型的融合效果。
其中，多个人体特征点可以包括但不限于：五官特征点、人体骨骼特征点和手部特征点，具体可视3D人体模型对应的第二用户的人体结构而定。另外，针对不同类型的候选商品，在人体上的试用位置是不同的。例如对于美妆类商品，试用位置通常在人体五官上，对于服装类商品，试用位置通常在人体上半身，与人体骨骼特征有关；如果是穿戴类商品，试用位置通常在人体手腕等部位。基于此，将候选商品对应的试用模型渲染至目标特征点位置上，以得到候选商品与3D人体模型的融合效果，可选的具体实施方式包括以下至少一种操作：
若候选商品为美妆类商品，目标特征点位置为五官特征点对应的位置，将美妆类商品对应的二维试用模型渲染至五官特征点对应的位置上，以得到美妆类商品与3D人体模型的融合效果。其中，五官特征点对应的位置可以是眉毛、眼睛、鼻子、嘴巴及下巴对应的位置。以口红为例，其目标特征点位置为嘴唇位置，由于口红只需涂抹嘴唇的表面，那么口红对应的试用模型为二维模型，在试装时，将口红对应的二维试用模型渲染至第二用户的嘴唇位置，得到口红与第二用户的嘴唇的融合效果。
若候选商品为服饰类商品，目标特征点位置为人体骨骼特征点对应的位置，将服饰类商品对应的三维试用模型渲染至人体骨骼特征点对应的位置上，以得到服饰类商品与3D人体模型的融合效果。其中，人体骨骼特征点对应的位置可以是肩周、胳膊肘、手肘、胯骨、膝盖及脚腕对应的位置点。以上衣外套为例，目标特征点为肩周、胳膊肘、手肘，将上衣外套对应的三维试用模型渲染至上述人体骨骼特征点对应的位置上，得到上衣外套与3D人体模型的融合效果。
若候选商品为穿戴类商品，目标特征点位置为手部特征点对应的位置，将穿戴类商品对应的三维试用模型渲染至手部特征点对应的位置上，以得到穿戴类商品与3D人体模型的融合效果。其中，手部特征点可以是手腕和手指关节对应的位置。对于穿戴类商品，特征点位置还可以是脖颈部位、耳部特征点对应的位置。以手链为例，目标特征点为手腕对应的位置，将手链对应的三维试用模型渲染至手腕对应的位置上，得到手链与3D人体模型的融合效果。
在本申请上述实施例中，并未限定选择候选商品的实施方式。在一可选实施例中，第一用户可以根据自己的喜好或需求在电商应用中搜索所需的商品作为候选商品。例 如，第一用户可以在搜索框内输入关键词“A品牌口红”，之后点击“搜索”控件发出搜索请求；第一终端设备响应第一用户发出的搜索请求，搜索出所有A品牌的口红商品并展示在搜索结果页面上。第一用户可以将搜索结果页面上的口红商品作为候选商品。或者，在另一可选实施例中，第一终端设备可以根据第一用户的历史操作数据，生成商品推荐列表，将商品推荐列表中的商品作为候选商品。除了这里列举的实施方式之外，还可以采用下述实施例提供的方式选择候选商品。
在本申请一可选实施例中，为了使电商购物场景更加丰富，进一步提高电商应用的灵活性，满足用户的购物需求，可以将第二用户的3D人体模型显示在第一界面上，第一界面可以是电商应用提供的任一页面、浮层或弹窗。第一界面上除了可以显示3D人体模型之外，还可以显示其他控件，以方便第一用户根据第二用户的3D人体模型为第二用户选择目标商品。可选地，可以将第一界面分为多个区域，例如划分为第一区域、第二区域及第三区域。其中，对第一区域、第二区域及第三区域的位置关系不做限定，例如三个区域分别分布在第一界面的左、中、右或上、中、下，或者三个区域还可以无序分布在第一界面的不同位置。在一可选实施例中，如图1b或图1c所示，可以将第二用户的3D人体模型显示于第一界面的第一区域中；相应地，在第一界面的第二区域中显示至少一个商品选择控件，进一步可选地，在第一界面的第三区域中显示分享控件、收藏控件、加购控件和下单控件中的至少一个。在图1b或图1c中，以第二区域、第一区域和第三区域分布在第一界面的左中右为例进行图示，但不限于此。
在一可选实施例中，不同商品选择控件可以对应不同的商品类型，在图1b或图1c中，示出试妆选择控件、试衣选择控件、眼镜试戴选择控件和手表试戴选择控件，但并不限于此。相应地，选择候选商品的又一种实施方式如下：响应于第一用户对任一商品选择控件的触发操作，确定展示与被触发的商品选择控件对应的商品类型下的至少一个商品；继续响应于第一用户的商品选择操作，确定被选择的商品作为候选商品。可选地，上述与被触发的商品选择控件对应的商品类型下的至少一个商品可以是第一终端设备根据第一用户针对该商品类型产生的历史操作数据所推荐的。这里的历史操作数据可以包括：第一用户加入购物车的属于该商品类型下的商品信息、之前购买过的该商品类型下的商品信息、收藏过的该商品类型下的商品信息以及多次浏览过过的该商品类型下的商品信息。
进一步可选地，第二区域中显示的每个一个商品选择控件可以是一类商品的总控件，每个商品选择控件还可以包括至少一个商品选择子控件，并且还可以根据至少一个商品选择子控件对应的商品的属性信息，适应性为至少一个商品选择子控件分别增 加下一级子控件，其中，属性信息可以是商品的品牌、颜色及材质等。在一可选实施例中，商品选择控件及对应的商品选择子控件可以形成层级关系，该层级关系与商品类目之间的层级关系对应，即商品选择控件对应第一级商品类目，每一层商品选择子控件对应一级子类目，直至到达叶子类目。相应地，选择候选商品，具体实施方式如下：响应于第一用户对第二区域中显示的任一商品选择控件的选择操作，展示与该商品选择控件对应的第一级商品选择子控件；继续响应于第一用户对每一级商品选择子控件中任一商品选择子控件的选择操作，直至到达与叶子类目对应的商品选择子控件；响应于对该叶子类目对应的商品选择子控件中任一商品选择子控件的选择操作，展示该叶子类目下可被选择的商品列表；响应于第一用户对该商品列表中的选择操作，确定被选择商品作为候选商品。
例如，假设多个商品选择控件至少包括：服饰类商品选择控件、穿戴类商品选择控件及美妆类商品选择控件，可选地，服饰类商品选择控件对应的下一级商品选择子控件至少包括：外套商品选择子控件、T恤商品选择子控件、裤装商品选择子控件及裙装商品选择子控件；相应地，穿戴商品选择控件对应的下一级商品选择子控件包括：帽子商品选择子控件、眼镜商品选择子控件、耳饰商品选择子控件、项链商品选择子控件、手链商品选择子控件、戒指商品选择子控件及手表商品选择子控件；美妆商品选择控件对应的下一级商品选择子控件包括：眉笔商品选择子控件、眼线商品选择子控件、眼影商品选择子控件、粉底液商品选择子控件及口红商品选择子控件等。以第一用户想要为第二用户选购口红为例，响应于第一用户对美妆商品选择控件的选择操作，展示眉笔商品选择子控件、眼线商品选择子控件、眼影商品选择子控件、粉底液商品选择子控件及口红商品选择子控件等；继续响应于第一用户对口红商品选择子控件的选择操作，继续展示下一级选择子控件，假设下一级选择子控件是对应不同口红品牌的选择子控件；继续响应于第一用户对口红品牌子控件的选择操作，继续展示下一级选择子控件，假设下一级选择子控件是对应不同口红颜色的选择子控件；响应于对口红颜色子控件的选择操作，展示被选择的口红品牌下的被选择的口红颜色对应的口红商品列表；响应于第一用户对该口红商品列表中的某个口红商品的选择操作，确定被选择口红作为候选商品。
在一可选实施例中，第一界面的第三区域中可以显示分享控件、收藏控件、加购控件和下单控件中的至少一个；相应地，在从候选商品中选择与第二用户适配的目标商品之后，还可以进行以下至少一种操作：响应于第一用户对分享控件的触发操作，将目标商品的链接信息发送给第二用户的终端设备，以使第二用户购买目标商品；响应第一用户对收藏控件的触发操作，将目标商品的链接信息添加至收藏夹中；响应第 一用户对加购控件的触发操作，将目标商品添加至购物车中；响应第一用户对下单控件的触发操作，对目标商品进行下单操作，其中，下单操作对应的账号可以为第一用户自己的账号，下单操作对应的收货地址可以为第二用户的收货地址，或者，下单操作对应的收货地址也可以为第一用户的收货地址。进一步，在收货地址是第一用户的收货地址的情况下，第一用户收到商品后，可以将目标商品转交给第二用户。
在本申请上述一些可选实施例中，第二用户的3D人体模型可以是基于第二用户的隐式3D表征模型渲染得到的。在基于第二用户的隐式3D表征模型渲染得到第二用户的3D人体模型之前，需要基于第二用户的多张图像进行基于神经网络的三维重建，得到第二用户的隐式3D表征模型。其中，基于第二用户的多张图像进行基于神经网络的三维重建，得到第二用户的隐式3D表征模型的过程可以是第一终端设备执行，也可以是第二终端设备执行，还可以是服务端设备执行，对此不做限定。无论是得到隐式3D表征模型的执行主体是谁，其基于第二用户的多张图像进行基于神经网络的三维重建，得到第二用户的隐式3D表征模型的过程相同或相似。下面实施例将展开进行详细描述。
具体地，可以对处于真实世界中的第二用户的至少部分人体结构从不同拍摄角度进行拍摄，得到包含该第二用户的至少部分人体结构(例如头部或上半身或整个人体)的多张图像或者视频，从视频中提取包含第二用户的多张图像。进一步可选的，为了能够准确重建出第二用户的三维模型，可以采用绕第二用户360度的环绕方式进行拍摄，得到第二用户的多张图像。需要说明的是，不同图像对应不同的相机位姿，相机位姿包括拍摄设备在拍摄图像时的位置和姿态。其中，本实施例对拍摄设备不做限制，拍摄设备例如可以是但不限于：第二用户的终端设备，该终端设备具有摄像头。在获取到多张图像之后，分别计算每张图像对应的相机位姿，根据每张图像对应的相机位姿和相机内参等数据确定相机在拍摄每张图像时发射出来的多条第一视线以及每条第一视线的视角信息。在每条第一视线上进行空间点采样，得到多个空间点。应理解，从同一条第一视线上采样得到的空间点的视角信息均是该第一视线的视角信息。在得到多个空间点之后，利用多个空间点的空间坐标及其视角信息进行基于神经网络的三维重建，该过程可以是不断进行模型训练的过程，但不限于此，最终可得到第二用户的隐式3D表征模型。进一步，还可以根据多张图像，构建第二用户对应的人体mesh模型，人体mesh模型包括第二用户的人体表面点及其颜色信息。
在另一可选实施例中，还可以采用下述实施例得到第二用户的隐式3D表征模型。该实施方式包括以下步骤：首先，根据第二用户的多张图像进行基于神经网络的三维重建，得到对第二用户进行隐式3D表达的初始隐式三维表征模型，第二用户的人体表面点与对 应图像中的像素点对应，且与拍摄到该像素点的第一视线对应。得到初始隐式三维表征模型的三维重建过程是传统的基于神经网络的三维重建过程。接着，根据初始隐式三维表征模型和多张图像，构建第二用户对应的显式三维模型，显式三维模型包括第二用户的人体表面点的颜色信息，每个表面点的颜色信息是根据该表面点对应的第一视线的平均视角信息确定的。接着，随机生成显式三维模型上表面点对应的第二视线，并根据每个表面点的颜色信息分别生成每个表面点对应的第二视线对应的平均视角信息。最后，根据第二视线对应的平均视角信息和第二视线上空间点的空间坐标，基于初始隐式三维表征模型进行基于神经网络的三维重建，得到对目标物体进行隐式3D表达的目标隐式三维表征模型。该实施例中得到的目标隐式三维表征模型可作为上述各实施例中第二用户的隐式3D表征模型。
需要说明的是，对于一张图像上的每个像素点都会对应一条第一视线，相应地，样本图像中的像素点是由第一视线射到第二用户的一个人体表面点上成像得到的，该第一视线也就是拍摄到该像素点的视线。由此可知，第二用户的每个人体表面点与像素点以及拍摄到该像素点的第一视线之间存在对应关系。每张图像中的不同像素点与第二用户的不同人体表面点对应，不同人体表面点对应不同的第一视线。
在本实施例中，初始隐式三维表征模型或目标隐式三维表征模型能够对第二用户进行隐式三维表达，例如可以表达第二用户的人体形状、轮廓、皮肤纹理、颜色等多个维度人体信息。在本实施例中，初始隐式三维表征模型或目标隐式三维表征模型是一个全连接神经网络，全连接神经网络又称多层感知器((Multi-Layer Perceptron，MLP)。该初始隐式三维表征模型或目标隐式三维表征模型基于输入的空间点的空间坐标和视角信息，分别预测空间点的体积密度和颜色信息。其中，初始隐式三维表征模型或目标隐式三维表征模型可以表达为：
σ，c＝F(d，x)……(1)
其中，x＝(x,y,z)，x记为空间点的空间坐标(x,y,z)；d＝(θ,φ)，d＝(θ,φ)记为空间点的视角信息(θ,φ)，θ为方位角，φ为仰角。c＝(r,g,b)，c记为空间点的颜色信息(r,g,b)，r是指红色(Red，R)，g是指绿色(Green，G)，b是指蓝色(Blue，B)。σ记为空间点的体积密度。
实际应用中，初始隐式三维表征模型或目标隐式三维表征模型包括用于预测σ体积密度的Fσ网络和用于预测c颜色信息的Fc网络。于是，初始隐式三维表征模型或目标隐式三维表征模型可以进一步表达为：
Fσ：x→(σ，f)……(2)
Fc：(d，f)→c……(3)
值得注意的是，Fσ网络输入的是空间点的空间坐标x，输出的是空间点的体积密度和中间特征f。Fc网络输入的是中间特征f和空间点的视角信息d，输入的是空间点的颜色信息RGB值。也就是说，体积密度只和空间坐标x有关，颜色信息RGB值和空间坐标及视角信息相关。
在本实施例中，在获取到第二用户的多张图像之后，分别计算每张图像对应的相机位姿，根据每张图像对应的相机位姿和相机内参等数据确定相机在拍摄每张图像时发射出来的多条第一视线以及每条第一视线的视角信息。在每条第一视线上进行采样，得到多个空间点。在得到多个空间点之后，利用多个空间点的空间坐标及其视角信息进行基于神经网络的三维重建，该过程可以是分批多次执行的过程，最终可得到初始隐式3D表征模型。
具体地，可以采用不断迭代的方式进行基于神经网络的三维重建，例如每次可以随机选择k张图像，从k张图像中随机选择大小为m*n的图像块，利用k个图像块中各像素点对应的第一视线上空间点的空间坐标和视角信息进行基于神经网络的三维重建(或模型训练)，直到三维重建过程的损失函数符合设定要求时终止三维重建过程。其中，k是大于或等于1的自然数，且k小于或等于图像的总数；m、n是大于或等于1的自然数，m、n分别表示图像块在横向和纵向维度上的像素数，m小于或等于原始图像的宽度(宽度维度对应横向)，n小于或等于图像的长度(长度维度对应纵向)，m和n可以相同，也可以不同。可选地，可以采用等间隔方式在每条第一视线上采样多个空间点，即任意两个相邻空间点之间的采样间隔是相同的。也可以采用不同采样间隔在每条第一视线上采样多个空间点，采样间隔的大小不做限定。
在本实施例中，在得到对第二用户进行隐式三维表达的初始隐式三维表征模型之后，根据初始隐式三维表征模型和多张图像，可以构建第二用户对应的显式三维模型。在本实施例中，显式三维模型可以是指能够反映第二用户的表面特征且能够对第二用户进行显式三维表示的Mesh(网格)模型，该显式三维模型包括第二用户的人体表面点及每个人体表面点的空间坐标和颜色信息。在本实施例中，显式三维模型上每个人体表面点的颜色信息是根据该人体表面点对应的第一视线的平均视角信息确定的，表示该人体表面点对应的任何视线对应的平均视角信息。换而言之，显式三维模型上每个人体表面点的颜色信息并不是第二用户在光线照射下产生的真实颜色信息，而是与该人体表面点对应的各条第一视线的平均视角信息具有映射关系的颜色信息。
在一可选实现方式中，根据初始隐式3D表征模型和多张图像，构建第二用户对应的显式三维模型，包括：根据多张图像的图像特征，确定第二用户对应的空间范围；基于空间范围和初始隐式3D表征模型生成第二用户对应的初始三维模型，初始三维模型包括第二用户上的人体表面点；针对任一人体表面点，将该人体表面点对应的至少一条第一视线 的视角信息的平均值转换为该人体表面点的颜色信息，以得到显式三维模型。
在本实施例中，可以采用诸如运动恢复结构(Structure from Motion，SfM)算法处理多张图像的图像特征，以估计出第二用户对应的稀疏3D点位置，第二用户对应的稀疏3D点位置可以帮助确定第二用户在世界坐标系中的空间范围。该空间范围可以是具有长、宽和高的空间范围，例如可以是正方体空间或长方体空间，但不限于此。
进一步可选的，上述基于空间范围和初始隐式三维表征模型生成第二用户对应的初始三维模型的一种实施方式是：基于空间范围和初始隐式三维表征模型生成第二用户对应的标量场数据，标量场数据包括多个体积元素(Volume Pixel)，可简称为体素；对多个体积元素进行三角面解析，得到初始三维模型包含的多个三角面、多个三角面上的多个顶点及其空间坐标，多个三角面和多个顶点用于限定初始三维模型包含的各人体表面点。
其中，人体表面点的颜色信息可采用下述方式确定的：针对任一人体表面点，从不同相机位姿对应的第一视线中，确定该人体表面点对应的至少一条第一视线，需要说明的是，同一人体表面点在同一相机位姿下只会有一条第一视线对应该人体表面点，但是，在采用不同相机位姿拍摄多张图像过程中，同一人体表面点通常会被两个或两个以上的相机位姿拍摄到，也就是说通常会有两条或两条以上来自不同相机位姿下的第一视线对应同一人体表面点，但是也会存在特殊情况，即某个人体表面点仅在一个相机位姿下被拍摄到，即只有一条第一视线对应该人体表面点。进一步，计算该人体表面点对应的至少一条第一视线的视角信息的平均值，将该平均值转换为该人体表面点的颜色信息进行保存。
进一步可选的，为了便于快速获取表面点对应的第一视线的视角信息，还可以生成每张图像对应的视角预存图，所述视角预存图中存储有该张图像中各像素点对应的第一视线的视角信息。值得注意的是，基于拍摄图像的相机位姿和相机内参，不难确定从拍摄图像时的光心位置出射并穿过图像的像素点对应的表面点的第一视线的直线方程信息，基于第一视线的直线方程信息根据几何原理可以快速获知第一视线的视角信息。
相应地，针对任一人体表面点，将该人体表面点对应的至少一条第一视线的视角信息的平均值转换为人体表面点的颜色信息，以得到显式三维模型，包括：针对任一人体表面点，根据多张图像对应的相机位姿，结合初始三维模型，从多张图像中确定包含该人体表面点对应的目标像素点的至少一张目标图像；将至少一张目标图像对应的视角预存图中存储的该目标像素点对应的第一视线的视角信息的平均值转换为该人体表面点的颜色信息。
在本实施例中，在得到第二用户的初始隐式3D表征模型和显式三维模型之后，还可以随机生成显式三维模型上各人体表面点对应的不同于第一视线的虚拟视线，为了便于理解，将随机生成的虚拟视线称作为第二视线，应理解，相对于真实相机发射出的第一视线来说，第二视线是假设的虚拟相机发射出的虚拟视线。可选地，针对显式三维模型任一人 体表面点，可以随机生成该人体表面点对应的第二视线，并根据该人体表面点的颜色信息生成该人体表面点对应的第二视线对应的平均视角信息。
在本实施例中，针对显式三维模型上任一人体表面点，可以以该人体表面点对应的第一视线为参考视线，在该参考视线一定范围内随机生成该人体表面点对应的第二视线。进一步可选的，根据该人体表面点对应的第一视线随机生成该人体表面点对应的第二视线包括：根据该人体表面点的空间坐标和该人体表面点对应的第一视线的视角信息，随机生成一条经过该人体表面点且不同于该人体表面点对应的第一视线的视线作为第二视线。
具体而言，根据该人体表面点的空间坐标和该目标像素点对应的第一视线的视角信息，确定候选空间范围；在该候选空间范围中，随机生成一条经过该人体表面点且不同于该目标像素点对应的第一视线的视线作为第二视线。其中，候选空间范围可以是任意形状的空间范围。可选的，候选空间范围是以人体表面点的空间坐标为圆点，以穿过目标像素点对应的第一视线为中心线的椎体空间范围。在确定候选空间范围时，可以是第二视线与穿过人体表面点的第一视线之间的夹角范围为[-η,η]度。其中，η例如为30度。
在本实施例中，采用上述实施例的方法，可以针对多张图像中各像素点对应的人体表面点分别随机生成第二视线，即可得到随机产生多条第二视线，并得到多条第二视线对应的平均视角信息，进一步可以利用多条第二视线对应的平均视角信息和多条第二视线上空间点的空间坐标，继续基于初始隐式3D表征模型进行基于神经网络的三维重建(或模型训练)，得到目标隐式3D表征模型。
值得注意的是，在三维重建过程中，依次利用每条第二视线对应的平均视角信息和第二视线上空间点的空间坐标在该初始隐式3D表征模型的基础上继续进行三维重建，在每次利用上一批次的第二视线对应的平均视角信息和上一批次的第二视线上空间点的空间坐标执行一次重建操作后，采用立体渲染技术，利用预测出的上一批次中各条第二视线上各个空间点的体积密度分别对各条第二视线上各个空间点的RGB颜色信息进行积分，得到上一批次中各条第二视线对应的像素点的预测RGB颜色信息；基于上一批次中各条第二视线对应的像素点的预测RGB颜色信息与各条第二视线对应的像素点的实际RGB颜色信息(这里的实际RGB颜色信息是指相应样本图像中该像素点的颜色信息)计算损失函数，若损失函数收敛，至此完成三维重建(或模型训练)过程，若损失函数未收敛，则调整模型参数，并利用下一批次第二视线对应的平均视角信息和下一批次第二视线上空间点的空间坐标继续迭代训练，直至损失函数收敛。
本上述可选实施例中，以第二用户的多张图像为基础分别进行基于神经网络的三维重建和传统的三维重建，得到初始隐式三维表征模型和显式三维模型；基于显式三维模型进行随机视线和平均视角的生成，基于随机视线和平均视角在初始隐式3D表征模型的基础 上继续进行基于神经网络的三维重建，得到目标隐式3D表征模型。其中，初始隐式3D表征模型和目标隐式3D表征模型都是对第二用户进行隐式三维表示的神经网络模型。在三维重建过程中，通过产生随机视线并以随机视线对应的平均视角信息代替其真实视角信息的方式，利用随机视线及其对应的平均视角信息增强视线数据，基于增强后的视线数据继续进行基于神经网络的三维重建，可以得到对视线具有较强鲁棒性的隐式3D表征模。
在本申请上述实施例中，第一用户可以通过第一终端设备的电商应用基于第二用户的3D人体模型为第二用户选择适配的目标商品，需要说明的是，除了该种方式外，每个用户还可以根据候选商品与自己的3D人体模型的融合效果，查看候选商品是否适合自己，进而决定是否购买相应的商品。基于此，本申请实施例还提供了另一商品信息处理方法。
如图2a为本申请另一示例性实施例提供的商品信息处理方法的流程示意图。如图2a所示，该方法包括：
201、获取用户的隐式3D表征模型，隐式3D表征模型是根据用户的多张图像进行基于神经网络的三维重建得到的；
202、基于隐式3D表征模型渲染出用户的3D人体模型，并显示3D人体模型；
203、选择候选商品，将候选商品渲染至3D人体模型上，以得到候选商品与3D人体模型的融合效果。
本实施例提供的方法适用于任一用户的终端设备，例如可以是上述实施例中第一用户的第一终端设备，也可以是第二用户的第二终端设备。如果是第一用户的第一终端设备，则隐式3D表征模型是指第一用户的隐式3D表征模型；如果是第二用户的第二终端设备，则隐式3D表征模型是指第二用户的隐式3D表征模型。
进一步可选地，该方法还包括：从候选商品中选择融合效果符合要求的目标商品。
进一步可选地，该方法还包括：针对目标商品执行以下至少一种操作：将目标商品的链接信息添加至收藏夹中；将目标商品添加至购物车中；对目标商品进行下单操作；将目标商品的链接信息分享给其它用户，以使其它用户购买目标商品。
在本申请实施例中，并不限定将目标商品的链接信息分享给其它用户的实施方式，例如可以通过应用内消息的方式将目标商品的链接信息分享给其它用户，或者，通过淘口令的方式将目标商品的链接信息分享给其它用户，等等。
可选地，获取用户的隐式3D表征模型，包括：获取用户的多张图像，根据用户的多张图像进行基于神经网络的三维重建，以得到用户的隐式3D表征模型；根据用户的隐式3D表征模型，渲染出用户的3D人体模型。
需要说明的是，本实施例提供的商品信息处理方法中各步骤的详细实施方式可参见图1a-图1c所示实施例中的相应内容，此处不再赘述。
在本实施例中，每个用户可以根据用户自身的3D人体模型，渲染出候选商品与用户的3D人体模型的融合效果，据此可以了解候选商品是否适合自己，进而用户可以根据候选商品与用户的3D人体模型的融合效果，为自己选择适配的目标商品。其中，用户基于自己的3D人体模型与候选商品的融合效果为自己选择商品，相当于通过VR预先看到了候选商品在自己身上的试用效果，可以准确地为自己选择商品，能够做到精准匹配，给用户带来新的购物体验。
另外，在本实施例中，在得到用户的隐式3D表征模型之后，还可以保存用户的隐式3D表征模型，这样，当用户自己需要选购商品时，或者其它用户需要为该用户选购商品时，可以直接获取该用户的隐式3D表征模型，并渲染出3D人体模型，进而基于该3D人体模型进行候选商品的试用，得到候选商品与该3D人体模型融合效果，基于该融合效果进行商品选择或选购。对用户来说，只需一次打开摄像头，拍摄相关的图像进行人体模型的三维重建，在选择商品过程中，可以直接使用，无需再次打开摄像头，可以有效解决传统AR试用需要用户实时打开摄像头所引发的各种问题。例如，本申请实施例可以解决多次打开摄像头造成的隐私泄露问题，还可以解决实时进行图像处理所占用的内存和计算资源较大的问题，还可以解决因拍摄环境复杂造成的虚拟和现实叠加效果不佳的问题，而且还可以解决一些因为没有摄像头或者不具备拍摄环境或条件导致无法进行AR试用的问题等。进一步，在一些可选实施例中，可以采用基于神经网络的三维重建方式，构建用户的三维人体模型，相比于传统三维重建方式，重建效果更加有效可靠，适用范围更广。
下面结合一电商购物场景，对本申请实施例的技术方案进行详细阐述。
用户的终端设备，例如手机上安装了电商APP，在电商APP中增加了基于3D人体模型选购商品的功能。对用户来说，可以打开电商APP，在电商APP的相关页面找到该新增的基于3D人体模型选购商品的功能，然后启用该功能。此时，在页面上可显示拍摄控件，以引导用户完成3D人体模型的创建操作。具体地，用户可以点击拍摄控件，终端设备响应于用户对该页面上的拍摄控件的触发操作，调用终端设备的摄像头，并提示用户利用摄像头围绕用户采集包含至少部分人体外貌特征的视频或多张图像，这些图像可以从不同视角反映相同部位的人体外貌特征。在完成用户图像的采集操作之后，该页面上可显示一生成控件，响应于用户对该生成控件的触发操作，基于采集到的多张图像进行基于神经网络的三维重建，生成用户的隐式3D表征模型，并将用户的隐式3D表征模型保存在本地或上传至服务端设备。在本实施例中，隐式3D表征模型可以是NERF模型。经过NERF模型进行推理和神经渲染，可以输出与用户真人基本相似的3D人体模型，或称为3D虚拟人。
在用户有为自己选购商品的需求时，进入电商APP的试妆页面，该页面分为三个区域， 分别为第一区域、第二区域及第三区域，第一区域中用于显示由NERF模型进行推理和神经渲染得到的用户的3D人体模型，第二区域中显示至少一个商品选择控件，第三区域中显示分享控件、收藏控件、加购控件和下单控件。在显示试妆页面过程中，可以加载用户的NERF模型并基于该NERF模型进行推理和神经渲染得到的用户的3D人体模型，将3D人体模型显示在第一区域中。之后，用户根据自己的需求通过第二区域中展示的商品控件选择候选商品，即响应于用户对一商品控件的触发操作，会展示该商品类目下的商品列表，商品列表可以以子控件的形式展示，用户可以从商品列表中选择自己需要的商品，响应于用户对商品列表中某一商品子控件的选择操作，可将该商品的试妆模型渲染至用户的3D人体模型上，展示出一张试妆效果图，用户可以基于对试妆效果的满意度决定是否要选购该商品。需要说明的是，用户还可以对3D人体模型进行选择放大、查看细节等操作。
用户可以基于对试妆效果的满意度决定是否要选购该商品时，如果用户对于本次试妆效果不满意，可以直接忽略该商品；如果用户对本次试妆效果较满意，但是还需要考虑是否购买，可以触发第三区域的加购物车或收藏控件，将该商品加入购物车或收藏，以便于后续查看；如果用户对本次试妆效果很满意，且决定购买，可以触发第三区域的下单控件，购买该商品。
在本实施例中，用户除了可以通过自己的NERF模型为自己选购商品外，还可以将自己的NERF模型授权给其它用户，这样其它用户就可以根据该用户的NERF模型，渲染出该用户的3D虚拟人，基于该3D虚拟人为该用户选购商品，实现送礼物等社交目的。相应地，该用户也可以获取其它用户的NERF模型，基于其它用户的NERF模型为其它用户选购商品，达到送礼物等社交目的。
用户通过自己的终端设备获取到其它用户的NERF模型，并保存于本地，当需要为其它用户选择商品时，可以进入试妆页面，基于其它用户的NERF模型进行推理和神经渲染得到的其它用户的3D人体模型，并将其它用户的3D人体模型显示在试妆页面的第一区域中；之后，通过第二区域中展示的商品控件选择候选商品进行试妆，基于试妆效果确定候选商品是否符合其它用户，进而选择出其它用户满意的目标商品，在选择好目标商品后，用户可以将目标商品添加至购物车或收藏夹，或者将目标商品的链接分享给其它用户，或者，直接下单帮助其它用户购买该目标商品，并将收货地址修改为其它用户的地址，以便于商品直接邮寄给其它用户。
需要说明的是，用户可以在3D人体模型上试用一种候选商品，也可以同时使用两种或两种以上的候选商品，两种或两种以上的候选商品对应不同的试妆位置。可选地，以用户想要试用眼镜和T恤为例，眼镜的目标特征点位置为鼻梁和耳朵对应的位 置，响应于用户对眼镜控件的选择操作，将眼镜的三维试用模型渲染至鼻梁和耳朵对应的位置上，以得到眼镜与3D人体模型的融合效果。T恤的目标特征点是肩周、手臂、胸部及腰部，响应于用户对T恤控件的选择操作，将T恤的三维试用模型渲染至上述人体骨骼特征点对应的位置上，得到T恤与3D人体模型的融合效果，渲染效果如图2b所示。
本实施例中，基于3D人体模型的试妆方法属于VR试妆范畴，基于NERF的真人重建技术可以得到3D虚拟人，在该过程中，只需一次打开摄像头，拍摄相关的照片进行三维重建即可。完成3D虚拟人之后，即可进行试用使用，无需再打开摄像头，可以有效解决传统AR商品试用面临的各种问题。另外，基于NERF的真人重建技术，通过拍摄人体多视角的照片作为输入，经过NERF模型进行神经渲染，渲染出逼真的人体3D模型，相比于传统方案更加有效可靠，运用最新的神经渲染技术重建的三维人体模型，适用范围更广，具备更好的普适性。
图3为本申请一示例性实施例提供的商品信息处理装置的结构示意图，可适用于第一用户终端设备。如图3所示，该装置包括：显示模块31、选择模块32和渲染模块33。
显示模块31，用于显示第二用户的三维3D人体模型，3D人体模型是根据第二用户的多张图像进行三维重建得到的。选择模块32，用于选择候选商品。渲染模块33，用于将候选商品渲染至3D人体模型上，以得到候选商品与3D人体模型的融合效果。选择模块32还用于：根据候选商品与3D人体模型的融合效果，从候选商品中选择与第二用户适配的目标商品。
进一步地，显示模块31在用于显示第二用户的三维3D人体模型时，具体用于：获取第二用户的隐式3D表征模型，隐式3D表征模型是根据第二用户的多张图像进行基于神经网络的三维重建得到的，是对第二用户的人体的三维隐式表征；根据第二用户的隐式3D表征模型，渲染出第二用户的3D人体模型；或者，获取第二用户的人体网格mesh模型，根据人体mesh模型渲染出第二用户的3D人体模型。
进一步地，显示模块31在用于获取第二用户的隐式3D表征模型或人体mesh模型时，具体用于：获取用于标识第二用户的隐式3D表征模型或人体mesh模型的模型标识；根据模型标识，从服务端获取第二用户的隐式3D表征模型或人体mesh模型，服务端维护有各个用户的隐式3D表征模型或人体mesh模型；或者，获取第二用户的多张图像，根据第二用户的多张图像进行基于神经网络的三维重建或传统的三维重建，以得到第二用户的隐式3D表征模型或人体mesh模型。
进一步地，渲染模块33在用于根据第二用户的隐式3D表征模型，渲染出第二用 户的3D人体模型时，具体用于：根据第二用户的多张图像的图像特征，确定第二用户对应的空间范围；基于空间范围和隐式3D表征模型生成第二用户对应的初始三维模型，初始三维模型包括第二用户上的表面点；将初始三维模型上每个表面点对应的第一视线的视角信息的平均值，分别转换为每个表面点的颜色信息，以得到第二用户的3D人体模型，第一视线是指每张图像中拍摄到各像素点的视线。
进一步地，渲染模块33在用于将候选商品渲染至3D人体模型上，以得到候选商品与3D人体模型的融合效果时，具体用于：对3D人体模型进行特征估计，得到多个人体特征点及其位置信息；根据多个人体特征点及其位置信息，确定3D人体模型上与候选商品适配的目标特征点位置；将候选商品对应的试用模型渲染至目标特征点位置上，以得到候选商品与3D人体模型的融合效果。
进一步地，多个人体特征点包括：五官特征点、人体骨骼特征点和手部特征点；
渲染模块33在用于将候选商品对应的试用模型渲染至目标特征点位置上，以得到候选商品与3D人体模型的融合效果时，具体用于一下至少一种操作：若候选商品为美妆类商品，目标特征点位置为五官特征点对应的位置，将美妆类商品对应的二维试用模型渲染至五官特征点对应的位置上，以得到美妆类商品与3D人体模型的融合效果；若候选商品为服饰类商品，目标特征点位置为人体骨骼特征点对应的位置，将服饰类商品对应的三维试用模型渲染至人体骨骼特征点对应的位置上，以得到服饰类商品与3D人体模型的融合效果；若候选商品为穿戴类商品，目标特征点位置为手部特征点对应的位置，将穿戴类商品对应的三维试用模型渲染至手部特征点对应的位置上，以得到穿戴类商品与3D人体模型的融合效果。
进一步地，显示模块31在用于显示第二用户的三维3D人体模型时，具体用于：在第一界面的第一区域中显示第二用户的3D人体模型，第一界面的第二区域中显示有至少一个商品选择控件，不同商品选择控件对应不同的商品类型；相应地，选择模块32在用于选择候选商品时，具体用于：响应于对任一商品选择控件的触发操作，确定展示与被触发的商品选择控件对应的商品类型下的至少一个商品；响应于商品选择操作，确定被选择的商品作为候选商品。
进一步地，第一界面的第三区域中显示有分享控件、收藏控件、加购控件和下单控件中的至少一个；在从候选商品中选择与第二用户适配的目标商品之后，商品信息处理装置还用于以下至少一种操作：响应于对分享控件的触发操作，将目标商品的链接信息发送给第二用户的终端设备，以使第二用户购买目标商品；响应对收藏控件的触发操作，将目标商品的链接信息添加至收藏夹中；响应对加购控件的触发操作，将目标商品添加至购物车中；响应对下单控件的触发操作，对目标商品进行下单操作， 下单操作对应的收货地址为第二用户的收货地址。
这里需要说明的是：本实施例提供的商品信息处理装置可实现上述图1a方法实施例中描述的技术方案，上述各模块或单元具体实现的原理可参见上述各方法实施例中的相应内容，此处不再赘述。
图4为本申请一示例性实施例提供的另一商品信息处理装置的结构示意图。如图4所示，该装置包括：获取模块41、渲染模块42和选择模块43。
获取模块41，用于获取用户的隐式三维3D表征模型，隐式3D表征模型是根据用户的多张图像进行基于神经网络的三维重建得到的。渲染模块42，用于基于隐式3D表征模型渲染出用户的3D人体模型，并显示3D人体模型。选择模块43，用于选择候选商品。渲染模块42还用于：将候选商品渲染至3D人体模型上，以得到候选商品与3D人体模型的融合效果。
进一步可选地地，该商品信息处理装置还用于：从候选商品中选择融合效果符合要求的目标商品。
进一步可选地，该商品信息处理装置还用于针对目标商品执行以下至少一种操作：将目标商品的链接信息添加至收藏夹中；将目标商品添加至购物车中；对目标商品进行下单操作；将目标商品的链接信息分享给其它用户，以使其它用户购买目标商品。
这里需要说明的是：本实施例提供的商品信息处理装置可实现上述图1a方法实施例中描述的技术方案，上述各模块或单元具体实现的原理可参见上述各方法实施例中的相应内容，此处不再赘述。
图5为本申请一示例性实施例提供的终端设备的结构示意图。如图5所示，该终端设备包括：存储器50a和处理器50b；存储器50a，用于存储计算机程序，处理器50b与存储器50a耦合，用于执行计算机程序，以用于实现一下步骤：
显示第二用户的三维3D人体模型，3D人体模型是根据第二用户的多张图像进行三维重建得到的；选择候选商品；将候选商品渲染至3D人体模型上，以得到候选商品与3D人体模型的融合效果；根据候选商品与3D人体模型的融合效果，从候选商品中选择与第二用户适配的目标商品。
进一步地，处理器50b在用于显示第二用户的三维3D人体模型时，具体用于：获取第二用户的隐式3D表征模型，隐式3D表征模型是根据第二用户的多张图像进行基于神经网络的三维重建得到的，是对第二用户的人体的三维隐式表征；根据第二用户的隐式3D表征模型，渲染出第二用户的3D人体模型；或者，获取第二用户的人体网格mesh模型，根据人体mesh模型渲染出第二用户的3D人体模型。
进一步地，处理器50b在用于获取第二用户的隐式3D表征模型或人体mesh模型 时，具体用于：获取用于标识第二用户的隐式3D表征模型或人体mesh模型的模型标识；根据模型标识，从服务端获取第二用户的隐式3D表征模型或人体mesh模型，服务端维护有各个用户的隐式3D表征模型或人体mesh模型；或者，获取第二用户的多张图像，根据第二用户的多张图像进行基于神经网络的三维重建或传统的三维重建，以得到第二用户的隐式3D表征模型或人体mesh模型。
进一步地，处理器50b在用于根据第二用户的隐式3D表征模型，渲染出第二用户的3D人体模型时，具体用于：根据第二用户的多张图像的图像特征，确定第二用户对应的空间范围；基于空间范围和隐式3D表征模型生成第二用户对应的初始三维模型，初始三维模型包括第二用户上的表面点；将初始三维模型上每个表面点对应的第一视线的视角信息的平均值，分别转换为每个表面点的颜色信息，以得到第二用户的3D人体模型，第一视线是指每张图像中拍摄到各像素点的视线。
进一步地，处理器50b在用于将候选商品渲染至3D人体模型上，以得到候选商品与3D人体模型的融合效果时，具体用于：对3D人体模型进行特征估计，得到多个人体特征点及其位置信息；根据多个人体特征点及其位置信息，确定3D人体模型上与候选商品适配的目标特征点位置；将候选商品对应的试用模型渲染至目标特征点位置上，以得到候选商品与3D人体模型的融合效果。
进一步地，多个人体特征点包括：五官特征点、人体骨骼特征点和手部特征点；处理器50b在用于将候选商品对应的试用模型渲染至目标特征点位置上，以得到候选商品与3D人体模型的融合效果时，具体用于一下至少一种操作：若候选商品为美妆类商品，目标特征点位置为五官特征点对应的位置，将美妆类商品对应的二维试用模型渲染至五官特征点对应的位置上，以得到美妆类商品与3D人体模型的融合效果；若候选商品为服饰类商品，目标特征点位置为人体骨骼特征点对应的位置，将服饰类商品对应的三维试用模型渲染至人体骨骼特征点对应的位置上，以得到服饰类商品与3D人体模型的融合效果；若候选商品为穿戴类商品，目标特征点位置为手部特征点对应的位置，将穿戴类商品对应的三维试用模型渲染至手部特征点对应的位置上，以得到穿戴类商品与3D人体模型的融合效果。
进一步地，处理器50b在用于显示第二用户的三维3D人体模型时，具体用于：在第一界面的第一区域中显示第二用户的3D人体模型，第一界面的第二区域中显示有至少一个商品选择控件，不同商品选择控件对应不同的商品类型；相应地，处理器50b在用于选择候选商品时，具体用于：响应于对任一商品选择控件的触发操作，确定展示与被触发的商品选择控件对应的商品类型下的至少一个商品；响应于商品选择操作，确定被选择的商品作为候选商品。
进一步地，第一界面的第三区域中显示有分享控件、收藏控件、加购控件和下单控件中的至少一个；在从候选商品中选择与第二用户适配的目标商品之后，处理器50b还用于以下至少一种操作：响应于对分享控件的触发操作，将目标商品的链接信息发送给第二用户的终端设备，以使第二用户购买目标商品；响应对收藏控件的触发操作，将目标商品的链接信息添加至收藏夹中；响应对加购控件的触发操作，将目标商品添加至购物车中；响应对下单控件的触发操作，对目标商品进行下单操作，下单操作对应的收货地址为第二用户的收货地址。
进一步地，如图5所示，该终端设备还包括：通信组件50c、显示器50d、电源组件50e、音频组件50f等其它组件。图5中仅示意性给出部分组件，并不意味着终端设备只包括图5所示组件。
这里需要说明的是：本实施例提供的终端设备可实现上述图1a方法实施例中描述的技术方案，上述各模块或单元具体实现的原理可参见上述各方法实施例中的相应内容，此处不再赘述。
本申请一示例性实施例提供存储有计算机程序或指令的计算机可读存储介质，当计算机程序或指令被处理器执行时，致使处理器能够实现以上方法中的步骤，此处不再赘述。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
在一个典型的配置中，计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。
还需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、商品或者设备中还存在另外的相同要素。
以上所述仅为本申请的实施例而已，并不用于限制本申请。对于本领域技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本申请的权利要求范围之内。

Claims (14)

  1. 一种商品信息处理方法，适用于第一用户的终端设备，其特征在于，所述方法包括：
    显示第二用户的三维3D人体模型，所述3D人体模型是根据第二用户的多张图像进行三维重建得到的；
    选择候选商品，将所述候选商品渲染至所述3D人体模型上，以得到所述候选商品与所述3D人体模型的融合效果；
    根据所述候选商品与所述3D人体模型的融合效果，从所述候选商品中选择与所述第二用户适配的目标商品。
  2. 根据权利要求1所述的方法，其特征在于，显示第二用户的三维3D人体模型，包括：
    获取第二用户的隐式3D表征模型，所述隐式3D表征模型是根据所述第二用户的多张图像进行基于神经网络的三维重建得到的，是对所述第二用户的人体的三维隐式表征；根据所述第二用户的隐式3D表征模型，渲染出所述第二用户的3D人体模型；
    或者
    获取第二用户的人体网格mesh模型，根据所述人体mesh模型渲染出所述第二用户的3D人体模型。
  3. 根据权利要求1所述的方法，其特征在于，获取第二用户的隐式3D表征模型或人体mesh模型，包括：
    获取用于标识第二用户的隐式3D表征模型或人体mesh模型的模型标识；根据所述模型标识，从服务端获取所述第二用户的隐式3D表征模型或人体mesh模型，所述服务端维护有各个用户的隐式3D表征模型或人体mesh模型；
    或者
    获取所述第二用户的多张图像，根据所述第二用户的多张图像进行基于神经网络的三维重建或传统的三维重建，以得到所述第二用户的隐式3D表征模型或人体mesh模型。
  4. 根据权利要求2所述的方法，其特征在于，根据所述第二用户的隐式3D表征模型，渲染出所述第二用户的3D人体模型，包括：
    根据所述第二用户的多张图像的图像特征，确定所述第二用户对应的空间范围；
    基于所述空间范围和所述隐式3D表征模型生成所述第二用户对应的初始三维模型，所述初始三维模型包括所述第二用户上的表面点；
    将所述初始三维模型上每个表面点对应的第一视线的视角信息的平均值，分别转 换为每个表面点的颜色信息，以得到所述第二用户的3D人体模型，所述第一视线是指每张图像中拍摄到各像素点的视线。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，将所述候选商品渲染至所述3D人体模型上，以得到所述候选商品与所述3D人体模型的融合效果，包括：
    对所述3D人体模型进行特征估计，得到多个人体特征点及其位置信息；
    根据所述多个人体特征点及其位置信息，确定所述3D人体模型上与所述候选商品适配的目标特征点位置；
    将所述候选商品对应的试用模型渲染至所述目标特征点位置上，以得到所述候选商品与所述3D人体模型的融合效果。
  6. 根据权利要求5所述的方法，其特征在于，所述多个人体特征点包括：五官特征点、人体骨骼特征点和手部特征点；
    将所述候选商品对应的试用模型渲染至所述目标特征点位置上，以得到所述候选商品与所述3D人体模型的融合效果，包括以下至少一种操作：
    若所述候选商品为美妆类商品，所述目标特征点位置为五官特征点对应的位置，将所述美妆类商品对应的二维试用模型渲染至所述五官特征点对应的位置上，以得到所述美妆类商品与所述3D人体模型的融合效果；
    若所述候选商品为服饰类商品，所述目标特征点位置为人体骨骼特征点对应的位置，将所述服饰类商品对应的三维试用模型渲染至所述人体骨骼特征点对应的位置上，以得到所述服饰类商品与所述3D人体模型的融合效果；
    若所述候选商品为穿戴类商品，所述目标特征点位置为手部特征点对应的位置，将所述穿戴类商品对应的三维试用模型渲染至所述手部特征点对应的位置上，以得到所述穿戴类商品与所述3D人体模型的融合效果。
  7. 根据权利要求1所述的方法，其特征在于，显示第二用户的三维3D人体模型，包括：在第一界面的第一区域中显示第二用户的3D人体模型，所述第一界面的第二区域中显示有至少一个商品选择控件，不同商品选择控件对应不同的商品类型；
    相应地，选择候选商品，包括：响应于对任一商品选择控件的触发操作，确定展示与被触发的商品选择控件对应的商品类型下的至少一个商品；响应于商品选择操作，确定被选择的商品作为候选商品。
  8. 根据权利要求7所述的方法，其特征在于，所述第一界面的第三区域中显示有分享控件、收藏控件、加购控件和下单控件中的至少一个；
    在从所述候选商品中选择与所述第二用户适配的目标商品之后，所述方法还包括以下至少一种操作：
    响应于对所述分享控件的触发操作，将所述目标商品的链接信息发送给所述第二用户的终端设备，以使第二用户购买所述目标商品；
    响应对所述收藏控件的触发操作，将所述目标商品的链接信息添加至收藏夹中；
    响应对所述加购控件的触发操作，将所述目标商品添加至购物车中；
    响应对所述下单控件的触发操作，对所述目标商品进行下单操作，所述下单操作对应的收货地址为所述第二用户的收货地址。
  9. 一种商品信息处理方法，其特征在于，包括：
    获取用户的隐式三维3D表征模型，所述隐式3D表征模型是根据用户的多张图像进行基于神经网络的三维重建得到的；
    基于所述隐式3D表征模型渲染出所述用户的3D人体模型，并显示所述3D人体模型；
    选择候选商品，将所述候选商品渲染至所述3D人体模型上，以得到所述候选商品与所述3D人体模型的融合效果。
  10. 根据权利要求9所述的方法，其特征在于，还包括：
    从所述候选商品中选择融合效果符合要求的目标商品；以及针对所述目标商品执行以下至少一种操作：
    将所述目标商品的链接信息添加至收藏夹中；
    将所述目标商品添加至购物车中；
    对所述目标商品进行下单操作；
    将所述目标商品的链接信息分享给其它用户，以使其它用户购买所述目标商品。
  11. 一种商品信息处理装置，可应用于第一用户的终端设备，其特征在于，所述装置包括：
    显示模块，用于显示第二用户的三维3D人体模型，所述3D人体模型是根据第二用户的多张图像进行三维重建得到的；
    选择模块，用于选择候选商品；
    渲染模块，用于将所述候选商品渲染至所述3D人体模型上，以得到所述候选商品与所述3D人体模型的融合效果；
    所述选择模块还用于：根据所述候选商品与所述3D人体模型的融合效果，从所述候选商品中选择与所述第二用户适配的目标商品。
  12. 一种商品信息处理装置，其特征在于，所述装置包括：
    获取模块，用于获取用户的隐式三维3D表征模型，所述隐式3D表征模型是根据用户的多张图像进行基于神经网络的三维重建得到的；
    渲染模块，用于基于所述隐式3D表征模型渲染出所述用户的3D人体模型，并显示所述3D人体模型；
    选择模块，用于选择候选商品；
    所述渲染模块还用于：将所述候选商品渲染至所述3D人体模型上，以得到所述候选商品与所述3D人体模型的融合效果。
  13. 一种终端设备，其特征在于，包括：存储器和处理器；所述存储器，用于存储计算机程序，所述处理器与所述存储器耦合，用于执行所述计算机程序，以用于实现权利要求1-10任一项所述方法中的步骤。
  14. 一种存储有计算机程序的计算机可读存储介质，其特征在于，当所述计算机程序被处理器执行时，致使所述处理器能够实现权利要求1-10任一项所述方法中的步骤。
PCT/CN2023/071992 2022-05-27 2023-01-13 商品信息处理方法、装置、终端设备及存储介质 WO2023226454A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210593701.5A CN114841783A (zh) 2022-05-27 2022-05-27 商品信息处理方法、装置、终端设备及存储介质
CN202210593701.5 2022-05-27

Publications (1)

Publication Number Publication Date
WO2023226454A1 true WO2023226454A1 (zh) 2023-11-30

Family

ID=82571336

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/071992 WO2023226454A1 (zh) 2022-05-27 2023-01-13 商品信息处理方法、装置、终端设备及存储介质

Country Status (2)

Country Link
CN (1) CN114841783A (zh)
WO (1) WO2023226454A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114841783A (zh) * 2022-05-27 2022-08-02 阿里巴巴（中国）有限公司 商品信息处理方法、装置、终端设备及存储介质
CN115359192B (zh) * 2022-10-14 2023-03-28 阿里巴巴（中国）有限公司 三维重建与商品信息处理方法、装置、设备及存储介质
CN115358828B (zh) * 2022-10-14 2023-03-28 阿里巴巴（中国）有限公司 基于虚拟试穿的信息处理与交互方法、装置、设备及介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150154691A1 (en) * 2013-12-02 2015-06-04 Scott William Curry System and Method For Online Virtual Fitting Room
KR20200019389A (ko) * 2018-08-14 2020-02-24 주식회사 케이티 가상 피팅 서비스를 제공하는 사용자 단말, 방법 및 가상 피팅 서비스 제공 서버
CN111598657A (zh) * 2020-05-12 2020-08-28 深圳小佳科技有限公司 一种基于图像处理的商品推荐方法、设备及存储介质
CN111681070A (zh) * 2020-05-25 2020-09-18 深圳市商汤科技有限公司 线上商品的购买方法、购买装置、存储装置及购买设备
CN114119855A (zh) * 2020-08-26 2022-03-01 北京新氧科技有限公司 一种虚拟换装方法、装置、电子设备及存储介质
CN114266621A (zh) * 2021-12-16 2022-04-01 阿里巴巴（中国）有限公司 图像处理方法、图像处理系统及电子设备
CN114841783A (zh) * 2022-05-27 2022-08-02 阿里巴巴（中国）有限公司 商品信息处理方法、装置、终端设备及存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW200828043A (en) * 2006-12-29 2008-07-01 Cheng-Hsien Yang Terminal try-on simulation system and operating and applying method thereof
CN106339390A (zh) * 2015-07-09 2017-01-18 中兴通讯股份有限公司 基于人体特征数据的匹配方法及装置
JP2019133611A (ja) * 2018-02-04 2019-08-08 光俊 秋谷 情報ネットワークマネキンシステム
CN109523343A (zh) * 2018-10-16 2019-03-26 深圳希智电子有限公司 购物数据处理方法、装置、计算机设备和存储介质
CN114331602A (zh) * 2021-12-21 2022-04-12 北京三快在线科技有限公司 一种基于迁移学习的模型训练方法、信息推荐方法及装置
CN114119839B (zh) * 2022-01-24 2022-07-01 阿里巴巴（中国）有限公司 三维模型重建与图像生成方法、设备以及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150154691A1 (en) * 2013-12-02 2015-06-04 Scott William Curry System and Method For Online Virtual Fitting Room
KR20200019389A (ko) * 2018-08-14 2020-02-24 주식회사 케이티 가상 피팅 서비스를 제공하는 사용자 단말, 방법 및 가상 피팅 서비스 제공 서버
CN111598657A (zh) * 2020-05-12 2020-08-28 深圳小佳科技有限公司 一种基于图像处理的商品推荐方法、设备及存储介质
CN111681070A (zh) * 2020-05-25 2020-09-18 深圳市商汤科技有限公司 线上商品的购买方法、购买装置、存储装置及购买设备
CN114119855A (zh) * 2020-08-26 2022-03-01 北京新氧科技有限公司 一种虚拟换装方法、装置、电子设备及存储介质
CN114266621A (zh) * 2021-12-16 2022-04-01 阿里巴巴（中国）有限公司 图像处理方法、图像处理系统及电子设备
CN114841783A (zh) * 2022-05-27 2022-08-02 阿里巴巴（中国）有限公司 商品信息处理方法、装置、终端设备及存储介质

Also Published As

Publication number Publication date
CN114841783A (zh) 2022-08-02

Similar Documents

Publication Publication Date Title
US20200380333A1 (en) System and method for body scanning and avatar creation
WO2023226454A1 (zh) 商品信息处理方法、装置、终端设备及存储介质
US11164381B2 (en) Clothing model generation and display system
US11244223B2 (en) Online garment design and collaboration system and method
US11640672B2 (en) Method and system for wireless ultra-low footprint body scanning
US10628666B2 (en) Cloud server body scan data system
GB2564745B (en) Methods for generating a 3D garment image, and related devices, systems and computer program products
US20200066052A1 (en) System and method of superimposing a three-dimensional (3d) virtual garment on to a real-time video of a user
CN111787242B (zh) 用于虚拟试衣的方法和装置
US20110298897A1 (en) System and method for 3d virtual try-on of apparel on an avatar
CN109671141B (zh) 图像的渲染方法和装置、存储介质、电子装置
US11836867B2 (en) Techniques for virtual visualization of a product in a physical scene
US12026843B2 (en) Systems and methods for using machine learning models to effect virtual try-on and styling on actual users
WO2022142816A1 (en) System and method for virtual fitting
CN106779774A (zh) 虚拟试衣系统及虚拟试衣方法
US11948057B2 (en) Online garment design and collaboration system and method
CN110349269A (zh) 一种目标穿戴物试戴方法及系统
WO2018182938A1 (en) Method and system for wireless ultra-low footprint body scanning
CN114445271B (zh) 一种生成虚拟试穿3d图像的方法
CN114339434A (zh) 货品试穿效果的展示方法及装置
US20240161423A1 (en) Systems and methods for using machine learning models to effect virtual try-on and styling on actual users
Tharaka Real time virtual fitting room with fast rendering
WO2021237169A1 (en) Online garment design and collaboration and virtual try-on system and method
Malathi et al. Cost Efficient Virtual Trial Rooms
WO2024033943A1 (en) Method and system for displaying three-dimensional virtual apparel on three-dimensional avatar for real-time fitting

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23810531

Country of ref document: EP

Kind code of ref document: A1