KR102646540B1 - Method for providing unmanned goods service - Google Patents

Abstract

The present invention discloses a method for providing unmanned product services. The unmanned product service providing method includes receiving user data in which the user's personal information is recorded, receiving a selection of a product from the user, and virtually fitting the product to the user, wherein the virtual fitting is performed by the user. A step including generating a virtual fitting image by combining the deformed appearance of and the image of the product, receiving a purchase request for the product, and the location of an affiliated store selling a linked product for the product and the location of the user. It includes the step of providing a list of affiliated stores using .

Description

{Method for providing unmanned goods service}

The present invention relates to a method of providing unmanned product services using a deep learning module. Specifically, the present invention relates to an unmanned product service providing method that can generate a virtual fitting image by combining the user's image with the image and modified appearance of the product selected by the user.

The content described in this section simply provides background information for this embodiment and does not constitute prior art.

Recently, as the Internet has developed, the number of people selling and purchasing products online has increased, and online stores are growing larger. Accordingly, sellers are registering various types of products in online stores, and buyers are purchasing products online more frequently than offline.

In the past, buyers would try on glasses frames offline, purchase the frames that best fit them, measure their visual acuity, and then purchase glasses lenses.

However, as online stores expanded, buyers had a wider choice of glasses frames, but the satisfaction rate with the products was low because they could not try on the frames themselves. Accordingly, there was a need to increase the buyer's satisfaction rate with the product by combining the glasses frame with the buyer's image and virtually fitting it to the buyer.

Additionally, when purchasing a spectacle frame, there was the inconvenience of having to go to an offline store where one could purchase spectacle lenses that can be combined with the spectacle frame. Accordingly, there is a need for a system that can provide an offline store that sells eyeglass lenses for eyeglass frames using the buyer's eyeglass frame information.

The object of the present invention is a method of providing an unmanned product service that creates a virtual fitting image by virtually fitting a product selected by a user in an online store to the user's image, and allows direct fitting of the virtually fitted product at an unmanned store. is to provide.

In addition, the purpose of the present invention is to provide a method of providing an unmanned product service that generates a virtual fitting image by modifying the deformable appearance of the user included in the user image and combining the image of the product with the user image with the modified appearance. .

Additionally, the purpose of the present invention is to provide an unmanned product service providing method that recommends products that are likely to be preferred by the user and items with a changeable appearance using deep learning based on user data and fitting data.

Additionally, the purpose of the present invention is to provide an unmanned product service providing method that can recommend an offline affiliated store where a user can purchase eyeglass lenses that can be combined with the eyeglasses frame purchased.

The objects of the present invention are not limited to the objects mentioned above, and other objects and advantages of the present invention that are not mentioned can be understood by the following description and will be more clearly understood by the examples of the present invention. Additionally, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof indicated in the patent claims.

An unmanned product service providing method according to an embodiment of the present invention includes receiving user data in which the user's personal information is recorded, receiving a selection of a product from the user, and virtually fitting the product to the user, The virtual fitting includes generating a virtual fitting image by combining the user's deformed appearance and an image of the product, receiving a purchase request for the product; and providing a list of affiliated stores using the location of the affiliated store selling a product linked to the product and the location of the user.

In addition, the step of generating a virtual fitting image by combining the user's deformed appearance and the image of the product includes receiving the user's image through an application installed on the user's user terminal, and applying the user's image to the user's image. It may include creating the modified appearance by applying a transformation and combining the received image of the product with the modified appearance.

In addition, the product calculates the expected preference of each product using virtual fitting data including the user data and the user's virtual fitting history, and is a product recommended to the user based on the calculated expected preference. may include

In addition, the step of generating a virtual fitting image by combining the user's deformed appearance and the image of the product includes receiving the user's image through a kiosk and applying a deformation to the user's image to create the deformed appearance. It may include generating and combining an image of the product with the modified appearance.

In addition, the user's actual fitting process captured through a camera is included, image data associated with the actual fitting data is received, and the actual fitting data is used together with virtual fitting data containing the user's virtual fitting history to It may further include calculating the user's expected preference.

Additionally, the product may include a product recommended to the user based on the predicted preference.

Additionally, the actual fitting data may include at least one of the user's fitting history, fitting time, or fitting order extracted from the image data.

In addition, the step of generating a virtual fitting image by combining the user's deformed appearance and the image of the product includes combining an external image with the user's image, wherein the external image is applied to hairstyle, makeup, clothing, accessories, or a mask. It may include at least one of the images for

Additionally, the step of combining the user's external image may include displaying a category for the external image and collecting items of the external image using crawling for the category.

Additionally, the step of providing the affiliated store list may include calculating the distance between the user's location and the affiliated store, and generating the affiliated store list by listing the affiliated stores in descending order of the calculated distance.

The unmanned product service providing method of the present invention can receive a user's image through an application and create a virtual fitting image by combining the user's image with the image of the product selected by the user. Through this, the user's satisfaction with product selection and purchase can be increased by virtually fitting the product. Additionally, by reducing the manpower and costs required to operate a store, an increase in the store's profitability can be expected.

In addition, the unmanned product service providing method of the present invention can display a virtual appearance category for a changeable appearance in a user image and combine the appearance image with the user image based on selection and reception of items in the virtual appearance category. Next, a virtual fitting image can be created by combining the product image with the user image whose appearance has been modified. Accordingly, fashion completion and user satisfaction can be improved by checking and purchasing products that match the hairstyle, makeup, clothing, and accessories.

In addition, the unmanned product service provision method of the present invention can derive expected preferences for each user using a deep learning module based on user data and fitting data, and generate product lists and appearance items based on the derived expected preferences. there is. Through this, products that reflect the user's preferences can be recommended to save the user's time in purchasing products and improve user convenience and satisfaction.

In addition, the unmanned product service providing method of the present invention can provide a list of affiliated stores that sell spectacle lenses that can be combined with the spectacle frame based on the user's spectacle frame information by listing them in order of distance adjacent to the user. Through this, the effect of automatically connecting online and offline stores can be expected to improve the user's convenience in purchasing eyeglass lenses and increase the user's service utilization rate for online stores.

In addition to the above-described content, specific effects of the present invention are described below while explaining specific details for carrying out the invention.

1 is a conceptual diagram illustrating an optical store system that performs an unmanned product service providing method according to an embodiment of the present invention.
Figure 2 is a block diagram for explaining each component of the optical store system of Figure 1.
Figure 3 is a flowchart for explaining a method of providing unmanned product services according to some embodiments of the present invention.
FIG. 4 is an example diagram for explaining a method of generating a virtual fitting image in step S130 of FIG. 1.
FIG. 5 is an example diagram illustrating a method of generating a virtual fitting image by modifying the user's appearance in step S130 of FIG. 1.
FIG. 6 is an example diagram illustrating a method of generating a virtual fitting image through a kiosk in step S130 of FIG. 1.
FIG. 7 is a flowchart explaining a method of deriving actual fitting data through the camera of FIG. 6.
FIG. 8 is an example diagram illustrating the affiliated store list provided in step S150 of FIG. 1.
Figure 9 is a diagram showing the configuration of a deep learning module used in the process of deriving a product to recommend to the user in step S110 of Figure 1.

Terms or words used in this specification and patent claims should not be construed as limited to their general or dictionary meaning. According to the principle that the inventor can define terms or word concepts in order to explain his or her invention in the best way, it should be interpreted with a meaning and concept consistent with the technical idea of the present invention. In addition, the embodiments described in this specification and the configurations shown in the drawings are only one embodiment of the present invention and do not completely represent the technical idea of the present invention, so they cannot be replaced at the time of filing the present application. It should be understood that there may be various equivalents, variations, and applicable examples.

Terms such as first, second, A, and B used in the present specification and claims may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term 'and/or' includes any of a plurality of related stated items or a combination of a plurality of related stated items.

The terms used in the specification and claims are merely used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "include" or "have" should be understood as not precluding the existence or addition possibility of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification. .

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the present invention pertains.

Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

Additionally, each configuration, process, process, or method included in each embodiment of the present invention may be shared within the scope of not being technically contradictory to each other.

Hereinafter, with reference to FIGS. 1 to 9, we will look in detail at a method of providing unmanned product services to users based on user data and fitting data.

1 is a conceptual diagram illustrating an optical store system that performs an unmanned product service providing method according to an embodiment of the present invention.

Referring to FIG. 1, an optical store system according to some embodiments of the present invention may include an optical store server 100, a user terminal 200, an affiliated store management server 300, and an affiliated store terminal 400.

The optical store server 100 (hereinafter referred to as server) according to the present invention can receive user data, product data, and fitting data in which the user's personal information is recorded. At this time, the user data may include the user's age, gender, region, and preferred color. However, this embodiment is not limited to this.

Product data may include sales product data and linked product data. The sales product data may be data on products sold by the server 100 (e.g., eyeglass frames), and the linked product data may be data on linked products (e.g., eyeglass lenses) of products sold at an affiliated store. .

Specifically, the server 100 may receive linked product data through the affiliated store management server 300, which manages affiliated stores that sell linked products. The affiliated store management server 300 may transmit linked product data received from the affiliated store terminal 400 to the server 100. Here, a plurality of affiliated stores can pre-register linked products available for sale for each affiliated store on the affiliated store terminal 400. The affiliated store management server 300 may provide linked product data registered by the affiliated store terminal 400 to the server 100 in real time. The affiliated store management server 300 may transmit the user's purchase product data received through the user terminal 200 to the affiliated store terminal 400.

Additionally, the fitting data may include virtual fitting data and/or actual fitting data. Here, the virtual fitting data may include the user's virtual fitting history through an application and/or kiosk, and the actual fitting data may be the actual fitting history of the user directly fitting a product in an unmanned store.

The server 100 may derive a product list using user data, product data, and fitting data. At this time, the server 100 can calculate the user's expected preference for each product using deep learning, and list and display the products based on the calculated expected preference. The server 100 may receive the user's selection of a product displayed in the product list.

Next, the server 100 may receive the user image through an application or kiosk. The server 100 may generate a virtual fitting image by combining the user image with the image of the product selected by the user.

Additionally, the server 100 may recognize a deformable appearance of the user image and display a virtual appearance category. The virtual appearance category may be a category for appearance items that the server 100 can modify by virtual fitting among the user's appearance included in the user image. Here, the virtual appearance category may include hairstyle, makeup, clothing, accessories, and mask categories.

At this time, the server 100 may collect data about appearance items to be displayed for each virtual appearance category from an external server through crawling. Here, crawling is a technology that collects information by extracting data distributed across numerous web pages, and is also called scraping. External information includes information accessible through portal site search information (e.g., Google, etc.), SNS (e.g., Facebook, etc.), communities (e.g., Naver Cafe), and personal blogs.

The server 100 may calculate the user's preference for appearance data collected through crawling, list appearance items in order of preference, and display them in a virtual appearance category.

Subsequently, the server 100 may generate a virtual fitting image by combining the appearance image corresponding to the appearance item selected by the user with the user image. At this time, the virtual fitting image may also include an image of the product. The server 100 may store the virtual fitting history as virtual fitting data in the user account.

When creating a virtual fitting image through a kiosk in an unmanned store, the server 100 may further provide actual fitting based on the user's request for the product.

Specifically, the server 100 can directly provide the product to the user by unlocking the storage box of the product for which the fitting request has been received. Users can actually try on the product.

At this time, the server 100 may receive video data of the user's actual fitting process through a camera installed in the unmanned store. The server 100 may analyze the received image data and extract actual fitting data including fitting details, fitting time, or fitting order. The server 100 may store the extracted actual fitting data in the user account.

Subsequently, the server 100 may receive a purchase request for a product included in the product list. The server 100 may deliver the product to the user's delivery address included in the user data, but the present invention is not limited thereto.

Subsequently, the server 100 may receive affiliated store data of an affiliated store that sells a linked product for the product purchased by the user from the affiliated store management server 300. The server 100 can calculate the distance between the user and the affiliated store using the location of the affiliated store and the user's location included in the affiliated store data. The server 100 may generate a list of affiliated stores based on the calculated distance and provide it to the user.

At this time, the server 100 and the user terminal 200 may be implemented as a server-client system. The server 100 can transmit and receive data with the user terminal 200 through a wired or wireless network.

Here, the terminal application may be a dedicated application for providing a product list and a virtual fitting image, or a web browsing application for providing a product list and a virtual fitting image through a web page. Here, the dedicated application for providing product lists and virtual fitting images may be an application built into the user terminal 200 or an application downloaded from an application distribution server and installed on the user terminal 200.

That is, the user terminal 200 may provide the product list and virtual fitting image provided from the server 100 to the user through a terminal application installed on the user terminal 200.

Hereinafter, for convenience of explanation, it is assumed that the user terminal 200 displays the interface, product list, and virtual fitting image provided from the server 100 on the screen using a pre-installed terminal application (hereinafter referred to as application). Let's do it.

At this time, the user terminal 200 refers to a communication terminal capable of operating a terminal application in a wired or wireless communication environment. The user terminal 200 may be a user's portable terminal. In FIG. 1, the user terminal 200 is shown as a smart phone, a type of portable terminal, but the present invention is not limited thereto, and can be applied without limitation to any device capable of mounting a terminal application as described above. . For example, the user terminal 200 may include various types of electronic devices such as a personal computer (PC), laptop, tablet, mobile phone, smartphone, wearable device (e.g., watch-type terminal), and kiosk. .

In addition, although only one user terminal 200 is shown in the drawing, the present invention is not limited thereto, and the server 100 may operate in conjunction with a plurality of user terminals 200.

Additionally, the user terminal 200 includes an input unit that receives user input, a display unit that displays visual information, a communication unit that transmits and receives signals to and from the outside, processes data, and controls each unit inside the user terminal 200. It may include a control unit that controls data transmission/reception between units. Hereinafter, the commands that the control unit executes within the user terminal 200 according to the user's commands are collectively referred to as those performed by the user terminal 200.

Meanwhile, the communication network 500 serves to connect the server 100, the user terminal 200, and the affiliated store management server 300. In other words, the communication network 500 refers to a communication network that provides a connection path so that user terminals 200 can connect to the server 100 and the affiliated store management server 300 and then transmit and receive data. The communication network 500 is, for example, a wired network such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), and ISDNs (Integrated Service Digital Networks), wireless LANs, CDMA, Bluetooth, satellite communication, etc. may cover wireless networks, but the scope of the present invention is not limited thereto.

More specifically, in an embodiment of the present invention, the server 100 can calculate expected preference using deep learning based on user data, generate a product list based on the expected preference, and provide it to the user terminal 200. there is.

In this process, the server 100 may provide a product list using artificial neural networks based on user data, virtual fitting data, and actual fitting data. A detailed description of the deep learning module using an artificial neural network will be described in detail later with reference to FIG. 9.

Below, the specific configuration of the optician's system will be described in detail.

Figure 2 is a block diagram for explaining each component of the optical store system of Figure 1.

Referring to FIG. 2, the server 100 may include a data management unit 110, a product management unit 120, a fitting unit 130, a deep learning unit 140, and a control unit 150.

Specifically, the data management unit 110 may store and manage user data and fitting data.

Here, the user data may include the user's personal information such as the user's age, gender, region, preferred color, and preferred design.

Fitting data may include virtual fitting data and/or actual fitting data. Here, virtual fitting data may include the user's virtual fitting history through applications and kiosks.

Additionally, actual fitting data may include the user's actual fitting history for products for which actual fitting has been requested through the kiosk. Actual fitting data may include at least one of the user's fitting history, fitting time, or fitting order. At this time, actual fitting data can be received through a camera installed in a store equipped with a kiosk.

The product management unit 120 may store and manage product data provided from the optical store and affiliated store management server 300. Here, product data may include sales product data and linked product data.

Sales product data may be data about glasses frames and lenses sold at an optical store. Sales product data may include product name, price, color, design, and product image.

Linked product data may be data about spectacle lenses pre-registered in the affiliated store management server 300. Linked product data may include the type, price, and sales franchise information of eyeglass lenses.

The fitting unit 130 can fit a product to a user.

Specifically, when a product selection is received from the user, the fitting unit 130 may receive product data for the selected product from the product management unit 120. Next, the fitting unit 130 may receive the user's image. The fitting unit 130 may generate a virtual fitting image by combining the user's image with the product image of the product data.

Additionally, the fitting unit 130 may further modify the user's appearance displayed in the user's image. At this time, the deformable appearance may include hairstyle, makeup, clothing, accessories, and mask. When a fitting request for an appearance is received from a user, the fitting unit 130 may generate a virtual fitting image by combining the user's image with the appearance image selected by the user.

As another example, when an actual fitting request for a product is received through a kiosk in an offline unmanned store, the fitting unit 130 may receive product data for the selected product from the product management unit 120. Next, the fitting unit 130 can unlock the storage box based on the product number of the product data and provide the actual product that can be directly fitted to the user.

Here, the fitting unit 130 may receive video data of the user's fitting process through a camera installed in the unmanned store. The fitting unit 130 may transmit the received image data to the deep learning unit 140.

The deep learning unit 140 can analyze user preferences for products using deep learning. Specifically, expected preferences for products can be calculated using a deep learning module including artificial neural networks based on user data, fitting data, and product data.

Additionally, the deep learning unit 140 may extract actual fitting data by analyzing the image data received from the fitting unit 130. Details about this will be explained in detail in FIGS. 6 and 7.

The control unit 150 may derive a product list based on the expected preference calculated through a deep learning module and provide it to the user terminal 200. Additionally, a list of affiliated stores that sell linked products can be derived from the product list based on the product selected by the user.

Below, a method for providing unmanned product services according to an embodiment of the present invention will be described.

Figure 3 is a flowchart for explaining a method of providing unmanned product services according to some embodiments of the present invention. FIG. 4 is an example diagram for explaining a method of generating a virtual fitting image in step S130 of FIG. 1. FIG. 5 is an example diagram illustrating a method of generating a virtual fitting image by modifying the user's appearance in step S130 of FIG. 1. FIG. 6 is an example diagram illustrating a method of generating a virtual fitting image through a kiosk in step S130 of FIG. 1. FIG. 7 is a flowchart explaining a method of deriving actual fitting data through the camera of FIG. 6. FIG. 8 is an example diagram illustrating the affiliated store list provided in step S150 of FIG. 1.

Here, <a1> in FIG. 4 represents a product list provided to the user, and <a2> represents a virtual fitting image in which the product is virtually fitted. Additionally, <b1> in FIG. 5 represents a virtual fitting interface that can change the user's appearance, and <b2> represents a virtual fitting image in which the user's appearance has been completely modified.

Hereinafter, the description will be made assuming that the product is a glasses frame.

Referring to Figures 1 to 8, the server 100 receives user data (S110). Here, the server 100 may receive user data based on the user's login information. The server 100 may receive fitting data for the user based on user data. Subsequently, the server 100 may generate a product list (ML) using user data, fitting data, and product data.

For example, when receiving a user's login to an application, the server 100 may receive user data, virtual fitting data, and product data for the user. Based on user data, virtual fitting data, and product data, the server 100 can analyze the user's preference for products using a deep learning module (DL). The server 100 may calculate the user's expected preference for each product included in the product data. Based on the expected preference, the server 100 may display products to be recommended to the user in the product list (ML).

As another example, when receiving a user's login to a kiosk, the server 100 may receive user data, virtual fitting data, actual fitting data, and product data. The server 100 can generate a product list (ML) by calculating the expected preference for each product through a deep learning module (DL).

Next, the server 100 receives the user's selection of the product (S120).

Specifically, referring to <a1> in FIGS. 1, 3, and 4, the server 100 may display the generated product list (ML). The server 100 may list and display products in order of highest expected preference based on expected preference. The server 100 may display the product list (ML) in the following order: A glasses frame (MO1), B glasses frame (MO2), C glasses frame (MO3), and D glasses frame (MO4).

At this time, the server 100 may display additional product images in the product list (ML). The server 100 includes an A image (MI1), a B image (MI2) corresponding to A glasses frame (MO1), B glasses frame (MO2), C glasses frame (MO3), and D glasses frame (MO4) in the product list (ML). C images (MI3) and D images (MI4) can be displayed.

The server 100 may receive the user's selection of the A eyeglasses frame (MO1) among the A eyeglasses frame (MO1), B eyeglasses frame (MO2), C eyeglasses frame (MO3), and D eyeglasses frame (MO4) displayed in the product list (ML). there is.

Next, the product is virtually fitted to create a virtual fitting image (S130).

For example, referring to <a2> in FIGS. 1, 3, and 4, the server 100 may receive a user image (UI) through an application or kiosk. The server 100 can recognize a face in a user image (UI). The server 100 may generate a virtual fitting image by combining the A image (MI1) of the A eyeglasses frame (MO1) with the user's face in the user image (UI). The server 100 may display the generated virtual fitting image.

However, the present invention is not limited to this, and the server 100 may recognize the user's eyes, ears, or nose of the user image (UI) in various ways and combine the A image (MI1).

Additionally, referring to FIGS. 1, 3, and 5, the server 100 can recognize the appearance of the user image (UI). The server 100 may display a virtual appearance category (OC) for a changeable appearance among the recognized appearances. That is, the virtual appearance category (OC) may include items that can be modified by virtual fitting among the user's appearance included in the user image (UI).

Specifically, the server 100 may recognize hair, makeup, clothing, and accessories as deformable appearances in the user image (UI). The server 100 may display the hair category (HC), makeup category (MC), clothing category (CC), and accessory category (AC) as a virtual appearance category (OC). Subsequently, the server 100 may receive a selection of a virtual appearance category (OC) to be modified from the user.

For example, the server 100 may receive the user's selection for a hair category (HC). When receiving a selection for a hair category (HC), the server 100 may derive a hairstyle to be displayed as a hair category (HC) as a hair item.

The server 100 may collect hairstyles from an external server through crawling. Here, external information includes information accessible through portal site search information (e.g., Google, etc.), SNS (e.g., Facebook, etc.), communities (e.g., Naver Cafe), and personal blogs. .

Next, based on the hairstyle data and user data collected through crawling, the server 100 can calculate the user's hairstyle preference using deep learning. At this time, the server 100 may calculate hairstyle preferences by reflecting the preferences of other users whose data is similar to the user data based on the user's gender, age, and preferred color included in the user data. The server 100 may calculate a ranking for each hairstyle based on hairstyle preference.

Alternatively, the server 100 may crawl hairstyles uploaded to external information for a preset period. The server 100 may calculate a ranking for each hairstyle in order of the amount of hairstyle data collected through crawling.

Alternatively, the server 100 may receive ranked hairstyle data (for example, information on the 1st to 5th most popular men's hairstyles, etc.) through crawling.

Here, the server 100 may calculate the ranking for each hairstyle using at least one of the three methods listed above, but the present invention is not limited thereto.

Subsequently, the server 100 may display first to fourth hairs (HO1 to HO4), which are hairstyle items included in the hair category (HC), based on the calculated rankings for each hairstyle. At this time, hairstyle items may be listed based on the calculated ranking of each hairstyle. The present invention is not limited to this, and hairstyle items may be listed and displayed in various ways, such as listed in name order.

Subsequently, the server 100 may receive a selection for the first hair (HO1) from the user. The server 100 may change the user's hairstyle displayed in the user image (UI) into an image of the first hair (HO1) (for example, a brown dyed hairstyle) and display it.

Subsequently, the server 100 may generate a virtual fitting image by combining the A image (MI1) of the A glasses frame (MO1) that has received a pre-selection with the user's face of the user image (UI) in which the first hair (HO1) is displayed. there is.

However, the present invention is not limited to this, and the server 100 may change the appearance of the user image (UI) while combining the product image with the user image (UI).

Through this, the unmanned product service provision method can transform the hairstyle included in the user's image into the hairstyle desired by the user and create a virtual fitting image by combining the product image with the user image displaying the modified hairstyle image. . Accordingly, changes in the user's image according to changes in the user's hairstyle can be confirmed, and user satisfaction can be increased by allowing the user to select a product that matches the user's image.

Below, only the differences will be described, excluding overlapping content.

As another example, the server 100 may receive the user's selection for a makeup category (MC). When receiving a selection for a makeup category (MC), the server 100 may derive a makeup style to be displayed as a makeup category (MC) as a makeup item. Here, the makeup style may include skin tone and lip color. Hereinafter, skin tone is assumed to be a makeup style.

The server 100 may collect skin tone data from an external server through crawling.

Next, based on the skin tone data and user data collected through crawling, the server 100 can calculate the user's skin tone preference using deep learning. At this time, the server 100 may calculate hairstyle preferences by reflecting the preferences of other users similar to the user data, and calculate rankings by skin tone.

Alternatively, the server 100 may crawl skin tones uploaded to external information for a preset period and calculate a ranking for each hairstyle in order of the amount of skin tone data collected. The server 100 may receive ranked hairstyle data (for example, information on the 1st to 5th most popular male skin tones, etc.) through crawling.

Subsequently, the server 100 may display skin tone items (for example, No. 21 and No. 23, etc.) included in the makeup category (MC) based on the calculated rankings for each hairstyle.

Subsequently, based on the user's selection of the skin tone item, the server 100 may change and display the skin tone for the face of the user image (UI). The server 100 may generate a virtual fitting image by combining the user image (UI) with the changed skin tone with the A image (MI1) of the A glasses frame (MO1) that has received a pre-selection.

Through this, the unmanned product service provision method can create a virtual fitting image by combining the user's image with the modified skin tone and product image. Due to the nature of the product called Glasses Terra, the user's impression can be greatly influenced by skin tone and color matching. Accordingly, the user can easily select a product whose color matches the user's skin tone or the user's preferred skin tone, thereby leading to the purchase of the user's preferred product.

As another example, the server 100 may receive the user's selection for a clothing category (CC). When receiving a selection for a clothing category (CC), the server 100 may derive clothing information to be displayed as a clothing category (CC) as a clothing item. At this time, the clothing category (CC) may include the user's outerwear or top displayed in the user image (UI) as clothing items.

Specifically, the server 100 may collect clothing data through crawling.

Next, based on the clothing data and user data collected through crawling, the server 100 can calculate the user's clothing preference using deep learning. At this time, the server 100 may calculate a ranking for each clothing based on preference. Alternatively, the server 100 may crawl uploaded clothing for a preset period, calculate a ranking for each hairstyle in order of the amount of clothing data collected, or receive clothing data including the ranking.

Next, the server 100 may display clothing items included in the clothing category (CC) (for example, in the case of tops, dress shirts, short-sleeved shirts, long-sleeved shirts, etc.) based on the calculated clothing rankings.

Subsequently, based on the user's selection of the clothing item, the server 100 may change and display the user's clothing image included in the user image (UI). The server 100 may generate a virtual fitting image by combining the A image (MI1) of the A glasses frame (MO1) with the user image (UI) with changed clothing.

Through this, the unmanned product service provision method can create a virtual fitting image by combining the product image with the user's image displayed with transformed clothing. Accordingly, users can easily check products that match the clothes they wear, thereby improving user convenience in product selection.

As another example, the server 100 may receive the user's selection for the accessory category (AC). When receiving a selection for an accessory category (AC), the server 100 may derive an accessory item. At this time, accessory items may include necklaces, hats, and masks. Accessory items may be displayed combined with the user image (UI) regardless of whether or not there are accessories in the user image (UI).

The server 100 may collect accessory data through crawling. Based on accessory data and user data collected through crawling, the server 100 can calculate the user's preference for each accessory using deep learning. Alternatively, the amount of accessory data collected for each accessory can be calculated by crawling the uploaded accessories for a preset period. Additionally, accessory data including rankings can be received.

Next, the server 100 may calculate a ranking for each accessory and display accessory items included in the accessory category (AC).

Based on the user's selection of an accessory item, the server 100 may change the accessory image included in the user image (UI) or combine the accessory images. The server 100 may generate a virtual fitting image by combining the A image (MI1) with the user image (UI) combined with the accessory image.

Through this, the unmanned product service provision method can create a virtual fitting image by combining the user image with the accessory image and product image. Accordingly, the user can create a sense of unity between the accessory and the product by purchasing and wearing a product that matches the accessory they want to wear.

Additionally, the virtual appearance category (OC) may further include a mask category. The server 100 may receive the user's selection of a mask category and derive a mask item. Here, mask items may include 3D three-dimensional masks, bird-beak 2D masks, and flat masks (eg, dental masks and disposable masks). At this time, the mask item may display masks of various colors such as white and black, but the present invention is not limited thereto.

The mask item can be displayed combined with the user image (UI) regardless of whether the user is wearing a mask in the image (UI).

The server 100 may collect mask data through crawling. Based on mask data and user data collected through crawling, the server 100 can calculate the user's preference for each type of mask using deep learning. Next, the server 100 may calculate the ranking for each mask type and display mask items included in the mask category. However, the present invention is not limited thereto and may be implemented in various modifications.

Based on the user's selection of the mask item, the server 100 may change the mask image included in the user image (UI) to the selected mask image. Additionally, if the user image (UI) does not include a mask, the selected mask image can be combined. The server 100 may generate a virtual fitting image by combining the A image (MI1) with the user image (UI) combined with the mask image.

In addition, referring to FIGS. 1, 6, and 7, when a user goes to an unmanned store, the server 100 generates a virtual fitting image of the user through the kiosk (KO) and an image of the product used for virtual fitting. We can provide actual product wear for .

Specifically, the server 100 may receive user account login and user image (UI) through the kiosk (KO). Here, upon logging in to the user account, the server 100 can take pictures of the user using a camera (CM) installed in the unmanned store. However, the present invention is not limited to this, and when the server 100 receives a fitting request for an actual product from a user, it can start taking pictures through the camera (CM).

Subsequently, the server 100 may display the received user image (UI) on the screen of the kiosk (KO). The server 100 may change the appearance for which a change request has been received through the virtual appearance category (OC) displayed on the kiosk (KO). Additionally, a virtual fitting image may be generated by combining a product image (eg, A image (MI1)) for a product selected by the user with a user image (UI) displayed with a modified appearance.

Subsequently, the server 100 may receive an actual fitting request for a product from the user through the kiosk (KO). The server 100 can extract the location of the storage box (OL) where the actual product is stored based on product data for the product for which the fitting request was actually received.

The server 100 can unlock the storage box (OL) corresponding to the extracted location and provide the actual product to the user. Users can actually try fitting the product by wearing the actual product. The server 100 can receive video data of the user's fitting process captured through a camera (CM) installed in an unmanned store.

The server 100 may extract actual fitting data using the received image data.

Specifically, the server 100 receives video data of the actual fitting process through the camera (CM) (S231). The server 100 may receive video data from the time of receiving the user's login to the time of receiving the user's logout through the kiosk (KO).

Next, the image data is analyzed to derive at least one of fitting details, fitting time, or fitting order (S233).

Specifically, the server 100 may perform image analysis using deep learning on the image of the storage box (OL) where the actual product contained in the video data is stored or the image of the actual product worn by the user.

For example, the server 100 may extract an image of the storage box (OL) opened while the user takes out the actual product or an image of the storage box (OL) opened after the actual product is taken out from the video data. Subsequently, the server 100 may extract the location of the open storage box (OL) included in the image. The server 100 may derive the storage box (OL) located in the middle of the second row. The server 100 may derive the product by comparing the actual product storage location included in the product data with the location of the derived storage box (OL).

As another example, the server 100 may extract an image of the actual product worn by the user through video data. Next, the server 100 may compare the image of the actual product with the product image included in the product data to derive the product.

Subsequently, the server 100 can extract all products derived from the user's image data as the user's fitting details. Additionally, the derivation order of products extracted from video data can be extracted as a fitting order.

Additionally, the server 100 can calculate the fitting time using the user's wearing and detaching times for the product. The server 100 may recognize the user's face included in the image data. The server 100 may derive the first time when the actual product is worn on the user's face. That is, the server 100 can extract an image of video data in which the user's face image is combined with the image of the actual product. The server 100 may derive the first time when the image of the extracted video data was captured.

Subsequently, the server 100 may derive a second view of the user detaching and detaching the actual product from the user's face. The server 100 can calculate the fitting time for each product using the first time and the second time.

The derived fitting details, fitting time, and fitting order are saved as the user's actual fitting data (S235). The server 100 may store actual fitting data for each product in the user's account.

Virtual fitting data and actual fitting data for each product can be used later to suggest new products to users or to regularly suggest additional purchase advertisements. In other words, if the predicted preference for a new product is high according to the predicted preference using virtual fitting data and actual fitting data, the server can send a purchase proposal to the user. In this way, when a purchase proposal is transmitted using predicted preference, the purchase rate can be higher than when it is not sent.

Next, the server 100 receives a purchase request for a product (S140).

Here, the server 100 may receive a purchase request for a product through an application or kiosk. The server 100 can request delivery of a product to the user's address included in the user data.

However, the present invention is not limited to this, and when a purchase request is received through a kiosk, the server 100 can check whether the product remains in stock at the unmanned store where the kiosk is installed. If inventory remains, the server 100 can immediately provide the product to the user.

Provides a list of affiliated stores selling product-linked products (S150).

Specifically, referring to FIGS. 1, 3, and 8, the server 100 can derive a linked product linked to a product for which a purchase request has been received from the user. For example, when receiving a purchase request for a glasses frame, the server 100 may derive glasses lenses that can be combined with the glasses frame as a linked product.

Next, the server 100 may derive an affiliated store that sells spectacle lenses based on linked product data received through the affiliated store management server 300. The server 100 can derive A franchise store (FO1), B franchise store (FO2), and C franchise store (FO3) that sell spectacle lenses. At this time, the server 100 can derive an affiliated store that has one or more spectacle lenses in stock included in the linked product data.

Subsequently, the server 100 may receive affiliated store data for affiliated store A (FO1), affiliated store B (FO2), and affiliated store C (FO3). The server 100 can calculate the distance between each affiliated store and the user (hereinafter referred to as distance between them) using the affiliated store location information included in the affiliated store data and the user location information included in the user data.

The server 100 can display the A affiliated store (FO1), B affiliated store (FO2), and C affiliated store (FO3) on the affiliated store list (FL) using the calculated distance. For example, the server 100 may list and display affiliated stores in order of short distance between them. A franchise (FO1) whose distance is calculated to be 0m can be displayed at the top of the franchise list (FL). Subsequently, the server 100 may display the B affiliated store (FO2) whose distance is 50 m and the C affiliated store (FO3) whose distance is 100 m.

Additionally, the server 100 may further display the prices of eyeglass lenses for each affiliated store included in the linked product data in the affiliated store list (FL). The server 100 may set the display order of affiliated stores in the affiliated store list (FL) in order of distance or price according to the user's settings. However, the present invention is not limited thereto.

Figure 9 is a diagram showing the configuration of a deep learning module used in some embodiments of the present invention.

Referring to FIG. 9, the deep learning module (DL) can use an artificial neural network learned based on big data to derive predicted preferences and actual fitting data required for an unmanned product service provision method.

To explain in more detail, Deep Learning technology, a type of Machine Learning, learns at a deep level in multiple stages based on data. In other words, deep learning represents a set of machine learning algorithms that extract key data from a plurality of data at increasing levels.

The deep learning module (DL) can use various known deep learning structures. For example, a deep learning module (DL) may use structures such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Deep Belief Network (DBN), and Graph Neural Network (GNN).

Meanwhile, artificial neural network learning of the deep learning module (DL) can be accomplished by adjusting the weight of the connection line between nodes (adjusting the bias value if necessary) so that the desired output is produced for a given input. Additionally, artificial neural networks can continuously update weight values through learning. Additionally, methods such as back propagation can be used to learn artificial neural networks.

Meanwhile, an artificial neural network previously trained through machine learning may be installed in the memory of the server 100.

In other words, the deep learning module (DL) can perform the operation of outputting predicted preferences based on machine learning, using user data, fitting data, and product data as input data. Additionally, the deep learning module (DL) can perform an operation to output actual fitting data using image data as input data based on machine learning.

At this time, both semi-supervised learning and supervised learning can be used as machine learning methods for artificial neural networks. Additionally, the deep learning module (DL) can be controlled to automatically update the artificial neural network structure for outputting predicted preference and actual fitting data after learning according to settings.

Additionally, although not clearly shown in the drawing, in another embodiment of the present invention, the operation of the deep learning module (DL) may be performed in the server 100 or a separate cloud server (not shown).

Here, the deep learning module (DL) includes an input layer (input) with user data, fitting data, and/or product data as input nodes, an output layer (output) with the user's expected preference as an output node, and an input layer. It includes M hidden layers placed between the and output layers.

Here, weights may be set on the edges connecting the nodes of each layer. The presence or absence of these weights or edges can be added, removed, or updated during the learning process. Therefore, through the learning process, the weights of nodes and edges arranged between k input nodes and i output nodes can be updated.

Before the deep learning module (DL) performs learning, all nodes and edges can be set to initial values. However, when information is input cumulatively, the weights of nodes and edges change, and in this process, the parameters input as learning factors (i.e., user data, fitting data, product data, and/or image data) and output nodes are changed. Matching between assigned values (expected preference and/or actual fitting data) can be achieved.

Additionally, when using a cloud server, the deep learning module (DL) can receive and process a large number of parameters. Therefore, the deep learning module (DL) can perform learning based on massive data.

In addition, the weights of nodes and edges between the input nodes and output nodes constituting the deep learning module (DL) may be updated by the learning process of the deep learning module (DL). In addition, of course, the parameters output from the deep learning module (DL) can be further expanded to various data in addition to expected preferences.

The above description is merely an illustrative explanation of the technical idea of the present embodiment, and those skilled in the art will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are not intended to limit the technical idea of the present embodiment, but rather to explain it, and the scope of the technical idea of the present embodiment is not limited by these examples. The scope of protection of this embodiment should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of this embodiment.

Claims (10)

In a method of providing unmanned product services performed by a server including a deep learning module,
Receiving user data in which the user's personal information is recorded; Calculating expected preference using the deep learning module based on the user data;
Generating a product list based on the expected preference;
providing the product list to the user through a user terminal;
Receiving a selection of a product from the user;
generating fitting data of the user for the selected product;
Receiving a purchase request for the product; and
Including providing a list of affiliated stores using the location of the affiliated store selling a linked product for the product and the location of the user,
Generating the user's fitting data for the selected product,
Including generating virtual fitting data and generating actual fitting data,
Generating the virtual fitting data includes:
Receive the user's image through a kiosk,
Applying transformations to the user's image to create a transformed appearance,
Generating virtual fitting data for the user by combining the image of the selected product with the modified appearance,
Generating the actual fitting data is,
Receiving a request for actual fitting of the selected product from the user through the kiosk,
Extract from the server the location of the storage box where the selected product is stored,
the server unlocks the locker so that the user actually obtains the selected product;
Photographing the actual fitting process of the user's selected product at a location different from the kiosk through a camera,
Including receiving the actual fitting process as image data and generating actual fitting data,
Including using the actual fitting data and the virtual fitting data together to calculate the user's expected preference through the deep learning module,
The actual fitting data includes the user's fitting time extracted from the image data,
The fitting time is calculated using the first time when the user puts on the product to the second time when the user detaches the product,
How to provide unmanned product services.
delete delete delete delete delete delete According to claim 1,
Generating virtual fitting data by combining the deformed appearance and the image of the product,
Combining an appearance image with the user's image, wherein the appearance image includes at least one of an image for a hairstyle, makeup, clothing, accessories, or a mask.
How to provide unmanned product services.
According to clause 8,
Combining the external image with the user's image,
Displays a category for the external image,
Including collecting items of the appearance image using crawling for the category.
How to provide unmanned product services.
According to claim 1,
The step of providing the affiliated store list is,
Calculate the distance between the user's location and the affiliated store,
Including generating the affiliated store list by listing the affiliated stores in descending order of the calculated distance.
How to provide unmanned product services.

Images