KR102060972B1 - System for virtual wearing of jewelry based on augmented reality and method thereof - Google Patents

Abstract

The present invention relates to a virtual wearing system for jewelry based on augmented reality and a method for the same. The virtual wearing system for jewelry based on augmented reality enables a user to experience virtual wearing of desired jewelry by photographing a human body part wearing the jewelry without installing an exclusive application for virtual wearing of jewelry in a user terminal, to provide a virtual wearing image of the jewelry to the user based on augmented reality. The method for the same includes: an identification cover information extracting step; an identification cover information providing step; a step of providing a jewelry image and jewelry information to a virtual wearing device; and an augmented reality image generating step.

Description

Augmented reality-based jewelry virtual wearing system and its method {SYSTEM FOR VIRTUAL WEARING OF JEWELRY BASED ON AUGMENTED REALITY AND METHOD THEREOF}

The present invention relates to an augmented reality-based jewelry virtual wearing system and method, and more specifically, to provide a user with a virtual reality wearing a jewelry image based on augmented reality, do not install a dedicated app for the virtual wearing jewelry on the user terminal The present invention relates to an augmented reality-based jewelery virtual wearing system and a method for photographing a body part to be worn without a jewel, so that a virtual wearing experience can be performed for a desired jewel.

Virtual reality (VR) is known to require a device, while augmented reality (AR) can be used using a smartphone without a separate device, so it is known that a device-related entry barrier is low from the consumer's point of view. In this context, changes in shopping trends are actively taking place in the way that most of the general public possesses smart phones through augmented reality devices.

In particular, in the case of jewelry, the growth of online sales has steadily reduced the existing direct trading methods, suggesting that demand patterns and shopping methods are changing. In particular, augmented reality e-commerce apps provide consumers with a new shopping experience, and consumers are increasingly using augmented reality e-commerce apps to expand their choices. In addition, shopping using social networking is becoming more common.

In other words, even if the brand awareness is low due to the low barriers to entry and the jewelery market with many competitors, if the product is promoted through the online media with unique design and style, it will be well received in the offline market. In particular, the famous influencers are spreading their influence through social network networks.

Augmented reality is a field of virtual reality (VR) that can be useful for synthesizing virtual objects (jewelry) or information (jewelry information) in the real environment (human body) and making them look like objects in the original environment. Technology. In other words, using augmented reality, by synthesizing a virtual object on the basis of the real world, it is possible to reinforce and provide additional information for information or marketing purposes that are difficult to obtain only in the real world. Unlike virtual reality, which presupposes a complete virtual world, the effect of reality is further increased by combining virtual objects on the environment of the real world.

Although the jewelry virtual wear service is provided based on augmented reality, if a dedicated application program is downloaded and installed on the user terminal to provide augmented reality, the virtual reality is provided even though the augmented reality is provided through the user terminal. Compared to this, the advantages and effects of lowering device entry barriers are halved.

Therefore, the present invention is to provide a virtual virtual wear service based on augmented reality in the user terminal without installing a separate dedicated app on the user terminal.

More specifically, in order to provide a user with a virtual virtual wearing jewelry image on the basis of augmented reality, to shoot (scan) the body parts to wear jewelry without installing a dedicated app for the virtual wearing jewelry on the user terminal We present augmented reality-based jewelry virtual wearing system and method for enabling virtual wearing experience for jewelry.

According to the present invention, by providing the augmented reality image without transmitting the image of the user's body part to the virtual wear server in the virtual wear device, to protect the privacy of the user, and between the virtual wear server and the virtual wear device This has the effect of generating as little traffic as possible.

Next, the prior art existing in the technical field of the present invention will be briefly described, and then the technical matters to be made differently from the prior art will be described.

First of all, Korean Patent No. 1085762 B1 (2011.11.15.) Shows the appearance of wearing virtual jewelry directly in a department store or jewelry store remotely using augmented reality, without having to wear various jewelry items in store. It's about making technology visible. In particular, when the user selects the desired jewelry from the jewelry list and shoots with the identifier attached to the part to be worn, the jewelry is worn using augmented reality, which is suitable for displaying the user image with the selected jewelry attached at the identifier position. A display device and a method thereof are provided.

The prior art allows the user to see the appearance of wearing virtual jewelery in department stores or jewelry stores using augmented reality without having to wear various jewelery items in a store, or wearing virtual jewelery on a web page without visiting the store. By letting you see one aspect, you can buy jewelry that suits you in a short time without any hassle.

In addition, Japanese Patent No. 4880350B2 (2011.12.09.) Allows a virtual mirror image (image on a mirror) to be displayed on the display surface when the system is in operation, and the user is in contact with the display surface by an optical imaging system. In association with each virtual object, one or more marker objects driven by a predetermined or user driven object, the object generator provides corresponding object data to the synthesizing means. The synthesizing means generates corresponding virtual image data including the virtual object. The corresponding virtual image data is then provided to the output means in order to give the user an impression as a mirror image. In addition, to expand the output and / or input functions of the system, for example, speakers, microphones, and other new input / output means may be provided. For example, in the case of selecting a specific virtual object to replace the marker object of the virtual image, the voice recognition system can be mounted on the input / output means in order for the user to control the data manipulation system.

The prior art is to use a marker, when wearing and photographing the marker on the body part, attaching the marker to the body part to wear the jewelry, and shooting, by displaying the jewelry image on the body part where the marker is located on the actual body part By synthesizing and displaying the jewel image on an image, a virtual wearing system for augmented reality is configured.

However, the present invention does not use a marker like the prior arts, but has an identification mark encoding an address for accessing a web server, and if the camera scans without a dedicated app, the identification mark is decoded based on this and connected to a web server. In order to automatically receive a virtual wearing service by accessing a web server from a user terminal, the present invention and the prior art are clearly different in technical features.

In addition, the virtual wear device according to the present invention generates the augmented reality image for the jewelry virtual wear service itself, and at least such information as the jewelry image or jewelry information is supported by the virtual wear server, reducing traffic generation, user O, there is an advantage that can provide virtual wearing service while protecting privacy.

The present invention was created to solve the above problems, without having to install a dedicated app for the virtual wearing the jewelry on the user terminal, attaching the identification mark rather than the conventional marker on the body to wear jewelry to the user terminal By scanning, the aim is to enable a virtual wearing experience for the desired jewelry based on augmented reality.

In another aspect, the present invention using the identification mark including access information, jewelry information, identification mark information for the virtual wearing server for providing augmented reality image to the user terminal, the user automatically scans the identification mark to the user terminal virtual It is connected to the wear server, receives the jewelry image and jewelry information from the virtual wearing server to generate and output the virtual wearing augmented reality image in the user terminal, providing augmented reality-based jewelry virtual wearing service to the user through the user terminal For that purpose.

In addition, the present invention to minimize the traffic with the virtual wearing server, and to protect the user's anonymity and privacy without providing the user's body image to the virtual wearing server in the user terminal, and also install a separate dedicated app Another purpose.

Augmented reality-based jewelery virtual wearing method according to an embodiment of the present invention, the identification label information extraction step of extracting the access information and the identification mark information for the virtual wearing server by scanning the identification mark of the jewelry products with the camera in the virtual wearing device Accessing the virtual wearing server with the extracted access information, and providing identification label information for providing the jewelry information and the identification label information to the virtual wearing server; the jewelry image and the jewelry corresponding to the identification label information in the virtual wearing server. And providing information to the virtual wearing device, and generating an augmented reality image by synthesizing the jewel image and the body image of the user in the virtual wearing device.

Wherein the augmented reality image includes information of the specific jewelry product, and the information includes the use, wearing, carrying, possession, placement, display, delivery, delivery, distribution, or a combination thereof. And a method for combining, wherein the identification mark includes access information and identification mark information for the virtual wear server, wherein the identification mark information is stored in association with a jewelry image and jewelry information for a particular jewelry product. If the user scans the identification mark with the camera of the virtual wear device, the virtual wear server is automatically connected to the virtual wear server, the identification mark information is provided, and the virtual wear server is applied to the virtual wear according to the identification mark information provided. Provide necessary jewelry images and jewelry information to the virtual wearing device.

The augmented reality-based jewelry virtual wearing method, the virtual wearing device recognizes a change to the virtual wearing information of the user, and transmitting the jewelry image and jewelry information according to the changed virtual wearing information to the virtual wearing server and And transmitting the jewelry image and the jewelry information to the virtual wear device according to the changed virtual wear information in the virtual wear server.

In addition, the augmented reality image generation step, extracting a jewelry image for a particular jewelry product from the identification mark information, by combining the extracted jewelry image and the actual image of the body of a specific user, generating the augmented reality image The jewelry image includes a 2D image, a 3D image, an animation image, or a combination thereof.

The augmented reality-based jewelery virtual wearing method further includes a body image extraction step of removing a band portion including an identification mark from the image taken by the camera in the virtual wearing device and extracting a body image of the user, wherein the band portion Removing may be performed by replacing the band with an image of a portion where the band is located or an adjacent portion of the body image of the user, or by replacing the band with a specific color component.

In addition, the augmented reality-based jewelry virtual wearing system according to another embodiment of the present invention, receiving a connection request from the plurality of virtual wearing devices that recognize the identification mark according to the access information included in the identification mark, the corresponding identification mark information And a jewelry image processing unit for extracting jewelry information according to the received identification label information, and a jewelry image providing unit for extracting jewelry images according to the received identification label information. Scans the identification mark with a camera and automatically accesses the virtual wearing server, provides the identification mark information, and displays the jewelry image and jewelry information necessary for the virtual wearing by the virtual wearing server according to the provided identification mark information. Provided as a virtual wearable device, combined with the user's body image augmented reality image Characterized in that the sex and output by providing the AR based wear jewelery virtual service.

The augmented reality-based jewelry virtual wearing system, when the virtual wearing device recognizes a change to the virtual wearing information of the user, and transmits the jewelry image and jewelry information according to the changed virtual wearing information to the virtual wearing server, the The method may further include transmitting, by the virtual wear server, the jewelry image and the jewelry information to the virtual wear device according to the changed virtual wear information.

In addition, the augmented reality-based jewelry virtual wearing system, receives a command for jewelry search or recommendation from the virtual wear device, and provides the corresponding jewelry information by searching for jewelry based on artificial intelligence for the requested search or recommendation The electronic device may further include a jewelry search recommending unit that recommends the jewelry according to the characteristics of a specific user and provides a corresponding recommendation list to the virtual wear device.

In addition, the augmented reality-based jewelry wearing device according to another embodiment of the present invention, the image capturing unit for recording the band and the body image of the user wearing the band, identification from the photographed body image Identification mark recognition unit for extracting the access information and the identification mark information for the virtual wearing server by recognizing the cover, the virtual wearing server connection unit for connecting to the virtual wearing server using the extracted connection information, the virtual based on the extracted identification label information A jewelry image receiving unit for receiving a jewelry image provided by a wear server, a jewelry information receiving unit for receiving jewelry information provided by the virtual wearing server based on the extracted identification mark information, and the received jewelry image and jewelry information of the body image Augmented reality image generating unit to generate an augmented reality image by combining with Characterized in that it also.

The augmented reality-based jewelry virtual wear device, the virtual wear device recognizes a change to the virtual wearing information of the user, and transmits the jewelry image and jewelry information according to the changed virtual wear information to the virtual wear server, the And receiving the jewelry image and the jewelry information according to the changed virtual wearing information in the virtual wear server.

The augmented reality image generating unit may include generating the augmented reality image by extracting a jewelry image for a particular jewelry product from the identification mark information and combining the extracted jewelry image with a reality image of a specific user's body. The jewelry image may include a 2D image, a 3D image, an animation image, or a combination thereof.

The augmented reality-based jewelery virtual wearing device further includes a body image separation unit for removing a body part to extract the body image of the user to remove the band portion including the identification mark from the image taken by the camera in the virtual wearing device, Removing the band portion may be performed by replacing the band portion with an image of a portion or an adjacent portion in which the band is located in the body image of the user, or by replacing the band portion with a specific color component.

As described above, the augmented reality-based jewelery virtual wearing system and method of the present invention do virtual wearing experience for jewelery by augmented reality by scanning body parts to be worn by the user's terminal without visiting the jewelry store directly. It can be effective.

In addition, the present invention in particular by attaching an identification mark rather than a conventional marker on the body parts to wear jewelry, by photographing the body parts, without the user installing a dedicated app for virtual jewelry wearing on his user terminal, There is an effect that you can experience the virtual wear on the jewelry you want.

In addition, the present invention, without providing the user's body image to the virtual wearing server in the user terminal, by accessing the virtual wearing server receives the jewelry information and the jewelry image, by generating and outputting augmented reality image itself, virtual wearing server and Minimize traffic and protect user's anonymity and privacy while providing augmented reality-based jewelry virtual wearing service.

1 is a conceptual diagram of providing or receiving a jewelry virtual wearing service through an augmented reality based jewelry virtual wearing system according to an embodiment of the present invention.
FIG. 2 is a table showing in detail jewelry information and identification mark information used in the virtual reality wearing system based on augmented reality according to an embodiment of the present invention.
3 is a conceptual diagram illustrating a flow in which a virtual wear device and a virtual wear server provide a jewelry virtual wear service in an augmented reality based jewelry virtual wear system according to an embodiment of the present invention.
Figure 4 is an exemplary view showing the configuration of the band used for jewelry virtual wearing according to an embodiment of the present invention.
5 is a signal flow diagram illustrating a process in which a virtual wear server and a virtual wear device are connected to each other and repeatedly provide a virtual wear service according to an embodiment of the present invention.
Figure 6 is a block diagram showing the configuration of the virtual wearing server in the augmented reality-based jewelry virtual wearing system according to an embodiment of the present invention.
Figure 7 is a block diagram showing the configuration of the virtual wearing device in the augmented reality based jewelry virtual wearing system according to an embodiment of the present invention.
8 is a flowchart of providing a jewelry virtual wearing service through an augmented reality based jewelry virtual wearing system according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the augmented reality-based jewelry virtual wearing system and method of the present invention. Like reference numerals in the drawings denote like elements. In addition, specific structural to functional descriptions of the embodiments of the present invention are only illustrated for the purpose of describing the embodiments according to the present invention, and unless otherwise defined, all terms used herein including technical or scientific terms These have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and are not construed in ideal or excessively formal meanings unless expressly defined herein. It is preferable not to.

1 is a conceptual diagram of providing or receiving a jewelry virtual wearing service through an augmented reality based jewelry virtual wearing system according to an embodiment of the present invention.

As shown in FIG. 1, the augmented reality-based jewelery virtual wearing system 10 according to an embodiment of the present invention is a wired state in which a jewelery virtual wearing server 100 and a virtual wearing device 200 are connected to a communication network. Alternatively, the virtual wearing device 200 having a wireless communication function receives the jewelry image and the jewelry information from the virtual wearing server 100, generates and displays an augmented reality image, thereby enabling a virtual wearing experience.

That is, the virtual wearing device 200 scans an identification mark worn by a user and automatically connects to the virtual wearing server 100, and provides the information included in the identification mark to the virtual wearing server. The wearing server provides the jewelry image and the jewelry information to the virtual wearing device with reference to the identification mark. The virtual wearing device generates augmented reality image by synthesizing the user's body image and the jewelry image and the jewelry information. . This process is processed and serviced in real time.

The main function supported by the virtual wearing server 100 generates and manages information on the identification mark provided to the user, and supports the connection when the virtual wearing device requests the connection based on the access information included in the identification mark. . In this case, the identification mark is distributed to the user through online and offline. When a plurality of virtual wear devices access the virtual wear server according to the present invention through the identification mark, the connection (connection) is enabled, and the user interface is provided so that the user can be provided with the AR-based virtual wear service. . In addition, it provides a database interface and a server (device) interface for the user to access the database 300 and the server through the virtual wearing device, respectively.

In addition, in the present invention, the virtual wearing server 100 provides a jewelry image to the virtual wearing device as a two-dimensional, three-dimensional and animated image. In particular, when the user changes posture or moves, the jewel image must also be changed. The jewel image is tracked by the user's movement, and the jewel image according to the movement is provided as an animation. When the jewelry image is provided, jewelry information or a jewelry list may also be provided. The jewelry list is listed by the jewelry name, jewelry identification number, thumbnail, and the like.

Furthermore, according to the present invention, the virtual wear server 100 provides a similar jewelry search and recommendation service. Since the similar jewelry search and recommendation is performed based on the image, classify according to the characteristics of the jewelry through machine learning, including CNN (Convolutional Neural Network), RNN (Recursive Neural Network), and learning model for each It generates and allows you to search for the desired jewelry through the generated learning model. In addition, by learning the combination of the body image (fashion) of the specific user and wearing the jewelry as a learning model, through the learned learning model provides a service for recommending jewelry to a specific user. The body image of the user includes a fashion image of the user. Therefore, the body image of the user includes a user image of wearing clothes.

The virtual wearing device 200 includes various terminals used by users who receive the virtual wearing service by accessing the virtual wearing server 100 such as the user terminal 210 and the seller terminal 220. That is, the virtual wearing device 200 may be a seller terminal 220 which is a terminal that conducts wholesale and retail business for a general user terminal 210 or jewelry, and there is no limitation. Those who use the virtual wearing device 200 include a jewelry distributor, a jewelry producer, a jewelry broker, in addition to a general user and a jewelry seller including a jewelry consumer. The virtual wearable device may play any role as long as it is a user terminal capable of wired / wireless communication and connected to a camera.

The database 300 related to the jewelry includes an augmented reality database 310, an identification mark database 320, and the like. The augmented reality database 310 may be not only a two-dimensional image of the jewelry, but also an animation or a three-dimensional image, and includes design information, history information, feature information, price information, AS information, and the like related to a particular jewelry. . In addition, the identification mark database 320 includes jewelry information and identification mark information for a particular jewelry product. It may further include various other information, there is no limitation.

FIG. 2 is a table showing in detail jewelry information and identification mark information used in the virtual reality wearing system based on augmented reality according to an embodiment of the present invention.

As shown in FIG. 2, the identification label information includes a unique identification number, a production date of the identification label, a production site, a manufacturer, and price information. In addition, the identification label information includes more information related to the identification label, and it is preferable that the identification label information represents information as a merchandise sold on or offline by the band including the identification label.

In addition, the jewelry information includes jewelry type, unique identification number, wearing type, design information, production information and the like. In addition, a lot of information can be included, if the features are equal, there is no limit. The identification label information and the jewelry information may cross search for mutual information by a unique identification number. This is why it contains a unique identification number in common. Of course, the only managed name can cross-search each other.

3 is a conceptual diagram illustrating a flow in which a virtual wear device and a virtual wear server provide a jewelry virtual wear service in an augmented reality based jewelry virtual wear system according to an embodiment of the present invention.

As shown in FIG. 3, the user scans the identification mark with the camera of the virtual wearing device 200 which is the user terminal while wearing a band including the identification mark on the part of the body where the user wears the jewelry (①). Here, the band may be a ring, bracelet, necklace, earrings, tiara, or the like, and is manufactured to be attached to or worn on a finger, a cuff, a neck, an ear, a head, or the like. Accordingly, the identification label information is information about the band, and the band can also be commercialized and sold.

For example, the identification mark can be produced in a band as shown in FIG. Figure 4 is an exemplary view showing the configuration of the band used for jewelry virtual wearing according to an embodiment of the present invention.

4 is one embodiment and is designed to be attached or worn on a finger, cuff, neck, ear, head, or the like. For example, you can virtually wear a watch instead of a bracelet.

The band includes various kinds of colors, patterns, rulers, and the like, so that the user can measure the size of the actual jewelry when the user actually wears the jewelry. The virtual wearing device or the virtual wearing server can recognize information including color, pattern, ruler, and the like provided in the band, so that when the user purchases jewelry, the information is different from the actual wearing through the virtual wearing. Make sure that

In the case of the color, it may be recognized as a preferred color for the user's jewelry selection, and the color may be recognized as specific beneficial information to indicate the manufacturer or price of the band in the virtual wear device and the virtual wear server. In addition, a serial number may be added to an identification mark included in the band to make it impossible to reproduce the band. The line spacing (ruler) displayed on the band provides useful information for measuring the finger thickness, neck thickness, wrist thickness, head thickness, etc. of the actual user.

The identification mark is scanned to recognize the access address and identification mark information of the virtual wear server (②), and from this, a virtual wear server including a URL (Uniform Resource Locator), a URI (Uniform Resource Identifier) or an IP (Internet Protocol) address. Connect to the virtual wear server with a connection address for 100. In addition, the identification mark information is transmitted to the virtual wearing server (③). In addition, the virtual wear server performs a device authentication procedure (③). However, the authentication procedure may not be performed as an option.

The virtual wear server 100 authenticates the virtual wear device that has requested a connection, and then provides the jewelry image and jewelry information to the corresponding device, and then repeatedly provides the virtual wear service to the virtual wear device. This process is described in detail with reference to FIG.

5 is a signal flow diagram illustrating a process in which a virtual wear server and a virtual wear device are connected to each other and repeatedly provide a virtual wear service according to an embodiment of the present invention.

As shown in FIG. 5, the virtual wearing device 200 is distributed to the user for free or free of charge and scans (shoots) an identification mark (for example, a QR code) from a band attached to the user's body and wears the virtual wearing server 100. Internet access information and identification label information of Access to the virtual wear server based on the obtained access information, and transmits the identification mark information together. The virtual wear server 100 has a function of a web server, and in response to a connection request of the virtual wear device, stores the received identification mark information, and confirms the existence of an identification key with the virtual wear device. (Request to execute key () function) Here, the key () function performs a function of requesting that the web server provided in the virtual wear server request the virtual wear device to search for the existence of an identification key and return it.

In the virtual wear device, the key () function is executed according to a markup (markup-xml, html, etc.) command received from the virtual wear server. Doing this in the key () function checks for the existence of a private key (private_key) on the virtual wear device. Then, if the identification key exists, the identification key is assigned to the key value and transmitted to the virtual wear server. If not, the identification key (eg, null) is assigned to the key value and transmitted. The virtual wear server may authenticate the virtual wear device through the identification key.

Through this process, the virtual wear device 200 authenticates the virtual wear server 100 and provides information required for the virtual wear device 200.

After completing the device authentication process, the virtual wear server 100 generates a jewelry image, jewelry information, and a jewelry list according to a command of a mark-up (markup-xml, html, etc.) language to the virtual wear device 200. It provides, and sends the necessary application so that the jewelry image is displayed in accordance with the user's body image. The virtual wearing device executes the application so that the virtual wearing device can generate and output an augmented reality image.

In this way, the virtual wear device connected to the virtual wear server for the first time and output the default jewelry virtual wear augmented reality image. After this process, the virtual wear device and the virtual wear server do not need to maintain a session, and the virtual wear device experiences the virtual wear of the jewelry with the application and the jewel image, the jewelry information, and the jewelry list provided from the virtual wear server. .

Then, if the user changes the virtual wearing information such as changing the state of the jewelry, changing the body image, or selecting another jewelry from the jewelry list, to further expand the jewelry virtual wearing experience on the virtual wearing device, the changed virtual wearing information is virtualized. Ask the wear server. As a result, the virtual wearing device and the virtual wearing server are connected again, which is possible by using previous connection information.

The virtual wearing server 100 extracts the required jewel image and the jewelery information from the database according to the request and provides it to the virtual wearing device, and matches the new jewel image and the jewelery information with the body image to the augmented reality image. Create and print

As this process is continuously repeated, the virtual wearing service is repeatedly provided interactively.

Next, with reference to FIG. 3, it demonstrates continuously. That is, the virtual wear server 100 recognizes the identification mark transmitted from the virtual wear device 200 to authenticate the access address, or stores or recognizes the identification mark information and processes it (④), using the identification mark information. With reference to the identification mark database 320 (⑤), and requests a default jewelry image, jewelry information and jewelry list to the augmented reality database 310 (⑥).

The virtual wear server 100 provides the default jewelry image, jewelry information, and jewelry list to the virtual wear device 200 (⑦). Subsequently, the virtual wearing device captures (scans) the body image of the user wearing the band (⑧), removes the band from the body image, synthesizes the default jewelry image, generates an augmented reality image, and outputs the same. Jewelery information is also output according to the application command or outputs only the augmented reality image (⑨).

The virtual wearing device 200 changes the virtual wearing information according to the user's movement or the user's jewelry selection while the virtual wearing service of the jewelry is provided (⑩). The virtual wearing device transmits the changed virtual wearing information to the virtual wearing server 100 to request a new jewelry image, jewelry information, and a jewelry list (⑪).

Subsequently, the virtual wearing server receives and processes new jewelry images and jewelry information requests (⑫), requests new jewelry images, jewelry information, and jewelry lists from the database 300 (에), and the new jewelry images and jewelry information. And providing a jewelry list to the virtual wearing device (iii).

The virtual wearing device may generate and output an augmented reality image from the provided jewelery image, the jewelery information, and the jewelery list, output new jewelry information, and list the updated jewelry list (⑮). By repeating this process, the jewelry virtual wearing service is provided.

Here, the virtual wearing device 200 generates augmented reality image by outputting a synthetic image to the user's body image by requesting the jewelry image, jewelry information stored in the augmented reality database 310 through the virtual wearing server 100, but generated The augmented reality image may be stored in the augmented reality database 310 to be reused. In addition, the reusable augmented reality image can be stored and managed in the augmented reality database.

In addition, the augmented reality database 310 stores not only two-dimensional images of jewelry, but also animated images, three-dimensional images, or a combination thereof. The animation image of the jewelry is synthesized such that the three-dimensional image of the jewelry prepared in advance is continuously updated according to a change in the coordinates of movement of the user's body part. To this end, the 3D image of the jewelry is stored in association with the coordinates of the body part wearing the jewelry, and is classified and stored according to the type of jewelry, the body part, the size information, and the like. Therefore, the 3D animation image of the jewel image can be extracted and displayed immediately.

Next will be described the configuration of the virtual wearing server in the augmented reality based jewelry virtual wearing system according to the present invention.

Figure 6 is a block diagram showing the configuration of the virtual wearing server in the augmented reality-based jewelry virtual wearing system according to an embodiment of the present invention.

As shown in FIG. 6, the virtual wearing server 100 of the augmented reality-based jewelery virtual wearing system according to the present invention includes an identification mark information processing unit 110, a jewelry information processing unit 120, a web server 130, and a jewelry image. It includes a provider 140 and a jewelry search recommender 150.

The identification label information processing unit 110 receives identification label information from at least one virtual wearing device, and separates information necessary for extracting corresponding jewelry information. The identification mark information may include a unique identification number for identifying jewelry and a serial number for distinguishing the identification mark. It may also contain a variety of information beyond the above-described range. In other words, it includes information about the band. Based on this information, the user can extract the jewelry image, the jewelry information, and the jewelry list to be used for the virtual wearing.

The identification mark according to the present invention enables to immediately read the type and size of the jewelry and the position of the user's jewelry wearing position by the information encoded and recorded on the identification mark without using a complicated image recognition algorithm. There is an advantage.

The jewelry information processor 120 stores and manages various jewelry information including jewelry type, wear type, design information, production information, and the like in association with the identification mark information.

Since the identification mark information is stored together with the virtual wearing device information and stores the body image of the user and the corresponding identification mark according to the information of the virtual wearing device, the virtual wearing device and the body part of the user through the identification label information. Are related to each other.

In addition, the jewelry information processing unit 120 serves to provide the user with the same type of jewelry for the identified jewelry or to classify the jewelry selected by the user and store the jewelry in the augmented reality database 310. That is, the jewelry information may be exchanged with at least one virtual wear device, and a jewelry list may be provided according to a user's request.

A plurality of the virtual wearable devices may transmit and receive information to and from the virtual wear server through the web server 130. When the virtual wearing device is connected for the first time, the device is authenticated and provided with identification mark information, which provides a path for providing necessary jewelry images, jewelry information, and jewelry lists. In addition, when a new request is received from the virtual wear device, it provides the processing module with the result and serves to provide the result back to the virtual wear device.

The jewelry image providing unit 140 searches for an augmented reality database 310 based on the identification mark information and the new virtual wearing information to provide necessary jewelry images, jewelry information, jewelry lists, and the like.

The jewel image and the body image may change according to a user's movement. By predicting such a change, the synthesis of the jewel image according to the user's movement is performed in the virtual wearing device.

It is also possible to provide a corresponding 2D jewelery image as a jewelery image according to the movement of the user, or may provide a 3D jewelery image corresponding to the user's body. The 3D image is animated while changing according to the movement of the user. Of course, by continually changing only important parts of jewelry, users can experience various jewelry virtual wearing in a short time. Through the provision of such a jewelry image, the virtual wearing device may generate an augmented reality image.

The virtual wearing device may generate the augmented reality image by receiving a corresponding jewelry image and combining a reality image of a specific user's body with respect to a particular jewelry product extracted from the identification mark information.

The jewelry search recommender 150 searches for similar jewelry for the jewelry selected by the user, and recommends jewelry according to the user's characteristics. Since the search and recommendation of jewelry can be performed based on the image, classification of the jewelry according to the characteristics of the jewelry through machine learning, including CNN (Convolutional Neural Network) and RNN (Recursive Neural Network), etc. Create a learning model. The user may search for the desired jewelry through the generated learning model. In addition, it learns the combination of the image of the specific user and the wearing of jewelry, and recommends jewelry suitable for the specific user through the learning model.

When the user's body information is anonymized, the user's body is classified and stored according to parts, sizes, genders, and races of the user's body, and the preferred jewelry is matched according to the classification through machine learning. In this case, the preferred jewelry is recommended based on the image of the body part of the specific user.

Subsequently, the virtual wearing device receives the jewelry image, the jewelry information and the jewelry list from the virtual wearing server, and describes the virtual wearing device for synthesizing and outputting the body image photographed by the camera.

Figure 7 is a block diagram showing the configuration of the virtual wearing device in the augmented reality based jewelry virtual wearing system according to an embodiment of the present invention.

As shown in FIG. 7, the virtual wearing device 200 according to the present invention includes an image photographing unit 210, an identification mark recognition unit 220, a virtual wearing server connection unit 230, and a body image separating unit 240. , A jewelry image receiver 250, a jewelry information receiver 260, an augmented reality image generator 270, and a display 280.

The image capture unit 210 scans an identification mark with a camera provided in the virtual wearing device or takes a picture of a user's body image. The image capturing unit 210 is performed through an image capturing application provided as a basis of the virtual wearing device.

The identification mark recognition unit 220 recognizes the identification mark displayed on the band attached to the body part of the user, which is scanned (photographed) by the camera through the image capturing unit 110, thereby recognizing a URL (Uniform Resource Locator) and a URI (Uniform Resource). An access address for the virtual wear server 100 including an identifier or an IP (Internet Protocol) address is extracted.

The virtual wear server connection unit 230 communicates with the virtual wear server 100, transmits identification label information, receives a default jewelry image and jewelry information thereof, and then transmits and receives interactive data. To perform.

The body image separator 240 removes the band from the body image including the band input through the image capturing unit 210 and separates only the body image.

In the past, a band is attached to a user's body part, and the band may be removed from the user's body image to synthesize a new jewelry image with the body image. Usually, the jewel image may be overlapped with the band and synthesized, but in this case, it is difficult to generate a harmonious composite image. Therefore, the jewel image is synthesized on the user's body part with the band removed to increase the effect of augmented reality. Specifically, it is preferable to synthesize a jewel image based on a texture image of a user's body part. Alternatively, it may be better to replace the body image without wearing the band, but may replace the portion where the band is located with the same image of the user's skin, clothes, and the like and then synthesize the jewelry image.

The jewelry image receiver 240 requests, receives, and buffers a jewelry image from the virtual wearing server 100 in real time, and places the jewelry in the body image so that the jewelry image is suitable for being synthesized with the actual body image. It is desirable to index and sort according to the storage. The jewel image is divided into body parts (left finger, middle finger, etc.) and positions to be worn in various sizes. This detailed image should be combined with the user's body part.

It is also possible to prepare a corresponding 2D jewelery image as a virtual wearing image according to a user's movement, and may prepare to display the 3D jewelery image in combination with a 3D image of the user's body. The 3D image is prepared in advance so as to be animated while changing according to the movement of the user. Of course, by continually changing only important parts of jewelry, users can experience various jewelry virtual wearing in a short time.

The jewelry image generating unit 270 generates an augmented reality image by combining the body image received by the body image separating unit 240 and the image for a particular jewelry product received by the jewelry image receiving unit 250. do. Wherein the augmented reality image includes information of the specific jewelry product, and the information includes the use, wearing, carrying, possession, placement, display, delivery, delivery, distribution, or a combination thereof. Methods for combinations.

In addition, the jewel image and the body image may change according to the movement of the user. The jewel image is synthesized according to the movement of the user by predicting such a change. The predicted synthetic image may also be stored in the augmented reality database 310.

Accordingly, it is also possible to synthesize a corresponding two-dimensional jewel image according to the user's movement to an augmented reality image, and to combine the three-dimensional jewel image with a three-dimensional image of the user's body. The 3D image is animated according to a predicted position while changing according to the movement of the user. It also involves constantly changing important parts of the jewelry so that the user can experience various virtual wearing of jewelry in a short time.

That is, the augmented reality image generating unit 270 extracts an image of a particular jewelry product from the identification mark and combines the extracted image with a reality image of a specific user's body to generate the augmented reality image. Do this. The augmented reality image generated in this way is output through the display unit of the virtual wearing device.

Hereinafter will be described with respect to the flow of providing a virtual wear service in the augmented reality-based virtual wear system according to the present invention.

8 is a flowchart of providing a jewelry virtual wearing service through an augmented reality based jewelry virtual wearing system according to an embodiment of the present invention.

As shown in FIG. 8, the virtual wear device 200 scans the identification mark of the jewelry product with the camera to extract the access information and the identification mark information for the virtual wear server (S110). The virtual access server is connected to the extracted access information, and the jewelry information and the identification mark information are provided to the virtual wear server (S120). Here, the identification mark is printed on a ring, a necklace, a bracelet, and an earring made in the form of a band or a sticker, and the band may be printed with a color, a ruler, etc., which may include various kinds of information in addition to the identification mark.

Next, the jewelry image and the jewelry information corresponding to the identification mark information are extracted from the virtual wear server and provided to the virtual wear device (S130).

The virtual wearing device removes the band portion including the identification mark from the body image photographed by the camera and extracts the user's body image (S140).

Finally, the virtual wearing device synthesizes the jewel image and the user's body image to generate an augmented reality image, and outputs it to a display for display.

The generated augmented reality image is generated in the virtual wearing device is output through the display, the process that does not go through the virtual wearing server has the advantage that the user's body image is leaked to invade privacy or the fear of personal information leakage There is this. In the virtual wear server, jewelry images that require a large amount of computation such as 2D, 3D, and animation are pre-processed or input and stored in a database. Therefore, the virtual wear device uses minimal computing power to generate augmented reality images. It can be done.

That is, the present invention is automatically structured by the user terminal connected to the virtual wear server, and receives and performs the application for synthesizing the augmented reality image for virtual wear, it is not necessary to provide the user's body image to the virtual wear server . This minimizes traffic to the virtual wear server and protects the anonymity and privacy of the user.

As described above, the augmented reality-based jewelery virtual wearing system and method of the present invention by virtually scanning the jewelery by augmented reality by scanning the body parts to be worn by their user terminal without having to visit the jewelry store directly There is an effect that you can experience.

In addition, the present invention in particular by attaching an identification mark rather than a conventional marker on the body parts to wear jewelry, by photographing the body parts, without the user installing a dedicated app for virtual jewelry wearing on his user terminal, There is an effect that you can experience the virtual wear on the jewelry you want.

Although the above has been described above with reference to a preferred embodiment according to the present invention, the technical idea of the present invention is not limited thereto, and each component of the present invention is changed or modified within the technical scope of the present invention for achieving the same object and effect. Could be.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

10: Virtual reality wearing system based on augmented reality
100: virtual wear server 200: virtual wear device
300: database 110: identification label information processing unit
120: jewelry information processing unit 130: web server
140: jewelry image providing unit 150: jewelry search recommendation unit
210: Image capturing unit 220: Identification mark recognition unit
230: virtual wearing server connection unit 240: body image separation unit
250: jewelry image receiving unit 260: jewelry information receiving unit
270: augmented reality image generation unit 280: display unit

Claims (15)

In the augmented reality based jewelry wearing method,
An identification label information extraction step of extracting the connection information and the identification label information for the virtual wearing server by scanning the identification mark of the jewelry product with the camera in the virtual wearing device;
An identification mark information providing step of accessing the virtual wear server with the extracted connection information from the virtual wear device and providing the identification mark information to the virtual wear server;
Providing, at the virtual wear server, a jewelry image and jewelry information corresponding to the identification mark information to the virtual wear device; And
Augmented reality image generation step of generating an augmented reality image by synthesizing the jewel image and the user's body image in the virtual wearing device;
The augmented reality image generation step,
Extracting the jewelry image for a particular jewelry product from the identification mark information, and combining the extracted jewelry image with a reality image of a specific user's body to generate the augmented reality image,
The jewelry image,
It includes two-dimensional image, three-dimensional image, animated image or a combination thereof,
The identification mark,
Includes access information and identification mark information for the virtual wear server,
The augmented reality-based jewelry virtual wearing method,
When the user scans the identification mark with the camera of the virtual wearing device, the user is automatically connected to the virtual wearing server, provides the identification mark information, and the jewel image required for the virtual wearing by the virtual wearing server according to the provided identification mark information. And virtual jewelry wearing method for providing virtual jewelry to the virtual wearing device. The method according to claim 1,
The augmented reality image,
Information about a particular jewelery product, the information including a method for use, wearing, carrying, carrying, placing, displaying, delivering, delivering, distributing, or a combination thereof, or a combination thereof. Augmented reality-based jewelry virtual wearing method characterized in that. The method according to claim 1,
The augmented reality-based jewelry virtual wearing method,
In the virtual wear server, further comprising the step of authenticating the virtual wear device;
Authenticating the virtual wear device,
In the virtual wear server, in response to the connection request of the virtual wear device, with reference to the provided identification label information, to store the predetermined identification key to the virtual wear device, or the presence of the identification key to be stored Requesting confirmation; And
In the virtual wear device, the identification key is automatically searched for the confirmation request, and if the identification key exists, the searched identification key is returned to the virtual wear server. null) returning a value;
Augmented reality-based jewelry virtual wear method characterized in that the virtual wearing server, by authenticating the virtual wearing device by placing the identification key in the virtual wearing device and requesting the identification key. The method according to claim 1,
The augmented reality-based jewelry virtual wearing method,
In the virtual wear device, recognizing a change in the virtual wear information of the user and transmitting the changed virtual wear information to the virtual wear server; And
And transmitting, by the virtual wear server, new jewelry images and jewelry information to the virtual wear device according to the changed virtual wear information. The method according to claim 1,
The augmented reality-based jewelry virtual wearing method,
At the virtual wear server, further providing a jewelry list according to the identification mark information provided from the virtual wear device;
In the virtual wear device, the user selecting a particular jewelry from the provided jewelry list; And
In the virtual wearing server, the step of searching for and recommending similar jewelry for the selected jewelry; further comprising,
The recommendation of the similar jewelery includes: learning a combination of wearing jewelry according to a user's characteristics through machine learning to generate a learning model, and recommending the jewelry to a specific user through the generated learning model. Augmented reality-based jewelry virtual wearing method. The method according to claim 1,
The identification mark is printed on a band or sticker made in the form of a ring, necklace, bracelet or earring,
The band or sticker, augmented reality-based jewelry virtual wearing method, characterized in that it further comprises additional information including a specific color, pattern, ruler (ruler), serial number, or a combination thereof in addition to the identification mark. In the augmented reality-based jewelry virtual wearing system,
An identification mark information processing unit which receives a connection request from the plurality of virtual wearing devices recognizing the identification mark according to the connection information included in the identification mark, and receives the identification mark information;
A jewelry information processor extracting jewelry information according to the received identification mark information; And
And a jewelry image providing unit configured to extract a jewelry image according to the received identification mark information.
In the virtual wear device, the identification mark is scanned by a camera and automatically connected to the virtual wear server to provide the identification mark information, and according to the provided identification mark information, the virtual wearing server requires a jewel image and the virtual wear. Providing jewelry information to the virtual wearing device, generates and outputs an augmented reality image by combining with the user's body image,
The jewelry image includes a 2D image, a 3D image, an animation image, or a combination thereof.
The identification mark, augmented reality-based jewelry virtual wearing system, characterized in that it comprises access information and identification mark information for the virtual wearing server. delete The method according to claim 7,
The augmented reality virtual jewelry wearing system,
When the virtual wear device recognizes a change to the virtual wear information of the user and transmits the jewel image and the jewelry information according to the changed virtual wear information to the virtual wear server, the virtual wear server according to the changed virtual wear information. Augmented reality-based jewelery virtual wearing system further comprising the step of transmitting the jewelry image and jewelry information to the virtual wearing device. The method according to claim 7,
The augmented reality virtual jewelry wearing system,
Receiving a command for a jewelry search or recommendation from the virtual wear device, and for the requested search, the corresponding jewelry is searched to provide corresponding jewelry information, and for the requested recommendation, the jewelry according to the characteristics of a specific user The augmented reality-based jewelry virtual wearing system further comprises; jewelry search recommendation unit for providing a recommendation list to the virtual wearing device to recommend. An image capturing unit for photographing a body image of a user wearing a band or a sticker made in the form of a ring, necklace, bracelet or earring including an identification mark;
An identification mark recognition unit recognizing the identification mark in the photographed body image and extracting access information and identification mark information for the virtual wearing server;
A virtual wear server connecting unit which connects to the virtual wear server using the extracted access information;
A jewelry image receiver configured to receive a jewelry image provided by the virtual wearing server based on the extracted identification mark information;
A jewelry information receiver configured to receive jewelry information provided by the virtual wear server based on the extracted identification mark information; And
And augmented reality image generating unit for generating an augmented reality image by combining the received jewelry image and jewelry information with the body image.
The augmented reality image generating unit,
Extracting a jewelry image for a particular jewelry product from the identification mark information, and combining the extracted jewelry image with a reality image of a specific user's body, thereby generating the augmented reality image;
The jewelery image, augmented reality-based jewelery device, characterized in that it comprises a two-dimensional image, three-dimensional image, animation image or a combination thereof. The method according to claim 11,
The augmented reality image,
Information about a particular jewelery product, the information including how to use or wear, carry, carry, carry, place, display, deliver, deliver, distribute, or a combination thereof. Augmented reality-based jewelry virtual wearing device characterized in that. The method according to claim 11,
The augmented reality-based jewelry virtual wearing device,
The virtual wear device recognizes a change in the virtual wear information of the user, and transmits the jewelry image and the jewelry information according to the changed virtual wear information to the virtual wear server, and according to the changed virtual wear information in the virtual wear server. Augmented reality-based jewelery virtual wearing device comprising receiving the jewelry image and jewelry information. delete The method according to claim 11,
The augmented reality-based jewelry virtual wearing device,
And a body image separator for removing a band or sticker portion including an identification mark from the image photographed by the camera in the virtual wearing device and extracting a body image for extracting a body image of the user.
Removing the band or sticker portion may be removed by replacing the band or sticker with an image of a portion or an adjacent portion where the band or sticker is located in the body image of the user,
Augmented reality-based jewelry virtual wear device, characterized in that by removing the band or sticker portion by replacing a specific color component.

Images