KR20230109017A - Method of providing a customized service for a product using a terminal and a service system therefor - Google Patents

Abstract

In a method for providing a product customized service through a terminal according to an embodiment of the present invention, in a state in which a product image customized by a user is output through a screen of the terminal, the product image is displayed through a camera of the terminal. Recognizing a body part for virtual wearing, and generating synthesized image data obtained by synthesizing the product image data with an image corresponding to the recognized body part, and outputting it in real time through the screen of the terminal. , The outputting may be performed when the user clicks a button that is activated when the result of the body part recognition is equal to or higher than a specific level at which the product image can be virtually worn.

Description

Method for providing customized service through terminal and service system therefor

The present invention relates to a method for providing a product customized service, and more particularly, to a user using a terminal to more intuitively and easily select and purchase a desired product through artificial intelligence (AI) and augmented reality (AR). : It relates to a method for providing ICT (Information & Communication Technology) convergence customized services including augmented reality or blockchain, and a service system for providing a service platform therefor.

Conventionally, a consumer visits an offline store, directly checks a product or service provided by a store manager, and then selects/purchases a desired product/service. However, since such a conventional purchase method requires a direct visit to an offline store, there are many limitations of time and space, which cause some inconvenience to consumers.

In recent years, with the development of information and communication technology, online transaction methods are very active. However, when using such an online method, it is possible to solve the problem of the conventional offline method, which is the limitation of time and space, to some extent. There is a new problem of deterioration in purchase satisfaction due to

In addition, current online and offline transactions mainly target only finished products for sale, and thus, there is a limit to the selection of products by consumers, which causes a decrease in satisfaction with the purchase / use of products or services. In relation to this, some services are provided so that parts or parts can be selected separately, but there is still a shortage to solve the above problems, which is not enough to completely solve the needs of consumers. Accordingly, there is a demand for a method for solving the above problems.

Korean Patent Publication No. 10-2020-0032090 (published date: March 25, 2020)

An object of the present invention is to use artificial intelligence (AI), augmented reality (AR) or block-chain (Block-chain) to enable users to select and purchase desired products more intuitively and easily using a terminal. It is to provide a method for providing a customized ICT (Information & Communication Technology) convergence service including a).

Another object of the present invention is to provide a service platform or service system for the method.

The technical problems to be achieved in the present invention are not limited to the above technical problems, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

A method for providing a product customized service through a terminal according to an embodiment of the present invention for solving the above-described problem is a product image customized by a user is output through a screen of the terminal, and the camera of the terminal is output. Recognizing a body part for virtual wearing of the product image and generating synthesized image data obtained by synthesizing the product image data with an image corresponding to the recognized body part and outputting the synthesized image data in real time through the screen of the terminal. The outputting may be performed when the user clicks a button that is activated when the result of the body part recognition is equal to or higher than a specific level at which the product image can be worn virtually.

In the method according to an embodiment of the present invention, guideline data related to the product image is output on the screen of the terminal, and a specific level at which the product image can be worn virtually is determined by the body part corresponding to the guideline data. It may be in a state of being aligned beyond a predetermined standard.

In the method according to an embodiment of the present invention, when the result of recognizing the body part is below a certain level at which the product image can be virtually worn, at least one image frame for the body part is generated for a predetermined time, and then Based on this, the step of generating an image corresponding to the body part and then outputting the image may be performed. In the method according to an embodiment of the present invention, the guideline data may be in the form of either a still image or a moving image. In the method according to an embodiment of the present invention, when the user continuously clicks the activated button for more than a predetermined time, the real-time output is made of a video,

When the video is output in real time, it may be possible to set an effect filter so that output with special effects is applied. In the method according to an embodiment of the present invention, the body part is recognized while the real-time output is being performed If the result of is changed below a certain level at which virtual wearing of the product image is possible, the real-time output may be stopped.

In the method according to an embodiment of the present invention, when the product image data is changed to other product image data during the real-time output, the changed product image data is synthesized with the image corresponding to the body part. Composite image data can be output in real time.

An apparatus for providing a customized product service according to an embodiment of the present invention for solving the above problems includes a data communication unit for transmitting and receiving data with a user's terminal and a control unit for providing the customized product service, The control unit recognizes a body part for virtual wearing of the product image through the camera of the terminal in a state in which a product image customized by the user is output through the screen of the terminal, and corresponds to the recognized body part. Control to generate synthesized image data obtained by synthesizing the product image data with an image and output the result in real time through the screen of the terminal, provided that the result of recognizing the body part is at least a certain level at which virtual wearing of the product image is possible. It may be performed when the user clicks a button that is activated when the user clicks the button.

According to the present invention as described above, the effects described below can be obtained. However, effects obtainable through the present invention are not limited thereto.

First, according to the present invention, a method for providing ICT (Information & Communication Technology) convergence customized services including artificial intelligence (AI), augmented reality (AR) or blockchain is provided Accordingly, there is an effect that the user can more intuitively and easily select and purchase a desired product using the terminal.

Second, according to the present invention, there is an effect that a service platform for the above method can be provided.

1 is a schematic diagram of a product customized service system according to an embodiment of the present invention.
2 is a configuration block diagram of a server according to an embodiment of the present invention.
3 is a detailed configuration block diagram of a product customized service unit within a server according to an embodiment of the present invention.
4 is a diagram illustrating a mobile-based jewelry design, stone material, and metal customized service according to an embodiment of the present invention.
5 is a diagram for explaining a 360-degree rotation image viewer service for mobile use using render farm technology according to an embodiment of the present invention.
6 is a diagram illustrating a mobile-only real-time 3D image viewer service using Web RTC technology according to an embodiment of the present invention.
7 is a diagram for explaining the provision of mobile-based 2D and 3D virtual wearing experience services according to an embodiment of the present invention.
8 is a diagram illustrating provision of a virtual wearing experience service using non-marker-based 2D synthesis technology according to an embodiment of the present invention.
9 is a diagram illustrating an augmented reality (AR) marker band for marker-based virtual wearing experience service according to an embodiment of the present invention.
10 is a diagram for explaining the provision of virtual wearing experience service based on augmented reality (AR) recognition markers according to an embodiment of the present invention.
11 is a diagram illustrating a chatbot service related to providing a customized product recommendation service according to an embodiment of the present invention.
12 is a diagram illustrating a chatbot service related to image analysis and similar product recommendation using image recognition technology according to an embodiment of the present invention.
13 is a diagram for explaining rights management and product assurance service provision using blockchain technology according to an embodiment of the present invention.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. The detailed description set forth below in conjunction with the accompanying drawings is intended to describe exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced.

Only these embodiments are provided to complete the disclosure of the present invention and to fully inform those skilled in the art of the scope of the invention to which the present invention belongs, and the present invention will be defined by the scope of the claims. only

In some cases, in order to avoid obscuring the concept of the present invention, well-known structures and devices may be omitted or may be shown in block diagram form centering on core functions of each structure and device. In addition, the same reference numerals are used to describe like components throughout this specification.

Throughout the specification, when a part is said to "comprising" or "including" a certain element, it means that it may further include other elements, not excluding other elements, unless otherwise stated. do.

In addition, the term "?? unit" described in the specification means a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software. Further, "a or an", "one", and similar related terms, in the context of describing the present invention, are both singular and plural unless otherwise indicated herein or clearly contradicted by context. It can be used in a meaning including.

In addition, specific terms used in the embodiments of the present invention are provided to aid understanding of the present invention, and unless otherwise defined, all terms used herein, including technical or scientific terms, are intended to support the present invention. It has the same meaning as commonly understood by a person of ordinary skill in the art to which it pertains. The use of these specific terms may be changed in other forms without departing from the spirit of the present invention.

In the present specification, a method for providing an ICT (Information & Communication Technology) convergence customized service including artificial intelligence (AI), augmented reality (AR) or block-chain according to the present invention And various embodiments that allow a user to more intuitively and easily select and purchase a desired product using a terminal through a service platform therefor will be described.

In addition, one embodiment of the present specification is that the terminal uses an application provided by the service platform, but is not necessarily limited thereto. These applications can be used by being downloaded and installed by the terminal, and application data can be updated periodically/non-periodically.

The application data may include data such as various images and video UI/UX for product customized services provided by the service platform and various data for application services.

In relation to the present invention, product customized service data may be provided through a service platform, that is, a terminal that communicates data with a server in real time. In this case, the terminal may replace part or all of the role of the server. For example, when the terminal updates related data from the server in advance, the terminal can support the product customized service of the present invention even when it is not connectable to the server, and provides basic services, but through communication with the user. Only when an additional service is required, it may be supported through data communication with the server.

In addition, in the present specification, products are described as examples of jewelry products such as rings, earrings, necklaces, bracelets, etc., to help the understanding of the present invention and for convenience of explanation by the applicant, but are not necessarily limited thereto. Therefore, the present invention can be used for customized services for various products or product groups, such as mobile phones, automobiles, and computers, in addition to the jewelry products.

In addition, as described above, the present invention is an information & communication technology (ICT) convergence customized service including artificial intelligence (AI), augmented reality (AR) or block-chain. As for the artificial intelligence (AI), augmented reality (AR), or blockchain technology, the previously known content may be applied to the description and configuration, and detailed description thereof is omitted in this specification, and only necessary contents are provided. Make the description clear in advance.

Hereinafter, various embodiments according to the present invention will be described with reference to the accompanying drawings.

1 is a schematic diagram of a product customized service system 100 according to an embodiment of the present invention, and for convenience of description, only key components of the product customized service system 100 are shown and described, but the present invention is not limited thereto. no.

Referring to FIG. 1 , a product customized service system 100 may include a terminal 120 and a server 130 .

Although not shown in FIG. 1 , the server 130 is a concept including a payment server or can be linked with an external payment server.

According to the embodiment, although only one terminal 120 is illustrated for convenience in FIG. 1 , a plurality of terminals may be provided in relation to the present invention. In this case, each terminal may directly perform data communication with the server 130 or perform data communication with the server 130 through another terminal or device. In addition, product customized service data related to the present invention may be exchanged between terminals through the server 130 or directly. For example, a customized service for one product can be simultaneously provided through the server 130 even between mobile terminals of two users who are friends or family members from a long distance or a short distance.

Meanwhile, as described above, the terminal 120 and the server 130 may exchange various customized service data according to the present invention through a wired/wireless communication network.

The terminal 120 described in this specification, including FIG. 1, collectively refers to all types of devices capable of providing a product customized service according to the present invention, including fixed terminals and mobile terminals.

The terminal 120 will be described as including a display for outputting or providing product customized service data, but is not necessarily limited thereto. For example, it is sufficient if the terminal 120 can be connected to an external display and through which the product customized service data can be output. At this time, the display is not limited according to the size or shape of the display, and it is sufficient as long as it can output data for the product customized service according to the present invention to the user. In addition, the display may include a touch screen, and may include other functions such as a keyboard, a mouse, and a stylus to meet user convenience.

Meanwhile, the terminal 120 is described as including at least one image capturing device for capturing the object 110 in relation to the present invention, but is not necessarily limited thereto. For example, the terminal 120 may or may not have an external image capture device attached to the front, rear, side, etc. of the terminal. At this time, the image capture device collectively refers to means for acquiring object data such as an image sensor and a camera sensor, and is hereinafter referred to as a camera sensor for convenience.

According to an embodiment, the terminal 120 includes a configuration for or supporting biometric input and/or output such as voice, gesture, text, touch, iris, etc. for or related to the product customized service according to the present invention. can do. According to an embodiment, the terminal 120 may support the service through a combination of at least two or more of the above inputs for the product customized service according to the present invention.

According to an embodiment, the terminal 120 may mean a device including sensing modules for sensing various data for a product customized service according to the present invention. Here, the sensing module may include an image capturing sensor such as a camera sensor.

When a plurality of terminals exist in a wired/wireless network, each terminal does not need to have the same type, property, or performance. However, according to the embodiment, it is preferable to satisfy the minimum specifications required to properly process and provide the product customized service data according to the present invention.

As the terminal 110, at least one of a display for outputting image data and a speaker for outputting audio data, such as a smartphone, PC, tablet PC, TV, monitor, wearable device, device dedicated to product customized services, etc. Any device can be included. However, the scope of the present invention is not limited by the above examples.

According to an embodiment, the terminal 110 includes a terminal device, a terminal, a mobile station (MS), a mobile subscriber station (MSS), a subscriber station (SS), an advanced mobile station (AMS), and a wireless terminal (WT). ), at least one of a machine-type communication (MTC) device, a machine-to-machine (M2M) device, and a device-to-device (D2D device) device.

The aforementioned terminal is only an example, and the terminal in the present invention should be construed as a concept including all devices capable of transmitting data or signals that are currently developed and commercialized or will be developed in the future, in addition to the above-described examples.

In addition, the terminal 120 may include an application program interface (API) for processing application data. In this case, the application data may include data related to the application for the product customized service according to the present invention downloaded and installed in the terminal 120 and data updated later.

Depending on the embodiment, the product customized service according to the present invention may be provided through the execution of an application installed in the terminal 120 or may be provided in the form of a web service such as a web page or a progressive web app. However, in this specification, for convenience of description, the provision of a product customized service using an electronic application, that is, a description will be given.

Therefore, after the terminal 120 executes the application, the terminal 120 transmits (or uploads) data about or for the product customized service according to the present invention to the server 130, and includes the response data and related service data. may be received (or downloaded) from the server 130.

Next, the server 130 refers to hardware/software that generates, processes, stores, or supports product customized service data according to the present invention.

Depending on the embodiment, the server 130 may be named and described as a processor, etc., but it should be noted in advance that the limited interpretation should not be based on the name.

The server 130 may process or support various data for or related to the product customized service platform according to the present invention.

According to an embodiment, the server 130 may include a Cloud server, an IP Multimedia Subsystem (IMS) server, a Telephony Application (Telephony Application) server, an Instant Messaging (IM) server, a Media Gateway Control Function (MGCF) server, It may be any one of a Messaging Gateway (MSG) server, a Call Session Control Function (CSCF) server, and the like.

In addition, even if it is only described as a 'terminal' for convenience in this specification, it can be interpreted in a plurality of meanings depending on the context and/or embodiment, and the same applies to 'server'.

Meanwhile, in the wired/wireless communication network, various product customized service data according to the present invention, such as text, digital image, digital video, and digital voice information, are transmitted between the components of the product customized service system 100. It refers to a network that supports communication between the terminal 120 and the server 130, and the type is not particularly limited.

Depending on the embodiment, the wired/wireless communication network may be an Internet Protocol (IP) network supporting mass data communication through Internet Protocol (IP) or an All IP network integrating different IP networks. there is.

According to the embodiment, the wired / wireless communication network includes a wired network, a mobile communication network including WCDMA, a mobile communication network including High Speed Downlink Packet Access (HSDPA) and Long Term Evolution (LTE), and LTE advanced (LTE-A) It may be formed by one of a mobile communication network, a satellite communication network, and a Wi-Fi network, or a combination of at least two of them. Here, Bluetooth communication may also be supported.

2 is a configuration block diagram of a server 130 according to an embodiment of the present invention, and FIG. 3 is a detailed configuration block diagram of a product customized service part 220 in the server according to an embodiment of the present invention.

Referring to FIG. 2 , the service platform by the server according to the present invention may be largely implemented by including one or more of a product data creation part 210, a product customization service part 220, and a blockchain part 240. The product customized service part 220 may be implemented by including the chatbot service part 225 and the augmented reality part 230, but is not necessarily limited thereto.

The product data generation part 210 is based on Web technologies such as progressive web and WebGL, and product design registration, management, open store support, custom data creation, 3D (Dimensional) jewelry design framework creation, support, etc. For example, data related to various product data, UI/UX, etc. shown in FIGS. 4 to 13 may be generated and provided. On the other hand, this product data generation part 210 is one component constituting the service platform, but does not necessarily have to be one component of the server 130, and serves to receive external components, that is, product-related data generated from the outside. can also be performed.

As described above, the product customization service part 220 is implemented by including the chatbot service part 225 and the augmented reality part 230, creating and managing data related to various embodiments of FIGS. 4 to 13 , processing and various service scenarios can be controlled or supported.

First, the chatbot service part 225 performs similar product analysis, user taste analysis, recommendation, expert product recommendation, and analysis related to other social network service products based on artificial intelligence (AI) technology.

As described above, the chatbot service part 225 provides a UI/UX or a communication window as shown in FIGS. 11 and 12 exclusively for product customized service applications, but is not necessarily limited thereto.

For example, UI/UX or conversation windows shown in FIGS. 11 to 12 may be provided through a messenger application or communication application previously downloaded and installed in the terminal 120 regardless of whether the product customized service application is executed or not. there is. In this case, an icon (not shown) for linking with the product customized service application is provided on the dialog window of the corresponding messenger application or communication application, and the product customized service according to the present invention is provided through linking through the icon. can be provided.

In addition, although not shown in FIGS. 11 and 12, the chatbot service part 225 can provide service support so that a plurality of users can use the same product customization service or different product customization services through a conversation window. . Therefore, the terminals of the plurality of users do not necessarily have to be located in a short distance, and it is not relevant even if they are remote.

In addition, although not shown, the chatbot service part 225 transmits some or all of the UI/UX provided in each drawing of FIGS. 4 to 12 to other interlocked terminals or conversely in each drawing of FIGS. Some of the provided UI/UX may be provided together with data transmitted from the other terminal in a manner such as overlay or audio provision.

For example, referring to FIG. 11, a conversation window is shown in FIGS. 11(a) to 11(c), and the first user has difficulty in selecting a chatbot service through the first terminal If another user's opinion is needed, the second user may be allowed to participate in the chatbot service, or it is possible to allow the second user to participate in the chatbot service from the beginning. In this case, the chatbot service data of the second user may be in the form of a general messenger application or communication application, and in addition, a separate chatbot service dialog window may be provided in the form of a pop-up or overlay. In FIG. 12 , a service may be provided in the same manner.

In addition, the chatbot service shown in FIGS. 11 and 12 may be called and used as needed at any stage of or continuing to each step of the product customized service according to the present invention.

Meanwhile, FIG. 11 illustrates a chatbot service for selecting a product category for a product customized service after the first product customized service application is executed.

11(a) illustrates a dialog window for a process of selecting a target product according to artificial intelligence (AI) guidance, and in FIG. 11(b), a first detailed product selection of the target product is illustrated. A dialog window for the process is illustrated, and in (c) of FIG. 11, a dialog window for the second detailed product selection process of the target product is illustrated.

In addition, in the case of FIG. 12, for example, instead of sequentially inducing target product selection from a large product group to a detailed product group as shown in FIG. 11 (a), the user As in (b), images of the same product or similar products, as shown in (c) to 12 (d) of FIGS. We present a chatbot service that provides convenience of choice by providing

Meanwhile, the user may purchase a product only with the chatbot service, that is, with only the steps of FIG. 11 and FIG. 12 . However, in the present specification, steps (a) to 11 (c) of FIG. 11 or steps (a) to 12 (d) of FIG. Accordingly, a virtual experience service provided by the augmented reality part 230 to be described later may be implemented to be provided.

4 is a diagram illustrating a mobile-based jewelry design, stone material, and metal customized service according to an embodiment of the present invention.

For convenience of description, FIGS. 4(a) to 4(d) assume that the steps of the customization service for selecting the final customized product (eg, a ring as one embodiment) are sequentially configured. However, it is not limited thereto.

First, in (a) of FIG. 4, a service screen is configured and provided as a ring style customization process, that is, to determine a desired ring style. In the first region of the service screen of FIG. 4 (a), various styles of ring lists are provided, and at this time, it is possible to select at least one style of ring according to a left and right swipe, and in the second region An enlarged screen of the ring style selected in the first area may be provided, and a button for moving to a step after the ring style is selected, that is, FIG. 4 (b), is provided in the third area.

4(b) is a stone customization process, and a stone customizing service screen based on the ring style selected in FIG. 4(a) is configured and provided. In the service screen of FIG. 4 (b), buttons for selecting various stone groups (eg, STONE 1, etc.) are provided in the first area, and stones belonging to the stone group determined in the first area are provided in the second area. A list is provided and it is possible to select at least one stone by swiping left and right, and in the third area, the ring having the style selected in FIG. 4 (a) and the stone determined by the area (s) are synthesized. A ring is provided, and in the fourth area, a button for moving to a step after stone customization is completed, that is, FIG. 4(c), is provided.

Figure 4 (c) is a metal customization process, based on the stone determined in Figure 4 (b) on the ring of the style determined in Figure 4 (a), a service screen for customizing the metal of the ring is configured and provided do. In the service screen of FIG. 4 (c), buttons for selecting various metal groups (eg, RING1, etc.) are provided in the first area, and in the second area, a list of metals belonging to the metal group determined in the first area is provided, and at this time it is possible to select at least one metal by swiping left and right. In addition, in the third area, a ring of the style determined in FIG. 4 (a), a ring in which the stone determined in FIG. 4 (b) and a metal determined by the area (s) are synthesized are provided. A button for moving to a step after metal customization is completed, that is, (d) of FIG. 4 is provided.

The order of the processes of (b) of FIG. 4 and (c) of FIG. 4 may be reversed, and only one specific process may be provided, but one of the processes may be applied to another process. It may not affect you.

In (d) of FIG. 4, a final view of the product customized through the process of (a) to (c) of FIG. 4 is provided. Here, in the first area of FIG. 4 (d), an enlarged image of the customized product is provided, and in the second area, product views of various angles created in advance for the customized product are provided to be selectable. In the area, icons for a stereoscopic view and/or a 360-degree view of the customized product provided in the first area are provided.

Meanwhile, in steps (a) to (d) of FIG. 4 , customization may be restarted by returning to a specific step among previous steps at any time according to the user's selection. In this case, customized decisions of other steps other than those of the corresponding step may not be changed according to the user's selection and setting. However, this may vary depending on the stage for reselection.

For example, when returning from step (d) of FIG. 4 to step (c) of FIG. 4, the predetermined metal is initialized, but the ring style previously determined in step (a) of FIG. 4 and (b) of FIG. ) step, the stone previously determined may not be changed.

On the other hand, in step (d) of FIG. 4, if you return to step (a) of FIG. 4, the customizing process can be re-performed from setting the ring style, and unlike this, the stone without changing the style of the previously determined ring. Only the customization process or the metal customization process may be re-performed. At this time, if there is a change in the ring style, the most recently determined stone of FIG. 4(b) and metal of FIG. 4(c) may be preferentially reflected. In this way, in the case of the correction step or the re-execution step, although not shown, for the convenience of the user's selection, the selection previously determined in each step or the image of the previously finally determined ring may be output together.

5 is a diagram for explaining a 360-degree rotation image viewer service for mobile use using render farm technology according to an embodiment of the present invention.

For example, (a) of FIG. 5 is the same as the customized ring view of (d) of FIG. 4 described above. Here, referring to (b) of FIG. 5, when the 360-degree rotation view icon provided in the third area is selected as described above, after going through the loading screen as shown in (c) of FIG. 5 (d) ), a customized ring view service screen according to 360 degree rotation may be provided. Here, the rotation may be determined by sensing a tilt of the terminal as well as a touch, voice, and text, and calculating a rotation amount accordingly.

On the other hand, FIG. 6 is a diagram illustrating a mobile-only real-time 3D image viewer service using Web RTC technology according to an embodiment of the present invention.

For example, (a) of FIG. 6 is the same as the customized ring view of (d) of FIG. 4 described above. Here, referring to (b) of FIG. 6, when the 3D view icon provided in the third area is selected as described above, after going through the loading screen as shown in (c) of FIG. 6, (d) of FIG. As such, a customized ring view service screen according to 360 degree rotation may be provided. Here, the rotation may be determined by sensing a tilt of the terminal as well as a touch, voice, and text, and calculating a rotation amount accordingly.

As described above, if a customized product is selected in this way, as shown in (d) of FIG. 4, FIG. 5 and FIG. 6, virtual wearing may be performed, or alternatively, an order may be placed immediately.

If virtual wearing is selected, service data implemented by the augmented reality part 230 may be provided.

At this time, the augmented reality part 230, based on the augmented reality (AR) technology, supports a service capable of virtually wearing jewelry products such as rings, earrings, necklaces, and bracelets through a terminal. In this case, the virtual wearing service can provide both 2D and 3D.

In addition, the augmented reality part 230 may also support both a marker method and a non-marker method.

Any one of the screens of FIGS. 4 to 6 may be provided according to a user's selection or setting during or after the process of FIGS. 11 to 12 or regardless of it.

3 illustrates an embodiment of a detailed configuration block diagram of the product customized service part 220, but is not necessarily limited thereto. At least one of the components shown in FIG. 3 may be provided in the terminal 120, and in this case, the function of the product customized service part 220 of the server may be replaced as necessary.

The product customized service part 220 includes a data communication unit 301, a virtual wearing item configuration unit 302, a real object information extraction unit 303, a synthesis unit 304, a chatbot service unit 305, and a control unit ( 306) can be implemented. Here, a database unit 310 in which various data are stored may be further included. However, the database unit 310 may be built-in or external, and may be one or more. When a plurality of database units 310 are used for the product customized service according to the present invention, some may be built-in and others may be external, and this is according to the system implementation method, and capacity or performance is considered, but there is no particular limitation.

Two or more of the components of FIG. 3 may be modularized into one component, and vice versa. In addition, although not shown, one or more components may be further added to implement the service system according to the present invention, or vice versa, according to system implementation examples.

According to the embodiment, the server 130 does not directly have a memory or database as hardware, that is, does not directly store data, but temporarily stores only address data such as a URL (Uniform Resource Locator) in which the corresponding data is stored, and stores it as a terminal ( 120) may also serve. In this case, it is possible to directly download related data by accessing the URL received from the terminal 120 .

The data communication unit 301 may communicate with the terminal 120 to exchange various data in relation to the product customized service. At this time, the server 130 may perform data communication with various external devices in addition to the terminal 120, and the data communication unit 301 supports this.

The virtual wearing item configuration unit 302 creates a corresponding virtual wearing item when there is a request for a product or part of the product through the terminal 120 or various routes through the data communication unit 301, for example. In addition, the virtual wearing item configuration unit 302 may create a virtual wearing item for a manufactured product or a suggested product or item through the jewelry design framework 210 .

The real object information extractor 303 receives object image data captured from the terminal 120 and extracts an object to be matched with the virtual worn item created in the virtual worn item configuration unit 302 therefrom. To this end, a feature point extraction method of the photographed object image received from the terminal 120 may be used. Alternatively, an object to be matched with a virtual wearing item may be extracted using provided guideline data shown in FIGS. 7 and 8 . Alternatively, as shown in FIGS. 9 to 10 , an object to be matched with a virtual wearing item may be extracted using an augmented reality (AR) band recognition code. In the present invention, other latest object recognition and extraction technologies not shown or described may be utilized.

The synthesis unit 304 may synthesize the two data to generate a virtual wearing image to be output to the display of the terminal 120 by using the generated virtual wearing item and the extracted real object information. At this time, before the synthesis, the size, format, etc. of the two data may be converted if necessary, or other data may be converted based on one data from the beginning.

Meanwhile, (b) of FIG. 8 or (d) of FIG. 8 can be regarded as an example of a synthesized image. The synthesis unit 304 may convert data synthesized by the server 130 and provide the data to the terminal 120 in consideration of the performance of the terminal 120 . In addition, the synthesis unit 304 can process both still image and/or moving image, and a known technique of overlapping a plurality of images may be used.

The chatbot service unit 305 generates data and/or UI data such as text, image, video, audio, etc. for providing a chatbot service based on artificial intelligence (AI) in relation to the aforementioned chatbot service, and , It can be temporarily stored in the database unit 310.

The control unit 306 generally controls the above components to provide a product customized service.

7 is a diagram for explaining the provision of mobile-based 2D and 3D virtual wearing experience services according to an embodiment of the present invention.

As in FIG. 4(d), FIG. 5 and FIG. 6, when the user selects virtual wearing in FIG. 7(a), a UI for virtual wearing experience may be provided as shown in FIG. 7(b). there is.

Referring to (b) of FIG. 7 , the virtual wearing experience UI provided overlaid can be largely divided into a 2D virtual wearing experience UI and a 3D virtual wearing experience UI. First, referring to the 2D virtual wearing experience UI, a guideline is provided as shown in (c) of FIG. The 2D virtual experience can be achieved by filming it through .

On the other hand, as shown in (d) of FIG. 7, in the case of the 3D virtual wearing experience, after wearing the augmented reality (AR) band, an object (eg, hand, neck) wearing the augmented reality (AR) band is detected through a camera sensor. etc.) is recognized through the camera sensor, the 3D virtual wearing experience can be performed.

Meanwhile, the size and shape of the guideline data output according to an embodiment of the present invention are determined according to the selected product, and may be in a fixed form within the display screen. The user may acquire a captured image corresponding to the guideline by adjusting and aligning an object (eg, a body part such as a hand or neck) to fit the guideline.

However, the present invention is not necessarily limited thereto. For example, the output guideline data may be increased or decreased according to an object sensed through a camera sensor of the terminal without being fixed within the screen. For example, since an object may have various shapes and sizes depending on a person, in this case, if a fixed guide line is provided, it may be difficult for the user to align within a predefined time. In this way, if the alignment fails several times and cannot move to the next step, the service is not provided eventually, which may cause inconvenience. In addition, if the object is not completely fixed, it may be difficult to align or it may be difficult to obtain valid data due to the short aligning time. Arbitrary alignment may be performed by modifying the guideline data, or modified guideline data may be provided to help the user's alignment convenience.

In this case, the guideline data may be deformed to a greater extent by a predetermined size based on the reference object image, thereby helping the user's alignment convenience.

In this specification, 2D and 3D can be regarded as concepts that are almost the same as commonly referred to as flat and three-dimensional concepts.

Hereinafter, 2D and 3D virtual wearing experience services will be described in more detail with reference to the accompanying drawings.

8 is a diagram for explaining the provision of a virtual wearing experience service using a non-marker-based 2D synthesis technology according to an embodiment of the present invention.

Referring to (a) of FIG. 8 , when 2D synthesis is selected or requested through the terminal 120, a guideline for recognizing and extracting an object to be worn by a virtual wearing item to be synthesized is provided on the terminal 120. do. Thereafter, the previously created virtual wearing item is combined with the object information recognized and extracted through the provided guideline so as to be overlapped at a predetermined position, and provided as shown in FIG. 8(b). At this time, referring to (b) of FIG. 8 , an icon for selecting and controlling a combination of an object and a virtual wearing item may be additionally provided on one area of the screen. For example, if the object is the left hand and the virtual wearing item is a ring, the provided icon can select which finger of the left hand to wear the ring on, and from this icon access, a composite image is created using the object and the virtual wearing item. can provide this.

On the other hand, even if it is 2D synthesis, since the virtual wearing experience in a specific direction may have its limitations, according to an embodiment of the present invention, FIG. am. For example, FIG. 8(c) shows that the guide line is provided in a form in which the left hand (object) is rotated 90 degrees to the left when compared to FIG. 8(a). Also, (d) of FIG. 8 illustrates that an image obtained by synthesizing an object and a virtual wearing item is provided through the guideline provided in (c) of FIG. 8 .

Although not shown, the guide line may be for obtaining object data in a representative image or direction (or angle), as in the case of FIG. 8 (a) or FIG. 8 (c), but is not necessarily limited thereto. no.

The guide line may be in the form of a flat still image, but is not necessarily limited thereto. For example, it may be different depending on the object, and furthermore, the guideline itself may be provided in the form of a video image, and accordingly, by allowing the user to match the object, object data of various angles or directions may be displayed at once or several times. It is possible to provide a 3D-like image synthesized through 2D synthesis technology by recognizing it over and extracting and synthesizing it as needed or according to the user's motion. For example, in the state of FIG. 8 (b), even if the guide line is not provided again as shown in (c) of FIG. 8, the synthesized image as shown in (d) of FIG. .

Also, even if the guideline is provided, it may not be easy for the user to align the objects to the guideline at once. In this case, the terminal or server may recognize and extract image data acquired in the process before the object is completely aligned with the guideline, and may use it for synthesis. For example, a predefined number of image frames are stored together before and after the object and guideline are completely aligned, but may be used for object image recognition and extraction for synthesis. This may be used in the form of reference data or the like when an object image is blurred or blur occurs despite the alignment.

9 is a diagram illustrating an augmented reality (AR) marker band for a marker-based virtual wearing experience service according to an embodiment of the present invention, and FIG. 10 is an augmented reality (AR) marker band according to an embodiment of the present invention. ) It is a diagram shown to explain the provision of a recognition marker-based virtual wearing experience service.

Figure 9 (a) shows an augmented reality (AR) band produced including a recognition code, Figure 9 (b) shows a person wearing the augmented reality (AR) band shown in (a) of FIG. It depicts an object.

When an augmented reality (AR) band is worn on an object as shown in (b) of FIG. 9 , a 3D virtual wearing experience image may be provided as shown in (c) of FIG. 9 . At this time, the 3D virtual wearing experience image is captured through the camera sensor of the terminal at the moment when the object wears the augmented reality (AR) band or the object wearing the augmented reality (AR) band after requesting the 3D virtual wearing experience. It can be provided by recognizing. At this time, if the augmented reality (AR) band is normally recognized through a recognition code, etc., it can notify the success of normal recognition through voice data or vibration data, as shown in FIGS. 9(c) to 9(f). A 3D virtual wearing experience image may be provided.

After recognizing the code dedicated to augmented reality (AR) in the augmented reality (AR) band, that is, the recognition code, even if the object is freely moved, tilt recognition, side recognition, You can feel a natural virtual fit, such as front face recognition.

Meanwhile, in (a) of FIG. 10 , augmented reality (AR) markers of various shapes or shapes according to objects are shown in relation to providing an augmented reality (AR) recognition marker-based virtual wearing experience service. For example, a sticker type for earrings, a pendant type for necklaces, and a medium and small band type for bracelets have been shown. The marker may be recognized through a camera sensor of the terminal.

FIG. 10(b) is an object image when at least one marker is worn, and FIG. 10(c) shows a 3D virtual wearing experience image corresponding to each marker of FIG. 10(b). In particular, in the case of (c) of FIG. 10 , if a plurality of markers are recognized at once or simultaneously, the corresponding 3D virtual wearing experience image may be enlarged or provided to be distinguished from the object image. Alternatively, a corresponding part of the object image may become flickering and only the corresponding image may be enlarged and provided according to the user's selection. In this case, a corresponding product-related service UI may be provided around the enlarged image.

Although not shown, when any one 3D virtual wearing experience image is selected in (c) of FIG. 10 or a corresponding wearing marker is accessed, a separate service screen related to the product may be overlaid or provided as a full screen. there is.

13 is a diagram for explaining rights management and product assurance service provision using blockchain technology according to an embodiment of the present invention.

13 illustrates an example of a product guarantee recorded in a block chain and provided to a terminal.

In this way, by using blockchain technology, management of rights such as copyright of jewelry design, which is the producer side, for example, 3D jewelry design file, registration date, designer, transaction information, etc. can be done

In addition, in relation to the jewelry product warranty, a guarantee is issued to the consumer who has purchased the jewelry, but it is recorded in the block chain, and if there is a request for after service, the purchase information of the jewelry (name, contact information, etc.) It is possible to provide related information after determining whether there is a legitimate authority through inquiry (unique code), and in this way, information security and management convenience can be expected by managing each jewelry product with a unique code and recording it in the block chain.

In addition, in relation to the transaction management system, if the service platform operator maliciously reduces or manipulates transaction performance, there may be profit or payment inconvenience on the producer or consumer side. Transparent transaction management can be achieved by making it impossible to modify or delete records.

A product customized service providing method through a terminal according to an embodiment of the present invention may be performed as follows, but the method of providing a product customized service through the terminal is performed while omitting one or more of the following steps. It could be.

The server 130 may recognize a body part for virtual wearing of the product image through the camera of the terminal in a state in which the product image customized by the user is output through the screen of the terminal. At this time, synthesized image data obtained by combining the product image data with an image corresponding to the recognized body part may be generated and outputted in real time through the screen of the terminal, and the output may be the result of recognizing the body part It may be performed when the user clicks a button that is activated when the virtual wearing of the product image is above a certain level.

In other words, in the process of recognizing a body part, if the body part is not recognized smoothly due to shaking or movement of the body part, even if a service for virtual wearing is provided in real time by generating synthetic image data, high-quality output is obtained. Since it may be difficult to provide a screen, when the result of recognizing the body part is above a predetermined level, that is, above a certain level at which virtual wearing of the product image is possible, synthetic image data is generated to provide a service for virtual wearing in real time. are making it

On the other hand, when the result of the body part recognition is below a certain level at which the product image can be virtually worn, at least one image frame for the body part is generated for a predetermined time, and based on this, at least one image frame corresponding to the body part is generated. The step of generating an image and then outputting the image may be performed.

In the present invention, guideline data related to the product image may be output on the screen of the terminal, and the guideline data may be in the form of either a still image or a moving image. In this case, a specific level at which the product image can be virtually worn may be a state in which the body part is aligned with the guideline data at a predetermined standard or higher.

Meanwhile, when the result of the recognition of the body part is equal to or higher than a certain level at which the product image can be worn virtually, a button on the screen of the terminal may be activated, and when the user clicks the corresponding button, the image corresponding to the recognized body part is activated. Synthesized image data synthesized with the product image data may be generated and output in real time through the screen of the terminal. At this time, when the user continuously clicks the button for a predetermined time or more, the real-time output at this time may consist of a video. In addition, in the case where the video is output in real time, an effect filter may be set so that a special effect is applied (eg, a shining effect, etc.).

In addition, in the present invention, while the real-time output is being performed, if the result of the body part recognition changes below a specific level at which the product image can be virtually worn, the real-time output may be stopped, and the real-time output may be stopped while the real-time output is being performed. When product image data is changed to other product image data, modified synthesized image data obtained by combining the changed product image data with an image corresponding to the body part may be output in real time, and the synthesized image data output in real time is The user can control the position and size, and can also rotate and check.

Meanwhile, the above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable medium. In addition, the structure of data used in the above method can be recorded on a computer readable medium through various means. Computer-readable media storing executable computer codes for performing various methods of the present invention include magnetic storage media (eg, ROM, floppy disk, hard disk, etc.), optical readable media (eg, CD-ROM, DVD) etc.).

Those skilled in the art related to the embodiments of the present invention will be able to understand that it may be implemented in a modified form without departing from the essential characteristics of the above description. Therefore, the disclosed methods are to be considered in an illustrative rather than a limiting sense. The scope of the present invention is shown in the claims rather than the detailed description of the invention, and all differences within the equivalent scope should be construed as being included in the scope of the present invention.

The present invention relates to a product customized service providing method and a service platform therefor, which can be applied to devices and systems therefor.

Claims (8)

A method for providing a product customized service through a terminal,
Recognizing a body part for virtual wearing of the product image through a camera of the terminal in a state in which the product image customized by the user is output through the screen of the terminal; and
Generating synthesized image data obtained by synthesizing the product image data with an image corresponding to the recognized body part and outputting it in real time through the screen of the terminal;
The outputting may be performed when the user clicks a button that is activated when the result of the body part recognition is equal to or higher than a specific level at which the product image can be virtually worn, a product customized service through a terminal. How to provide. According to claim 1,
Guideline data related to the product image is output on the screen of the terminal,
The specific level at which the virtual wearing of the product image is possible is a state in which the body part is aligned with a predetermined standard or more in the guideline data, the product customized service providing method through the terminal. According to claim 1,
When the result of the body part recognition is below a certain level at which the product image can be virtually worn, at least one image frame for the body part is created for a predetermined time, and then an image corresponding to the body part is generated based on this. And then, characterized in that performing the outputting step, a product customized service providing method through a terminal. According to claim 2,
The guideline data is a method of providing a product customized service through a terminal, in the form of any one of a still image or a moving image. According to claim 1,
When the user continuously clicks the activated button for more than a predetermined time, the real-time output is made of a video,
A method of providing a product customized service through a terminal, wherein an effect filter can be set so that a special effect is applied when the video is output in real time. According to claim 1,
While the real-time output is being performed, when a result of the body part recognition changes below a specific level at which the product image can be virtually worn, the real-time output is stopped. According to claim 1,
When the product image data is changed to other product image data while the real-time output is being performed, the changed synthesized image data obtained by combining the changed product image data with the image corresponding to the body part is output in real time through a terminal. How to provide product customization service. In the apparatus for providing a product customized service,
a data communication unit for transmitting and receiving data with a user's terminal; and
Including; a control unit for providing the product customized service;
The control unit recognizes a body part for virtual wearing of the product image through a camera of the terminal in a state in which a product image customized by a user is output through a screen of the terminal,
Control to generate synthesized image data obtained by synthesizing the product image data with an image corresponding to the recognized body part and output it in real time through the screen of the terminal,
The output is performed when the user clicks a button that is activated when a result of the body part recognition is equal to or higher than a specific level at which virtual wearing of the product image is possible.