KR102659455B1 - Method and system for providing live broadcasting - Google Patents

Abstract

The present invention relates to a method and system for providing live broadcasting to more effectively provide information about products in live commerce. A method of providing a live broadcast according to the present invention includes the steps of receiving a broadcast image of a live broadcast from a live broadcast server, providing the broadcast image on a display unit of a user terminal based on the reception, and providing the broadcast image on the display unit. In a state where the broadcast image is being provided, receiving an activation request for an augmented reality function related to the live broadcast from the user terminal, in response to the activation request, a camera received from a camera provided in the user terminal For the video, performing augmented reality graphic processing related to a product linked to the live broadcast, and in a state of receiving the broadcast video at the user terminal, together with the augmented reality graphic processed camera image, related to the product It may include displaying option information on the display unit.

Description

Method and system for providing live broadcasting {METHOD AND SYSTEM FOR PROVIDING LIVE BROADCASTING}

The present invention relates to a method and system for providing live broadcasting to more effectively provide information about products in live commerce.

As technology develops, the spread of electronic devices (e.g., smartphones, tablet PCs, automated devices, etc.) has become popular, and as a result, dependence on electronic devices is gradually increasing in many aspects of daily life.

Furthermore, the popularization of electronic devices has promoted the development of Internet-based services, and through this, consumption patterns that were previously highly dependent on offline are gradually moving to online.

In line with this change in trend, it is becoming common for industrial fields, which are based off-line due to the nature of services such as product sales, to attract customers through various online marketing.

Accordingly, in recent years, various product sales services have been provided online, and their utilization is also greatly increasing. However, these online product sales services only provide pre-uploaded product information, and consumers still want to be provided with various information about products.

To address these needs, a method of selling products using live broadcasts is being introduced. The online product selling method using real-time interactive video (reference patent, Korean Patent Publication No. 10-2009-0001838) can immediately provide product information that consumers want by communicating with consumers in real time using online live broadcasting. It came to be.

However, this method of selling products using live broadcasts also has the disadvantage of not allowing consumers to try the products themselves. Accordingly, there is still a need for a service that allows customers to test products using only the electronic devices they own.

The method and system for providing live broadcasting according to the present invention are intended to provide a user experience that can more effectively provide information about products to users watching live broadcasting.

Furthermore, the method and system for providing live broadcasting according to the present invention are intended to provide a service that allows users watching live broadcasting to experience or test products sold through live broadcasting.

In particular, the method and system for providing live broadcasting according to the present invention are intended to provide a service that allows users to experience or test products sold through live broadcasting without stopping the live broadcast being watched.

Furthermore, the method and system for providing live broadcasting according to the present invention are intended to provide a service that can provide various information about products sold through live broadcasting to users who watch live broadcasting.

Furthermore, the method and system for providing live broadcasting according to the present invention are intended to provide a service where users watching live broadcasting or users experiencing products can easily purchase products.

A method of providing a live broadcast according to the present invention includes the steps of receiving a broadcast image of a live broadcast from a live broadcast server, providing the broadcast image on a display unit of a user terminal based on the reception, and providing the broadcast image on the display unit. In a state where the broadcast image is being provided, receiving an activation request for an augmented reality function related to the live broadcast from the user terminal, in response to the activation request, a camera received from a camera provided in the user terminal Performing augmented reality graphic processing related to a product linked to the live broadcast on the video; And while the user terminal is receiving the broadcast image, it may include displaying option information related to the product on the display unit along with the augmented reality graphic-processed camera image.

Furthermore, the live broadcasting providing system according to the present invention is a live broadcasting providing system that works in conjunction with a user terminal including a communication unit, a camera, and a display unit, and receives a broadcast image of a live broadcast from a live broadcasting server through the communication unit, and a control unit providing the broadcast image on a display unit of a user terminal, wherein the control unit controls an augmented reality function related to the live broadcast from the user terminal while the broadcast image is being provided on the display unit. Receiving an activation request for, and in response to the activation request, performing augmented reality graphic processing related to a product linked to the live broadcast on a camera image received from a camera provided in the user terminal, While receiving the broadcast video, optional information related to the product can be displayed on the display unit along with the camera image processed with augmented reality graphics.

Furthermore, according to the present invention, a program executed by one or more processes in an electronic device and stored in a computer-readable recording medium includes the steps of receiving a broadcast image of a live broadcast from a live broadcast server, based on the reception. Thus, providing the broadcast image on the display unit of the user terminal, while the broadcast image is being provided on the display unit, receiving a request to activate an augmented reality function related to the live broadcast from the user terminal. In response to the activation request, performing augmented reality graphic processing related to a product linked to the live broadcast on a camera image received from a camera provided in the user terminal, and performing augmented reality graphic processing related to a product linked to the live broadcast, and the broadcast image in the user terminal While receiving, commands for displaying optional information related to the product on the display unit along with an augmented reality graphic-processed camera image can be issued.

The method and system for providing live broadcasting according to the present invention can utilize an augmented reality function to provide a user experience that allows users to experience products introduced in the live broadcast while watching the live broadcast.

According to the method and system for providing live broadcasting according to the present invention, consumers can experience products even without directly visiting the store, saving time on visiting the store and purchasing products. You can do it more carefully.

Furthermore, according to the method and system for providing live broadcasting according to the present invention, sellers can induce consumers to more actively purchase products by providing consumers with opportunities to experience products in a variety of ways. Furthermore, even if sellers do not directly operate a store, they can meet various consumers and experience products through live broadcasts, thereby saving on the operating costs of running a store and having various opportunities to sell products.

Meanwhile, the method and system for providing live broadcasting according to the present invention can provide the provision of live broadcasting, product experience, and product purchase process in a unified manner while live broadcasting is in progress. Through this, consumers can experience and purchase products more conveniently just by watching the live broadcast according to the present invention.

1 is a conceptual diagram for explaining live broadcasting according to the present invention.
Figure 2 is a conceptual diagram for explaining a live broadcasting provision system according to the present invention.
Figure 3 is a flowchart illustrating a method for providing live broadcasting according to the present invention.
FIGS. 4A and 4B are conceptual diagrams illustrating activation of an augmented reality function in a method for providing live broadcasting according to the present invention.
Figure 5 is a conceptual diagram for explaining a broadcast video provided through live broadcasting according to the present invention.
Figure 6 is a conceptual diagram for explaining the camera image provided through the augmented reality function in the present invention.
Figure 7 is a conceptual diagram for explaining the DB according to the present invention.
Figures 8, 9, and 10 are conceptual diagrams to explain the user environment provided to the seller's terminal in the present invention.
11 and 12 are conceptual diagrams for explaining a method of providing additional item information in the present invention.
13 to 15 are conceptual diagrams for explaining an augmented reality service related to the present invention.
Figure 16 is a flowchart for explaining a method of providing option information for products linked to live broadcasting in the present invention.
17, 18, 19, 20, 21, 22, 23a, 23b, 24a, 24b, 25, 26a and 26b show services linked to option information according to the present invention. These are conceptual diagrams to explain the user environment that provides.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings. However, identical or similar components will be assigned the same reference numbers regardless of drawing symbols, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves. Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

The present invention relates to a method of providing a live broadcast, and in particular, by providing an augmented reality service along with the provision of a live broadcast, so that a user can watch the live broadcast and at the same time sell products on the live broadcast. It provides a service that allows you to experience or test.

Here, live broadcasting is a streaming broadcast that introduces products while communicating with consumers in real time online, and can also be called streaming broadcasting, live commerce, or shopping live. You can.

Here, Augmented Reality is a technology that displays a single image by overlapping a virtual image on a real image or background. It synthesizes images by overlapping a virtual image on a real image, or at least part of a real image. It may mean transforming . Here, the real image may mean a camera image received through a camera provided in the user terminal.

In the present invention, combining information related to an item (specific item) or product to the camera image 20, or changing at least part of the camera image 20 to reflect information related to the item (specific item) or product. Through this, augmented reality graphics processing can be performed.

In the present invention, while a broadcast image of a live broadcast is being provided to a user terminal, augmented reality graphics processing related to products sold in the live broadcast is performed on the camera image received from the camera, and the camera image is processed with augmented reality graphics. By providing, it is possible to provide a method of providing live broadcasting that allows viewing live broadcasting and product experience (testing) at the same time.

Here, the “camera image 20” may include at least one of a photo (or image), a video, and a preview screen captured through a camera. That is, in the present invention, augmented reality graphic processing can be performed on at least one of a photo (or image), video, and preview screen received through a camera. However, in the following, they will not be distinguished and all will be explained by expressing them as camera images 20. 1 is a conceptual diagram for explaining live broadcasting according to the present invention.

As shown in FIG. 1, the live broadcast providing system 100 according to the present invention can provide a broadcast image 10 of a live broadcast on the display unit of the user terminal 200.

The broadcast video 10 of this live broadcast is filmed through the user terminal of the seller (or user corresponding to the seller) involved in selling the product, and is transmitted through the user terminal of the consumer (or user corresponding to the consumer) involved in purchasing the product. It can be provided (or output) to .

In the present invention, in order to avoid confusion in terminology, the user involved in product sales is named “seller” or “(live broadcast) host”, and the user terminal used by the seller is called “seller terminal (or seller’s terminal).” , “Host terminal”, “Live broadcasting progress terminal”, “Live commerce progress terminal”, “Sales related terminal”, etc.

In addition, users involved in purchasing products are named “consumers,” “buyers,” or “viewers,” and user terminals used on the consumer side are called “consumer terminals (or consumer terminals),” “buyer terminals,” and “viewers.” It can be named “Terminal”, etc.

As shown in FIG. 1, the broadcast image 10 is displayed on the display unit of the user terminal 200 along with various related information and related graphic objects 11a to 11f associated with the broadcast image 10. It may be provided (or printed).

In the present invention, providing (or outputting) the broadcast video 10 and the related information and the related graphic objects 11a to 11f together means that the related information and the related graphic objects 11a to 11f are provided in at least one area of the broadcast video 10. 11f) may be understood as being overlapped or provided (or output) in an overlapping manner.

The related information and related graphic objects 11a to 11f overlapping with the broadcast video 10 include i) information pre-entered by the seller (ex: live broadcast title information 11a, product information 11b, Augmented reality function icon (or graphic object, 11c) for augmented reality service, ii) various icons (11d, 11e) related to live broadcast, ii) chat input by at least one of the seller and the consumer (or a plurality of consumers) (Chatting) may include at least one of the information (11f).

Meanwhile, in the method of providing live broadcasting according to the present invention, a series of processes may be performed by a system implemented in the form of an application or software (hereinafter, the terms application and software will not be distinguished). (unified and named application). That is, the live broadcast providing system 100 according to the present invention may also be called a “live broadcast providing application,” “live commerce application,” or “shopping live application.”

Below, we will look at the live broadcasting provision system according to the present invention in more detail along with the attached drawings. Figure 2 is a conceptual diagram for explaining a live broadcasting provision system according to the present invention.

As shown in FIG. 2, the live broadcasting providing system 100 according to the present invention can receive broadcasting video from the live broadcasting server 300 and provide it on the user terminal 200. Furthermore, it is also possible for the live broadcast video to be provided directly from the seller terminal to the user terminal 200.

The live broadcast server 300 is a server that provides a live broadcast streaming service, and can receive broadcast video from a specific user terminal and provide the received broadcast video to the live broadcast system 100 according to the present invention. there is.

In particular, the live broadcast server 300 according to the present invention can provide a live broadcast streaming service in conjunction with a shopping service. In this case, the specific user terminal corresponds to the seller terminal, and the live broadcast providing system 100 may be a live broadcast providing application implemented in the consumer terminal.

Accordingly, the live broadcast server 300 according to the present invention may be interchangeably named as “server”, “broadcast server”, “live server”, “live commerce server”, “commerce server”, etc.

Furthermore, the live broadcast server 300 provides at least one of i) broadcast information, ii) item information, iii) chat information, and vi) function icons (or graphic objects) related to the broadcast video. The live broadcast providing system according to the present invention ( 100).

Here, item information is information about an item, and an item may mean a product that is subject to sale or advertisement (promotion) through live broadcasting.

Broadcast information may be various information related to a specific broadcast video. For example, broadcast information includes title information of a specific broadcast video (ex: “Monthly Gift Shop Episode 8: Black Friday Special”), host information (“Barbie Cosmetics”), viewer information (ex: “331,425 views”) may include broadcast type information (ex: live), etc.

Item information may be various information related to at least one item associated with a specific broadcast. For example, item information may include a thumbnail image of an item sold or introduced in a live broadcast of a specific broadcast video, item name information, detailed item information, item price information, etc.

Chat information can be understood as messages exchanged in real time with the live broadcast of a specific broadcast video by a seller terminal and a plurality of consumer terminals.

A function icon (or graphic object) may be an icon (graphic object) associated with various functions related to a specific broadcast video or shopping service.

The broadcast information, item information, chat information, and function icons (or graphic objects) are based on the present invention based on at least one of information received from a seller terminal related to a specific broadcast video and information stored in the live broadcast server 300. It may be provided to the live provision system 100 according to the following.

Information stored in the live broadcasting server 300 may be information input by an administrator of the live broadcasting server 300.

In this way, the live broadcast server 300 according to the present invention can provide various information related to the broadcast video of the live broadcast in order to provide a live broadcast streaming service linked to a shopping service.

Meanwhile, in the present invention, the live broadcast providing system 100 is installed in the user terminal 200 in the form of an application, as shown in FIG. 2, and performs a process of providing an augmented reality service along with broadcast images of a live broadcast. It can be done. This application may be installed on the user terminal 200 at the request of the user on the user terminal 200, or may exist installed on the user terminal 200 at the time the user terminal 200 is shipped. An application implementing such a live broadcasting provision system may be downloaded from an external data storage (or external server) through data communication and installed on the user terminal 200. . And, when an application implementing the live broadcast providing system according to the present invention is executed on the user terminal 200, a series of processes for providing an augmented reality service along with broadcast images of the live broadcast may proceed.

The user terminal 200 referred to in the present invention may be any electronic device capable of operating the live broadcasting provision system 100 according to the present invention, and there is no particular limitation on its type. For example, the user terminal 200 is a mobile phone, a smart phone, a laptop computer, a laptop computer, a slate PC, a tablet PC, and an ultrabook. (ultrabook), desktop computer, digital broadcasting terminal, PDA (personal digital assistants), PMP (portable multimedia player), navigation, wearable device (e.g., smartwatch), glass type Terminals (smart glass), HMD (head mounted display), etc. may be included.

Meanwhile, the live broadcast providing system 100 according to the present invention, which can be implemented in the form of an application, may include at least one of a communication unit 110, a storage unit 120, and a control unit 130. The above components are software components and can perform functions in conjunction with hardware components of the user terminal 200.

For example, the communication unit 110 uses the configuration of communication modules (e.g., mobile communication module, short-range communication module, wireless Internet module, location information module, broadcast reception module, etc.) provided in the user terminal 200. Thus, it can perform the role of transmitting and receiving information (or data) related to the present invention with at least one external device (or live broadcast server, 100a).

In addition, the storage unit 120 uses at least one of the memory provided in the user terminal 200 and an external storage (or external server 100a) to store information related to providing live broadcasting and augmented reality services, or to store information related to providing live broadcasting and augmented reality services. Information or commands related to can be stored.

In the present invention, “stored” in the storage unit 120 may mean that information is physically stored in the memory of the user terminal 200 or in an external storage (or external server, 100a).

Meanwhile, when the application corresponding to the present invention is deleted from the user terminal 200, information stored in the storage unit 120 may also be deleted from the user terminal 200. Additionally, the external storage 100a may be a cloud-type storage.

In the present invention, the memory of the user terminal 200 and the external storage (or external server, 100a) are not distinguished, and both are described by referring to the storage unit 120.

Meanwhile, the control unit 130 can perform overall control for carrying out the present invention using a CPU (Central Processing Unit) provided in the user terminal 200. The components examined above can operate under the control of the control unit 130, and of course, the control unit 130 can control the physical components of the user terminal 200.

For example, the control unit 130 may control the broadcast image of a live broadcast to be output through the display unit 210 provided in the user terminal 200. Additionally, the control unit 130 may control the taking of a video or photo (or image) through the camera 220 provided in the user terminal 200. Additionally, the control unit 130 may receive information from the user through an input unit (not shown) of the user terminal 200.

There is no particular limitation on the types of display unit 210, camera 220, and input unit (not shown) provided in the user terminal 200. For example, the input unit (not shown) may be a mechanical input means (or mechanical key, for example, mouse, joystick, physical button, dome switch, jog wheel). , jog switch, etc.) and a touch input means. As an example, the touch input means consists of a virtual key, soft key, or visual key displayed on the touch screen through software processing, or on a part other than the touch screen. It can be done with a touch key placed.

Meanwhile, the virtual key or visual key can be displayed on the touch screen in various forms, for example, graphic, text, icon, video or these. It can be made up of a combination of . At this time, when the input unit (not shown) includes a touch screen, the display unit 110 of the user terminal 200 may be made of a touch screen and performs both the role of outputting information and the role of receiving information. can do.

Furthermore, the control unit 130 may receive a request for activation of an augmented reality function related to the live broadcast from the user while the broadcast image of the live broadcast is being provided through the display unit 210 of the user terminal. In response to the user's request to activate the augmented reality function, the control unit 130 performs graphics processing related to items related to the live broadcast on the camera image 20 received from the camera 220 provided in the user terminal. can do. Additionally, the control unit 130 may display the camera image 20 on which augmented reality graphic processing has been performed along with the broadcast image on the display unit 210 of the user terminal. Through this, users can watch the broadcast video of the live broadcast and at the same time experience (or experience, test) the items sold in the live broadcast. Hereinafter, the live broadcast service and the augmented reality service are simultaneously provided in the live broadcast providing system 100a according to the present invention so that the user can experience or test items related to the live broadcast while watching the broadcast video of the live broadcast. Let us explain in more detail how to provide it.

FIG. 3 is a flowchart for explaining a method for providing live broadcasting according to the present invention, and FIGS. 4A and 4B are conceptual diagrams for explaining activation of an augmented reality function in the method for providing live broadcasting according to the present invention. Furthermore, FIG. 5 is a conceptual diagram for explaining a broadcast image provided through a live broadcast according to the present invention, and FIG. 6 is a conceptual diagram for explaining a camera image provided through an augmented reality function in the present invention.

Furthermore, Figure 7 is a conceptual diagram for explaining the product DB according to the present invention, and Figures 8, 9, and 10 are conceptual diagrams for explaining the user environment provided to the seller terminal in the present invention. FIGS. 11 and 12 are conceptual diagrams for explaining a method of providing additional item information in the present invention, and FIGS. 13 to 15 are conceptual diagrams for explaining an augmented reality service related to the present invention.

In the method for providing live broadcasting according to the present invention, a process of receiving broadcast video of a live broadcast may be performed from the live broadcast server 300 (S310, see FIG. 3).

Reception of broadcast video of a live broadcast may be performed in the user terminal 200.

As described above, the live broadcast server 300 can receive broadcast video of a live broadcast from a seller terminal and provide it to the live broadcast providing system 100 according to the present invention.

The communication unit 110 may receive broadcast video of a live broadcast conducted in real time from the live broadcast providing server 300 using a streaming technique.

That is, the control unit 130 can control the communication unit 110 so that the communication unit 110 continues to receive broadcast images while a live broadcast is in progress.

Next, in the method for providing live broadcasting according to the present invention, based on receiving the broadcast image from the live broadcast server 300, the received broadcast image 10 is displayed on the display unit 210 of the user terminal 200. It can be provided (S320, see FIG. 3).

As shown in (a) of FIG. 4A, the control unit 130 operates the display unit 210 so that the received broadcast video 10 is provided (played, output, or displayed) through the display unit of the user terminal 200. can be controlled.

In this way, the control unit 130 performs first data processing to control the display unit of the user terminal 200 so that the broadcast image 10 is provided (played, output, or displayed) through the display unit of the user terminal 200. You can. This first data processing may be performed simultaneously with second data processing related to the augmented reality service described later.

Meanwhile, the broadcast image 10 provided from the display unit may be a broadcast image 10 corresponding to a broadcast channel selected by the user.

Next, in the method for providing live broadcasting according to the present invention, while the broadcast image 10 is being provided on the display unit of the user terminal 200, the augmented reality function related to the live broadcast is received from the user terminal 200. The process of receiving an activation request may proceed (S330, see FIG. 3).

As discussed above, a request to activate the augmented reality function may be received based on the icon 410 being selected from the user terminal 200.

When the icon 410 is selected, the control unit 130 may determine that there is a request for an activation function for an augmented reality function associated with an item sold in a live broadcast of a broadcast video.

As shown in (a) of FIG. 4A, the control unit 130 activates the augmented reality function in one area of the display unit while the broadcast image 10 is being provided on the display unit of the user terminal 200. The linked icon 410 can be controlled to be displayed.

The control unit 130 may control the display unit of the user terminal 200 so that the icon 410 linked to the augmented reality function is displayed in an overlapping manner on the broadcast video 10 played in real time.

The icon 410 may be linked to an augmented reality function linked to an item sold (or introduced) in a live broadcast of a broadcast video provided with the icon 410.

When the icon 410 is selected, an augmented reality function for the item currently being introduced in the broadcast video may be provided to the user terminal.

In the present invention, the augmented reality function can provide an augmented reality function linked to the item according to the item being introduced in the broadcast video.

As an example, when selling the first item (ex: “lipstick”) in a live broadcast, item information (or product information) of the first item may be provided in the broadcast video, and in this case, the person selling the first item The icon 410 displayed together with the broadcast image of the live broadcast may be linked to an augmented reality function linked to the first item.

Specification of the item being introduced in the broadcast video can be made based on item information 430 displayed together with the broadcast video 10, as shown in FIG. 4A.

That is, the control unit 130 may provide an augmented reality function corresponding to the item information 430 according to the item information 430 provided in the broadcast video 10. For example, when selling a second item (ex: “foundation”) in a live broadcast, item information (or product information) of the first item may be provided, and in this case, the live broadcast selling the second item may be provided. The icon 410 displayed together with the broadcast image 10 may be linked to an augmented reality function linked to the second item.

Meanwhile, in the broadcast video 10, there is a specified area where the item information 430 is provided, and the item information for which item is provided in the specific area where the item information 430 is specified to be provided. Accordingly, different augmented reality graphics processing can be performed. In this way, the control unit 130 is based on the fact that the broadcast image 10 provided on the display unit of the user terminal 200 changes from a broadcast image selling the first item to a broadcast image selling the second item. Thus, when the icon 410 on the display unit of the user terminal 200 is selected, augmented reality graphic processing linked to the second item can be performed.

That is, when the icon 410 is selected while a broadcast image selling the first item is being output, the control unit 130 performs augmented reality graphic processing linked to the first item on the camera image 20, If the icon 410 is selected while a broadcast image selling the second item is being output, the control unit 130 may perform augmented reality graphic processing linked to the second item on the camera image 20.

Meanwhile, the specification of which item in the broadcast video is being output can be determined based on which item the item information provided with the broadcast video is for.

That is, as described above, depending on which item the item information is output to in the specific area where the item information is output, the control unit 130 determines the camera image corresponding to the item information being output in the specific area. Augmented reality graphics processing can be performed.

Meanwhile, if the augmented reality function is not linked to the item according to the item information 430 being output in a specific area, the icon 410 for activating the augmented reality function may not be output.

That is, in the present invention, whether or not the augmented reality function is applied may differ for each item, and the control unit 130 displays an augmented reality function icon in the broadcast video only when an item linked to the augmented reality function is introduced in the broadcast video. can be provided.

The “request to activate an augmented reality function” described below can be understood as a request to activate an augmented reality function related to a product sold in a live broadcast of a broadcast video provided through the display unit of the user terminal 200.

Additionally, the “augmented reality function” described below can be understood as an augmented reality function related to products sold in a live broadcast of a broadcast video provided through the display unit of the user terminal 200.

Meanwhile, the live broadcasting providing system 100 according to the present invention is a live broadcasting server 300 through software (ex: web browser (wep-browser), mobile browser) that displays web information on the screen. The broadcast video 10 received from can be provided to the display unit of the user terminal 200.

The control unit 130 controls the augmented reality related to the live broadcast from the user terminal 200 while the broadcast image 10 is displayed on the display unit of the user terminal 200 through software that displays web information on the screen. You may receive activation requests for features.

In this case, as shown in (a) of FIG. 4B, the control unit 130 is an application or software that provides an augmented reality function (hereinafter referred to as an augmented reality application) based on receiving an activation request for the augmented reality function. ) to display on the display unit of the user terminal 200 a pop-up window containing a guidance phrase (ex: “Do you want to go to the shopping live app to preview AR makeup?”). You can.

The control unit 130 executes the augmented reality function application based on receiving an augmented reality function application execution request from the user terminal 200 while the guide text 440 is displayed on the user terminal 200. You can.

In addition, if the augmented reality function application is not installed in the user terminal 200, the control unit 130 displays a page related to installing the augmented reality application to the user terminal to induce installation of the augmented reality function application. (200) can be provided.

Next, in the method for providing live broadcasting according to the present invention, in response to a request for activation of the augmented reality function, an item linked to the live broadcast is selected for the camera image 20 received from the camera provided in the user terminal 200. A process of performing augmented reality graphic processing related to may proceed (S340, see FIG. 3).

Even when there is a request to activate the augmented reality function, the control unit 130 may continue to perform first data processing related to providing broadcast images and perform second data processing related to providing augmented reality services.

Here, the second data processing related to providing augmented reality service can be understood as activating a camera in the user terminal 200 and performing augmented reality graphic processing on the camera image 20 received from the activated camera. there is.

At this time, the control unit 130 may access the camera provided in the user terminal 200 based on the access authority to the camera provided in the user terminal 200.

The control unit 130 may perform control related to activation of a camera provided in the user terminal 200. The control unit 130 may check access rights to the camera provided in the user terminal 200 in the live provision system 100 according to the present invention in order to perform control related to camera activation.

As a result of confirmation, if there is no access right to the camera provided in the user terminal 200, the control unit 130 uses (b) in FIG. 4B to obtain access right to the camera provided in the user terminal 200. ), information related to obtaining access rights may be output on one area of the display unit of the user terminal 200. For example, the control unit 130 may output a pop-up or modal window (or modal window) related to obtaining access rights.

The pop-up or modal window may include information related to obtaining access rights (ex: “Camera access is required to use the real-time virtual makeup preview shooting function”).

The control unit 130 may obtain access rights to the camera provided in the user terminal 200 based on user input to one area of the pop-up or modal window.

Such a pop up or modal window may be exposed only once when there is a first activation request for the augmented reality function.

After gaining access to the camera provided in the user terminal 200 once, the control unit 130 displays the pop-up or modal window even if there is a request to activate the augmented reality function. It can be controlled not to be output on the display unit of the user terminal 200.

Meanwhile, as a result of confirmation, if access rights to the camera provided in the user terminal 200 exist, the control unit 130 may control the camera provided in the user terminal 200 to be activated. That is, the control unit 130 can run the camera provided in the user terminal 200.

In this case, the control unit 130 may control activation of either a first type camera provided on the front of the user terminal 200 or a second type camera provided on the back. For example, the control unit 130 may control the user terminal 200 so that the first type of camera is activated in priority over the second type of camera.

The control unit 130 may set the default value (or default value) to activate the first type of camera provided on the front of the user terminal 200 based on an activation request for the augmented reality function.

In this way, based on a request for activation of the augmented reality function, the control unit 130 activates the camera of the user terminal in conjunction with the activation of the augmented reality function so that a camera image is received from the camera of the user terminal 200. You can.

Furthermore, the control unit 130 may receive the camera image 20 from the camera of the activated user terminal 200.

Here, the “camera image 20” may include at least one of a photo (or image), a video, and a preview screen captured through a camera. That is, in the present invention, augmented reality graphic processing can be performed on at least one of a photo (or image), video, and preview screen received through a camera. However, in the following, they will not be distinguished and all will be explained by expressing them as camera images 20.

The control unit 130 may perform augmented reality graphics processing on the camera image 20 based on the camera image 20 being received from the camera provided in the user terminal 200.

In the present invention, performing graphic processing means visual modification of at least a portion of the camera image 20 or visual modification of the camera image 20 based on product information that exists in matching with an item sold in a live broadcast. It may mean synthesizing information.

As shown in FIG. 7, in a pre-specified DB (database), mapping information 720 including product information (or item information, 621, 622) for each item associated with a specific live broadcast 700 is provided. It can exist. Product information 621 of a specific item (ex: first item) included in this mapping information 720 may include graphic information 721c about graphics processing performed in the augmented reality function related to the specific item. You can. The graphic information 621c of a specific item (ex: first item) may include “subject information” that specifies a part of the subject that needs to be visually transformed in the camera image 20. This subject information can be set based on the type of item.

For example, if the item is a product for lip makeup, the subject information includes information specifying the lip portion of the user's face, and if the item is a product for eye makeup, the subject information includes information specifying the lip portion of the user's face. The information may include information specifying the eye portion of the user's face.

As another example, if the item is a product for an ear accessory, the subject information includes information specifying the ear part of the user's face, and if the item is a product for a hand accessory, the subject information may include information specifying the hand part of the user's body.

As another example, if the item is a product for tops (clothing worn on the upper half of the body), the subject information includes information specifying the upper half (or upper body) of the user's body, and the item is for bottoms (lower body). In the case of a product for clothing (clothing worn on the lower half of the body), the subject information may include information specifying the lower half (or lower body) of the user's body.

The control unit 130 may use the subject information to specify a part corresponding to the subject information in the camera image and perform augmented reality graphic processing focusing on the specified part.

Meanwhile, it is also possible that graphic information for graphics processing performed in a related augmented reality function does not include subject information. In this case, the control unit 130 can use image recognition technology to recognize a subject in a camera image and specify a part that is the target of augmented reality graphic processing based on the type of item in the recognized subject. In this case, the control unit 130 can specify a part in the camera image that corresponds to the type of item based on artificial intelligence technology.

Furthermore, the graphic information 621c of a specific item (ex: first item) may include “type information” about the type of graphic processing for visual transformation of the camera image 20. This graphic type information can be set based on information received from the seller terminal.

Here, examples of graphic processing types for visual transformation are very diverse, for example, i) a graphic processing type that performs color conversion (or color calibration) on the camera image 20, ii) camera A graphic processing type that adjusts at least one of the brightness, contrast, sharpness, and color conversion values of the image 20, iii) inserting a graphic object such as text, a static image, or a dynamic image into the camera image 20 ( or compositing) graphics processing types may exist. The control unit 130 may perform graphics processing on the camera image according to one type or a plurality of types among the various graphics processing types described above.

Furthermore, the graphic information 621c of a specific item (ex: first item) includes “color information” about the color that should be expressed in the area corresponding to a part of the subject by visual transformation of the camera image 20. You can. For example, if the item is a lip makeup product, the color information includes the color that should be expressed in the area corresponding to the lip area (ex: “Color code B46143” or “AR setting code AR123”) ) information may be included.

Furthermore, the shape of a graphic object that must be synthesized (or inserted) into the graphic information 621c of a specific item (ex: first item) in an area corresponding to a part of the subject by visual transformation of the camera image 20. Information may be included. For example, if the item is a product for an ear accessory, the shape information may include information about a graphic object in the shape of an ear accessory.

Meanwhile, the control unit 130 can check whether the camera image 20 includes a part of the subject that needs to be visually transformed, based on the graphic information discussed above.

As a result of confirmation, if the camera image 20 includes a part of the subject that needs to be visually transformed, the control unit 130 selects a specific part (or specific area) of the camera image 20 corresponding to the part of the subject. It can be specified.

For example, if the graphic information 621c of a specific item (ex: first item) includes information specifying the “lip part” of the user's face as the part that needs to be visually transformed, the control unit 130 uses the camera The user's face can be recognized in the video 20. And as shown in (b) of FIG. 4A, the control unit 130 can specify a specific part (or specific area, 420) corresponding to the lips of the user's face in the camera image 20.

As another example, if the graphic information 621c of a specific item (ex: first item) includes information specifying the eye part of the user's face as a part that needs to be visually transformed, the control unit 130 may control the camera image ( In 20), a specific part (or specific area) corresponding to the eyes of the user's face can be specified.

Furthermore, the control unit 130 may perform graphics processing using a graphics processing type according to matching information matched to a specific portion (or specific area, 420) of the camera image 20.

If the graphics processing type matched to the specific part is a type that performs color conversion (or color calibration) on the camera image 20, the control unit 130 controls the color of the specific part (or specific area). can be changed to correspond to color information.

For example, as shown in (b) of FIG. 4A, a specific part (or specific area, 420) of the camera image 20 corresponding to the lip portion of the user's face is displayed so that the color corresponding to the color information is visible. Colors may change.

Through this color change or color correction, users can experience the same effect as applying makeup to their face image included in the camera video.

As another example, when the graphics processing type is a type that inserts (or combines) a graphic object into the camera image 20, the control unit 130 inserts a graphic object corresponding to shape information in a specific part (or specific area). Can be inserted (or synthesized).

For example, the control unit 130 inserts a graphic object of an ear accessory (ex: earring) corresponding to shape information into a specific part (or specific area) of the camera image 20 corresponding to the ear part of the user's face. (or can be synthesized).

By inserting these graphic objects, users can experience the same effect as wearing an accessory on their own image included in the camera image.

In this way, the control unit 130 may perform graphics processing on the camera image 20 in response to a request to activate the augmented reality function from the user terminal 200.

Meanwhile, product information for items that are not linked to the augmented reality function may not include graphic information.

Furthermore, the specified DB discussed above may exist and be stored on a live broadcast server or an external server linked to the live broadcast server.

Meanwhile, product information of items such as the product information of the first and second items discussed above may exist and be stored in the item DB of the live broadcast server. Furthermore, this item DB can be classified in connection with a specific seller account. That is, item information on items sold by a specific seller account can be divided based on each specific seller account to form an item DB.

Meanwhile, product information included in the pre-specified DB can be selected from the seller or live broadcast server from this item DB and included in the pre-specified DB.

Next, in the method for providing live broadcasting according to the present invention, while the broadcast image 10 is being received at the user terminal 200, the camera image 20 on which augmented reality graphic processing has been performed is sent to the user terminal 200. The process of displaying on the display unit may proceed (S350, see FIG. 3).

As described above, the control unit 130 may simultaneously perform first data processing related to providing the broadcast image 10 and second data processing related to providing the augmented reality service even if there is a request to activate the augmented reality function. . That is, the control unit 130 can control the user terminal 200 so that the first data processing and the second data processing are performed in parallel without stopping the first data processing.

As shown in (b) of FIG. 4A, the control unit 130 displays the camera image 20 on which augmented reality graphic processing has been performed along with the broadcast image 10 on the display unit of the user terminal 200. This can be controlled as much as possible, and in the present invention, this can be called “PIP (Picture In Picture) mode”.

In response to a request to activate the augmented reality function, the control unit 130 controls the display unit of the user terminal 200 in PIP mode to display the broadcast image 10 and the camera image 20 on which augmented reality graphic processing has been performed. Can be provided together.

As such, in the present invention, the broadcast image 10 and the camera image 20 on which augmented reality graphic processing has been performed can be simultaneously provided through the PIP mode by performing the first data processing and the second data processing in parallel. there is. Through this, users can experience or test products sold in the live broadcast through augmented reality without stopping watching the live broadcast.

Meanwhile, the live provision system 100 according to the present invention improves the user's interest (or interest) in the live broadcast, intuitively provides the user with information about products sold through the live broadcast, and allows the user to It can provide a variety of user environments that allow users to conveniently use augmented reality services. Hereinafter, various user environments provided by the live broadcasting provision system 100 according to the present invention will be described in detail.

As shown in (b) of FIG. 4A, the display unit of the user terminal 200 may include at least one of a main display area (A) and a sub-display area (B).

The sub-display area (B) may be displayed overlapping at least a portion of the main display area (A).

The position where the sub-display area (B) overlaps with the main display area (A) can be set by the system administrator or changed based on user input. Furthermore, of course, the output size of at least one of the main display area (A) and the sub-display area (B) can also be set and changed based on system administrator and user input.

In the above example, the case where the sub-display area (B) overlaps the main display area (A) was examined, but the present invention is not limited to the above example. For example, the display unit of the user terminal 200 may be divided into a plurality of display areas, one of which may be the main display area (A) and the other may be the sub-display area (B). At this time, the main display area (A) and the sub-display area (B) may not overlap each other, or may only partially overlap.

Meanwhile, the control unit 130 may control the broadcast image 10 to be displayed in at least one of the main display area (A) and the sub-display area (B) based on whether the augmented reality function is activated.

As shown in (a) of FIG. 4A, when the augmented reality function is deactivated, the control unit 130 can control the broadcast image 10 to be displayed on the main display area (A).

On the other hand, as shown in (b) of FIG. 4A, in the activated state of the augmented reality function, the control unit 130 displays the broadcast image 10 in the sub-display area (B) and displays the camera image (or The camera image 20) on which graphic processing has been performed may be displayed in the main display area (A). Hereinafter, for convenience of explanation, the camera image 20 and the camera image 20 on which graphic processing has been performed will be used interchangeably.

Based on the augmented reality function being switched from the deactivated state to the activated state, the control unit 130 operates the display unit 210 so that the broadcast image 10 disappears from the main display area (A) and the camera image 20 is displayed. You can control it. Additionally, the control unit 130 may control the display unit 210 to display the broadcast image 10 in the sub-display area (B). That is, the control unit 130 operates the display unit so that the broadcast image 10 displayed in the main display area (A) continues to be displayed through the sub-display area (B) even if the augmented reality function is switched from the deactivated state to the activated state. (210) can be controlled. Through this, the user can continue to watch the broadcast video 10 being watched without interruption while receiving the augmented reality service.

Meanwhile, as shown in (a) of FIG. 5, the control unit 130 operates in a state where the broadcast image 10 is provided on the display unit (or main display area A) of the user terminal 200. , it can be controlled so that various information related to the broadcast image 10 is displayed in at least one area of the display unit of the user terminal 200.

The control unit 130 may control various information to be displayed together with the broadcast video 10 of a specific live broadcast, based on various information that matches the specific live broadcast in a pre-specified DB.

In the present invention, as shown in (a) of FIG. 5, various information along with the broadcast image 10 are displayed through the display unit of the user terminal 200 by “at least one area of the broadcast image 10.” It can be expressed as “displaying (or outputting or providing) information” or “displaying (or outputting or providing) a broadcast video (10) containing various information.”

As seen above, in a pre-specified DB, channel information 710 and mapping information 720 related to a specific live broadcast 700 may be matched and exist.

Channel information 710 related to a specific live broadcast 700 may include information used to proceed and watch the specific live broadcast 700. For example, the channel information 710 includes i) title information 711 of a specific live broadcast 700, ii) viewer information watching a specific live broadcast (ex: number of viewers, 712), iii) It may be included in the host information 713 conducting a specific live broadcast.

In the channel information 710, i) title information 711 and iii) host information 713 may be set based on information received from the seller terminal. On the other hand, in the channel information 710, ii) viewer information 712 may be generated (or set) by counting the number of viewers watching a specific live broadcast 700 by the live broadcast server 300. there is.

The control unit 130 controls the user for which the broadcast video 10 of the specific live broadcast 700 is displayed, as shown in (a) of 5a, based on the channel information 710 related to the specific live broadcast 700. The channel information 710 may be displayed in at least one area of the display unit of the terminal 200. At this time, the control unit 130 may display the channel information 710 on the display unit as text, a graphic object, or an icon.

For example, as shown in (a) of 5a, the first area 510 of the display unit 510 of the user terminal 200 where the broadcast video 10 is displayed includes title information (ex. : “Monthly Gift Shop Episode 8: Black Friday Special”), viewer information (ex: “Number of viewers: 331, 425”) watching a specific live broadcast (700) may be displayed. A user watching a specific live broadcast can intuitively recognize information about the live broadcast being watched just by looking at the display unit.

Furthermore, as shown in FIG. 7, the mapping information 720 related to a specific live broadcast 700 may include product information 721 and 722 related to each of at least one item associated with the specific live broadcast. . Taking a specific item (ex: first item) as an example and describing the product information 721 related to the specific item, the product information 721 includes i) linkage information 721a indicating that the specific item is linked to the augmented reality function; , ii) Item information that can be used for the shopping service of a specific item (ex: product name of a specific item, price information of a specific item, detailed information of a specific item used to describe a specific item, 721b), iii) Specific item and It may include at least one of graphic information (ex: color information, 721c) used to perform related graphic processing, and vi) target information (721d) that induces the experience or purchase of a specific item. The product information 721 may be set based on information received from the seller terminal.

The control unit 130 controls the user for whom the broadcast video 10 of the specific live broadcast 700 is displayed, as shown in (a) of 5, based on the mapping information 720 related to the specific live broadcast 700. The mapping information 720 may be displayed in at least one area of the display unit of the terminal 200. At this time, the control unit 130 may display the mapping information 710 on the display unit as text, a graphic object, or an icon.

As shown in (a) of FIG. 5, in the second area 510 of the display unit 510 of the user terminal 200 where the broadcast image 10 is displayed, an augmented reality display related to an item associated with a specific live broadcast 700 is displayed. A function icon 410 may be displayed to provide a function. The function icon 410 may be linked to an augmented reality function for at least one item associated with a specific live broadcast 700.

The icon 410 linked to the augmented reality function displays the broadcast image 10 and the broadcast image 10 on the display unit of the user terminal 200, based on whether the item linked to the live broadcast is matched to the augmented reality function. can be displayed together.

When an item linked to a live broadcast is matched to the augmented reality function, the control unit 130 displays the augmented reality function in the first area 520 of the display unit 520 of the user terminal 200 where the broadcast image 10 is displayed. An icon 410 linked to can be displayed.

More specifically, the control unit 130 includes augmented reality function linkage information indicating that at least one item related to the specific live broadcast 700 is linked to the augmented reality function in the mapping information 720 of the specific live broadcast 700. Based on the inclusion of 721a), an icon 410 linked to the augmented reality function can be displayed in the second area 520.

On the other hand, if the item linked to the live broadcast is not matched to the augmented reality function, the control unit 130 controls the user terminal 200 on which the broadcast image 10 is displayed, as shown in (b) of FIG. 5. ) may be restricted from displaying icons linked to the augmented reality function in the first area 520 of the display unit.

More specifically, the control unit 130 operates the second area ( Display of the icon 410 linked to the augmented reality function in 520 may be restricted.

Furthermore, as shown in (a) of FIG. 5, on the display unit of the user terminal 200, around the icon 410 linked to the augmented reality function, a guidance message to induce activation of the augmented reality function ( ex: “Experience the color with AR makeup preview”530) may be displayed.

This guidance message 530 may be output on the display unit 210 only for a preset time after the product information of the item linked to the augmented reality function is output on the broadcast video.

Furthermore, this guidance message 530 may be output on the display unit 210 at preset time intervals and then disappear while product information of an item linked to the augmented reality function is output on a broadcast video.

Furthermore, this information message 530 is displayed on the display unit 210 only when the augmented reality function has never been activated on the user terminal, or when the augmented reality function has never been activated on the user terminal while watching a live broadcast. It can be printed on the screen. Meanwhile, the user can use a service to experience or test a product (or item) sold in a specific live broadcast 700 that is currently being watched through an information message 530 or an icon 410 linked to the augmented reality function. You can intuitively recognize that it can be used.

Subsequently, as shown in (a) of FIG. 5, in the third area 530 of the display unit 530 of the user terminal 200 where the broadcast image 10 is displayed, there is at least one image associated with a specific live broadcast 700. Various item information about the product (or item) may be displayed. As seen above, the icon 520 linked to the augmented reality function is displayed in the second area 520 only when the item according to the item information output in the third area 530 is linked to the augmented reality function. can be printed.

Meanwhile, the product information displayed in the third area 530 includes the product name of the item introduced in the live broadcast (ex: “Lira Powder Matte Lip”), the price information of the item (ex: “40,000 won”), and the description of the item. It may contain at least one piece of information.

Continuing, as shown in (a) of FIG. 5, in the fourth area 540 of the display unit of the user terminal 200 where the broadcast video 10 is displayed, chatting related to a specific live broadcast 700 is displayed. ) information can be displayed. Chat information may include messages received from a seller terminal or a plurality of consumer terminals while a specific live broadcast 700 is in progress. The control unit 130 may display messages received in real time from a seller terminal or a plurality of consumer terminals through the fourth area 540 of the display unit.

In this way, while the broadcast image 10 of a specific live broadcast is displayed on the display unit of the user terminal 200, the control unit 130 provides various information related to the specific live broadcast on the display unit. , users can recognize various information related to a specific live broadcast at once through one display unit.

Meanwhile, in the present invention, various information related to the augmented reality function is provided along with the camera image 20 to the display unit (or main display area (A)) of the user terminal 200, so that the user can view items through augmented reality. Experience services can be used more conveniently and intuitively.

As shown in FIG. 6, the control unit 130 performs augmentation on at least one area of the display unit of the user terminal 200 while the camera image 20 is provided on the display unit of the user terminal 200. You can control the display of various information related to reality functions.

As shown in FIG. 6, various information associated with the augmented reality function along with the camera image 20 are displayed through the display unit of the user terminal 200, which means “information in at least one area of the camera image 20.” It can be expressed as “display (or output or provide)” or “display (or output or provide) a camera image 10 containing various information.” At this time, the camera image 20 may be a camera image 20 on which augmented reality graphic processing has been performed. Various information associated with the augmented reality function may also be displayed on the display unit based on various information that exists in a pre-specified DB in matching with a specific live broadcast.

As shown in FIG. 7, mapping information 720 including product information 721 and 722 for each item associated with a specific live broadcast 700 may exist in the pre-specified DB. Each product information 721, 722 of a plurality of items (ex: first item and second item) includes graphic information 721c, 722c about graphics processing performed in an augmented reality function related to each of the plurality of items. may be included.

As described above, the graphic information 721c and 722c of a specific item includes i) “subject information” that specifies a portion of the subject for which visual transformation of the camera image 20 is to be performed in relation to the specific item, ii) In relation to a specific item, “type information” about the type of graphics processing for visual transformation of the camera image 20, iii) in relation to a specific item, corresponding to a part of the subject by visual transformation of the camera image 20 “Color information” about the color that must be expressed in the area, and vi) in relation to a specific item, a graphic object that must be synthesized (or inserted) in the area corresponding to a part of the subject by visual transformation of the camera image 20 At least one of the “shape information” may be included.

The control unit 130 displays the camera image 20, as shown in (b) of FIG. 5, based on the graphic information 721c and 721c of each of the plurality of items associated with the specific live broadcast 700. The graphic information 721c and 722c may be displayed in at least one area of the display unit of the user terminal 200. At this time, the control unit 130 may display the graphic information 721c and 722c as text, a graphic object, or an icon on the display unit.

These graphic information 721c and 722c may each correspond to a plurality of options set for the item. Here, the option is selection information for specifying the item you want to purchase when purchasing an item. If the item is a product corresponding to lip makeup (ex: lipstick), the option is the color of the lip tip. It could be information.

Hereinafter, for convenience of explanation, an example will be given where the item associated with a specific live broadcast 700 is a lip makeup product.

As shown in FIG. 6, in the first area 610 of the display unit of the user terminal 200 where the camera image 20 is displayed, an item associated with a specific live broadcast 700 (or an item associated with an augmented reality function) is displayed. A graphic object (for example, reference numerals 611 and 612) corresponding to an option set in an item may be output. The graphic object corresponding to these options may be a graphic object corresponding to color information corresponding to each option.

In this case, the camera image 20 may be a camera image 20 on which graphic processing has been performed using any one 611 of color information corresponding to a plurality of options corresponding to the item. For example, the camera image 20 is a camera image 20 that has been color converted to correspond to color information corresponding to the graphic object 611 included in a specific partial area 610a of the first area 610. may apply. For example, in the camera image 20, the area 420 corresponding to the lip portion of the user's face is color converted into color information corresponding to the graphic object 611 included in the specific partial area 610a. There may be.

Furthermore, the control unit 130, based on receiving a preset user input applied to the first area 610 while the camera image 20 is displayed on the display unit of the user terminal 200, Augmented reality graphic processing for the camera image 20 can be changed.

More specifically, the control unit 130 is displaying a camera image 20 on which graphic processing has been performed corresponding to color information corresponding to the first option of a specific item on the display unit of the user terminal 200. In, based on the graphic object 612 corresponding to the color information of the second option different from the first option being selected by a preset user input applied to the first area 610, the camera image 20 Graphics processing according to the color information of the second option may be performed.

For example, in a state where a graphic object 611 corresponding to the color information of the first option is displayed in the specific partial area 610a, the Since the graphic object 612 corresponding to the color information of the second option is displayed in the specific partial area 610a, the graphic object 612 corresponding to the color information of the second option may be selected.

That is, the control unit 130 moves the graphic object 612 corresponding to the color information of the second option to the specific partial area 610a based on a preset user input (e.g., drag or swipe input). When moving, the graphic object 612 corresponding to the color information of the second option may be processed as selected. In this case, the control unit 130 may perform augmented reality graphic processing on the camera image using the color information of the second option.

As another example, while the graphic object 611 corresponding to the color information of the first option is displayed in the specific partial area 610a, a graphic object corresponding to the color information of the second option is sent from the user terminal 200 ( When 612) is selected, the control unit 130 may process the graphic object 612 corresponding to the second color information as being selected.

In this case, the control unit 130 may perform augmented reality graphic processing on the camera image using the color information of the second option.

Meanwhile, in the specific partial area 610a, a graphic object 612 corresponding to the color information of the second option may be located instead of the graphic object 611 corresponding to the color information of the first option. In this way, the user can select one of the plurality of graphic objects 611 and 612 displayed in the first area 610 and experience various items.

Meanwhile, when a graphic object located in the specific partial area 610a is selected from the user terminal 200, the control unit 130 may capture a camera image processed with augmented reality graphics. In this case, the camera image processed with augmented reality graphics may be stored in the storage of the user terminal or an external server (live broadcast server or cloud server, etc.).

Meanwhile, as shown in FIG. 6, in the display unit of the user terminal 200 on which the camera image 20 is displayed, in the second area 620 around the specific partial area 610a, the specific partial area 610a is displayed. Among the descriptive text (ex: “301 Spring Date”) that describes the option corresponding to the graphic object 611 included in (610a) and the recommendation text that recommends the item corresponding to the option (ex: “Popular Color”) At least one may be displayed.

These explanatory phrases and guidance phrases may also exist in matching with the item in the previously specified product DB discussed above. For example, the explanatory text and guidance text may be included in target information 621d that induces experience or purchase of the item.

Continuing, as shown in FIG. 6, in the third area 630 of the display unit of the user terminal 200 where the camera image 20 is displayed, at least one product (or Various item information about the item is displayed, and can be used as the product name of the item (ex: “Lira Powder Matte Lip”), used as price information of the item (ex: “40,000 won”), or used to describe the item. It can be used in various ways.

At this time, the third area 630 may include option information corresponding to a specific item on which graphic processing has been performed on the camera image 20. The control unit 130 is configured to display the camera image 20 on which graphic processing has been performed according to the color information of the first option on the display unit of the user terminal 200, in the third area 630. It is possible to control the display of the first option information corresponding to the item. In addition, the control unit 130 performs graphic processing according to the color information of the second option on the camera image 20 based on a preset user input for the first area 610, and performs graphic processing according to the color information of the second option. 3 It can be controlled to display information corresponding to the second option on area 630.

Meanwhile, as described above, when the augmented reality function is deactivated, the broadcast image 10 is displayed in the main display area (A) of the user terminal 200 (see (a) in FIG. 4A), and the augmented reality function is activated. In this activated state, the camera image 20 may be displayed on the main display area (A) of the user terminal 200 (see (b) in FIG. 4A).

As shown in FIG. 4A, the control unit 130 displays, independently of whether the augmented reality function is activated, an item related to the live broadcast in at least one area on the main display area (A) of the user terminal 200. You can control item information to be displayed continuously.

More specifically, the control unit 130 displays a camera image ( Even if 20) is displayed, the item information 430 displayed together with the broadcast image 10 can be controlled to continue to be displayed together with the camera image 20.

Through this, the user can continue to check what item is being experienced through augmented reality even when the augmented reality function is activated.

Meanwhile, as described above, the live broadcast providing system 100 according to the present invention not only can display various information on the display unit of the user terminal 200, based on a pre-specified DB, but also can display various information in the live broadcast. You can decide whether to provide augmented reality features for your product.

The pre-specified product DB according to the present invention can be constructed based on information received from the seller terminal, and can be the basis for providing various information and augmented reality functions in the live broadcast providing system 100 according to the present invention. there is. That is, in the present invention, a pre-specified product DB can be constructed based on information received from the seller terminal, and a live broadcast provision service and an augmented reality function provision service can be performed based on the constructed pre-specified product DB.

This pre-specified product DB can be built by the live broadcast server 300 or by a separate external server 100a linked to the live broadcast providing system 100 according to the present invention. Hereinafter, the live broadcast server 300 and the separate external server 100a linked to the live broadcast providing system 100 according to the present invention are not distinguished, and the pre-specified product DB is stored by the live broadcast server 300. Let's explain it by constructing it.

The live broadcast server 300 builds a pre-specified product DB using the information received from the seller terminal, and includes it in the pre-specified product DB in response to a request from the live provision system 100 according to the present invention. Various information (or data) can be provided to the live provision system 100 according to the present invention.

The live broadcast server 300, in order to receive at least one of information related to the live broadcast, information about the product sold in the live broadcast, and information related to the augmented reality function of the product, from the seller terminal, as shown in FIGS. 8, 9, and As shown in Figure 10, seller pages 800, 900, and 1000 can be provided to the seller terminal.

Here, the seller pages (800, 900, 1000) allow sellers who want to sell products through live broadcasting to provide i) information related to the live broadcast (ex: broadcast title information, broadcast host information, etc.), ii) products for sale, and Enter and set related information (ex: item type information, item name information, item price information, item manufacturer information, recommendation information for recommending an item, item sales statistics information, etc.) and ii) various information related to the augmented reality function. It can be understood as a page (or screen) that provides functions that allow

In particular, the live broadcast server 300 can provide a function to receive various information for matching augmented reality functions to items through the seller pages 800, 900, and 1000.

As shown in FIG. 8, the seller page 800 may include a function icon 810 that can set whether or not the augmented reality function matches for a specific item.

Based on receiving a user input for the function icon 810, the cloud server displays a screen (ex: pop-up) linked to a page for matching augmented reality functions in one area of the seller page 800. , 820) can be displayed.

The live broadcast server 300 responds to receiving a request for a page for augmented reality function matching (ex: “Settings” selection, 821) through the screen 820, as shown in FIG. 9. , A seller page 800 can be provided for matching augmented reality functions for a specific item.

The seller page 900 for matching the augmented reality function for a specific item includes i) an area 910 for registering specific item information (ex: “product number”) matching the augmented reality function, ii) the specific item’s Area 920 for registering an image, iii) Area 930 for registering an option name (ex: item name or item number) related to a specific item, vi) inputting graphic information (ex: color information) related to the specific item It may include at least one of areas 940 and 950 and v) an area 960 where information used to recommend or explain the specific item is input.

The live broadcast server 300 matches the augmented reality function to a specific item based on the information input (or received) through the seller page 900, and links various information related to the specific item and the augmented reality function. A DB for already specified products can be built.

Furthermore, as shown in FIG. 10, on the seller page 1000, an icon 1010 that provides a modification function for a specific item previously matched to the augmented reality function, an icon 1020 that provides a delete function, and history information It may include at least one of the icons 1030 matched to the provision.

The user can edit or add information (ex: color information) about a specific item already matched to the augmented reality function by using the icon 1010 that provides the editing function on the seller page 1000 shown in FIG. 10. there is. Additionally, the user can select the icon 1020 that provides a delete function so that a specific item is no longer matched to the augmented reality function.

Furthermore, the live broadcast server 300, in response to a user input for the icon 1030 matched to provide history information, provides setting history information of the augmented reality function for a specific item on one area of the seller page 1000. (1040) can be provided.

As such, in the present invention, a service can be provided that allows the seller (or host) to directly set the settings for whether or not to provide the augmented reality function and the product experience provided through the augmented reality function, and the seller (or host)'s settings can be adjusted. Based on this, augmented reality functions can be provided for specific items.

Accordingly, sellers can provide consumers with the opportunity to easily experience their products through the seller page by simply entering information.

Meanwhile, as described above, item information related to items associated with a live broadcast may be continuously displayed on the display unit of the user terminal 200, regardless of whether the augmented reality function is activated.

The item information may include item information (ex: specific item name, price, etc.) related to one of a plurality of items associated with the live broadcast. The user can receive information about a specific item through the item information. Furthermore, the user may wish to receive additional item information about a wider variety of items in addition to item information about a specific item.

Accordingly, in the present invention, item information of additional items linked to the live broadcast can be provided on the display unit of the user terminal 200, independently of whether the augmented reality function is activated.

Here, “additional items” can be set in various ways. Among the plurality of items linked to the live broadcast, it may be understood as an item other than the item of item information displayed together with the broadcast video 10. For example, assume that item information of the first lip makeup item among the first lip makeup items and second lip makeup items associated with the live broadcast is displayed together with the broadcast video 10. In this case, the additional item can be understood as an item for the second lip makeup.

Furthermore, “additional items” can be understood as items related to items linked to the live broadcast. For example, let's assume that the item linked to the live broadcast is an item about lip makeup. In this case, the additional item may correspond to an item for eye makeup.

In the present invention, while item information of a specific item is being displayed on the display unit of the user terminal 200, item information of an additional item may be provided in response to a user request for item information of an additional item.

As shown in (a) of FIG. 11, the control unit 130 displays a graphic object linked to the additional item list while the broadcast image 10 is displayed on the main display area A of the user terminal 200. User selection for (ex: “See more”, 1140) can be received.

And, as shown in FIG. 11(b), in response to the user selection, the control unit 130 displays a plurality of additional item information items 1110, 1120, and 1130 in at least one area of the main display area A. ) can display an additional item information list 1100 including.

Each item information item (1110, 1120, 1130) of a plurality of items included in the additional item information list (1100) includes a thumbnail image of the item corresponding to each item, an item name (ex: “Lira Spicy Volume Mat”), It may include at least one of price information (ex: “39,000 won”) and event information.

Furthermore, based on the fact that the augmented reality function is matched to a specific item among the plurality of items included in the additional item information list 1100, an augmented reality function related to the specific item is displayed in one area 1120a of the item item of the specific item. An icon 1120b linked to the function may be included.

Meanwhile, even when the augmented reality function is activated, the control unit 130 may provide item information about additional items in response to a user request for item information about additional items.

As shown in (a) of FIG. 12, the control unit 130 displays a graphic object linked to the additional item list while the camera image 20 is displayed on the main display area A of the user terminal 200. User selection for 1240 may be received.

And, as shown in FIG. 12(b), in response to the user selection, the control unit 130 displays a plurality of additional item information items 1210, 1220, and 1230 in at least one area of the main display area A. An additional item information list 1200 including can be displayed.

The item information items 1210, 1220, and 1230 of each of the plurality of items included in the additional item information list 1200 include a thumbnail image of the item corresponding to each item, an item name (ex: “Lira Spicy Volume Mat”), It may include at least one of price information (ex: “39,000 won”) and event information.

Furthermore, based on the fact that the augmented reality function is matched to a specific item among the plurality of items included in the additional item information list 1200, an augmented reality function related to the specific item is displayed in one area 1220a of the item item of the specific item. An icon 1240 linked to the function may be included.

In this way, the user can be provided with the augmented reality function for a specific item through the icon 410 linked to the augmented reality function displayed on the display unit of the user terminal 200 where the broadcast video 10 is displayed. Additionally, the user can be provided with the augmented reality function for the additional item through the icons 1120b and 1220b linked to the augmented reality function included in the additional item list 1100 and 1200.

The control unit 130 can activate the camera of the user terminal as long as the icon linked to the augmented reality function exposed in various situations is selected from the user terminal 200. Additionally, the control unit 130 may perform augmented reality graphic processing on the camera image received through the activated camera to provide the user with an experience of the item.

Meanwhile, in the present invention, data processing related to ending Picture In Picture (PIP) mode can be performed based on the augmented reality function switching from an activated state to a deactivated state.

The control unit 130 may switch the augmented reality function from the activated state to the deactivated state in response to receiving a deactivation request for the augmented reality function from the user terminal 200.

As shown in (a) of FIG. 13, the request for deactivation of the augmented reality function is based on selection of a graphic object (or icon, 1300) matched to the deactivation function of the augmented reality function from the user terminal 200. can be received.

More specifically, with the camera image 20 displayed in the main display area (A) and the broadcast image 10 displayed in the sub-display area (B), it is located in one area of the main display area (A). Based on the graphic object 1300 being selected, the control unit 130 may receive a deactivation request for the augmented reality function.

The graphic object 1300 matched to this augmented reality function deactivation function may be variously named “augmented reality function end button”, “PIP mode end button”, “return to broadcast video button”, etc.

As shown in (b) of FIG. 13, in response to receiving a deactivation request for the augmented reality function, the control unit 130 causes the sub-display area B to disappear on the display unit of the user terminal 200. You can control it. Also, the control unit 130 can control the display part of the user terminal 200 to include only the main display area (A).

Furthermore, the control unit 130 can control the broadcast image 10 to be displayed in the main display area A where the camera image 20 was displayed. Additionally, the control unit 130 may restrict the display of the camera image 20 on the display of the user terminal 200.

In this case, the broadcast image 10 displayed in the main display area (A) may be the broadcast image 10 displayed in the sub-display area (B).

That is, when the augmented reality function is switched from the activated state to the deactivated state, the control unit 130 can control the broadcast image 10 displayed in the sub-display area (B) to continue to be displayed in the main display area (A). .

When the augmented reality function is activated, the user can continue to watch the live broadcast he was watching and at the same time experience products sold in a specific live broadcast through the camera image 20 on which augmented reality graphic processing has been performed. Furthermore, when the augmented reality function is disabled, you can continue to watch only the live broadcast you were watching.

Meanwhile, in the present invention, a product purchase page for purchasing products can be provided to the user terminal 200 so that users watching a live broadcast can purchase products sold through the live broadcast.

The control unit 130 receives a preset input for specific item information from the user terminal 200 while at least one of the broadcast image 10 and the camera image 20 is displayed on the display unit of the user terminal 200. can receive.

As shown in (a) of FIG. 14, receiving a preset input for the specific item information includes item information (ex: representative image or thumbnail of the specific item, 1410) of the specific item from the user terminal 200. may be received based on a preset input being applied to the displayed area. The item information of the specific item may be linked to a product purchase page 1400 selling the specific item, as shown in (b) of FIG. 14 .

As shown in (b) of FIG. 14, the control unit 130, based on receiving a preset input for item information 1410 of a specific item from the user terminal 200, A product purchase page 1400 for purchasing a product may be provided on the user terminal 200. At this time, the product purchase page 1400 may be provided in a modal window in one area of the display unit of the user terminal 200.

Meanwhile, the product purchase page 1400 may be linked to item information displayed overlapping the broadcast image 10 when the augmented reality function is deactivated.

Furthermore, the product purchase page 1400 may be linked to item information displayed overlapping the camera image 20 when the augmented reality function is activated. That is, in the present invention, specific item information displayed on the display unit of the user terminal 200 may be linked to a product purchase page 1400 where a product corresponding to the specific item information can be purchased.

Meanwhile, as shown in (b) of FIG. 14, the product purchase page 1400 is displayed on the user terminal 200 while the user terminal 200 is receiving the broadcast video 10 of a live broadcast. It may be provided on the side.

The broadcast image 10 of the live broadcast may be continuously displayed in the background area (or screen) of the modal window including the product purchase page 1400.

When the broadcast image 10 is obscured by the modal window, the control unit 130 controls the broadcast sound provided by the broadcast image 10 to be continuously output through the user terminal 200. can do.

Furthermore, the control unit 130 can control the product payment process corresponding to a specific item sold on the product sales page to proceed based on a payment request being received from the user terminal 200 through the product purchase page 1400. there is.

This payment process can be performed by the electronic payment system according to the present invention. The control unit 130 may control the communication unit 110 so that the communication unit 110 communicates with a financial institution (bank or credit card) server. The control unit 130 may request a payment amount corresponding to a specific item from an account associated with the user account to the financial institution through the communication unit 110. However, the above-described procedure is only an example, and the procedure initially performed for the electronic payment process is not limited thereto.

Meanwhile, the user cannot activate the camera of the user terminal 200 or uses the broadcast video 10 rather than the camera video 20 received from the camera provided in the user terminal 200 to select a specific item. There may be a need to receive experience about . For example, a user may want to experience an item by applying lip makeup to the seller's face rather than to the user's own face. Additionally, the host of the live broadcast can also apply lip makeup items to his or her face and introduce the items to consumers.

Accordingly, the present invention can provide an augmented reality function that performs augmented reality graphic processing on the broadcast video 10.

As shown in (a) of FIG. 15, the control unit 130 controls the broadcast image 10 from the user terminal 200 while the broadcast image 10 is being provided on the display unit of the user terminal 200. ) can receive a request for activation of the augmented reality function.

A request to activate the augmented reality function using the broadcast image 10 may be received from the user terminal 200 based on selection of the icon 1500 linked to the augmented reality function using the broadcast image 10.

As shown in (b) of FIG. 15, the control unit 130 activates the broadcast image 20 received from the live broadcast server 300 based on a request for activation of the augmented reality function using the broadcast image 10. Augmented reality graphics processing can be performed.

The broadcast image 10 on which graphic processing has been performed may continue to be provided through the main display area A even if there is a request for activation of the augmented reality function. Additionally, in this case, the sub-display area B may not be displayed on the user terminal 200.

Meanwhile, although not shown, a first icon and a second icon linked to the augmented reality function may be displayed on the display unit of the user terminal 200. The first icon is an icon (410, see (a) in FIG. 4A) linked to the augmented reality function using the camera image 20 received from the camera of the user terminal 200, and the second icon is a broadcast image. It may be an icon 1610 (see (a) of FIG. 16) linked to an augmented reality function using (20).

The control unit 130 selects the second icon 1610 among the first icon 410 and the second icon 1610 and sends the camera image 20 based on receiving a request for activation of the augmented reality function. Augmented reality graphics processing can be performed.

In addition, although not shown, when the control unit 130 receives a request to activate the augmented reality function based on the icon linked to the augmented reality function using the broadcast image 10 from the user terminal 200 is selected. , a screen for selecting any one of the broadcast image 10 and the camera image 20 (e.g., a screen including a first graphic object corresponding to the broadcast image and a second graphic object corresponding to the camera image) ) can be provided.

Based on whether one of the broadcast image 10 and the camera image 20 is selected from the user terminal 200, the control unit 130 may perform augmented reality graphics processing on the selected image.

In addition, although not shown, the control unit 130 provides augmented reality graphics processing to the consumer's user terminal 200 when augmented reality graphics processing is performed on the broadcast image 10 in the seller's user terminal (or seller's terminal). A broadcast image 10 on which realistic graphics processing has been performed can be provided.

Also, although not shown. Assume that a request to activate an augmented reality function using the broadcast video 10 is received from the first user terminal while the broadcast video 10 of a live broadcast is displayed on the first user terminal and the second user terminal, respectively. let's do it. The control unit 130 may display the broadcast image 10 on which graphic processing has been performed on the first user terminal based on a request to activate the augmented reality function. In contrast, the display of camera images on which graphic processing has been performed may be restricted from being displayed on the second user terminal.

That is, in the present invention, when a request for activation of the augmented reality function using the broadcast image 10 is received from the seller terminal, the broadcast image on which graphics processing is performed on each of the first user terminal and the second user terminal watching the live broadcast (10) can be displayed.

On the other hand, when a request for activation of the augmented reality function using the broadcast image 10 is received from the first user terminal watching the live broadcast, the first user terminal displays the broadcast image 10 on which graphic processing has been performed, and , display of the broadcast image 20 on which graphic processing has been performed may be restricted in the second user terminal.

In addition, although not shown, while the broadcast image 10 on which graphic processing has been performed is displayed on the main display area A of the user terminal 200, the augmented reality function is received from the user terminal 200. An activation request may be received. In this case, the request to activate the augmented reality function may be a request to activate the augmented reality function using the camera image of the user terminal 200.

Based on the request to activate the augmented reality function, the control unit 130 displays the camera image 20 on which augmented reality graphic processing has been performed in the main display area (A) of the user terminal 200, and displays the camera image 20 on which augmented reality graphic processing has been performed. In area B, a broadcast image 10 on which augmented reality graphic processing has been performed can be displayed.

For example, in the main display area (A), a camera image with makeup applied to the user's face may be displayed, and in the sub-display area (B), a camera image with makeup applied to the live broadcast host's face may be displayed.

Meanwhile, the broadcast image of the live broadcast described above may be a real-time streaming image or may be a broadcast image stored in the live broadcast server 200 after the live broadcast has ended. Such broadcast video may also be expressed as recorded broadcast video. In the present invention, even when a recorded broadcast video is provided on the user terminal 200, an icon linked to the augmented reality function can be provided along with the recorded broadcast video. In this case, based on the icon being selected, the control unit 130 may provide a camera image processed with augmented reality graphics using a camera image from a camera provided in the user terminal, similar to the method discussed above. Accordingly, the method of providing augmented reality-related functions to recorded broadcasting images will be replaced with the explanation discussed above.

Meanwhile, as described above, a product (or item) according to the present invention may include at least one option information. In the present invention, various services linked to options can be provided so that users can easily experience augmented reality for each product option or purchase the product. Meanwhile, in the present invention, products and items can be used interchangeably.

Here, “option” is selection information for specifying the item you want to purchase when purchasing an item. If the item is a product corresponding to lip makeup (ex: lipstick), the option is a lip tip. It may be color information of .

Hereinafter, a method of providing live broadcasting linked to option information according to the present invention will be described along with the attached drawings. Figure 16 is a flowchart for explaining a method of providing option information for live broadcasts and products linked to live broadcasts in the present invention, and includes Figures 17, 18, 19, 20, 21, 22, and 23a. Figures 23b, 24a, 24b, 25, 26a, and 26b are conceptual diagrams for explaining a user environment that provides services linked to option information according to the present invention.

In the method for providing live broadcasting according to the present invention, a process of receiving broadcast video 10 of a live broadcast may be performed from the live broadcasting server 100 (S1610, see FIG. 16). The broadcast video 10 may be received from the live broadcast providing server 300 using a streaming technique.

In the method of providing live broadcasting according to the present invention, based on the reception of the broadcast image 10, a process of providing the broadcast image 10 on the main display area (A) of the display unit of the user terminal 200 may be performed. There is (S1620, see FIG. 16).

As described above, in the present invention, the display unit of the user terminal 200 may include either a main display area (A) or a sub-display area (B).

As shown in (b) of FIG. 4A discussed above, the sub-display area (B) may be displayed overlapping (or overlapping) at least a portion of the main display area (A).

In the present invention, when the display unit includes one display area, the one display area can be understood as the main display area (A). In the present invention, this can be explained as “the PIP mode of the display unit of the user terminal 200 is in a deactivated state.”

Meanwhile, in the present invention, the main display area (A) can be named “main area”, “first display area”, etc., and the sub display area (B) can be named “sub area”, “second display area”, etc. It can be named as

Next, in the method for providing live broadcasting according to the present invention, while the broadcast image 10 is being provided on the main display area (A), a request is made to activate an augmented reality function related to the live broadcast from the user terminal 200. The process of receiving may proceed (S1630, see FIG. 16).

As described above, the control unit 130 may receive an activation request for the augmented reality function based on the icon 410 linked to the augmented reality function being selected from the user terminal 200 (see Figure 4a). (see (a)).

The control unit 130 may switch (or change) the state of the augmented reality function from a deactivated state to an activated state based on receiving a request to activate the augmented reality function.

Furthermore, the control unit 130 may control the camera of the user terminal 200 to be activated in conjunction with the augmented reality function being activated.

Here, “camera activation” can be understood as enabling the camera to receive the camera image 20 from the camera.

Additionally, the control unit 130 may activate the camera of the user terminal 200 and receive the camera image 20 from the camera.

Here, the camera image 20 can be understood as at least one of a preview screen (or video) of an area recognized by the camera and a captured video (image or video) captured by the camera.

Next, in the method for providing live broadcasting according to the present invention, in response to the augmented reality function being activated according to the activation request, the camera image received from the camera provided in the user terminal 200 is linked to the live broadcast. A process of performing augmented reality graphic processing related to the item may proceed (S1640, see FIG. 16).

For example, the control unit 130 may change the color of the area 420 of the camera image 20 corresponding to the lips of the user's face based on color information matched to an item associated with a live broadcast (Figure (see (b) of 4a). That is, the control unit 130 can provide an effect as if makeup was applied to the user's face image included in the camera image 20 by changing the color or correcting the color of the camera image 20.

Next, in the method of providing live broadcasting according to the present invention, while receiving the broadcast image 10 at the user terminal 200, option information related to the product is provided along with the augmented reality graphic-processed camera image 20. A display process on the display unit of the user terminal 200 may proceed (S1650, see FIG. 16).

In response to the augmented reality function being activated, the control unit 130 displays product-related options along with the camera image 20 on which graphic processing has been performed on the main display area A where the broadcast image 10 was output. Information can be printed.

Users can intuitively recognize the types of options set for a product through option information and experience augmented reality for each product option, which helps them purchase the product. Hereinafter, a method of providing various option information related to a product together with the camera image 20 on which graphic processing has been performed will be described in more detail.

As shown in FIG. 17, a pre-specified product database 1700 may exist in the live broadcast server 300 according to the present invention.

In the product database 1700, option information associated with each of at least one option set for a product may exist in matching with each option.

In such option information, there may be attribute information that can specify the attribute (ex: visual attribute) of at least one option set for the product. For example, if the product is a makeup product for lips, the attribute information matched to the option may include color information (or color code, 1710) for the product corresponding to the option.

Furthermore, the option information may include option specific information that can specify a product corresponding to the option. Option-specific information may include at least one of the option name and unique number of the product corresponding to the option (ex: “301 Spring Data”, 1720).

Furthermore, the option information may include recommendation information (or suggestion phrases) for recommending a product corresponding to the option. Such option recommendation information may include information about the customer (or recommendation target) for whom a product corresponding to the option is recommended. For example, the option information includes a recommendation phrase (or suggestion phrase, “I recommend a warm tone”, 1740) including the skin type (ex: warm tone, cool tone), age, and gender of the customer for whom the product is recommended (or the person being recommended). It can be included.

Furthermore, the option information may include situational information about at least one situation related to the option.

Option situation information may be generated based on various information related to the product. More specifically, i) the context information of the option includes target customers (or target customer base) related to the product corresponding to the option, ii) purchasers who purchased the product corresponding to the option, and iii) sales of the product corresponding to the option ( It may be information generated in relation to at least one of (or sales volume).

For example, the context information of an option may include phrases (“popular color”, “steady seller”, “expected to sell out”, 1730, 1760) that include sales information (or sales volume information) of the product corresponding to the option. there is.

As another example, the context information of an option includes information about the main purchaser of the product corresponding to the option (ex: age, gender), a phrase containing the skin type information of the main purchaser (or “popular information”, “teenagers”) It is popular among people”, 1750).

Meanwhile, at least some of the option information existing in the product database 1700 may be preset (registered or stored) based on information received from the seller terminal. For example, the user enters option information (ex: color information, option name, unique number, recommendation information) for each product option through the seller page described above (see reference numerals 800, 900, and 1000 in FIGS. 8 to 10). Thus, it can be set (registered or stored) in advance on the product database 1700.

In addition, at least some of the option information existing in the product database 1700 is created and updated in real time based on information collected in real time related to the product while a live broadcast selling (or introducing) the product is in progress. It can be set (registered or saved) by matching each product option.

For example, while the live broadcast is in progress, the live broadcast server 300 provides sales information (ex: quantity information) for each option of the product sold (or introduced) through the live broadcast, purchase customer information (ex: age, gender, region, skin type, etc.), analyze and statistics the collected information, and update real-time sales aggregate information in the product database 1700.

As such, the present invention includes a product database 1700 that includes sales status or specialized information for at least one option set for a product, and based on this product database 1700, the user can easily recognize option information. In order to provide optional information, a graphical interface such as graphic objects, tooltips, and speech bubbles with various visual appearances can be provided.

As shown in FIG. 18, the control unit 130 displays (outputs or provides) optional information related to the product along with the camera image 20 on which graphic processing has been performed on the display unit of the user terminal 200. You can.

The option information may include a graphic object 1811 corresponding to an option registered in connection with a product, and the visual appearance of the graphic object 1811 may be determined based on the properties of the option corresponding to the graphic object. .

The visual appearance of a graphic object may vary depending on attribute information of options corresponding to the graphic object. For example, if the attribute information of an option is information about a specific color, the visual appearance of the graphic object may be made of a specific color. As another example, if the attribute information of the option is information about a specific texture, the visual appearance of the graphic object may be made of a specific texture.

The control unit 130 creates a graphic corresponding to the option based on the fact that the option registered in connection with the product includes color information (or color code, see reference numeral 1710 in FIG. 17) for the color corresponding to the option. The object 1811 may be displayed (output or provided) on the display unit of the user terminal 200 with a visual appearance corresponding to color information corresponding to the option.

The user can intuitively recognize the attribute (ex: color) of an option corresponding to the graphic object simply by looking at the graphic object displayed on the display unit of the user terminal 200.

As shown in FIG. 18, when there are multiple options registered in connection with a product, the option information may include a plurality of graphic objects 1811 and 1812 respectively corresponding to the plurality of options. The visual appearances of the plurality of graphic objects 1811 and 1812 may have different visual appearances based on different properties of each of the plurality of options.

The control unit 130 displays (outputs or provides) the first graphic object 1811 on the display unit of the user terminal 200 in a color corresponding to the color information of the first option, and the second graphic object 1812 ) can be displayed (output or provided) on the display unit of the user terminal 200 in a color corresponding to the color information of the second option.

Furthermore, the control unit 130 may arrange the plurality of graphic objects 1811 and 1812 side by side along one direction on the display unit of the user terminal 200.

For example, as shown in (a) of FIG. 18, the control unit 130 displays the first graphic object 1811 in the X-axis direction (i.e., from X1 to ) and the second graphic object 1812 can be placed side by side.

Meanwhile, in the present invention, a basic value (default value) may be set to one of a plurality of options set for a product. Hereinafter, the option with the default value set will be referred to as the pre-specified first option (or default option).

In response to receiving an augmented reality activation request for a product linked to a live broadcast from the user terminal 200, the control unit 130 augments the properties of a pre-specified first option among a plurality of options set for the product. Augmented reality graphics processing may be performed on the camera image 20 so that it appears in the real graphics processed camera image 20.

Performing augmented reality graphics processing on the camera image 20 means transforming, color correcting, and compositing (or inserting) a graphic object into at least a portion of the camera image 20, as described above. can do.

If the product linked to the live broadcast is a makeup product for lips, the control unit 130 determines the camera image based on the color information of the first option specified among the plurality of options set for the product. In (20), at least some areas (area 1800 corresponding to the lips of the user's face (see (a) of FIG. 18)) are displayed in a color corresponding to the color information of the pre-specified first option, Graphics processing can be performed on the camera image 20.

Meanwhile, as shown in (a) of FIG. 18, the first graphic object 1811 corresponding to the attribute of the pre-specified first option is displayed in a plurality of options, respectively, on the display unit of the user terminal 200. It may be located in the center (or center area, 1820) among a plurality of corresponding graphic objects.

In response to receiving an augmented reality activation request for a product linked to a live broadcast from the user terminal 200, the control unit 130 generates the first graphic object 1811 corresponding to the pre-specified first option, Among the plurality of graphic objects, they may be displayed (output or provided) by placing them in the center (or center area 1820), and the camera image 20 may be provided by performing graphics processing based on a pre-specified first option.

That is, the control unit 130 sets the camera image 20 to have a visual appearance corresponding to the attribute of the option corresponding to the first graphic object 1811 located at the center 1820 of the display unit 200. , Augmented reality graphic processing can be performed on the camera image 20.

As such, in the present invention, based on a request for activation of the augmented reality function from the user terminal 200, the camera image 20 that has performed data processing according to a pre-specified first option and attribute information of the pre-specified first option Graphic objects 1811 having visual appearances according to can be provided on the display unit of the user terminal 200 by linking them with each other. Through this, the user can intuitively recognize the properties of the first preset option on which augmented reality graphic processing has been performed on the camera image 20.

Meanwhile, while being provided with a camera image 20 on which graphics processing has been performed according to a pre-specified first option, the user selects a graphic object 1812 according to a second option different from the first option, The product experience can be expanded through the camera image 20 on which graphics processing has been performed according to the first option.

The control unit 130 receives from the user terminal 200 a graphic object 1812 corresponding to a second option different from the pre-specified first option among a plurality of graphic objects 1811 and 1812 respectively corresponding to a plurality of options. You can receive your selection.

Selection of the graphic object 1812 corresponding to the second option may be made based on various user inputs.

For example, as shown in (a) of FIG. 18, based on a user input (ex: touch) on the graphic object 1812 corresponding to the second option, the graphic object 1812 corresponding to the second option ) can be selected.

As another example, on the display unit of the user terminal 200, a plurality of graphic objects 1811 and 1812 are displayed on the display unit based on a preset user input (ex: swipe) received from the user terminal 200. It can be made to move along the one direction.

As a result of moving the plurality of graphic objects 1811 and 18212 based on the user input in the preset method, as shown in (b) of FIG. 18, the center (or central area, 1820) of the plurality of graphic objects ), the graphic object 1812 corresponding to the second option may be selected based on the location of the second graphic object 1811, which is different from the first graphic object 1811.

In response to the selection of the graphic object 1812 corresponding to the second option, the control unit 130 selects the camera image 20 on which graphic processing has been performed according to attribute information corresponding to the first option. Instead of this, graphics processing can be changed so that graphics processing is performed according to attribute information corresponding to the second option.

More specifically, when the product is a lip makeup product, the control unit 130 displays the color of the color information corresponding to the second option that is different from the color information corresponding to the first option in the camera image 20, Augmented reality graphic processing for the camera image 20 can be changed.

Meanwhile, the control unit 130 may position a graphic object corresponding to attribute information of a specific option on which graphic processing has been performed on the camera image 20 at the center (or center area, 1820) of a plurality of graphic objects.

That is, the control unit 130 controls the attribute information of a specific option on which graphic processing has been performed on the camera image 20 and the attributes of a graphic object located in the center (or center area, 1820) of a plurality of graphic objects to be linked. can do.

More specifically, the control unit 130 positions a graphic object 1811 corresponding to the first option to a graphic object 1812 corresponding to the second option in the center (or central area 1820) of the plurality of graphic objects. In conjunction with this, graphic processing for the camera image 20 can be controlled to change from the attributes according to the first option to the attributes according to the second option.

As shown in (b) of FIG. 18, at least a portion of the area (ex: the area corresponding to the lips of the user's face (1800, see (a) of FIG. 18)) of the camera image 20 is Graphics processing may be performed on the camera image 20 so that it is displayed in a color corresponding to the color information of the first option.

In this way, in the present invention, a graphic object corresponding to a specific attribute and a camera image 20 on which graphic processing has been performed according to a specific attribute are provided in conjunction on the display unit of the user terminal 200, so that the user can view the camera image. Not only can the graphics processing for (20) be easily changed, but it is also possible to intuitively recognize which option properties the graphics processing performed on the camera image (20) was based on.

Meanwhile, there may be a case where the user wants to keep the camera image 20 on which augmented reality graphic processing has been performed. For example, if the user likes the camera image 20 on which augmented reality graphic processing has been performed related to a lip makeup product, he or she may want to keep it.

Accordingly, the present invention can provide a service that captures the camera image 20 on which augmented reality graphic processing has been performed and stores it in the storage unit 120.

As shown in (a) of FIG. 18, one graphic object 1811 among a plurality of graphic objects 1811 and 1812 provided (displayed or output) on the display unit of the user terminal 200 is a camera. It may be linked to the shooting function for .

The control unit 130 may control a graphic object located (or arranged) in a specific area 1820 on the display unit of the user terminal 200 to be linked to the camera shooting function.

Here, “linked to the camera shooting function” can be understood as allowing the camera’s shooting function to be performed in response to the selection of a graphic object.

Based on the fact that the first graphic object 1811 is positioned (or placed) in a specific area 1820, the control unit 130 is changed to position the second graphic object 1812 in the specific area 1820, The camera shooting function can be controlled to be linked from the first graphic object 1812 to the second graphic object 1822.

That is, when the first graphic object 1811 is located (or placed) in a specific area 1820 and the second graphic object 1812 is changed to be located in the specific area 1820, the camera shooting function is linked. The graphic object may be changed from the first graphic object 1811 to the second graphic object 1812.

Furthermore, when one graphic object 1811 associated with the camera shooting function is selected among the plurality of graphic objects 1811 and 1812, the control unit 130 displays the camera image 20 on which augmented reality graphic processing has been performed. can be photographed and stored in at least one of the storage unit 120 or a server (ex: cloud server).

For example, in (a) of FIG. 18 , a camera image 20 on which graphic processing has been performed according to attribute information of the first option corresponding to the first graphic object 1811 may be captured and stored. Additionally, in (b) of FIG. 18 , the camera image 20 on which graphic processing has been performed according to attribute information of the second option corresponding to the second graphic object 1812 may be captured and stored.

Meanwhile, in the present invention, based on the product DB (1700, see FIG. 17), a speech bubble or speech bubble is added to the graphic object corresponding to the option around the option so that the user can easily check various information about a plurality of options set for the product. A tooltip can be provided.

These speech bubbles or tooltips may include option information related to options corresponding to the graphic object.

As described above, such option information includes i) attribute information that can specify the attribute (ex: visual attribute) of at least one option set for the product, ii) option specific information that can specify the product corresponding to the option , iii) recommendation information for recommending a product corresponding to the option, and iv) situational information about at least one situation related to the option.

Meanwhile, the attribute information, option specific information, recommendation information, and situation information may be interchangeably named depending on the type and attribute of the information. For example, an option name that specifies a product corresponding to an option may be referred to as option specific information and may also be referred to as attribute information.

As shown in (a) of FIG. 19, the control unit 130 selects one of the specific graphic objects 1811 among the plurality of graphic objects 1811 and 1812 displayed (output or provided) on the user terminal 200. Optional information (“301 Spring Date - Popular Color”, 1910) corresponding to one option may be provided around the specific graphic object 1811.

The option information 1910 is option information of an option corresponding to the graphic object 1811 located in the center among a plurality of graphic objects 1811 and 1812 arranged side by side along one direction on the display unit of the user terminal 200. It can be. In addition, the option information 1910 is an option corresponding to the graphic object 1811 located in a specific area (central area 1820 corresponding to the center of a plurality of graphic objects) on the display unit of the user terminal 200. This may be optional information. Additionally, the option information 1910 may be option information of an option corresponding to a graphic object linked to the camera shooting function. Hereinafter, for convenience of explanation, the option information 1910 will be described as an example of the option information of a graphic object located in a specific area 1810.

The control unit 130 is linked to the change of the graphic object located in the specific area 1820 from the first graphic object 1811 to the second graphic object 1812, as shown in (b) of FIG. 19. , Option information (“333 Closure-Reaction Like”) of the option corresponding to the second graphic object 1812 can be controlled to be displayed around the second graphic object 1812.

For example, when a user input (for example, a swipe or drag input) that moves a plurality of graphic objects 1811 and 1821 in one direction is received from the user terminal 200, the control unit 130 is linked to the user input. Thus, option information on graphic objects located in the specific area 1820 can be provided through the display unit of the user terminal 200. A user can check option information about a plurality of items for a product simply by moving a plurality of graphic objects. At this time, as seen above, augmented reality graphic processing for the camera image 20 may also be changed to correspond to the option linked to the second graphic object 1812.

Meanwhile, option information displayed around a graphic object may be generated (determined) based on the option information of the option corresponding to the graphic object located in the specific area 1820. As described above, such option information may be stored (or included) in the product database 1700 of the live broadcast server 300 (see FIG. 17).

Meanwhile, as shown in FIG. 20, the present invention can provide recommended option information based on the user's face image included in the camera image 20.

The control unit 130 may recognize the user's face image included in the camera image 20 and recommend item options appropriate for the recognized user's face. For example, if the item is related to skin makeup, the control unit 130 130 may extract information about the user's skin type (or skin color) from the camera image 20. A method of extracting information about skin type can be accomplished using various image analysis techniques.

The control unit 130 may display (output or provide) different option information on the display unit of the user terminal 200 based on the different skin types of users extracted from the camera image 20.

More specifically, in the product database, information about different skin types may be matched to each of the different options. For example, a first skin type (ex: cool tone) may be matched to the first option, and a second skin type (ex: warm tone) may be matched to the second option.

The control unit 130 may provide the user with an option corresponding to the user's skin type recognized from the camera image 20 among a plurality of options related to the item, with priority over other options. For example, the control unit 130 may provide augmented reality graphic processing corresponding to a specific option corresponding to the user's skin type among a plurality of options related to the item, with priority over other options.

For example, as shown in (a) of FIG. 20, the control unit 130 analyzes the camera image 20 and, when the user's skin type is the first skin type (ex: warm tone), the camera image For (20), augmented reality graphics processing can be performed using color information of the first option corresponding to the first skin type. In this case, around the first graphic object 1811 corresponding to the first option , recommendation information 2010 recommending the first option may be provided to the user.

For another example, as shown in (b) of FIG. 20, as a result of analyzing the camera image 20, the control unit 130 determines that if the user's skin type is the second skin type (ex: cool tone), the camera Augmented reality graphic processing may be performed on the image 20 using color information of the second option corresponding to the second skin type. In this case, the area around the first graphic object 1812 corresponding to the second option In , recommendation information 2020 recommending the second option may be provided to the user.

In this way, the control unit 130 can analyze the user's information in the camera image 20 that suits the user and specify an option that suits the user among a plurality of options related to the item. Additionally, a series of controls may be performed on the camera image 20 so that augmented reality graphic processing corresponding to the specified option is performed with priority.

For another example, the control unit 130 may provide recommendation information (suggestion information) related to the currently selected option based on recommendation information preset for each option, even if the camera image 20 is not analyzed. You can. As shown in (a) of FIG. 20, when the graphic object 1811 corresponding to the first option is selected from the user terminal, option information matching the first option is displayed around the first graphic object 1811 ( Alternatively, recommended information or suggested phrases) may be displayed. For example, as shown in (a) of FIG. 20, graphic processing related to the first option is performed on the area 2000a corresponding to the lips of the user's face included in the camera image 20, and the control unit 130 ) may display recommendation information (ex: “I recommend warm tones”, 2010) related to product options corresponding to the first option around the first graphic object 1811.

For another example, as shown in (b) of FIG. 20, when the graphic object 1812 corresponding to the second option is selected from the user terminal 200, a second graphic object 1812 is displayed around the second graphic object 1812. 2 Option information (or recommended information, suggested text) matching the option can be displayed. For example, as shown in (b) of FIG. 20, graphics processing related to the second option is performed on the area 2000b corresponding to the lips of the user's face included in the camera image 20, and the control unit 130 ) may display recommended information (ex: “What about cool tone?”, 2020) related to the option of the product corresponding to the second option around the second graphic object 1812.

Meanwhile, as shown in FIG. 21 , the control unit 130 may control context information related to options corresponding to the graphic object to be output (displayed or provided) around the graphic object.

Here, “situational information” may be based on various situations related to the product corresponding to the option. More specifically, the situation information is at least one of i) target customers (or target customer base) related to the product corresponding to the option, ii) purchasers who purchased the product corresponding to the option, and iii) sales of the product corresponding to the option. It may be information generated in relation to .

The situation information may be information generated in real time based on information collected related to the product while the live broadcast is in progress.

The live broadcast server 300 may collect information on the age range of users who purchased the product and the options selected by the users. The collection of this information may be collected from information on products sold in connection with the live broadcast while the live broadcast is in progress, or based on the sales history of the product through various sales channels before the live broadcast begins. You can.

The live broadcast server 300 may transmit to the user terminal 200 situation information including information on the product purchase age group for each product option collected in various ways. The control unit 130 may receive the context information from the live broadcast server 300 and provide the context information around the graphic object matched to the option corresponding to the context information. For example, as shown in (a) of FIG. 21, when a product corresponding to the first option is purchased by many people in their 20s, situation information (ex. : “Many people in their 20s chose it”, 2110) can be provided. For another example, as shown in (b) of FIG. 21, when a product corresponding to the second option is purchased by many teenagers, situation information is displayed around the graphic object 1812 corresponding to the second option. (ex: “It’s popular among teenagers”, 2120) can be provided.

Meanwhile, the live broadcast server 300 may collect sales volume information about products sold through live broadcast while the live broadcast is in progress. Additionally, the live broadcast server 300 may provide the collected sales volume information (sales information) to the user terminal 200. The control unit 130 may receive context information including sales or sales volume information from the live broadcast server 300 and provide context information around graphic objects matched to options corresponding to the context information. For example, For example, when the product corresponding to the first option is expected to be sold out, as shown in (a) of FIG. 22, around the graphic object 1811 corresponding to the first option, situation information related to the expected sold out ( “Expected to be out of stock.”, 2210) may be displayed. As another example, as shown in (b) of FIG. 22, around the graphic object 1812 corresponding to the second option, situation information related to the sales quantity of the product corresponding to the second option (“cumulative sales to date”) “There are 3503”, 2220) may be displayed.

As such, in the present invention, by providing the sales status or specialized information of products sold in a live broadcast in the form of a tooltip, the user can immediately recognize the real-time situation related to the live broadcast.

Meanwhile, a user may want to be provided with a plurality of camera images on which graphic processing has been performed according to attribute information for each of a plurality of options set for a product. For example, a user may want to see their face made up in different colors in order to check which of the makeup products for lips with different colors suits them best. Accordingly, in the present invention, a plurality of camera images on which augmented reality graphic processing has been performed according to the attribute information of each of the plurality of options can be provided.

As shown in FIG. 23A, while the camera image 20 on which augmented reality graphics processing has been performed according to the attribute information of the first option among the plurality of options set for the product is provided to the user terminal 200, the user From the terminal 200, a user input corresponding to a request for a camera image to which a plurality of options are respectively applied (hereinafter referred to as a “preview image request”) may be received.

For example, when a preset user input (ex: long press input (or long touch)) for a specific area 1820 is received on the user terminal 200, the control unit 130 receives a preview image request. This can be treated as accomplished. Meanwhile, as described above, when a preset user input (ex: short press input (or short touch)) for a specific area 1820 is received, the control unit 130 performs augmented reality graphic processing for the currently selected option. Capturing of camera images can be processed as if a request has been made.

In response to receiving a preview image request from the user terminal 200, the control unit 130 displays an area 2300 of the display unit 2300 of the user terminal 200, as shown in (a) of FIG. 23B, You can display (print or provide) a preview image (ex: 2300a, 2300b, etc.) corresponding to each option associated with the item.

When there are multiple options associated with an item, one area 2300 may include multiple preview images (ex: 2300a, 2300b, etc.).

Each of the plurality of preview images (ex: 2300a, 2300b, etc.) in one area 2300 may be understood as a camera image 20 on which augmented reality graphic processing has been performed in relation to each of the plurality of options set for the product. That is, among the plurality of preview images 2300a and 2300b, the first preview image 2300a is a camera image on which augmented reality graphics processing has been performed in relation to the first option, and the second preview image 2300b is the first preview image 2300b. Regarding option 2, it may be a camera image on which augmented reality graphics processing has been performed.

Meanwhile, as shown in (a) of FIG. 23B, the control unit 130 controls the user terminal 200 while a plurality of preview images 2300a and 2300b are displayed on the display unit of the user terminal 200. Based on the user's selection for 200, a selection for one preview image 2300b may be received.

In response to receiving the selection of one preview image 2300b, the control unit 130 displays the selected preview image 2300b as a large screen (or main screen) on the display unit of the user terminal 200. can be provided. That is, the control unit 130 provides the camera image 20 on which graphic processing related to the option corresponding to the selected preview image 2300b has been performed on the main display area of the user terminal 200. You can.

The user can select a preview image that he or she wishes to view on a larger screen among the plurality of preview images (2300a, 2300b) and receive the preview image on a larger screen.

Meanwhile, the plurality of images 2300a and 2300b provided on one area 2300 as seen above may be images that are images processed with augmented reality graphics captured on the user terminal 200 and stored in the storage unit 120. there is. That is, when an augmented reality graphic-processed image is captured based on the user's request, the control unit 130 can collect the captured images and provide them on the display unit. Through this, users can compare information about products they have experienced.

Meanwhile, as shown in FIG. 14, the product purchase page 1400 for purchasing a product is displayed on the display unit of the user terminal 200 while the camera image 20 on which graphic processing has been performed is displayed. The method of providing it was explained.

Hereinafter, a method of providing a product purchase page for purchasing a product to the user terminal 200 while a plurality of preview images 2300a and 2300b are displayed on the display unit of the user terminal 200 will be described. Let me explain.

As shown in (a) of FIG. 24A, the control unit 130 responds to receiving a preview image request from the user terminal 200 (ex: long touch input on the shooting function icon 1830), As shown in (b) of 24a, a preview image (ex: 2400a, 2400b, etc.) corresponding to each option associated with the item is displayed on one area 2400 of the display unit of the user terminal 200 ( print or provide).

In the present invention, each of the plurality of preview images 2400a and 2400b may be linked to a product purchase page corresponding to an option related to each preview image. More specifically, the first preview image 2400a may be linked to a purchase page for a product corresponding to the first option. Additionally, the second preview image 2400b may be linked to a purchase page for a product corresponding to the second option.

In a state where a plurality of preview images 2400a and 2400b are displayed on the display unit of the user terminal 200, the control unit 130 selects one of the plurality of preview images 2400a and 2400b (2400b). You can receive user selection for .

Based on the user selection, the control unit 130 displays the product purchase page 2410 corresponding to the option related to the selected preview image 2400b on the display unit 200, as shown in FIG. 24b. can be provided. At this time, the product purchase page 2410 may be provided in a modal window in one area of the display unit of the user terminal 200.

The control unit 130 may simultaneously provide a payment process for purchasing a product while providing a live broadcast to the user terminal 200 and the augmented reality function is activated.

Accordingly, users can receive unified viewing of live broadcasts, product experience through augmented reality functions, and product payment.

Meanwhile, the user changes at least one of the density, shading, saturation, and brightness (hereinafter described as shading) of the color information in the camera image 20 on which graphic processing has been performed according to the color information corresponding to the option. There may be cases where you want to do so. For example, let's assume that the product being sold (or introduced) in a live broadcast is a lip makeup product. There may be cases where a user wants to experience (or test) a lip makeup product in a light color or in a dark color. Accordingly, the present invention can provide a service that can change the shade of color information in the camera image 20 on which graphic processing has been performed based on color information.

As shown in (a) of FIG. 25, a camera image on which augmented reality graphic processing related to specific color information has been performed on a specific area (ex: area corresponding to the user's lips, 2500) of the display unit of the user terminal 200 In a state where (20) is displayed, the control unit 130 may display a graphic object 2510 for changing the degree of augmented reality graphic processing on one area of the display unit of the user terminal 200. This graphic object 2510 may be provided in the form of a progress bar. Based on the user input for the graphic object 2510, the control unit 130 can change the degree of augmented reality graphic processing. For example, the graphic object 2510 may be linked to a change in shade of specific color information corresponding to an option applied to the current camera image 20. The control unit 130 may vary the degree of graphic processing performed on the camera image 20 through user input applied to the graphic object 2510 through the user terminal 200. More specifically, the shading for a specific area 2500 that is the target of augmented reality graphic processing may be changed.

Different shading information may be matched to different positions of the graphic object 2510. For example, shading information corresponding to 10% of the color information is matched to the first specific position of the graphic object, and shading information corresponding to 150% of the color information is matched to the second specific position of the graphic object. You can.

Based on receiving a user input for selecting a specific location of a graphic object from the user terminal 200, the control unit 130 reflects shading information matching the specific location on the camera image 20, The graphic processing of the camera image 20 can be changed.

For example, assume that the control unit 130 has received location information of a graphic object corresponding to shade information corresponding to 150% of the color information from the user terminal 200. As shown in (b) of FIG. 25, the control unit 130 sets a color in which 150% of the shading information is reflected in a specific area (ex: area corresponding to the user's lips, 2500) on the display unit of the user terminal 200. A camera image 20 on which information-related graphics processing has been performed may be provided.

In this way, the control unit 130 can use the graphic object 2510 to vary the degree to which augmented reality is applied, thereby providing the user with a variety of experiences, such as actually experiencing a product.

Meanwhile, a user may have a need to experience multiple products simultaneously. For example, the user wants to experience the same effect as applying lip makeup and eye makeup at the same time by simultaneously applying a product for lip makeup and a product for eye makeup to the camera image 20 including the image of his or her face. can do.

Accordingly, in the present invention, graphic processing related to a plurality of different products can be performed on one camera image 20 and displayed on the display unit of the user terminal 200.

As shown in (a) of FIG. 26A, the control unit 130 displays an additional product list on the display unit of the user terminal 200 ( 2610) can be provided.

Products provided through the additional product list 2610 may be products linked to live broadcasting. Furthermore, products provided through the additional product list 2610 may be products that provide an augmented reality function.

The additional product list 2610 may include additional product items for products different from the product related to the camera image 20 on which augmented reality graphic processing was performed. For example, assume that graphic processing is performed on the camera image 20 in relation to the first lip makeup product among the first lip makeup products and second lip makeup products associated with the live broadcast. In this case, the additional product can be understood as a product for the second lip makeup.

In the present invention, while the additional product list 2610 is displayed on the display unit of the user terminal 200, a user selection of an additional product item corresponding to a product to perform additional augmented reality graphic processing can be received. there is.

Based on receiving the user's selection for an additional product to perform additional augmented reality graphic processing, the control unit 130 selects graphics corresponding to a plurality of options set for the additional product, as shown in (b) of FIG. 26A. Objects 2630, 2640, and 2560 may be provided on the display unit of the user terminal 200.

More specifically, the control unit 130, on the display unit of the user terminal 200, i) a plurality of graphic objects 2631, each corresponding to a plurality of options set for the first product (ex: eye makeup product) 2632), ii) a plurality of graphic objects (2641, 2642) corresponding to each of a plurality of options set in the second product (ex: product corresponding to lip makeup), iii) a third product (ex: product corresponding to blusher makeup) A plurality of graphic objects (2651, 2652) corresponding to each of the plurality of options set for the product can be displayed.

When the user wants to change the eye makeup effect in his/her face image included in the camera image 20, a plurality of graphics corresponding to each of a plurality of options set in the first product (ex: product for eye makeup) are displayed. Any one of objects 2631 and 2632 can be selected.

In addition, when the user wants to change the lip makeup effect in his/her face image included in the camera image 20, the user selects multiple options corresponding to each of a plurality of options set in the second product (ex: product for lip makeup). You can select any one of the graphic objects 2641 and 2642.

Meanwhile, the display unit of the user terminal 200 may include a function icon 2670 linked to a product purchase page 2660 (see FIG. 26B) where multiple products can be purchased.

The control unit 130 displays graphic objects 2630, 2640, and 2650 for option information set for each of the plurality of products on the display unit of the user terminal 200, and provides a product purchase function that allows all of the plurality of products to be purchased. The page 2660 may also display a linked function icon 2670.

Based on receiving the user's selection of the function icon 2670 to which the product purchase page 2660 (see FIG. 26B), where all of the plurality of products can be purchased, is linked, the control unit 130 selects the product as shown in FIG. 26B. , a product purchase page 2660 where all of the plurality of products can be purchased can be provided on the display unit of the user terminal 200.

The user can purchase multiple products in bulk through the product purchase page 2660 without the hassle of purchasing the products individually on the product purchase page that sells each of the multiple products.

As seen above, the method and system for providing live broadcasting according to the present invention can utilize the augmented reality function to provide a user experience that allows users to experience products introduced in the live broadcast while watching the live broadcast. there is.

According to the method and system for providing live broadcasting according to the present invention, consumers can experience products even without directly visiting the store, saving time on visiting the store and purchasing products. You can do it more carefully.

Furthermore, according to the method and system for providing live broadcasting according to the present invention, sellers can induce consumers to more actively purchase products by providing consumers with opportunities to experience products in a variety of ways. Furthermore, even if sellers do not directly operate a store, they can meet various consumers and experience products through live broadcasts, thereby saving on the operating costs of running a store and having various opportunities to sell products.

Meanwhile, the method and system for providing live broadcasting according to the present invention can provide the provision of live broadcasting, product experience, and product purchase process in a unified manner while live broadcasting is in progress. Through this, consumers can experience and purchase products more conveniently just by watching the live broadcast according to the present invention. Meanwhile, the present invention discussed above can be implemented as a program that is executed by one or more processes on a computer and can be stored in a medium that can be read by such a computer.

Furthermore, the present invention discussed above can be implemented as computer-readable codes or instructions on a program-recorded medium. That is, various control methods according to the present invention may be provided in the form of programs, either integrated or individually.

Meanwhile, computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is.

Furthermore, the computer-readable medium may be a server or cloud storage that includes storage and can be accessed by electronic devices through communication. In this case, the computer can download the program according to the present invention from a server or cloud storage through wired or wireless communication.

Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a CPU (Central Processing Unit), and there is no particular limitation on its type.

Meanwhile, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (18)

Receiving broadcast video of a live broadcast from a live broadcast server;
Based on the reception, performing first data processing to provide the broadcast image on a display unit of a user terminal;
Receiving an activation request for an augmented reality function related to the live broadcast from the user terminal while the broadcast image is being provided on the display unit according to the first data processing;
While the first data processing is being performed, when an activation request for the augmented reality function is received, a camera provided in the user terminal is activated in response to the activation request, and a camera image is received from the activated camera. Regarding, performing second data processing so that augmented reality graphic processing related to the product linked to the live broadcast is performed; and
Based on the first data processing and the second data processing, while the user terminal is receiving the broadcast image, the option information related to the product is displayed along with the camera image on which the augmented reality graphic processing has been performed. A method of providing live broadcasting, comprising the step of displaying on a display unit. According to paragraph 1,
Optional information related to the above product,
Contains a graphic object corresponding to an option registered in connection with the product,
A method for providing live broadcasting, wherein the visual appearance of the graphic object is determined based on properties of options corresponding to the graphic object. According to paragraph 2,
When there are a plurality of options registered in connection with the product, the graphic object includes a plurality of graphic objects respectively corresponding to the plurality of options. According to paragraph 3,
In the step of performing the augmented reality graphics processing,
A method for providing a live broadcast, characterized in that the augmented reality graphics processing is performed using the camera image so that an attribute of a pre-specified first option among the plurality of options appears in the augmented reality graphics processed camera image. According to clause 4,
The properties of each of the plurality of options correspond to different color information, respectively,
In the step of performing the augmented reality graphics processing,
Live broadcasting, characterized in that the augmented reality graphic-processed camera image is generated by converting the color of at least a portion of the camera image so that the color of color information corresponding to the predetermined first option is shown in the camera image. How to provide. According to clause 5,
selecting, from the user terminal, a graphic object corresponding to a second option different from the pre-specified first option among the plurality of graphic objects corresponding to each of the plurality of options; and
In response to selection of a graphic object corresponding to the second option, the color of the color information corresponding to the second option that is different from the color information corresponding to the first option is displayed in the camera image. A method of providing a live broadcast, further comprising the step of changing augmented reality graphics processing. According to paragraph 3,
While the broadcast image is being provided on the display unit, an icon linked to the augmented reality function is displayed in one area of the display unit,
A request for activation of the augmented reality function is received from the user terminal based on the icon being selected,
A method for providing a live broadcast, wherein one graphic object among the plurality of graphic objects is linked to a shooting function for the camera. In clause 7,
The plurality of graphic objects are arranged side by side along one direction,
The shooting function for the camera is,
A method of providing a live broadcast, characterized in that it is linked to a first graphic object located in the center among the plurality of graphic objects arranged side by side. According to clause 8,
A method for providing a live broadcast, wherein the camera image processed with augmented reality graphics is processed with augmented reality graphics to have a visual appearance corresponding to an attribute of an option corresponding to the first graphic object located at the center. According to clause 9,
The plurality of graphic objects are configured to move along the one direction on the display based on a preset user input received from the user terminal,
As a result of moving the plurality of graphic objects based on the user input in the preset method, when a second graphic object different from the first graphic object is located in the center of the plurality of graphic objects,
The shooting function for the camera is linked to the second graphic object,
A method for providing a live broadcast, characterized in that updating augmented reality graphic processing for the camera image so that the camera image reflects the visual appearance corresponding to the attribute of the option corresponding to the second graphic object. In clause 7,
When one graphic object associated with the camera shooting function is selected among the plurality of graphic objects,
Further comprising capturing the augmented reality graphic-processed camera image and storing it in a storage unit,
A method for providing a live broadcast, wherein the camera image processed with augmented reality graphics is processed with augmented reality graphics to have a visual appearance corresponding to an attribute of an option corresponding to the one graphic object. According to paragraph 2,
A method of providing a live broadcast, characterized in that option information related to options corresponding to the graphic object is output around the graphic object. According to clause 9,
A method of providing live broadcasting, characterized in that the option information is stored and exists in a database in connection with options registered in connection with the product. According to clause 13,
Context information related to options corresponding to the graphic object is output around the graphic object,
The above situation information is,
A method of providing live broadcasting, characterized in that it is related to at least one of target customers, purchasing customers, and sales related to the product corresponding to the option. According to clause 14,
The above situation information is,
A method of providing a live broadcast, characterized in that collection related to the product is collected while the live broadcast is in progress. According to clause 15,
The above situation information is,
A method of providing a live broadcast, characterized in that it is updated in real time based on information collected while the live broadcast is in progress. In a live broadcasting provision system linked to a user terminal including a communication unit, a camera, and a display unit,
A control unit that receives broadcast video of a live broadcast from a live broadcast server through the communication unit,
The control unit,
Based on the reception, perform first data processing to provide the broadcast image on a display unit of the user terminal,
While the broadcast image is being provided on the display unit according to the first data processing, a request for activation of an augmented reality function related to the live broadcast is received from the user terminal,
While the first data processing is being performed, when an activation request for the augmented reality function is received, a camera provided in the user terminal is activated in response to the activation request, and a camera image is received from the activated camera. For this, second data processing is performed so that augmented reality graphic processing related to the product linked to the live broadcast is performed,
Based on the first data processing and the second data processing, while the user terminal is receiving the broadcast image, the option information related to the product is displayed along with the camera image on which the augmented reality graphic processing has been performed. A live broadcasting provision system characterized by displaying on a display unit. A program that is executed by one or more processes in an electronic device and stored on a computer-readable recording medium,
The above program is,
Receiving broadcast video of a live broadcast from a live broadcast server;
Based on the reception, performing first data processing to provide the broadcast image on a display unit of a user terminal;
Receiving an activation request for an augmented reality function related to the live broadcast from the user terminal while the broadcast image is being provided on the display unit according to the first data processing;
While the first data processing is being performed, when an activation request for the augmented reality function is received, a camera provided in the user terminal is activated in response to the activation request, and a camera image is received from the activated camera. Regarding, performing second data processing so that augmented reality graphic processing related to the product linked to the live broadcast is performed; and
Based on the first data processing and the second data processing, while the user terminal is receiving the broadcast image, the option information related to the product is displayed along with the camera image on which the augmented reality graphic processing has been performed. A program stored in a computer-readable recording medium comprising instructions for performing display steps on a display unit.

Images