KR102659456B1 - Method and system for providing live broadcasting - Google Patents

Abstract

The present invention relates to a method and system for providing live broadcasting to more effectively provide information about products in live commerce. A method of providing a live broadcast according to the present invention includes the steps of receiving a broadcast image of a live broadcast from a live broadcast server, providing the broadcast image on the main display area of a display unit of a user terminal based on the reception, In a state where the broadcast image is being provided on the main display area, receiving an activation request for an augmented reality function related to the live broadcast from the user terminal, responding to activation of the augmented reality function according to the activation request Thus, performing augmented reality graphic processing related to an item linked to the live broadcast on a camera image received from a camera provided in the user terminal, and the augmented reality graphic processed camera image on the main display area. It may include the step of providing.

Description

Method and system for providing live broadcasting {METHOD AND SYSTEM FOR PROVIDING LIVE BROADCASTING}

The present invention relates to a method and system for providing live broadcasting to more effectively provide information about products in live commerce.

As technology develops, the spread of electronic devices (e.g., smartphones, tablet PCs, automated devices, etc.) has become popular, and as a result, dependence on electronic devices is gradually increasing in many aspects of daily life.

Furthermore, the popularization of electronic devices has promoted the development of Internet-based services, and through this, consumption patterns that were previously highly dependent on offline are gradually moving to online.

In line with this change in trend, it is becoming common for industrial fields, which are based off-line due to the nature of services such as product sales, to attract customers through various online marketing.

Accordingly, in recent years, various product sales services have been provided online, and their utilization is also greatly increasing. However, these online product sales services only provide pre-uploaded product information, and consumers still want to be provided with various information about products.

To address these needs, a method of selling products using live broadcasts is being introduced. The online product selling method using real-time interactive video (reference patent, Korean Patent Publication No. 10-2009-0001838) can immediately provide product information that consumers want by communicating with consumers in real time using online live broadcasting. It came to be.

However, this method of selling products using live broadcasts also has the disadvantage of not allowing consumers to try the products themselves. Accordingly, there is still a need for a service that allows customers to test products using only the electronic devices they own.

The method and system for providing live broadcasting according to the present invention are intended to provide a user experience that can more effectively provide information about products to users watching live broadcasting.

Furthermore, the method and system for providing live broadcasting according to the present invention are intended to provide a service that allows users watching live broadcasting to experience or test products sold through live broadcasting.

In particular, the method and system for providing live broadcasting according to the present invention are intended to provide a service that allows users to experience or test products sold through live broadcasting without stopping the live broadcast being watched.

Furthermore, the method and system for providing live broadcasting according to the present invention are intended to provide a service that can provide various information about products sold through live broadcasting to users who watch live broadcasting.

Furthermore, the method and system for providing live broadcasting according to the present invention are intended to provide a service where users watching live broadcasting or users experiencing products can easily purchase products.

A method of providing a live broadcast according to the present invention includes the steps of receiving a broadcast image of a live broadcast from a live broadcast server, providing the broadcast image on the main display area of a display unit of a user terminal based on the reception, In a state where the broadcast image is being provided on the main display area, receiving an activation request for an augmented reality function related to the live broadcast from the user terminal, responding to activation of the augmented reality function according to the activation request Thus, performing augmented reality graphic processing related to an item linked to the live broadcast on a camera image received from a camera provided in the user terminal, and the augmented reality graphic processed camera image on the main display area. Comprising the step of providing, wherein the broadcast image is provided to the user terminal through a sub-display area located on the display unit while the augmented reality graphic-processed camera image is provided on the main display area. You can.

Furthermore, the live broadcast providing system according to the present invention includes a communication unit that receives the broadcast image of the live broadcast from a live broadcast server, and a control unit that provides the broadcast image on the main display area of the display unit of the user terminal based on the reception. Includes. The control unit receives an activation request for an augmented reality function related to the live broadcast from the user terminal while the broadcast image is being provided on the main display area, and activates the augmented reality function according to the activation request. In response to activation, augmented reality graphics processing related to an item associated with the live broadcast is performed on a camera image received from a camera provided in the user terminal, and the augmented reality graphics processing is performed on the main display area. The broadcast image is provided to the user terminal through a sub-display area located on the display unit while the augmented reality graphic-processed camera image is provided on the main display area. Features a live broadcasting provision system.

Furthermore, according to the present invention, a program executed by one or more processes in an electronic device and stored in a computer-readable recording medium includes the steps of receiving a broadcast image of a live broadcast from a live broadcast server, based on the reception. Thus, providing the broadcast image on the main display area of the display unit of the user terminal, and in a state where the broadcast image is provided on the main display area, providing information on the augmented reality function related to the live broadcast from the user terminal. Receiving an activation request, in response to the augmented reality function being activated according to the activation request, augmented reality graphics related to an item associated with the live broadcast on a camera image received from a camera provided in the user terminal. Instructions for performing processing and providing a camera image processed with augmented reality graphics on the main display area, wherein the broadcast image is processed with augmented reality graphics on the main display area. While the camera image is being provided, it may be provided to the user terminal through the sub-display area located on the display unit.

The method and system for providing live broadcasting according to the present invention can utilize an augmented reality function to provide a user experience that allows users to experience products introduced in the live broadcast while watching the live broadcast.

According to the method and system for providing live broadcasting according to the present invention, consumers can experience products even without directly visiting the store, saving time on visiting the store and purchasing products. You can do it more carefully.

Furthermore, according to the method and system for providing live broadcasting according to the present invention, sellers can induce consumers to more actively purchase products by providing consumers with opportunities to experience products in a variety of ways. Furthermore, even if sellers do not directly operate a store, they can meet various consumers and experience products through live broadcasts, thereby saving on the operating costs of running a store and having various opportunities to sell products.

Meanwhile, the method and system for providing live broadcasting according to the present invention can provide the provision of live broadcasting, product experience, and product purchase process in a unified manner while live broadcasting is in progress. Through this, consumers can experience and purchase products more conveniently just by watching the live broadcast according to the present invention.

1 is a conceptual diagram for explaining live broadcasting according to the present invention.
Figure 2 is a conceptual diagram for explaining a live broadcasting provision system according to the present invention.
Figure 3 is a flowchart illustrating a method for providing live broadcasting according to the present invention.
FIGS. 4A and 4B are conceptual diagrams illustrating activation of an augmented reality function in a method for providing live broadcasting according to the present invention.
Figure 5 is a conceptual diagram for explaining a broadcast video provided through live broadcasting according to the present invention.
Figure 6 is a conceptual diagram for explaining the camera image provided through the augmented reality function in the present invention.
Figure 7 is a conceptual diagram for explaining the DB according to the present invention.
Figures 8, 9, and 10 are conceptual diagrams for explaining the user environment provided to the seller's terminal in the present invention.
11 and 12 are conceptual diagrams for explaining a method of providing additional item information in the present invention.
13 to 15 are conceptual diagrams for explaining an augmented reality service related to the present invention.
Figure 16 is a flowchart for explaining a method of providing live broadcasting in the PIP (Picture In Picture) method in the present invention.
FIGS. 17, 18, 19, 20, 21, 22, and 23 are conceptual diagrams for explaining the user environment provided when the augmented reality function according to the present invention is activated.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings. However, identical or similar components will be assigned the same reference numbers regardless of drawing symbols, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves. Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

The present invention relates to a method of providing a live broadcast, and in particular, by providing an augmented reality service along with the provision of a live broadcast, so that a user can watch the live broadcast and at the same time sell products on the live broadcast. It provides a service that allows you to experience or test.

Here, live broadcasting is a streaming broadcast that introduces products while communicating with consumers in real time online, and can also be called streaming broadcasting, live commerce, or shopping live. You can.

Here, Augmented Reality is a technology that displays a single image by overlapping a virtual image on a real image or background. It synthesizes images by overlapping a virtual image on a real image, or at least part of a real image. It may mean transforming . Here, the real image may mean a camera image received through a camera provided in the user terminal.

In the present invention, combining information related to an item (specific item) or product to the camera image 20, or changing at least part of the camera image 20 to reflect information related to the item (specific item) or product. Through this, augmented reality graphics processing can be performed.

In the present invention, while a broadcast image of a live broadcast is being provided to a user terminal, augmented reality graphics processing related to products sold in the live broadcast is performed on the camera image received from the camera, and the camera image is processed with augmented reality graphics. By providing, it is possible to provide a method of providing live broadcasting that allows viewing live broadcasting and product experience (testing) at the same time.

Here, the “camera image 20” may include at least one of a photo (or image), a video, and a preview screen captured through a camera. That is, in the present invention, augmented reality graphic processing can be performed on at least one of a photo (or image), video, and preview screen received through a camera. However, in the following, they will not be distinguished and all will be explained by expressing them as camera images 20. 1 is a conceptual diagram for explaining live broadcasting according to the present invention.

As shown in FIG. 1, the live broadcast providing system 100 according to the present invention can provide a broadcast image 10 of a live broadcast on the display unit of the user terminal 200.

The broadcast video 10 of this live broadcast is filmed through the user terminal of the seller (or user corresponding to the seller) involved in selling the product, and is transmitted through the user terminal of the consumer (or user corresponding to the consumer) involved in purchasing the product. It can be provided (or output) to .

In the present invention, in order to avoid confusion in terminology, the user involved in product sales is named “seller” or “(live broadcast) host”, and the user terminal used by the seller is called “seller terminal (or seller’s terminal).” , “Host terminal”, “Live broadcasting progress terminal”, “Live commerce progress terminal”, “Sales related terminal”, etc.

In addition, users involved in purchasing products are named “consumers,” “buyers,” or “viewers,” and user terminals used on the consumer side are called “consumer terminals (or consumer terminals),” “buyer terminals,” and “viewers.” It can be named “Terminal”, etc.

As shown in FIG. 1, the broadcast image 10 is displayed on the display unit of the user terminal 200 along with various related information and related graphic objects 11a to 11f associated with the broadcast image 10. It may be provided (or printed).

In the present invention, providing (or outputting) the broadcast video 10 and the related information and the related graphic objects 11a to 11f together means that the related information and the related graphic objects 11a to 11f are provided in at least one area of the broadcast video 10. 11f) may be understood as being overlapped or provided (or output) in an overlapping manner.

The related information and related graphic objects 11a to 11f overlapping with the broadcast video 10 include i) information pre-entered by the seller (ex: live broadcast title information 11a, product information 11b, Augmented reality function icon (or graphic object, 11c) for augmented reality service, ii) various icons (11d, 11e) related to live broadcast, ii) chat input by at least one of the seller and the consumer (or a plurality of consumers) (Chatting) may include at least one of the information (11f).

Meanwhile, in the method of providing live broadcasting according to the present invention, a series of processes may be performed by a system implemented in the form of an application or software (hereinafter, the terms application and software will not be distinguished). (unified and named application). That is, the live broadcast providing system 100 according to the present invention may also be called a “live broadcast providing application,” “live commerce application,” or “shopping live application.”

Below, we will look at the live broadcasting provision system according to the present invention in more detail along with the attached drawings. Figure 2 is a conceptual diagram for explaining a live broadcasting provision system according to the present invention.

As shown in FIG. 2, the live broadcasting providing system 100 according to the present invention can receive broadcasting video from the live broadcasting server 300 and provide it on the user terminal 200. Furthermore, it is also possible for the live broadcast video to be provided directly from the seller terminal to the user terminal 200.

The live broadcast server 300 is a server that provides a live broadcast streaming service, and can receive broadcast video from a specific user terminal and provide the received broadcast video to the live broadcast system 100 according to the present invention. there is.

In particular, the live broadcast server 300 according to the present invention can provide a live broadcast streaming service in conjunction with a shopping service. In this case, the specific user terminal corresponds to the seller terminal, and the live broadcast providing system 100 may be a live broadcast providing application implemented in the consumer terminal.

Accordingly, the live broadcast server 300 according to the present invention may be interchangeably named as “server”, “broadcast server”, “live server”, “live commerce server”, “commerce server”, etc.

Furthermore, the live broadcast server 300 provides at least one of i) broadcast information, ii) item information, iii) chat information, and vi) function icons (or graphic objects) related to the broadcast video. The live broadcast providing system according to the present invention ( 100).

Here, item information is information about an item, and an item may mean a product that is subject to sale or advertisement (promotion) through live broadcasting.

Broadcast information may be various information related to a specific broadcast video. For example, broadcast information includes title information of a specific broadcast video (ex: “Monthly Gift Shop Episode 8: Black Friday Special”), host information (“Barbie Cosmetics”), viewer information (ex: “331,425 views”) may include broadcast type information (ex: live), etc.

Item information may be various information related to at least one item associated with a specific broadcast. For example, item information may include a thumbnail image of an item sold or introduced in a live broadcast of a specific broadcast video, item name information, detailed item information, item price information, etc.

Chat information can be understood as messages exchanged in real time with the live broadcast of a specific broadcast video by a seller terminal and a plurality of consumer terminals.

A function icon (or graphic object) may be an icon (graphic object) associated with various functions related to a specific broadcast video or shopping service.

The broadcast information, item information, chat information, and function icons (or graphic objects) are based on the present invention based on at least one of information received from a seller terminal related to a specific broadcast video and information stored in the live broadcast server 300. It may be provided to the live provision system 100 according to the following.

Information stored in the live broadcasting server 300 may be information input by an administrator of the live broadcasting server 300.

In this way, the live broadcast server 300 according to the present invention can provide various information related to the broadcast video of the live broadcast in order to provide a live broadcast streaming service linked to a shopping service.

Meanwhile, in the present invention, the live broadcast providing system 100 is installed in the user terminal 200 in the form of an application, as shown in FIG. 2, and performs a process of providing an augmented reality service along with broadcast images of a live broadcast. It can be done. This application may be installed on the user terminal 200 at the request of the user on the user terminal 200, or may exist installed on the user terminal 200 at the time the user terminal 200 is shipped. An application implementing such a live broadcasting provision system may be downloaded from an external data storage (or external server) through data communication and installed on the user terminal 200. . And, when an application implementing the live broadcast providing system according to the present invention is executed on the user terminal 200, a series of processes for providing an augmented reality service along with broadcast images of the live broadcast may proceed.

The user terminal 200 referred to in the present invention may be any electronic device capable of operating the live broadcasting provision system 100 according to the present invention, and there is no particular limitation on its type. For example, the user terminal 200 is a mobile phone, a smart phone, a laptop computer, a laptop computer, a slate PC, a tablet PC, and an ultrabook. (ultrabook), desktop computer, digital broadcasting terminal, PDA (personal digital assistants), PMP (portable multimedia player), navigation, wearable device (e.g., smartwatch), glass type Terminals (smart glass), HMD (head mounted display), etc. may be included.

Meanwhile, the live broadcast providing system 100 according to the present invention, which can be implemented in the form of an application, may include at least one of a communication unit 110, a storage unit 120, and a control unit 130. The above components are software components and can perform functions in conjunction with hardware components of the user terminal 200.

For example, the communication unit 110 uses the configuration of communication modules (e.g., mobile communication module, short-range communication module, wireless Internet module, location information module, broadcast reception module, etc.) provided in the user terminal 200. Thus, it can perform the role of transmitting and receiving information (or data) related to the present invention with at least one external device (or live broadcast server, 100a).

In addition, the storage unit 120 uses at least one of the memory provided in the user terminal 200 and an external storage (or external server 100a) to store information related to providing live broadcasting and augmented reality services, or to store information related to providing live broadcasting and augmented reality services. Information or commands related to can be stored.

In the present invention, “stored” in the storage unit 120 may mean that information is physically stored in the memory of the user terminal 200 or in an external storage (or external server, 100a).

Meanwhile, when the application corresponding to the present invention is deleted from the user terminal 200, information stored in the storage unit 120 may also be deleted from the user terminal 200. Additionally, the external storage 100a may be a cloud-type storage.

In the present invention, the memory of the user terminal 200 and the external storage (or external server, 100a) are not distinguished, and both are described by referring to the storage unit 120.

Meanwhile, the control unit 130 can perform overall control for carrying out the present invention using a CPU (Central Processing Unit) provided in the user terminal 200. The components examined above can operate under the control of the control unit 130, and of course, the control unit 130 can control the physical components of the user terminal 200.

For example, the control unit 130 may control the broadcast image of a live broadcast to be output through the display unit 210 provided in the user terminal 200. Additionally, the control unit 130 may control the taking of a video or photo (or image) through the camera 220 provided in the user terminal 200. Additionally, the control unit 130 may receive information from the user through an input unit (not shown) of the user terminal 200.

There is no particular limitation on the types of display unit 210, camera 220, and input unit (not shown) provided in the user terminal 200. For example, the input unit (not shown) may be a mechanical input means (or mechanical key, for example, mouse, joystick, physical button, dome switch, jog wheel). , jog switch, etc.) and a touch input means. As an example, the touch input means consists of a virtual key, soft key, or visual key displayed on the touch screen through software processing, or on a part other than the touch screen. It can be done with a touch key placed.

Meanwhile, the virtual key or visual key can be displayed on the touch screen in various forms, for example, graphic, text, icon, video or these. It can be made up of a combination of . At this time, when the input unit (not shown) includes a touch screen, the display unit 110 of the user terminal 200 may be made of a touch screen and performs both the role of outputting information and the role of receiving information. can do.

Furthermore, the control unit 130 may receive a request for activation of an augmented reality function related to the live broadcast from the user while the broadcast image of the live broadcast is being provided through the display unit 210 of the user terminal. In response to the user's request to activate the augmented reality function, the control unit 130 performs graphics processing related to items related to the live broadcast on the camera image 20 received from the camera 220 provided in the user terminal. can do. Additionally, the control unit 130 may display the camera image 20 on which augmented reality graphic processing has been performed along with the broadcast image on the display unit 210 of the user terminal. Through this, users can watch the broadcast video of the live broadcast and at the same time experience (or experience, test) the items sold in the live broadcast. Hereinafter, the live broadcast service and the augmented reality service are simultaneously provided in the live broadcast providing system 100a according to the present invention so that the user can experience or test items related to the live broadcast while watching the broadcast video of the live broadcast. Let us explain in more detail how to provide it.

FIG. 3 is a flowchart for explaining a method for providing live broadcasting according to the present invention, and FIGS. 4A and 4B are conceptual diagrams for explaining activation of an augmented reality function in the method for providing live broadcasting according to the present invention. Furthermore, FIG. 5 is a conceptual diagram for explaining a broadcast image provided through a live broadcast according to the present invention, and FIG. 6 is a conceptual diagram for explaining a camera image provided through an augmented reality function in the present invention.

Furthermore, Figure 7 is a conceptual diagram for explaining the product DB according to the present invention, and Figures 8, 9, and 10 are conceptual diagrams for explaining the user environment provided to the seller terminal in the present invention. FIGS. 11 and 12 are conceptual diagrams for explaining a method of providing additional item information in the present invention, and FIGS. 13 to 15 are conceptual diagrams for explaining an augmented reality service related to the present invention.

In the method for providing live broadcasting according to the present invention, a process of receiving broadcast video of a live broadcast may be performed from the live broadcast server 300 (S310, see FIG. 3).

Reception of broadcast video of a live broadcast may be performed in the user terminal 200.

As described above, the live broadcast server 300 can receive broadcast video of a live broadcast from a seller terminal and provide it to the live broadcast providing system 100 according to the present invention.

The communication unit 110 may receive broadcast video of a live broadcast conducted in real time from the live broadcast providing server 300 using a streaming technique.

That is, the control unit 130 can control the communication unit 110 so that the communication unit 110 continues to receive broadcast images while a live broadcast is in progress.

Next, in the method for providing live broadcasting according to the present invention, based on receiving the broadcast image from the live broadcast server 300, the received broadcast image 10 is displayed on the display unit 210 of the user terminal 200. It can be provided (S320, see FIG. 3).

As shown in (a) of FIG. 4A, the control unit 130 operates the display unit 210 so that the received broadcast video 10 is provided (played, output, or displayed) through the display unit of the user terminal 200. can be controlled.

In this way, the control unit 130 performs first data processing to control the display unit of the user terminal 200 so that the broadcast image 10 is provided (played, output, or displayed) through the display unit of the user terminal 200. You can. This first data processing may be performed simultaneously with second data processing related to the augmented reality service described later.

Meanwhile, the broadcast image 10 provided from the display unit may be a broadcast image 10 corresponding to a broadcast channel selected by the user.

Next, in the method for providing live broadcasting according to the present invention, while the broadcast image 10 is being provided on the display unit of the user terminal 200, the augmented reality function related to the live broadcast is received from the user terminal 200. The process of receiving an activation request may proceed (S330, see FIG. 3).

As discussed above, a request to activate the augmented reality function may be received based on the icon 410 being selected from the user terminal 200.

When the icon 410 is selected, the control unit 130 may determine that there is a request for an activation function for an augmented reality function associated with an item sold in a live broadcast of a broadcast video.

As shown in (a) of FIG. 4A, the control unit 130 activates the augmented reality function in one area of the display unit while the broadcast image 10 is being provided on the display unit of the user terminal 200. The linked icon 410 can be controlled to be displayed.

The control unit 130 may control the display unit of the user terminal 200 so that the icon 410 linked to the augmented reality function is displayed in an overlapping manner on the broadcast video 10 played in real time.

The icon 410 may be linked to an augmented reality function linked to an item sold (or introduced) in a live broadcast of a broadcast video provided with the icon 410.

When the icon 410 is selected, an augmented reality function for the item currently being introduced in the broadcast video may be provided to the user terminal.

In the present invention, the augmented reality function can provide an augmented reality function linked to the item, depending on the item being introduced in the broadcast video.

As an example, when selling the first item (ex: “lipstick”) in a live broadcast, item information (or product information) of the first item may be provided in the broadcast video, and in this case, the person selling the first item The icon 410 displayed together with the broadcast image of the live broadcast may be linked to an augmented reality function linked to the first item.

Specification of the item being introduced in the broadcast video can be made based on item information 430 displayed together with the broadcast video 10, as shown in FIG. 4A.

That is, the control unit 130 may provide an augmented reality function corresponding to the item information 430 according to the item information 430 provided in the broadcast video 10. For example, when selling a second item (ex: “foundation”) in a live broadcast, item information (or product information) of the first item may be provided, and in this case, the live broadcast selling the second item may be provided. The icon 410 displayed together with the broadcast image 10 may be linked to an augmented reality function linked to the second item.

Meanwhile, in the broadcast video 10, there is a specified area where the item information 430 is provided, and the item information for which item is provided in the specific area where the item information 430 is specified to be provided. Accordingly, different augmented reality graphics processing can be performed. In this way, the control unit 130 is based on the fact that the broadcast image 10 provided on the display unit of the user terminal 200 changes from a broadcast image selling the first item to a broadcast image selling the second item. Thus, when the icon 410 on the display unit of the user terminal 200 is selected, augmented reality graphic processing linked to the second item can be performed.

That is, when the icon 410 is selected while a broadcast image selling the first item is being output, the control unit 130 performs augmented reality graphic processing linked to the first item on the camera image 20, If the icon 410 is selected while a broadcast image selling the second item is being output, the control unit 130 may perform augmented reality graphic processing linked to the second item on the camera image 20.

Meanwhile, the specification of which item in the broadcast video is being output can be determined based on which item the item information provided with the broadcast video is for.

That is, as described above, depending on which item the item information is output to in the specific area where the item information is output, the control unit 130 determines the camera image corresponding to the item information being output in the specific area. Augmented reality graphics processing can be performed.

Meanwhile, if the augmented reality function is not linked to the item according to the item information 430 being output in a specific area, the icon 410 for activating the augmented reality function may not be output.

That is, in the present invention, whether or not the augmented reality function is applied may differ for each item, and the control unit 130 displays an augmented reality function icon in the broadcast video only when an item linked to the augmented reality function is introduced in the broadcast video. can be provided.

The “request to activate an augmented reality function” described below can be understood as a request to activate an augmented reality function related to a product sold in a live broadcast of a broadcast video provided through the display unit of the user terminal 200.

Additionally, the “augmented reality function” described below can be understood as an augmented reality function related to products sold in a live broadcast of a broadcast video provided through the display unit of the user terminal 200.

Meanwhile, the live broadcasting providing system 100 according to the present invention is a live broadcasting server 300 through software (ex: web browser (wep-browser), mobile browser) that displays web information on the screen. The broadcast video 10 received from can be provided to the display unit of the user terminal 200.

The control unit 130 controls the augmented reality related to the live broadcast from the user terminal 200 while the broadcast image 10 is displayed on the display unit of the user terminal 200 through software that displays web information on the screen. You may receive activation requests for features.

In this case, as shown in (a) of FIG. 4B, the control unit 130 is an application or software that provides an augmented reality function (hereinafter referred to as an augmented reality application) based on receiving an activation request for the augmented reality function. ) to display on the display unit of the user terminal 200 a pop-up window containing a guidance phrase (ex: “Do you want to go to the shopping live app to preview AR makeup?”). You can.

The control unit 130 executes the augmented reality function application based on receiving an augmented reality function application execution request from the user terminal 200 while the guide text 440 is displayed on the user terminal 200. You can.

In addition, if the augmented reality function application is not installed in the user terminal 200, the control unit 130 displays a page related to installing the augmented reality application to the user terminal to induce installation of the augmented reality function application. (200) can be provided.

Next, in the method for providing live broadcasting according to the present invention, in response to a request for activation of the augmented reality function, an item linked to the live broadcast is selected for the camera image 20 received from the camera provided in the user terminal 200. A process of performing augmented reality graphic processing related to may proceed (S340, see FIG. 3).

Even when there is a request to activate the augmented reality function, the control unit 130 may continue to perform first data processing related to providing broadcast images and perform second data processing related to providing augmented reality services.

Here, the second data processing related to providing augmented reality service can be understood as activating a camera in the user terminal 200 and performing augmented reality graphic processing on the camera image 20 received from the activated camera. there is.

At this time, the control unit 130 may access the camera provided in the user terminal 200 based on the access authority to the camera provided in the user terminal 200.

The control unit 130 may perform control related to activation of a camera provided in the user terminal 200. The control unit 130 may check access rights to the camera provided in the user terminal 200 in the live provision system 100 according to the present invention in order to perform control related to camera activation.

As a result of confirmation, if there is no access right to the camera provided in the user terminal 200, the control unit 130 uses (b) in FIG. 4B to obtain access right to the camera provided in the user terminal 200. ), information related to obtaining access rights may be output on one area of the display unit of the user terminal 200. For example, the control unit 130 may output a pop-up or modal window (or modal window) related to obtaining access rights.

The pop-up or modal window may include information related to obtaining access rights (ex: “Camera access is required to use the real-time virtual makeup preview shooting function”).

The control unit 130 may obtain access rights to the camera provided in the user terminal 200 based on user input to one area of the pop-up or modal window.

Such a pop up or modal window may be exposed only once when there is a first activation request for the augmented reality function.

After gaining access to the camera provided in the user terminal 200 once, the control unit 130 displays the pop-up or modal window even if there is a request to activate the augmented reality function. It can be controlled not to be output on the display unit of the user terminal 200.

Meanwhile, as a result of confirmation, if access rights to the camera provided in the user terminal 200 exist, the control unit 130 may control the camera provided in the user terminal 200 to be activated. That is, the control unit 130 can run the camera provided in the user terminal 200.

In this case, the control unit 130 may control activation of either a first type camera provided on the front of the user terminal 200 or a second type camera provided on the back. For example, the control unit 130 may control the user terminal 200 so that the first type of camera is activated in priority over the second type of camera.

The control unit 130 may set the default value (or default value) to activate the first type of camera provided on the front of the user terminal 200 based on an activation request for the augmented reality function.

In this way, based on a request for activation of the augmented reality function, the control unit 130 activates the camera of the user terminal in conjunction with the activation of the augmented reality function so that a camera image is received from the camera of the user terminal 200. You can.

Furthermore, the control unit 130 may receive the camera image 20 from the camera of the activated user terminal 200.

Here, the “camera image 20” may include at least one of a photo (or image), a video, and a preview screen captured through a camera. That is, in the present invention, augmented reality graphic processing can be performed on at least one of a photo (or image), video, and preview screen received through a camera. However, in the following, they will not be distinguished and all will be explained by expressing them as camera images 20.

The control unit 130 may perform augmented reality graphics processing on the camera image 20 based on the camera image 20 being received from the camera provided in the user terminal 200.

In the present invention, performing graphic processing means visual modification of at least a portion of the camera image 20 or visual modification of the camera image 20 based on product information that exists in matching with the item sold in the live broadcast. It may mean synthesizing information.

As shown in FIG. 7, in a pre-specified DB (database), mapping information 720 including product information (or item information, 621, 622) for each item associated with a specific live broadcast 700 is provided. It can exist. Product information 621 of a specific item (ex: first item) included in this mapping information 720 may include graphic information 721c about graphics processing performed in the augmented reality function related to the specific item. You can. The graphic information 621c of a specific item (ex: first item) may include “subject information” that specifies a part of the subject that needs to be visually transformed in the camera image 20. This subject information can be set based on the type of item.

For example, if the item is a product for lip makeup, the subject information includes information specifying the lip portion of the user's face, and if the item is a product for eye makeup, the subject information includes information specifying the lip portion of the user's face. The information may include information specifying the eye portion of the user's face.

As another example, if the item is a product for an ear accessory, the subject information includes information specifying the ear part of the user's face, and if the item is a product for a hand accessory, the subject information may include information specifying the hand portion of the user's body.

As another example, if the item is a product for tops (clothing worn on the upper half of the body), the subject information includes information specifying the upper half (or upper body) of the user's body, and the item is for bottoms (lower body). In the case of a product for clothing (clothing worn on the lower half of the body), the subject information may include information specifying the lower half (or lower body) of the user's body.

The control unit 130 may use the subject information to specify a part corresponding to the subject information in the camera image and perform augmented reality graphic processing focusing on the specified part.

Meanwhile, it is also possible that graphic information for graphics processing performed in a related augmented reality function does not include subject information. In this case, the control unit 130 can use image recognition technology to recognize a subject in a camera image and specify a part that is the target of augmented reality graphic processing based on the type of item in the recognized subject. In this case, the control unit 130 can specify a part in the camera image that corresponds to the type of item based on artificial intelligence technology.

Furthermore, the graphic information 621c of a specific item (ex: first item) may include “type information” about the type of graphic processing for visual transformation of the camera image 20. This graphic type information can be set based on information received from the seller terminal.

Here, examples of graphic processing types for visual transformation are very diverse, for example, i) a graphic processing type that performs color conversion (or color calibration) on the camera image 20, ii) camera A graphic processing type that adjusts at least one of the brightness, contrast, sharpness, and color conversion values of the image 20, iii) inserting a graphic object such as text, a static image, or a dynamic image into the camera image 20 ( or compositing) graphics processing types may exist. The control unit 130 may perform graphics processing on the camera image according to one type or a plurality of types among the various graphics processing types discussed above.

Furthermore, the graphic information 621c of a specific item (ex: first item) includes “color information” about the color that should be expressed in the area corresponding to a part of the subject by visual transformation of the camera image 20. You can. For example, if the item is a lip makeup product, the color information includes the color that should be expressed in the area corresponding to the lip area (ex: “Color code B46143” or “AR setting code AR123”) ) information may be included.

Furthermore, the shape of a graphic object that must be synthesized (or inserted) into the graphic information 621c of a specific item (ex: first item) in an area corresponding to a part of the subject by visual transformation of the camera image 20. Information may be included. For example, if the item is a product for an ear accessory, the shape information may include information about a graphic object in the shape of an ear accessory.

Meanwhile, the control unit 130 can check whether the camera image 20 includes a part of the subject that needs to be visually transformed, based on the graphic information discussed above.

As a result of confirmation, if the camera image 20 includes a part of the subject that needs to be visually transformed, the control unit 130 selects a specific part (or specific area) of the camera image 20 corresponding to the part of the subject. It can be specified.

For example, if the graphic information 621c of a specific item (ex: first item) includes information specifying the “lip part” of the user's face as the part that needs to be visually transformed, the control unit 130 uses the camera The user's face can be recognized in the video 20. And as shown in (b) of FIG. 4A, the control unit 130 can specify a specific part (or specific area, 420) corresponding to the lips of the user's face in the camera image 20.

As another example, if the graphic information 621c of a specific item (ex: first item) includes information specifying the eye part of the user's face as a part that needs to be visually transformed, the control unit 130 may control the camera image ( In 20), a specific part (or specific area) corresponding to the eyes of the user's face can be specified.

Furthermore, the control unit 130 may perform graphics processing using a graphics processing type according to matching information matched to a specific portion (or specific area, 420) of the camera image 20.

If the graphics processing type matched to the specific part is a type that performs color conversion (or color calibration) on the camera image 20, the control unit 130 controls the color of the specific part (or specific area). can be changed to correspond to color information.

For example, as shown in (b) of FIG. 4A, a specific part (or specific area, 420) of the camera image 20 corresponding to the lip portion of the user's face is displayed so that the color corresponding to the color information is visible. Colors may change.

Through this color change or color correction, users can experience the same effect as applying makeup to their face image included in the camera video.

As another example, when the graphics processing type is a type that inserts (or combines) a graphic object into the camera image 20, the control unit 130 inserts a graphic object corresponding to shape information in a specific part (or specific area). Can be inserted (or synthesized).

For example, the control unit 130 inserts a graphic object of an ear accessory (ex: earring) corresponding to shape information into a specific part (or specific area) of the camera image 20 corresponding to the ear part of the user's face. (or can be synthesized).

By inserting these graphic objects, users can experience the same effect as wearing an accessory on their own image included in the camera image.

In this way, the control unit 130 may perform graphics processing on the camera image 20 in response to a request to activate the augmented reality function from the user terminal 200.

Meanwhile, product information for items that are not linked to the augmented reality function may not include graphic information.

Furthermore, the specified DB discussed above may exist and be stored on a live broadcast server or an external server linked to the live broadcast server.

Meanwhile, product information of items such as the product information of the first and second items discussed above may exist and be stored in the item DB of the live broadcast server. Furthermore, this item DB can be classified in connection with a specific seller account. That is, item information on items sold by a specific seller account can be divided based on each specific seller account to form an item DB.

Meanwhile, product information included in the pre-specified DB can be selected from the seller or live broadcast server from this item DB and included in the pre-specified DB.

Next, in the method for providing live broadcasting according to the present invention, while the broadcast image 10 is being received at the user terminal 200, the camera image 20 on which augmented reality graphic processing has been performed is sent to the user terminal 200. The process of displaying on the display unit may proceed (S350, see FIG. 3).

As described above, the control unit 130 may simultaneously perform first data processing related to providing the broadcast image 10 and second data processing related to providing the augmented reality service even if there is a request to activate the augmented reality function. . That is, the control unit 130 can control the user terminal 200 so that the first data processing and the second data processing are performed in parallel without stopping the first data processing.

As shown in (b) of FIG. 4A, the control unit 130 displays the camera image 20 on which augmented reality graphic processing has been performed along with the broadcast image 10 on the display unit of the user terminal 200. This can be controlled as much as possible, and in the present invention, this can be called “PIP (Picture In Picture) mode”.

In response to a request to activate the augmented reality function, the control unit 130 controls the display unit of the user terminal 200 in PIP mode to display the broadcast image 10 and the camera image 20 on which augmented reality graphic processing has been performed. Can be provided together.

As such, in the present invention, the broadcast image 10 and the camera image 20 on which augmented reality graphic processing has been performed can be simultaneously provided through the PIP mode by performing the first data processing and the second data processing in parallel. there is. Through this, users can experience or test products sold in the live broadcast through augmented reality without stopping watching the live broadcast.

Meanwhile, the live provision system 100 according to the present invention improves the user's interest (or interest) in the live broadcast, intuitively provides the user with information about products sold through the live broadcast, and allows the user to It can provide a variety of user environments that allow users to conveniently use augmented reality services. Hereinafter, various user environments provided by the live broadcasting provision system 100 according to the present invention will be described in detail.

As shown in (b) of FIG. 4A, the display unit of the user terminal 200 may include at least one of a main display area (A) and a sub-display area (B).

The sub-display area (B) may be displayed overlapping at least a portion of the main display area (A).

The position where the sub-display area (B) overlaps with the main display area (A) can be set by the system administrator or changed based on user input. Furthermore, of course, the output size of at least one of the main display area (A) and the sub-display area (B) can also be set and changed based on system administrator and user input.

In the above example, the case where the sub-display area (B) overlaps the main display area (A) was examined, but the present invention is not limited to the above example. For example, the display unit of the user terminal 200 may be divided into a plurality of display areas, one of which may be the main display area (A) and the other may be the sub-display area (B). At this time, the main display area (A) and the sub-display area (B) may not overlap each other, or may only partially overlap.

Meanwhile, the control unit 130 may control the broadcast image 10 to be displayed in at least one of the main display area (A) and the sub-display area (B) based on whether the augmented reality function is activated.

As shown in (a) of FIG. 4A, when the augmented reality function is deactivated, the control unit 130 can control the broadcast image 10 to be displayed on the main display area (A).

On the other hand, as shown in (b) of FIG. 4A, in the activated state of the augmented reality function, the control unit 130 displays the broadcast image 10 in the sub-display area (B) and displays the camera image (or The camera image 20) on which graphic processing has been performed may be displayed in the main display area (A). Hereinafter, for convenience of explanation, the camera image 20 and the camera image 20 on which graphic processing has been performed will be used interchangeably.

Based on the augmented reality function being switched from the deactivated state to the activated state, the control unit 130 operates the display unit 210 so that the broadcast image 10 disappears from the main display area (A) and the camera image 20 is displayed. You can control it. Additionally, the control unit 130 may control the display unit 210 to display the broadcast image 10 in the sub-display area (B). That is, the control unit 130 operates the display unit so that the broadcast image 10 displayed in the main display area (A) continues to be displayed through the sub-display area (B) even if the augmented reality function is switched from the deactivated state to the activated state. (210) can be controlled. Through this, the user can continue to watch the broadcast video 10 being watched without interruption while receiving the augmented reality service.

Meanwhile, as shown in (a) of FIG. 5, the control unit 130 operates in a state where the broadcast image 10 is provided on the display unit (or main display area A) of the user terminal 200. , it can be controlled so that various information related to the broadcast image 10 is displayed in at least one area of the display unit of the user terminal 200.

The control unit 130 may control various information to be displayed together with the broadcast video 10 of a specific live broadcast, based on various information that matches the specific live broadcast in a pre-specified DB.

In the present invention, as shown in (a) of FIG. 5, various information along with the broadcast image 10 are displayed through the display unit of the user terminal 200 by “at least one area of the broadcast image 10.” It can be expressed as “displaying (or outputting or providing) information” or “displaying (or outputting or providing) a broadcast video (10) containing various information.”

As seen above, in a pre-specified DB, channel information 710 and mapping information 720 related to a specific live broadcast 700 may be matched and exist.

Channel information 710 related to a specific live broadcast 700 may include information used to proceed and watch the specific live broadcast 700. For example, the channel information 710 includes i) title information 711 of a specific live broadcast 700, ii) viewer information watching a specific live broadcast (ex: number of viewers, 712), iii) It may be included in the host information 713 conducting a specific live broadcast.

In the channel information 710, i) title information 711 and iii) host information 713 may be set based on information received from the seller terminal. On the other hand, in the channel information 710, ii) viewer information 712 may be generated (or set) by counting the number of viewers watching a specific live broadcast 700 by the live broadcast server 300. there is.

The control unit 130 controls the user for which the broadcast video 10 of the specific live broadcast 700 is displayed, as shown in (a) of 5a, based on the channel information 710 related to the specific live broadcast 700. The channel information 710 may be displayed in at least one area of the display unit of the terminal 200. At this time, the control unit 130 may display the channel information 710 on the display unit as text, a graphic object, or an icon.

For example, as shown in (a) of 5a, the first area 510 of the display unit 510 of the user terminal 200 where the broadcast video 10 is displayed includes title information (ex. : “Monthly Gift Shop Episode 8: Black Friday Special”), viewer information (ex: “Number of viewers: 331, 425”) watching a specific live broadcast (700) may be displayed. A user watching a specific live broadcast can intuitively recognize information about the live broadcast being watched just by looking at the display unit.

Furthermore, as shown in FIG. 7, the mapping information 720 related to a specific live broadcast 700 may include product information 721 and 722 related to each of at least one item associated with the specific live broadcast. . Taking a specific item (ex: first item) as an example and describing the product information 721 related to the specific item, the product information 721 includes i) linkage information 721a indicating that the specific item is linked to the augmented reality function; , ii) Item information that can be used for the shopping service of a specific item (ex: product name of a specific item, price information of a specific item, detailed information of a specific item used to describe a specific item, 721b), iii) Specific item and It may include at least one of graphic information (ex: color information, 721c) used to perform related graphic processing, and vi) target information (721d) that induces the experience or purchase of a specific item. The product information 721 may be set based on information received from the seller terminal.

The control unit 130 controls the user for whom the broadcast video 10 of the specific live broadcast 700 is displayed, as shown in (a) of 5, based on the mapping information 720 related to the specific live broadcast 700. The mapping information 720 may be displayed in at least one area of the display unit of the terminal 200. At this time, the control unit 130 may display the mapping information 710 on the display unit as text, a graphic object, or an icon.

As shown in (a) of FIG. 5, in the second area 510 of the display unit 510 of the user terminal 200 where the broadcast image 10 is displayed, an augmented reality display related to an item associated with a specific live broadcast 700 is displayed. A function icon 410 may be displayed to provide a function. The function icon 410 may be linked to an augmented reality function for at least one item associated with a specific live broadcast 700.

The icon 410 linked to the augmented reality function displays the broadcast image 10 and the broadcast image 10 on the display unit of the user terminal 200, based on whether the item linked to the live broadcast is matched to the augmented reality function. can be displayed together.

When an item linked to a live broadcast is matched to the augmented reality function, the control unit 130 displays the augmented reality function in the first area 520 of the display unit 520 of the user terminal 200 where the broadcast image 10 is displayed. An icon 410 linked to can be displayed.

More specifically, the control unit 130 includes augmented reality function linkage information indicating that at least one item related to the specific live broadcast 700 is linked to the augmented reality function in the mapping information 720 of the specific live broadcast 700. Based on the inclusion of 721a), an icon 410 linked to the augmented reality function can be displayed in the second area 520.

On the other hand, if the item linked to the live broadcast is not matched to the augmented reality function, the control unit 130 controls the user terminal 200 on which the broadcast image 10 is displayed, as shown in (b) of FIG. 5. ), it is possible to limit the display of icons linked to the augmented reality function in the first area 520 of the display unit.

More specifically, the control unit 130 operates the second area ( Display of the icon 410 linked to the augmented reality function in 520 may be restricted.

Furthermore, as shown in (a) of FIG. 5, on the display unit of the user terminal 200, around the icon 410 linked to the augmented reality function, a guidance message to induce activation of the augmented reality function ( ex: “Experience the color with AR makeup preview”530) may be displayed.

This guidance message 530 may be output on the display unit 210 only for a preset time after the product information of the item linked to the augmented reality function is output on the broadcast video.

Furthermore, this guidance message 530 may be output on the display unit 210 at preset time intervals and then disappear while product information of an item linked to the augmented reality function is output on a broadcast video.

Furthermore, this information message 530 is displayed on the display unit 210 only when the augmented reality function has never been activated on the user terminal, or when the augmented reality function has never been activated on the user terminal while watching a live broadcast. It can be printed on the screen. Meanwhile, the user can use a service to experience or test a product (or item) sold in a specific live broadcast 700 that is currently being watched through an information message 530 or an icon 410 linked to the augmented reality function. You can intuitively recognize that it can be used.

Subsequently, as shown in (a) of FIG. 5, in the third area 530 of the display unit 530 of the user terminal 200 where the broadcast image 10 is displayed, there is at least one image associated with a specific live broadcast 700. Various item information about the product (or item) may be displayed. As seen above, the icon 520 linked to the augmented reality function is displayed in the second area 520 only when the item according to the item information output in the third area 530 is linked to the augmented reality function. can be printed.

Meanwhile, the product information displayed in the third area 530 includes the product name of the item introduced in the live broadcast (ex: “Lira Powder Matte Lip”), the price information of the item (ex: “40,000 won”), and the description of the item. It may contain at least one piece of information.

Continuing, as shown in (a) of FIG. 5, in the fourth area 540 of the display unit of the user terminal 200 where the broadcast video 10 is displayed, chatting related to a specific live broadcast 700 is displayed. ) information can be displayed. Chat information may include messages received from a seller terminal or a plurality of consumer terminals while a specific live broadcast 700 is in progress. The control unit 130 may display messages received in real time from a seller terminal or a plurality of consumer terminals through the fourth area 540 of the display unit.

In this way, while the broadcast image 10 of a specific live broadcast is displayed on the display unit of the user terminal 200, the control unit 130 provides various information related to the specific live broadcast on the display unit. , users can recognize various information related to a specific live broadcast at once through one display unit.

Meanwhile, in the present invention, various information related to the augmented reality function is provided along with the camera image 20 to the display unit (or main display area (A)) of the user terminal 200, so that the user can view items through augmented reality. Experience services can be used more conveniently and intuitively.

As shown in FIG. 6, the control unit 130 performs augmentation on at least one area of the display unit of the user terminal 200 while the camera image 20 is provided on the display unit of the user terminal 200. You can control the display of various information related to reality functions.

As shown in FIG. 6, various information associated with the augmented reality function along with the camera image 20 are displayed through the display unit of the user terminal 200, which means “information in at least one area of the camera image 20.” It can be expressed as “display (or output or provide)” or “display (or output or provide) a camera image 10 containing various information.” At this time, the camera image 20 may be a camera image 20 on which augmented reality graphic processing has been performed. Various information associated with the augmented reality function may also be displayed on the display unit based on various information that exists in a pre-specified DB in matching with a specific live broadcast.

As shown in FIG. 7, mapping information 720 including product information 721 and 722 for each item associated with a specific live broadcast 700 may exist in the pre-specified DB. Each product information 721, 722 of a plurality of items (ex: first item and second item) includes graphic information 721c, 722c about graphics processing performed in an augmented reality function related to each of the plurality of items. may be included.

As described above, the graphic information 721c and 722c of a specific item includes i) “subject information” that specifies a portion of the subject for which visual transformation of the camera image 20 is to be performed in relation to the specific item, ii) In relation to a specific item, “type information” about the type of graphics processing for visual transformation of the camera image 20, iii) in relation to a specific item, corresponding to a part of the subject by visual transformation of the camera image 20 “Color information” about the color that must be expressed in the area, and vi) in relation to a specific item, a graphic object that must be synthesized (or inserted) in the area corresponding to a part of the subject by visual transformation of the camera image 20 At least one of the “shape information” may be included.

The control unit 130 displays the camera image 20, as shown in (b) of FIG. 5, based on the graphic information 721c and 721c of each of the plurality of items associated with the specific live broadcast 700. The graphic information 721c and 722c may be displayed in at least one area of the display unit of the user terminal 200. At this time, the control unit 130 may display the graphic information 721c and 722c as text, a graphic object, or an icon on the display unit.

These graphic information 721c and 722c may each correspond to a plurality of options set for the item. Here, the option is selection information for specifying the item you want to purchase when purchasing an item. If the item is a product corresponding to lip makeup (ex: lipstick), the option is the color of the lip tip. It could be information.

Hereinafter, for convenience of explanation, an example will be given where the item associated with a specific live broadcast 700 is a lip makeup product.

As shown in FIG. 6, in the first area 610 of the display unit of the user terminal 200 where the camera image 20 is displayed, an item associated with a specific live broadcast 700 (or an item associated with an augmented reality function) is displayed. A graphic object (for example, reference numerals 611 and 612) corresponding to an option set in an item may be output. The graphic object corresponding to these options may be a graphic object corresponding to color information corresponding to each option.

In this case, the camera image 20 may be a camera image 20 on which graphic processing has been performed using any one 611 of color information corresponding to a plurality of options corresponding to the item. For example, the camera image 20 is a camera image 20 that has been color converted to correspond to color information corresponding to the graphic object 611 included in a specific partial area 610a of the first area 610. may apply. For example, in the camera image 20, the area 420 corresponding to the lip portion of the user's face is color converted into color information corresponding to the graphic object 611 included in the specific partial area 610a. There may be.

Furthermore, the control unit 130, based on receiving a preset user input applied to the first area 610 while the camera image 20 is displayed on the display unit of the user terminal 200, Augmented reality graphic processing for the camera image 20 can be changed.

More specifically, the control unit 130 is displaying a camera image 20 on which graphic processing has been performed corresponding to color information corresponding to the first option of a specific item on the display unit of the user terminal 200. In, based on the graphic object 612 corresponding to the color information of the second option different from the first option being selected by a preset user input applied to the first area 610, the camera image 20 Graphics processing according to the color information of the second option may be performed.

For example, in a state where a graphic object 611 corresponding to the color information of the first option is displayed in the specific partial area 610a, the Since the graphic object 612 corresponding to the color information of the second option is displayed in the specific partial area 610a, the graphic object 612 corresponding to the color information of the second option may be selected.

That is, the control unit 130 moves the graphic object 612 corresponding to the color information of the second option to the specific partial area 610a based on a preset user input (e.g., drag or swipe input). When moving, the graphic object 612 corresponding to the color information of the second option may be processed as selected. In this case, the control unit 130 may perform augmented reality graphic processing on the camera image using the color information of the second option.

As another example, while the graphic object 611 corresponding to the color information of the first option is displayed in the specific partial area 610a, a graphic object corresponding to the color information of the second option is sent from the user terminal 200 ( When 612) is selected, the control unit 130 may process the graphic object 612 corresponding to the second color information as being selected.

In this case, the control unit 130 may perform augmented reality graphic processing on the camera image using the color information of the second option.

Meanwhile, in the specific partial area 610a, a graphic object 612 corresponding to the color information of the second option may be located instead of the graphic object 611 corresponding to the color information of the first option. In this way, the user can select one of the plurality of graphic objects 611 and 612 displayed in the first area 610 and experience various items.

Meanwhile, when a graphic object located in the specific partial area 610a is selected from the user terminal 200, the control unit 130 may capture a camera image processed with augmented reality graphics. In this case, the camera image processed with augmented reality graphics may be stored in the storage of the user terminal or an external server (live broadcast server or cloud server, etc.).

Meanwhile, as shown in FIG. 6, in the display unit of the user terminal 200 on which the camera image 20 is displayed, in the second area 620 around the specific partial area 610a, the specific partial area 610a is displayed. Among the descriptive text (ex: “301 Spring Date”) that describes the option corresponding to the graphic object 611 included in (610a) and the recommendation text that recommends the item corresponding to the option (ex: “Popular Color”) At least one may be displayed.

These explanatory phrases and guidance phrases may also exist in matching with the item in the previously specified product DB discussed above. For example, the explanatory text and guidance text may be included in target information 621d that induces experience or purchase of the item.

Continuing, as shown in FIG. 6, in the third area 630 of the display unit of the user terminal 200 where the camera image 20 is displayed, at least one product (or Various item information about the item is displayed, and can be used as the product name of the item (ex: “Lira Powder Matte Lip”), used as price information of the item (ex: “40,000 won”), or used to describe the item. It can be used in various ways.

At this time, the third area 630 may include option information corresponding to a specific item on which graphic processing has been performed on the camera image 20. The control unit 130 is configured to display the camera image 20 on which graphic processing has been performed according to the color information of the first option on the display unit of the user terminal 200, in the third area 630. It is possible to control the display of the first option information corresponding to the item. In addition, the control unit 130 performs graphic processing according to the color information of the second option on the camera image 20 based on a preset user input for the first area 610, and performs graphic processing according to the color information of the second option. 3 It can be controlled to display information corresponding to the second option on area 630.

Meanwhile, as described above, when the augmented reality function is deactivated, the broadcast image 10 is displayed in the main display area (A) of the user terminal 200 (see (a) in FIG. 4A), and the augmented reality function is activated. In this activated state, the camera image 20 may be displayed on the main display area (A) of the user terminal 200 (see (b) in FIG. 4A).

As shown in FIG. 4A, the control unit 130 displays, independently of whether the augmented reality function is activated, an item related to the live broadcast in at least one area on the main display area (A) of the user terminal 200. You can control item information to be displayed continuously.

More specifically, the control unit 130 displays a camera image ( Even if 20) is displayed, the item information 430 displayed together with the broadcast image 10 can be controlled to continue to be displayed together with the camera image 20.

Through this, the user can continue to check what item is being experienced through augmented reality even when the augmented reality function is activated.

Meanwhile, as described above, the live broadcast providing system 100 according to the present invention not only can display various information on the display unit of the user terminal 200, based on a pre-specified DB, but also can display various information in the live broadcast. You can decide whether to provide augmented reality features for your product.

The pre-specified product DB according to the present invention can be constructed based on information received from the seller terminal, and can be the basis for providing various information and augmented reality functions in the live broadcast providing system 100 according to the present invention. there is. That is, in the present invention, a pre-specified product DB can be constructed based on information received from the seller terminal, and a live broadcast provision service and an augmented reality function provision service can be performed based on the constructed pre-specified product DB.

This pre-specified product DB can be built by the live broadcast server 300 or by a separate external server 100a linked to the live broadcast providing system 100 according to the present invention. Hereinafter, the live broadcast server 300 and the separate external server 100a linked to the live broadcast providing system 100 according to the present invention are not distinguished, and the pre-specified product DB is stored by the live broadcast server 300. Let's explain it by building it.

The live broadcast server 300 builds a pre-specified product DB using the information received from the seller terminal, and includes it in the pre-specified product DB in response to a request from the live provision system 100 according to the present invention. Various information (or data) can be provided to the live provision system 100 according to the present invention.

The live broadcast server 300, in order to receive at least one of information related to the live broadcast, information about the product sold in the live broadcast, and information related to the augmented reality function of the product, from the seller terminal, as shown in FIGS. 8, 9, and As shown in Figure 10, seller pages 800, 900, and 1000 can be provided to the seller terminal.

Here, the seller pages (800, 900, 1000) allow sellers who want to sell products through live broadcasting to provide i) information related to the live broadcast (ex: broadcast title information, broadcast host information, etc.), ii) products for sale, and Enter and set related information (ex: item type information, item name information, item price information, item manufacturer information, recommendation information for recommending an item, item sales statistics information, etc.) and ii) various information related to the augmented reality function. It can be understood as a page (or screen) that provides functions that allow

In particular, the live broadcast server 300 can provide a function to receive various information for matching augmented reality functions to items through the seller pages 800, 900, and 1000.

As shown in FIG. 8, the seller page 800 may include a function icon 810 that can set whether or not the augmented reality function matches for a specific item.

Based on receiving a user input for the function icon 810, the cloud server displays a screen (ex: pop-up) linked to a page for matching augmented reality functions in one area of the seller page 800. , 820) can be displayed.

The live broadcast server 300 responds to receiving a request for a page for augmented reality function matching (ex: “Settings” selection, 821) through the screen 820, as shown in FIG. 9. , A seller page 800 can be provided for matching augmented reality functions for a specific item.

The seller page 900 for matching augmented reality functions for a specific item includes i) an area 910 for registering specific item information (ex: “product number”) matching the augmented reality function, ii) the specific item’s Area 920 for registering an image, iii) Area 930 for registering an option name (ex: item name or item number) related to a specific item, vi) inputting graphic information (ex: color information) related to the specific item It may include at least one of areas 940 and 950 and v) an area 960 where information used to recommend or explain the specific item is input.

The live broadcast server 300 matches the augmented reality function to a specific item based on the information input (or received) through the seller page 900, and links various information related to the specific item and the augmented reality function. A DB for already specified products can be built.

Furthermore, as shown in FIG. 10, on the seller page 1000, an icon 1010 that provides a modification function for a specific item previously matched to the augmented reality function, an icon 1020 that provides a delete function, and history information It may include at least one of the icons 1030 matched to the provision.

The user can edit or add information (ex: color information) about a specific item already matched to the augmented reality function by using the icon 1010 that provides the editing function on the seller page 1000 shown in FIG. 10. there is. Additionally, the user can select the icon 1020 that provides a delete function so that a specific item is no longer matched to the augmented reality function.

Furthermore, the live broadcast server 300, in response to a user input for the icon 1030 matched to provide history information, provides setting history information of the augmented reality function for a specific item on one area of the seller page 1000. (1040) can be provided.

As such, in the present invention, a service can be provided that allows the seller (or host) to directly set the settings for whether or not to provide the augmented reality function and the product experience provided through the augmented reality function, and the seller (or host)'s settings can be adjusted. Based on this, augmented reality functions can be provided for specific items.

Accordingly, sellers can provide consumers with the opportunity to easily experience their products through the seller page by simply entering information.

Meanwhile, as described above, item information related to items associated with a live broadcast may be continuously displayed on the display unit of the user terminal 200, regardless of whether the augmented reality function is activated.

The item information may include item information (ex: specific item name, price, etc.) related to one of a plurality of items associated with the live broadcast. The user can receive information about a specific item through the item information. Furthermore, the user may wish to receive additional item information about a wider variety of items in addition to item information about a specific item.

Accordingly, in the present invention, item information of additional items linked to the live broadcast can be provided on the display unit of the user terminal 200, independently of whether the augmented reality function is activated.

Here, “additional items” can be set in various ways. Among the plurality of items linked to the live broadcast, it may be understood as an item other than the item of item information displayed together with the broadcast video 10. For example, assume that item information of the first lip makeup item among the first lip makeup items and second lip makeup items associated with the live broadcast is displayed together with the broadcast video 10. In this case, the additional item can be understood as an item for the second lip makeup.

Furthermore, “additional items” can be understood as items related to items linked to the live broadcast. For example, let's assume that the item linked to the live broadcast is an item about lip makeup. In this case, the additional item may correspond to an item for eye makeup.

In the present invention, while item information of a specific item is being displayed on the display unit of the user terminal 200, item information of an additional item may be provided in response to a user request for item information of an additional item.

As shown in (a) of FIG. 11, the control unit 130 displays a graphic object linked to the additional item list while the broadcast image 10 is displayed on the main display area A of the user terminal 200. User selection for (ex: “See more”, 1140) can be received.

And, as shown in FIG. 11(b), in response to the user selection, the control unit 130 displays a plurality of additional item information items 1110, 1120, and 1130 in at least one area of the main display area A. ) can display an additional item information list 1100 including.

Each item information item (1110, 1120, 1130) of a plurality of items included in the additional item information list (1100) includes a thumbnail image of the item corresponding to each item, an item name (ex: “Lira Spicy Volume Mat”), It may include at least one of price information (ex: “39,000 won”) and event information.

Furthermore, based on the fact that the augmented reality function is matched to a specific item among the plurality of items included in the additional item information list 1100, an augmented reality function related to the specific item is displayed in one area 1120a of the item item of the specific item. An icon 1120b linked to the function may be included.

Meanwhile, even when the augmented reality function is activated, the control unit 130 may provide item information about additional items in response to a user request for item information about additional items.

As shown in (a) of FIG. 12, the control unit 130 displays a graphic object linked to the additional item list while the camera image 20 is displayed on the main display area A of the user terminal 200. User selection for 1240 may be received.

And, as shown in FIG. 12(b), in response to the user selection, the control unit 130 displays a plurality of additional item information items 1210, 1220, and 1230 in at least one area of the main display area A. An additional item information list 1200 including can be displayed.

The item information items 1210, 1220, and 1230 of each of the plurality of items included in the additional item information list 1200 include a thumbnail image of the item corresponding to each item, an item name (ex: “Lira Spicy Volume Mat”), It may include at least one of price information (ex: “39,000 won”) and event information.

Furthermore, based on the fact that the augmented reality function is matched to a specific item among the plurality of items included in the additional item information list 1200, an augmented reality function related to the specific item is displayed in one area 1220a of the item item of the specific item. An icon 1240 linked to the function may be included.

In this way, the user can be provided with the augmented reality function for a specific item through the icon 410 linked to the augmented reality function displayed on the display unit of the user terminal 200 where the broadcast video 10 is displayed. Additionally, the user can be provided with the augmented reality function for the additional item through the icons 1120b and 1220b linked to the augmented reality function included in the additional item list 1100 and 1200.

The control unit 130 can activate the camera of the user terminal as long as the icon linked to the augmented reality function exposed in various situations is selected from the user terminal 200. Additionally, the control unit 130 may perform augmented reality graphic processing on the camera image received through the activated camera to provide the user with an experience of the item.

Meanwhile, in the present invention, data processing related to ending Picture In Picture (PIP) mode can be performed based on the augmented reality function switching from an activated state to a deactivated state.

The control unit 130 may switch the augmented reality function from the activated state to the deactivated state in response to receiving a deactivation request for the augmented reality function from the user terminal 200.

As shown in (a) of FIG. 13, the request to deactivate the augmented reality function is based on selecting a graphic object (or icon, 1300) matched to the augmented reality function deactivation function from the user terminal 200. can be received.

More specifically, with the camera image 20 displayed in the main display area (A) and the broadcast image 10 displayed in the sub-display area (B), it is located in one area of the main display area (A). Based on the graphic object 1300 being selected, the control unit 130 may receive a deactivation request for the augmented reality function.

The graphic object 1300 matched to this augmented reality function deactivation function may be variously named “augmented reality function end button”, “PIP mode end button”, “return to broadcast video button”, etc.

As shown in (b) of FIG. 13, in response to receiving a deactivation request for the augmented reality function, the control unit 130 causes the sub-display area B to disappear on the display unit of the user terminal 200. You can control it. Also, the control unit 130 can control the display part of the user terminal 200 to include only the main display area (A).

Furthermore, the control unit 130 can control the broadcast image 10 to be displayed in the main display area A where the camera image 20 was displayed. Additionally, the control unit 130 may restrict the display of the camera image 20 on the display of the user terminal 200.

In this case, the broadcast image 10 displayed in the main display area (A) may be the broadcast image 10 displayed in the sub-display area (B).

That is, when the augmented reality function is switched from the activated state to the deactivated state, the control unit 130 can control the broadcast image 10 displayed in the sub-display area (B) to continue to be displayed in the main display area (A). .

When the augmented reality function is activated, the user can continue to watch the live broadcast he was watching and at the same time experience products sold in a specific live broadcast through the camera image 20 on which augmented reality graphic processing has been performed. Furthermore, when the augmented reality function is disabled, you can continue to watch only the live broadcast you were watching.

Meanwhile, in the present invention, a product purchase page for purchasing products can be provided to the user terminal 200 so that users watching a live broadcast can purchase products sold through the live broadcast.

The control unit 130 receives a preset input for specific item information from the user terminal 200 while at least one of the broadcast image 10 and the camera image 20 is displayed on the display unit of the user terminal 200. can receive.

As shown in (a) of FIG. 14, receiving a preset input for the specific item information includes item information (ex: representative image or thumbnail of the specific item, 1410) of the specific item from the user terminal 200. may be received based on a preset input being applied to the displayed area. The item information of the specific item may be linked to a product purchase page 1400 selling the specific item, as shown in (b) of FIG. 14 .

As shown in (b) of FIG. 14, the control unit 130, based on receiving a preset input for item information 1410 of a specific item from the user terminal 200, A product purchase page 1400 for purchasing a product may be provided on the user terminal 200. At this time, the product purchase page 1400 may be provided in a modal window in one area of the display unit of the user terminal 200.

Meanwhile, the product purchase page 1400 may be linked to item information displayed overlapping the broadcast image 10 when the augmented reality function is deactivated.

Furthermore, the product purchase page 1400 may be linked to item information displayed overlapping the camera image 20 when the augmented reality function is activated. That is, in the present invention, specific item information displayed on the display unit of the user terminal 200 may be linked to a product purchase page 1400 where a product corresponding to the specific item information can be purchased.

Meanwhile, as shown in (b) of FIG. 14, the product purchase page 1400 is displayed on the user terminal 200 while the user terminal 200 is receiving the broadcast video 10 of a live broadcast. It may be provided on the side.

The broadcast image 10 of the live broadcast may be continuously displayed in the background area (or screen) of the modal window including the product purchase page 1400.

When the broadcast image 10 is obscured by the modal window, the control unit 130 controls the broadcast sound provided by the broadcast image 10 to be continuously output through the user terminal 200. can do.

Furthermore, the control unit 130 can control the product payment process corresponding to a specific item sold on the product sales page to proceed based on a payment request being received from the user terminal 200 through the product purchase page 1400. there is.

This payment process can be performed by the electronic payment system according to the present invention. The control unit 130 may control the communication unit 110 so that the communication unit 110 communicates with a financial institution (bank or credit card) server. The control unit 130 may request a payment amount corresponding to a specific item from an account associated with the user account to the financial institution through the communication unit 110. However, the above-described procedure is only an example, and the procedure initially performed for the electronic payment process is not limited thereto.

Meanwhile, the user cannot activate the camera of the user terminal 200 or uses the broadcast video 10 rather than the camera video 20 received from the camera provided in the user terminal 200 to select a specific item. There may be a need to receive experience about . For example, a user may want to experience an item by applying lip makeup to the seller's face rather than to the user's own face. Additionally, the host of the live broadcast can also apply lip makeup items to his or her face and introduce the items to consumers.

Accordingly, the present invention can provide an augmented reality function that performs augmented reality graphic processing on the broadcast video 10.

As shown in (a) of FIG. 15, the control unit 130 controls the broadcast image 10 from the user terminal 200 while the broadcast image 10 is being provided on the display unit of the user terminal 200. ) can receive a request for activation of the augmented reality function.

A request to activate the augmented reality function using the broadcast image 10 may be received from the user terminal 200 based on selection of the icon 1500 linked to the augmented reality function using the broadcast image 10.

As shown in (b) of FIG. 15, the control unit 130 activates the broadcast image 20 received from the live broadcast server 300 based on a request for activation of the augmented reality function using the broadcast image 10. Augmented reality graphics processing can be performed.

The broadcast image 10 on which graphic processing has been performed may continue to be provided through the main display area A even if there is a request for activation of the augmented reality function. Additionally, in this case, the sub-display area B may not be displayed on the user terminal 200.

Meanwhile, although not shown, a first icon and a second icon linked to the augmented reality function may be displayed on the display unit of the user terminal 200. The first icon is an icon (410, see (a) in FIG. 4A) linked to the augmented reality function using the camera image 20 received from the camera of the user terminal 200, and the second icon is a broadcast image. It may be an icon 1610 (see (a) of FIG. 16) linked to an augmented reality function using (20).

The control unit 130 selects the second icon 1610 among the first icon 410 and the second icon 1610 and sends the camera image 20 based on receiving a request for activation of the augmented reality function. Augmented reality graphics processing can be performed.

In addition, although not shown, when the control unit 130 receives a request to activate the augmented reality function based on the icon linked to the augmented reality function using the broadcast image 10 from the user terminal 200 is selected. , a screen for selecting any one of the broadcast image 10 and the camera image 20 (e.g., a screen including a first graphic object corresponding to the broadcast image and a second graphic object corresponding to the camera image) ) can be provided.

Based on whether one of the broadcast image 10 and the camera image 20 is selected from the user terminal 200, the control unit 130 may perform augmented reality graphics processing on the selected image.

In addition, although not shown, the control unit 130 provides augmented reality graphics processing to the consumer's user terminal 200 when augmented reality graphics processing is performed on the broadcast image 10 in the seller's user terminal (or seller's terminal). A broadcast image 10 on which realistic graphics processing has been performed can be provided.

Also, although not shown. Assume that a request to activate an augmented reality function using the broadcast video 10 is received from the first user terminal while the broadcast video 10 of a live broadcast is displayed on the first user terminal and the second user terminal, respectively. let's do it. The control unit 130 may display the broadcast image 10 on which graphic processing has been performed on the first user terminal based on a request to activate the augmented reality function. In contrast, the display of camera images on which graphic processing has been performed may be restricted from being displayed on the second user terminal.

That is, in the present invention, when a request for activation of the augmented reality function using the broadcast image 10 is received from the seller terminal, the broadcast image on which graphics processing is performed on each of the first user terminal and the second user terminal watching the live broadcast (10) can be displayed.

On the other hand, when a request for activation of the augmented reality function using the broadcast image 10 is received from the first user terminal watching the live broadcast, the first user terminal displays the broadcast image 10 on which graphic processing has been performed, and , display of the broadcast image 20 on which graphic processing has been performed may be restricted in the second user terminal.

In addition, although not shown, while the broadcast image 10 on which graphic processing has been performed is displayed on the main display area A of the user terminal 200, the augmented reality function is received from the user terminal 200. An activation request may be received. In this case, the request to activate the augmented reality function may be a request to activate the augmented reality function using the camera image of the user terminal 200.

Based on the request to activate the augmented reality function, the control unit 130 displays the camera image 20 on which augmented reality graphic processing has been performed in the main display area (A) of the user terminal 200, and displays the camera image 20 on which augmented reality graphic processing has been performed. In area B, a broadcast image 10 on which augmented reality graphic processing has been performed can be displayed.

For example, in the main display area (A), a camera image with makeup applied to the user's face may be displayed, and in the sub-display area (B), a camera image with makeup applied to the live broadcast host's face may be displayed.

Meanwhile, the broadcast image of the live broadcast described above may be a real-time streaming image or may be a broadcast image stored in the live broadcast server 200 after the live broadcast has ended. Such broadcast video may also be expressed as recorded broadcast video. In the present invention, even when a recorded broadcast video is provided on the user terminal 200, an icon linked to the augmented reality function can be provided along with the recorded broadcast video. In this case, based on the icon being selected, the control unit 130 may provide a camera image processed with augmented reality graphics using a camera image from a camera provided in the user terminal, similar to the method discussed above. Accordingly, the method of providing augmented reality-related functions to recorded broadcasting images will be replaced with the explanation discussed above.

Meanwhile, as described above, in the present invention, the PIP (Picture In Picture) mode of the display unit can be controlled to either an activated or deactivated state based on whether or not the augmented reality function is activated. In the present invention, “activation of the PIP mode of the display unit” can be used interchangeably with “executing the PIP mode (On)”, “operating in the PIP mode”, etc.

Below, together with the attached drawings, data processing for operating the PIP mode of the display unit in either an activated or deactivated state depending on whether augmented reality is activated will be described. Figure 16 is a flowchart for explaining a method of providing live broadcasting in the PIP (Picture In Picture) method in the present invention, and Figures 17, 18, 19, 20, 21, 22, and 23 are the present invention. These are conceptual diagrams to explain the user environment provided when the augmented reality function according to the invention is activated.

In the method for providing live broadcasting according to the present invention, a process of receiving broadcast video 10 of a live broadcast may be performed from the live broadcasting server 100 (S1610, see FIG. 16). The broadcast video 10 may be received from the live broadcast providing server 300 using a streaming technique.

In the method of providing live broadcasting according to the present invention, based on the reception of the broadcast image 10, a process of providing the broadcast image 10 on the main display area (A) of the display unit of the user terminal 200 may be performed. There is (S1620, see FIG. 16).

As described above, in the present invention, the display unit of the user terminal 200 may include either a main display area (A) or a sub-display area (B).

As shown in (b) of FIG. 4A, the sub-display area (B) may be displayed overlapping (or overlapping) at least a portion of the main display area (A).

In the present invention, when the display unit includes one display area, the one display area can be understood as the main display area (A). In the present invention, this can be explained as “the PIP mode of the display unit of the user terminal 200 is in a deactivated state.”

In addition, in the present invention, when the display unit includes two overlapping display areas, the area corresponding to the lower layer is understood as the main display area (B), and the area corresponding to the upper layer is understood as the sub display area. It can be understood as the display area (B). In the present invention, this can be explained as “the PIP mode of the display unit of the user terminal 200 is in a deactivated state.” That is, the control unit 130 can provide the broadcast image 10 to the main display area (A) while the PIP mode of the display unit is deactivated.

Meanwhile, in the present invention, the main display area (A) can be named “main area”, “first display area”, etc., and the sub display area (B) can be named “sub area”, “second display area”, etc. It can be named as

Next, in the method for providing live broadcasting according to the present invention, while the broadcast image 10 is being provided on the main display area (A), a request is made to activate an augmented reality function related to the live broadcast from the user terminal 200. The process of receiving may proceed (S1630, see FIG. 16).

As described above, the control unit 130 may receive an activation request for the augmented reality function based on the icon 410 linked to the augmented reality function being selected from the user terminal 200 (see Figure 4a). (see (a)).

The control unit 130 may switch (or change) the state of the augmented reality function from a deactivated state to an activated state based on receiving a request to activate the augmented reality function.

The control unit 130 may control the camera of the user terminal 200 to be activated in conjunction with the augmented reality function being activated.

Here, “camera activation” can be understood as enabling the camera to receive the camera image 20 from the camera.

Additionally, the control unit 130 may activate the camera of the user terminal 200 and receive the camera image 20 from the camera.

Here, the camera image 20 can be understood as at least one of a preview screen (or video) of an area recognized by the camera and a captured video (image or video) captured by the camera.

Next, in the method for providing live broadcasting according to the present invention, in response to the augmented reality function being activated according to the activation request, the camera image received from the camera provided in the user terminal 200 is linked to the live broadcast. A process of performing augmented reality graphic processing related to the item may proceed (S1630, see FIG. 16).

For example, the control unit 130 may change the color of the area 420 of the camera image 20 corresponding to the lips of the user's face based on color information matched to an item associated with a live broadcast (Figure (see (b) of 4a). That is, the control unit 130 can provide an effect as if makeup was applied to the user's face image included in the camera image 20 by changing the color or correcting the color of the camera image 20. Furthermore, in the present invention, the method of performing augmented reality graphics processing can synthesize graphic objects for camera images, and in addition, as discussed above, augmentation can be achieved by modifying or synthesizing parts of the camera images in a variety of ways. Real-world graphics processing can be performed.

Next, in the method for providing live broadcasting according to the present invention, the camera image 20 on which the graphic processing has been performed can be provided on the main display area (A).

In response to the augmented reality function being activated, the control unit 130 may output the graphically processed camera image 20 on the main display area A where the broadcast image 10 was output.

Furthermore, the control unit 130 operates the PIP (PIP) of the display unit of the user terminal 200 in order to continuously provide the broadcast image 10 that was being output on the main display area (A) through the display unit of the user terminal 200. Picture In Picture) mode can be switched from disabled to enabled.

The control unit 130 may provide two areas, a main display area (A) and a sub-display area (B), on the display unit of the user terminal 200 based on the PIP mode of the display unit being activated.

As described above, the sub-display area B can be understood as an area corresponding to the upper layer area among two overlapping layers (upper layer and lower layer).

That is, the control unit 130 sets up a sub-display in at least one area of the main display area (A) so that the broadcast image 10 and the augmented reality graphic-processed camera image 20 are provided together on the user terminal 200. The sub-display area B may be provided at a location where the area B overlaps.

And, in response to providing the sub-display area (B) on the display unit of the user terminal 200, the control unit 130 changes the provision area of the broadcast image 10 that was being provided in the display area (A). You can. More specifically, the control unit 130 can change the provision area of the broadcast image 10 from the main display area (A) to the sub-display area (B).

The control unit 130 may continue to provide the broadcast image 10 through the sub-display area B even if the augmented reality function is activated based on a change in the provision area of the broadcast image 10. That is, the broadcast image 10 can be continuously provided through the sub-display area (B) while the camera image 20 on which graphic processing has been performed is provided to the main display area (A).

As such, in the present invention, the state of the augmented reality function can be switched to the activated state based on receiving an activation request for the augmented reality function. Then, based on the state of the augmented reality function being switched to the activated state, the camera can be activated to perform augmented reality graphic processing on the camera image 20. In addition, in the present invention, the PIP mode of the display unit is also activated to simultaneously output the camera image 20 on which augmented reality graphic processing has been performed to the main display area (A) and the broadcast image 20 to the sub-display area (B). You can. Accordingly, through the present invention, a user can experience items associated with a live broadcast provided by the augmented reality function and continue to watch the live broadcast at the same time.

Meanwhile, in the present invention, the PIP mode can be provided by overlapping the sub-display area (B) on at least part of the main display area (A). Accordingly, as shown in (a) of FIG. 17, the main display area (A) may be relatively large in size (or area), and the sub-display area (B) may be relatively small in size (or area).

In the present invention, when the augmented reality function is activated, the graphically processed camera image 20 is displayed in the main display area (A) so that the user can view the graphically processed camera image 20 in a larger size. and a broadcast image 10 can be provided on the sub-display area (B).

However, there may be cases where the user wants to view the broadcast video 10 larger while experiencing the augmented reality function for the product. Additionally, there may be cases where the user wants to view the image 20 on which graphic processing has been performed again in a larger size. Accordingly, in the present invention, for the user's needs, as shown in (a) and (b) of Figure 17, the image provided (displayed or output) through the main display area (A) and the sub-display area (B) (or information) can be mutually converted. In this case, the first data processing for providing the broadcast video described above and the second data processing for providing the augmented reality function may continue to be performed simultaneously.

As shown in (a) of FIG. 17, the control unit 130 provides images (or information) provided in the main display area (A) and the sub-display image (B) on the sub-display area (B). A function icon 1700 matching the function for changing the location can be displayed. In the present invention, the function icon 1700 may be named “switch button”, “screen switch button”, etc.

The control unit 130 selects the image (or information) provided in the main display area (A) based on the function icon 1700 matching the image (or information) provision location change function through the user terminal 200. ) and the sub-display area (B) can change the location (or provision area) of the image (or information) provided.

For example, as shown in (a) of FIG. 17, a camera image 20 on which augmented reality graphic processing has been performed is provided on the main display area (A), and a broadcast image (20) on which augmented reality graphics processing has been performed is provided on the sub-display area (B). 10), while each is being provided, the icon 1700 matching the image (or information) provision location change function of the user terminal 200 may be selected.

In this case, the control unit 130 changes the provision location (or provision area) of the augmented reality graphic-processed camera image from the main display area (A) to the sub-display area (B) and provides the room image 10. The location (or provision area) can be changed from the sub-display area (B) to the main display area (A).

And, as shown in (b) of FIG. 17, the control unit 130 provides (displays or outputs) the broadcast image 10 in the main display area (A) and performs graphic processing in the sub-display area (B). The performed camera image 20 can be provided (displayed or output).

Furthermore, the control unit 130 bases the selection on the function icon 1700 matching the image (or information) provision location change function through the user terminal 200 in the state shown in (b) of FIG. 17. Thus, the camera image 20 on which graphic processing has been performed can be provided in the main display area (A), and the broadcast image 10 can be provided (displayed or output) in the sub-display area (B).

Meanwhile, in the present invention, a graphically processed camera image 20 is provided on the main display area (A) of the user terminal 200, and a broadcast image 10 is provided on the sub-display area (B). , a deactivation request for the augmented reality function may be received from the user terminal 200.

As described above, the control unit 130 may receive a deactivation request for the augmented reality function from the user terminal 200 based on the icon 1300 or 1710 linked to the deactivation request for the augmented reality function being selected. There is (see (a) in Figure 13, (a) in Figure 17).

The control unit 130 may switch (or change) the state of the augmented reality function from the activated state to the deactivated state based on receiving a deactivation request for the augmented reality function.

The control unit 130 may control the camera of the user terminal 200 to be deactivated in conjunction with the augmented reality function being deactivated.

Here, “camera deactivation” can be understood as restricting the reception of the camera image 20 from the camera.

The control unit 130 may end providing the camera image 20 on which augmented reality graphic processing has been performed based on the augmented reality function being deactivated (or the camera being deactivated).

In other words, the camera image 20 on which augmented reality graphic processing was performed that was being provided on the main display area (A) may end being provided in the main display area (A) based on the augmented reality function being deactivated. there is.

Furthermore, in order to no longer provide the camera image 20 through the display unit of the user terminal 200, the control unit 130 activates the PIP (Picture In Picture) mode of the display unit of the user terminal 200. It can be switched to a disabled state.

The control unit 130 may provide only one main display area (A) on the display unit of the user terminal 200 based on the PIP mode of the display unit being deactivated.

As described above, when the display unit includes one display area, the one display area can be understood as the main display area (A).

And, in response to the end of provision of the sub-display area (B) on the display unit of the user terminal 200, the control unit 130 selects the provision area of the broadcast image 10 that was being provided in the sub-display area (B). can be changed. More specifically, the control unit 130 can change the provision area of the broadcast image 10 from the sub-display area (B) to the main display area (A).

The control unit 130 may continue to provide the broadcast image 10 through the main display area (B) even if the augmented reality function is deactivated based on a change in the provision area of the broadcast image 10.

Furthermore, in the present invention, it is possible to receive a deactivation request for the augmented reality function in various ways, and as shown in (a) of FIG. 17, the function icon 1720 provided in the main display area (A) is used by the user. Based on what is selected from the terminal 200, the augmented reality function can be deactivated.

In this case, as discussed above, a broadcast image may be provided in the main display area (A).

As such, in the present invention, the state of the augmented reality function can be switched to the deactivated state based on receiving a deactivation request for the augmented reality function. Then, based on the state of the augmented reality function being switched to the disabled state, the PIP mode of the camera and display unit of the user terminal 200 is deactivated, and the broadcast image 10 is continuously provided through the main display area (A). can do. Accordingly, after experiencing items linked to the live broadcast through the augmented reality function according to the present invention, the user can continue to watch the live broadcast again on a large screen.

Meanwhile, as shown in (a) of FIG. 18, in the sub-display area (B), there is a function icon ( 1800) may be included. In the present invention, the function icon 1700 can be named “end button”, “end broadcast button”, “end broadcast viewing button”, etc.

The control unit 130 may receive a broadcast provision termination request from the user terminal 200 based on the function icon 1800 matching the broadcast video 10 provision termination function being selected through the user terminal 200. there is.

In response to receiving a broadcast provision termination request, the control unit 130 terminates provision of the sub-display area B from the display unit so that the broadcast image 10 is no longer provided on the user terminal 200. You can do it.

More specifically, the control unit 130 activates the PIP (Picture In Picture) mode of the display unit of the user terminal 200 in order to no longer provide the broadcast image 10 through the display unit of the user terminal 200. You can switch from state to disabled state.

At this time, the control unit 130 may maintain the activation state for the augmented reality function.

That is, when the control unit 130 receives a deactivation request for the augmented reality function from the user terminal 200, it switches both the augmented reality function and the PIP mode of the display unit to the deactivated state, but transmits the broadcast image from the user terminal 200. When the termination request of (10) is received, only the PIP mode of the display unit of the user terminal 200 can be switched to the deactivated state independently of the augmented reality function, while maintaining the activated state for the augmented reality function.

Additionally, the control unit 130 may provide only one main display area (A) on the display unit of the user terminal 200 based on the fact that the PIP mode of the display unit is deactivated.

Furthermore, in response to receiving a request to terminate broadcast provision, the control unit 130 may control the communication unit 110 to no longer receive the broadcast video 10 of the live broadcast from the live broadcast server 300.

Meanwhile, when receiving a request to end broadcast provision, the control unit 130 may end provision of the broadcast image 10 and continue to provide the camera image 20 on which graphic processing has been performed.

More specifically, the control unit 130 may continue to provide the augmented reality graphic-processed camera image 20 to the main display area (A) even if provision of the sub-display area (B) is terminated.

That is, even if the first data processing related to providing the broadcast image ends, the control unit 130 may continue to perform the second data processing related to providing the augmented reality function. In this case, the control unit 130 may continue to perform only the second data processing related to providing the augmented reality function.

Meanwhile, although not shown, in a state where only the camera image 20 with augmented reality graphic processing is provided on the display unit of the user terminal 200, the control unit 130 receives the broadcast image from the user terminal 200 ( 10) A resume request can be received. For example, a function icon linked to a request to provide a broadcast video 10 may be included on the display unit of the user terminal 200. The control unit 130 may receive a request to resume the broadcast video 10 from the user terminal 200 based on the user's selection of the function icon.

Based on receiving a request to resume the broadcast image 10, the control unit 130 may again simultaneously perform second data processing related to the augmented reality function and first data processing related to providing the broadcast image.

More specifically, based on receiving a request to resume the broadcast video 10, the control unit 130 causes the communication unit 110 to re-receive the broadcast video 10 of the live broadcast from the live broadcast server 300. (110) can be controlled.

Furthermore, based on receiving a request to resume the broadcast video 10, the control unit 130 switches the PIP mode of the display unit of the user terminal 200 from the inactive state to the active state again, and A sub-display unit (B) may be provided on the display unit. Also, the control unit 130 can provide the broadcast image 10 on the sub-display unit (B).

In this way, even when receiving a request to resume the broadcast image 10, the control unit 130 processes the first data related to providing the broadcast image and relates to the augmented reality function, as if receiving a request to activate the augmented reality function. The second data processing can be performed simultaneously.

However, unlike when receiving a request to activate the augmented reality function, when receiving a request to resume the broadcast image 10, the control unit 130 determines the provision area of the broadcast image 10 and the location where graphics processing is performed. Data processing to change the provision area of the camera image 20 may not be performed.

Meanwhile, as described above, the control unit 130 immediately displays the sub-display area (B) based on the function icon 1800 matching the broadcast video 10 provision end function being selected through the user terminal 200. The provision of can be terminated from the display unit, but as shown in (b) of FIG. 18, an opportunity to select another service (ex: ending broadcast, returning to PIP mode, ending augmented reality experience) can be provided. there is.

In this case, the control unit 130 selects the function icon 1700 included in the sub-display area (B) in (a) of FIG. 18, as shown in (b) of FIG. 18. A plurality of function icons (1810, 1820, 1830) corresponding to each possible option may be provided. Hereinafter, in order to avoid confusion in terminology, the function icon 1800 included in the sub-display area (B) in Figure 18 (a) is called the main function icon 1800, and the main icon 1800 is selected. The icons provided (output or displayed) in response to this are named sub-function icons (1810, 1820, 1830).

The sub-function icon is a first sub-function icon (ex: “Broadcast End”, 1810) that matches (or is linked to) the broadcast image 10 end function, and displays the broadcast image 10 through the sub display area (B). A second sub-function icon (ex: “PIP return”, 1820) matched (or linked) to the function being provided again, a third sub-function icon (ex: “PIP return”, 1820) matched (or linked) to the deactivating function for the augmented reality function : “Going out for an experience”, 1830).

The control unit 130 may perform a function linked to the selected function icon in response to selecting one of the plurality of sub-function icons 1810, 1820, and 1830. For example, when the second sub-function icon (ex: “PIP return”, 1820) is selected, the control unit 130 continues to display augmented reality while maintaining the augmented reality activation state and the PIP mode of the display unit being activated. The camera image 20 and the broadcast image 10 on which graphic processing has been performed can be provided through the display unit.

In this way, in the present invention, images (or information) provided (output or displayed) in the main display area (A) and the sub-display area (B) can be switched between each other. Accordingly, in the present invention, the services provided while the graphically processed camera image 20 is output in the main display area (A) and the broadcast image 10 is output in the sub-display area (B) are all: The broadcast image 10 may be output in the main display area (A), and the camera image 20 on which graphic processing has been performed may be output in the sub-display area (B).

Meanwhile, there may be cases where the user wishes to change the location of the sub-display area (B). For example, the user may want to view the camera image 20 that is covered by the sub-display area B. Accordingly, in the present invention, as shown in FIG. 19, the output position of the sub-display area (B) can be changed to a position desired by the user.

The output location (ex: specific output coordinates on the display unit) of the sub-display area (B) may be preset by the system administrator. Before receiving a user input for changing the position of the sub-display area B, the control unit 130 may provide (output or display) the sub-display area B at a preset position (default position).

Furthermore, when the control unit 130 receives a user input for changing the position of the sub-display area B through the user terminal 200, it can change the position of the sub-display area B.

The user input for changing the position of the sub-display area (B) may be a user input (ex: drag) using a preset method for the sub-display area (B).

As shown in (a) of FIG. 19, the control unit 130 controls the position of the sub-display area (B) on the main display area (A) in conjunction with a preset user input for the sub-display area (B). can be changed. Furthermore, as shown in FIG. 19(b), the control unit 130 may provide (display or output) the sub-display area B at the changed location based on a preset user input.

In this way, the output position of the sub-display area B may be changed from the first position 1910a to the second position 1910b based on the user input.

When the position of the sub-display area B is changed, the control unit 130 may store the changed position (ex: output coordinates on the display unit corresponding to the changed position) in the storage unit 120. Also, when the display unit of the user terminal 200 is later activated in PIP mode, the control unit 130 may provide (display or output) the sub-display area B at the changed location.

In contrast, even if the position of the sub-display area (B) is changed, the control unit 130 sets the sub-display to a preset position (default position) when the display unit of the user terminal 200 is later activated in PIP mode. Area (B) can be provided (displayed or output).

After changing the position of the sub-display area B, the location where the sub-display area B is provided (output or displayed) can be set by either the user or the system administrator.

Meanwhile, the control unit 130 receives a user input for at least one of the main display area (A) and the sub-display area (B), and even while the corresponding control is being performed, the control unit 130 provides a first display related to providing a broadcast image. While data processing is continuously performed, secondary data processing related to provision of augmented reality service may be performed.

Meanwhile, this may occur when the user wishes to change the output size (or width) of the sub-display area (B). For example, the user may want to view the sub-display area (B) a little larger than it is now, or may want it to be a little smaller than it is now. Accordingly, in the present invention, as shown in FIG. 20, the user can enlarge or reduce the output size (or area) of the sub-display area (B). It goes without saying that the “(output) size of the display area” described in the present invention can also be referred to as “display (output) width” and “display (output) shape.”

The size of the sub-display area (B) may be preset by the system administrator. Before receiving a user input to change the size of the sub-display area B, the control unit 130 may provide (output or display) the sub-display area B according to a preset size (default size). .

Furthermore, when the control unit 130 receives a user input for changing the size of the sub-display area (B) through the user terminal 200, it can change the size of the sub-display area (B).

A user input for changing the size of the sub-display area (B) may be a user input in a preset manner for the sub-display area (B).

The control unit 130 adjusts the size of the sub-display area (B) on the main display area (A) to a size corresponding to the user input in a pre-set manner in conjunction with a pre-set user input to the sub-display area (B). You can change it.

More specifically, as shown in (a) of FIG. 20, the control unit 130 is linked to a preset user input for the sub-display area (B), as shown in (b) of FIG. 20. , the size of the sub-display area (B) on the main display area (A) can be enlarged. At this time, the preset user input may be a gesture corresponding to enlarging the size of the sub-display area (B).

In addition, although not shown, in conjunction with a preset user input (or gesture) corresponding to reducing the size of the sub-display area (B), the control unit 130 , the size of the sub-display area (B) on the main display area (A) can be reduced.

Furthermore, when the size of the sub-display area B is changed, the control unit 130 may store the changed position (ex: output coordinates on the display unit corresponding to the changed size) in the storage unit 120. Also, when the display unit of the user terminal 200 is later activated in PIP mode, the control unit 130 may provide (display or output) the sub-display area B according to the changed size.

In contrast, even if the size of the sub-display area (B) is changed, the control unit 130 controls the sub-display area (B) according to the preset size (default position) when the display unit of the user terminal 200 is later activated in PIP mode. The display area (B) can be provided (displayed or output).

After changing the size of the sub-display area B, whether to change the size of the sub-display area B provided (output or displayed) can be set by either the user or the system administrator.

Meanwhile, the control unit 130 receives a user input for at least one of the main display area (A) and the sub-display area (B), and even while the corresponding control is being performed, the control unit 130 provides a first display related to providing a broadcast image. While data processing is continuously performed, secondary data processing related to provision of augmented reality service may be performed.

Meanwhile, although not shown, the sub-display area B may include a function icon matching a size change function for changing the size of the sub-display area B.

In a state where the sub-display area (B) is overlapped on the main display area (A), the control unit 130 controls the user to select a function icon matching the size change function for changing the size of the sub-display area (B). You can receive selection.

The control unit 130 may reduce the size of the sub-display area B to a preset size based on receiving a user selection for a size change function that changes the size of the sub-display area B. This preset size can be set in various ways by the system administrator.

The control unit 130 outputs visual information corresponding to the broadcast image 10 on the reduced sub-display area (B') in conjunction with reducing the size of the sub-display area (B) to a preset size. can be limited.

As an example, the control unit 130 may provide (output or display) a thumbnail of a live broadcast instead of the broadcast image 10 on the sub-display area (B') reduced to a preset size. there is.

As another example, the control unit 130 provides information indicating that the visual information of the broadcast image 10 is limited, instead of the broadcast image 10, on the sub-display area (B') reduced to a preset size ( output or display). The information may include, for example, phrases such as “LIVE,” “On Air,” or “On Air,” or graphic objects corresponding to the phrases.

Furthermore, the control unit 130 controls the output of visual information corresponding to the broadcast image 10 to continue even if the output of visual information corresponding to the broadcast image 10 is limited on the reduced sub-display area B'. can do. That is, even if the broadcast screen is not output through the display unit of the user terminal 200, the control unit 130 can control the output unit of the user terminal 200 to continue outputting broadcast sound. The user can be provided with the augmented reality function while temporarily minimizing the sub-display area (B) and only listening to the sound of the live broadcast.

Meanwhile, in the present invention, as shown in (b) of FIG. 21, the entire area of the display unit of the user terminal 200 is divided into two, and a first divided display area C and a second division display area C that do not overlap each other are formed on the display unit. Two split display areas (D) can be provided.

The display unit of the user terminal 200 may be divided into two, as shown in (b) of FIG. 21, so that the first divided display area (C) and the second divided display area (D) are arranged vertically. , This is just an example and can be done in various ways based on the settings of the user or system administrator. For example, the display unit of the user terminal 200 may be divided so that the first divided display area C and the second divided display area D are arranged left and right, and may be divided based on the diagonal line of the display unit.

As shown in (a) of FIG. 21, the control unit 130 operates in a state where the main display area (A) and the sub display area (B) are provided on the display unit through the user terminal 200, A request for dividing the display area may be received from the user terminal 200.

A request for division of the display unit may be received based on various user inputs.

For example, as shown in (a) of FIG. 21, when the size of the sub-display area (B) becomes the preset maximum size based on a preset user input to the sub-display area (B) , the control unit 130 can process data indicating a user request for division of the display area.

As another example, although not shown, based on the function icon matching (or linked to) the display area division function being selected, the control unit 130 may process data indicating that there is a user request for display area division. there is.

Based on receiving a request to divide the display area from the user terminal 200, the control unit 130 divides the entire display area of the user terminal 200 so that they do not overlap each other, as shown in (b) of FIG. 21. It can be divided into a first divided display area (C) and a second divided display area (D).

Furthermore, the control unit 130 can change the output area of the camera image 20 and the broadcast image 10 on which graphic processing was performed, which were being output to each of the main display area (A) and the sub-display area (B). Additionally, the control unit 130 may continue to provide the camera image 20 and the broadcast image 10 on which graphic processing has been performed on the changed output area.

At this time, the control unit 130 outputs the camera image 20 and the broadcast image 10 based on which division display area is matched to each of the main display area (A) and the sub display area (B). Each area can be changed to one of the first divided display area (C) and the second display area (D).

For example, the control unit 130 controls the camera image on which graphics processing has been performed provided in the main display area (A) based on the fact that the main display area (A) is matched to the first divided display area (C). The output area of (20) can be changed to the first divided display area (C). Additionally, the control unit 130 may continue to provide the camera image 20 on which graphic processing has been performed on the first divided display area C, as shown in (b) of FIG. 21 .

Meanwhile, although not shown, as shown in (b) of FIG. 21, graphics processing is performed in each of the first divided display area (C) and the second divided display area (D) of the user terminal 200. In a state where the camera image 20 and the broadcast image 10 are being output, based on receiving a display unit merging request from the user terminal 200, as shown in (a) of FIG. 21, the user terminal ( The camera image 20 and the broadcast image 10 on which graphic processing has been performed can be controlled to be provided on the main display area (A) and sub-display area (B) of 200).

Meanwhile, the user may have a need to receive an experience for a specific item using not only the camera image 10 captured in the user terminal 200 but also the broadcast image 10 of a live broadcast.

For example, assume that the special feature item is a makeup item for lips. The user may want to test the makeup on the lips on their own face, and they may want to test the makeup on the lips on the face of the live host (or seller). Accordingly, as previously described with reference to FIG. 15 , in the present invention, the broadcast video 10 can be provided by performing graphic processing related to a specific item.

Furthermore, in the present invention, as shown in FIG. 22, in a state where the display unit of the user terminal 200 is divided into a plurality of areas (ex: divided into two states), a camera on which augmented reality graphic processing related to a specific item is performed Video 20 and broadcast video 20 can be provided simultaneously.

Based on receiving a display unit division request from the user terminal 200, the control unit 130 divides the first divided display area C of the user terminal 200, as shown in (a) of FIG. 22. Provides (outputs or displays) a camera image 20 on which augmented reality graphics processing has been performed, and provides (outputs or displays) a broadcast image 10 on which augmented reality graphics processing has been performed in the second split display area (D). )can do.

Meanwhile, although not shown, the control unit 130 provides (outputs or displays) a broadcast image 10 on which augmented reality graphic processing has been performed to the first divided display area C of the user terminal 200, and It is natural that the camera image 20 on which augmented reality graphic processing has been performed can be provided (output or displayed) in the two-split display area D.

The control unit 130 sets an option to an item associated with a specific live broadcast (or an item associated with an augmented reality function) on one region of each of the first divided display area C and the second divided display area D. Graphic objects (for example, reference numerals 2211, 2212, 2221, and 2222) corresponding to can be output.

In this case, each of the camera image 20 and the broadcast image 10 is a camera on which augmented reality graphics processing has been performed to correspond to one of the color information (2211, 2221) corresponding to a plurality of options corresponding to the item. It may be video 20 and broadcast video 10.

For example, the camera image 20 is a camera image whose color has been converted to correspond to color information corresponding to the graphic object 2211 included in a specific partial area 2210 of the first divided display area C ( 20) may apply. For example, in the camera image 20, the area 2200a corresponding to the lip portion of the user's face is displayed on the graphic object 2211 included in the specific partial area 2210 of the first divided display area C. Color conversion may be performed using corresponding color information.

In addition, for example, the broadcast image 10 is a broadcast image in which the color has been converted to correspond to the color information corresponding to the graphic object 2221 included in the specific partial area 2220 of the second divided display area D. This may correspond to video 10. For example, in the broadcast image 10, the area 2200b corresponding to the lip part of the user's face corresponds to the graphic object 2221 included in the specific partial area 2220 of the second divided display area D. Color conversion may be performed using color information.

Furthermore, the control unit 130 displays the camera image 20 on the first divided display area (C) of the user terminal 200, and the broadcast image 10 is displayed on the second divided display area (D). In this state, based on receiving a preset user input applied to the first divided display area C, augmented reality graphic processing for the camera image 20 may be changed on the first divided display area C.

In this case, the controller 130 may not change the augmented reality graphic processing for the broadcast image 10 displayed on the second divided display area D.

That is, the control unit 130 can perform color change or color correction only on the camera image 20, providing an effect equivalent to modifying makeup only on the user's face image included in the camera image 20. .

On the other hand, the control unit 130 displays the camera image 20 on the first divided display area (C) of the user terminal 200, and displays the broadcast image 10 on the second divided display area (D). In this state, based on receiving a preset user input applied to the second divided display area (D), the augmented reality graphic processing for the broadcast image 10 can be changed on the second divided display area (C). .

In this case, the controller 130 may not change the augmented reality graphic processing of the camera image 10 displayed on the first divided display area C.

In other words, the control unit 130 performs color change or color correction only on the broadcast video 10, resulting in the same effect as modifying makeup only for the face image of the broadcast host (or seller) included in the broadcast video 10. can provide.

More specifically, the control unit 130 is displaying a broadcast image 10 on which graphic processing has been performed corresponding to color information corresponding to the first option of a specific item on the second divided display area D. In, based on the graphic object 2222 corresponding to the color information of the second option different from the first option being selected by a preset user input applied to the second divided display area D, the second Graphics processing according to the color information of the second option can be performed on the broadcast image 10 displayed on the split display area D.

For example, as shown in (a) of FIG. 22, a graphic object 2221 corresponding to color information of the first option is displayed in a specific partial area 2200 of the second divided display area D. In this state, when the graphic object 2222 corresponding to the color information of the second option is selected from the user terminal 200, the control unit 130 processes the graphic object 2222 corresponding to the second color information as selected. can do.

The control unit 130 displays the color of the second option in the specific partial area 2200, as shown in (b) of FIG. 22, in conjunction with the selection of the graphic object 2222 corresponding to the second color information. A graphic object 2222 corresponding to information may be displayed.

Additionally, the control unit 130 may perform augmented reality graphic processing using the color information of the second option only for the broadcast image 10 displayed in the second divided display area D.

As another example, in a state in which a graphic object 2221 corresponding to color information of the first option is displayed in a specific partial area 2200 of the second divided display area D, the second divided display area D In conjunction with a preset user input (ex: swipe input), a graphic object 2222 corresponding to the color information of the second option is displayed in the specific partial area 2200. The graphic object 612 corresponding to the color information of option 2 may be selected.

Additionally, the control unit 130 may perform augmented reality graphic processing using the color information of the second option only for the broadcast image 10 displayed in the second divided display area D.

The control unit 130 displays augmented reality graphics only for the camera image 20 displayed on the first divided display area C based on receiving a preset user input applied to the first divided display area C. The processing can be changed and may be the same as the method described above.

Meanwhile, in the present invention, items linked to live broadcasting can be changed. For example, in a live broadcast (or a specific live broadcast), a first item (ex: lip makeup item, hereinafter lipstick) is sold (or introduced), and then a second item (cheek makeup) that is different from the first item is sold (or introduced). When selling (or introducing) an item (hereinafter “blusher”), the item linked to the live broadcast may change.

In the present invention, information (or visual appearance) provided on the display unit of the user terminal 200 can be changed based on a change in an item related to a live broadcast.

As shown in (a) of FIG. 23, on one area 2300 of the main display area A of the user terminal 200, item information (ex: “Lira powder matte lip”) corresponding to the first item is displayed. , 2300a), the item associated with the live broadcast may be changed from the first item (lipstick) to the second item (blusher). The control unit 130 controls the user terminal 200, as shown in (b) of FIG. 23, based on the item linked to the live broadcast changing from the first item (lipstick) to the second item (blusher). On one area 2300 of the main display area A of You can control the display of “Blooming Cream Blush”, 2300b).

Furthermore, as shown in (a) of FIG. 23, the camera image 20a on which graphic processing related to the first item has been performed is displayed on the sub-display area B of the user terminal 200. , the item linked to the live broadcast may change from the first item (lipstick) to the second item (blusher). The control unit 130 controls the user terminal 200, as shown in (b) of FIG. 23, based on the item linked to the live broadcast changing from the first item (lipstick) to the second item (blusher). On the sub-display area (B) of the camera image (“image with lipstick makeup effect”, 20a) related to the first item (lipstick) disappears) and graphics processing related to the second item (ex: blur) is performed. You can control the display of the camera image (“Video with blusher (cheek touch) makeup effect” 20b).

Users can make real-time judgments about products sold (or introduced) in live broadcasts by being provided with information corresponding to the products sold (or introduced) in live broadcasts and experiencing augmented reality.

As seen above, the method and system for providing live broadcasting according to the present invention can utilize the augmented reality function to provide a user experience that allows users to experience products introduced in the live broadcast while watching the live broadcast. there is.

According to the method and system for providing live broadcasting according to the present invention, consumers can experience products even without directly visiting the store, saving time on visiting the store and purchasing products. You can do it more carefully.

Furthermore, according to the method and system for providing live broadcasting according to the present invention, sellers can induce consumers to more actively purchase products by providing consumers with opportunities to experience products in a variety of ways. Furthermore, even if sellers do not directly operate a store, they can meet various consumers and experience products through live broadcasts, thereby saving on the operating costs of running a store and having various opportunities to sell products.

Meanwhile, the method and system for providing live broadcasting according to the present invention can provide the provision of live broadcasting, product experience, and product purchase process in a unified manner while live broadcasting is in progress. Through this, consumers can experience and purchase products more conveniently just by watching the live broadcast according to the present invention. Meanwhile, the present invention discussed above can be implemented as a program that is executed by one or more processes on a computer and can be stored in a medium that can be read by such a computer.

Furthermore, the present invention discussed above can be implemented as computer-readable codes or instructions on a program-recorded medium. That is, various control methods according to the present invention may be provided in the form of programs, either integrated or individually.

Meanwhile, computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is.

Furthermore, the computer-readable medium may be a server or cloud storage that includes storage and can be accessed by electronic devices through communication. In this case, the computer can download the program according to the present invention from a server or cloud storage through wired or wireless communication.

Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a CPU (Central Processing Unit), and there is no particular limitation on its type.

Meanwhile, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (18)

Receiving broadcast video of a live broadcast from a live broadcast server;
Based on the reception, performing first data processing to provide the broadcast image on the main display area of the display unit of the user terminal;
Receiving an activation request for an augmented reality function related to the live broadcast from the user terminal while the broadcast image is being provided on the main display area according to the first data processing;
While the first data processing is being performed, when an activation request for the augmented reality function is received, a camera provided in the user terminal is activated in response to the activation request, and a camera image is received from the activated camera. For this, performing second data processing so that augmented reality graphic processing related to the item linked to the live broadcast occurs; and
Based on the first data processing and the second data processing, displaying the camera image on which the augmented reality graphic processing has been performed on the main display area while the user terminal is receiving the broadcast image. Including,
The broadcast video is,
A method of providing a live broadcast, characterized in that while the camera image on which the second data processing is performed is provided on the main display area, it is provided to the user terminal through a sub-display area located on the display unit. According to paragraph 1,
In response to the activation request, providing the sub-display area on the display unit; and
In response to provision of the sub-display area, the method further comprising changing the provision area of the broadcast image provided in the main display area to the sub-display area. According to paragraph 2,
So that the broadcast image and the augmented reality graphic-processed camera image are provided together on the user terminal,
A method of providing live broadcasting, wherein the sub-display area is positioned to overlap at least one area of the main display area. According to paragraph 2,
Receiving a deactivation request for the augmented reality function from the user terminal; and
Further comprising terminating provision of the sub-display area based on the augmented reality function being deactivated according to the deactivation request,
A method of providing a live broadcast, characterized in that when provision of the sub-display area is terminated, the broadcast image provided in the sub-display area continues to be provided in the main display area. According to clause 4,
The augmented reality graphic-processed camera image that was provided on the main display area based on activation of the augmented reality function,
A method of providing a live broadcast, characterized in that provision in the main display area is terminated based on the augmented reality function being deactivated. According to clause 5,
A method for providing a live broadcast, further comprising deactivating the camera so that the camera image is no longer received in conjunction with the augmented reality function being deactivated according to the deactivation request. According to clause 6,
The main display area includes a function icon linked to the deactivation request,
A method of providing a live broadcast, characterized in that deactivation of the augmented reality function is performed based on the function icon being selected. According to clause 4,
In the sub-display area,
A function icon matching the function for changing the location of information provided in each of the main display area and the sub-display area is included, (switching)
When the function icon is selected through the user terminal while the augmented reality graphic-processed camera image is provided on the main display area and the broadcast image is provided on the sub-display area,
A method of providing a live broadcast, characterized in that the provision positions of the augmented reality graphic-processed camera image and the broadcast image are changed. According to clause 4,
In the sub-display area,
A function icon matched to a termination function that terminates provision of the broadcast video provided in the sub-display area is included,
When the function icon is selected through the user terminal,
Terminating provision of the sub-display area from the display unit so that the broadcast image is no longer provided on the user terminal,
A method of providing a live broadcast, wherein even if the function icon is selected and provision of the sub-display area ends, the augmented reality graphic-processed camera image is continuously provided in the main display area. According to clause 4,
In the sub-display area,
A function icon matching the size change function for changing the size of the sub-display area is included,
When the function icon is selected through the user terminal,
A method for providing a live broadcast, characterized in that the size of the sub-display area is reduced to a preset size, and output of visual information corresponding to the broadcast image is limited in the reduced sub-display area. According to paragraph 3,
The output location of the sub-display area is,
A method of providing live broadcasting, wherein the main display area is changed based on a preset user input to the sub-display area applied through the user terminal. According to clause 3,
The output size of the sub-display area is,
A method of providing a live broadcast, characterized in that the sub-display area is changed based on a preset user input applied through the user terminal. According to paragraph 1,
While the broadcast image is being provided on the main display area, an icon linked to the augmented reality function is displayed in one area of the main display area,
A method for providing a live broadcast, wherein the activation request for the augmented reality function is received based on the icon being selected from the user terminal. According to clause 13,
A method of providing a live broadcast, characterized in that the icon linked to the augmented reality function is displayed on the main display area based on whether the item is matched to the augmented reality function. According to paragraph 1,
In the item, at least one color information is matched,
In the step of performing the augmented reality graphics processing,
A method for providing live broadcasting, characterized in that the color of at least a portion of the camera image is converted so that the color corresponding to the color information is displayed in the camera image. According to clause 15,
In the step of performing the augmented reality graphics processing,
Based on the type of the item, specify a specific part to change color in the camera image,
A method for providing live broadcasting, characterized in that the color of the specific part of the camera image is converted to correspond to the color information. In a live broadcasting provision system linked to a user terminal including a camera and a display unit,
a communication unit that receives broadcast images of a live broadcast from a live broadcast server; and
and a control unit that performs first data processing related to providing the broadcast image on the main display area of the display unit of the user terminal, based on the reception.
The control unit,
While the broadcast image is being provided on the main display area according to the first data processing, a request for activation of an augmented reality function related to the live broadcast is received from the user terminal,
While the first data processing is being performed, when an activation request for the augmented reality function is received, a camera provided in the user terminal is activated in response to the activation request, and a camera image is received from the activated camera. For this, second data processing is performed so that augmented reality graphic processing related to the item linked to the live broadcast is performed,
Based on the first data processing and the second data processing, while the user terminal is receiving the broadcast image, the camera image on which the augmented reality graphic processing has been performed is displayed on the main display area,
The broadcast video is,
A live broadcasting providing system, characterized in that while the camera image on which the second data processing is performed is provided on the main display area, it is provided to the user terminal through a sub-display area located on the display unit. A program that is executed by one or more processes in an electronic device and stored on a computer-readable recording medium,
The above program is,
Based on the reception, performing first data processing to provide the broadcast image on the main display area of the display unit of the user terminal;
Receiving an activation request for an augmented reality function related to the live broadcast from the user terminal while the broadcast image is being provided on the main display area according to the first data processing;
While the first data processing is being performed, when an activation request for the augmented reality function is received, a camera provided in the user terminal is activated in response to the activation request, and a camera image is received from the activated camera. For this, performing second data processing so that augmented reality graphic processing related to the item linked to the live broadcast occurs; and
Based on the first data processing and the second data processing, displaying the camera image on which the augmented reality graphic processing has been performed on the main display area while the user terminal is receiving the broadcast image. Contains instructions that perform,
The broadcast video is,
A record that can be read by a computer, characterized in that it is provided to the user terminal through a sub-display area located on the display unit while the camera image on which the second data processing has been performed is provided on the main display area. Program stored on media.

Images