KR101716617B1 - Method and system for providing augmented reality contents concerning homeshopping product of digital tv - Google Patents

Abstract

The present invention relates to a system providing augmented reality (AR) content about a home shopping product of digital data broadcasting on air through a digital TV to a smart terminal. The system comprises: a service server transmitting home shopping content of digital data broadcasting to a set-top box; a digital TV displaying the home shopping content received from the set-top box on a screen; an AR server transmitting AR content corresponding to the home shopping content on air to the set-top box; and a smart terminal which drives an AR application and receives the AR content from the set-top box.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and system for implementing augmented reality contents related to a home shopping product of digital data broadcasting on a digital TV,

The present invention relates to a method of implementing an augmented reality in digital data broadcasting.

Technology in the field of TV shopping is changing slowly. TV shopping is still centered on real-time channels, and viewers only look at the content provided in real-time channels. Therefore, conventional TV shopping is a passive shopping that can not be seen even if there is something more to see and can not ask even if there is something to ask.

On the other hand, the mobile market has exploded in recent years. The collective experience of new user experience through smart phone has changed not only the purpose of use of mobile terminal but also the structure of related industry. In particular, the emergence of smartphones has paved the way for mobile shopping.

Accordingly, there is a risk that the conventional TV shopping market may be encroached on mobile shopping characterized by convenience and promptness. Therefore, in order to solve the problems of the related art, it is necessary to introduce an innovative new user experience to the TV shopping market.

The inventor of the present invention has studied for a long time to solve such a problem, developed through trial and error, and finally completed the present invention.

It is an object of the present invention to implement a new concept of N-screen shopping. The data broadcasting home shopping is provided as it is through the screen of the TV, and a variety of information about the product being broadcast or the related product is provided through the augmented reality on the screen of the smartphone. You do not have to turn the TV home shopping channel or use your smartphone for a while to get the information you need.

It is an object of the present invention to propose a new method of providing various information related to a product of a data broadcasting home shopping using an augmented reality. The use of the Augmented Reality has the effect of enabling individual viewers to freely acquire and experience desired product information in addition to the contents being broadcast uniformly.

It is another object of the present invention to provide a method for receiving various information about a product that is being broadcasted easily and quickly, without searching or inputting the name of the product on the air, through the augmented reality.

On the other hand, other unspecified purposes of the present invention will be further considered within the scope of the following detailed description and easily deduced from the effects thereof.

According to an aspect of the present invention, there is provided a method for providing digital content broadcasting services, the method comprising: (a) transmitting a home shopping content of digital data broadcasting to a set top box by a service server;

(b) receiving a home shopping content from the set-top box and displaying the digital home TV on a screen;

(c) the smart terminal of the viewer drives an Augmented Reality (AR) application and takes a picture of the digital TV using the camera of the smart terminal;

(d) providing the augmented reality server with the augmented reality contents for the home shopping contents being broadcast on the digital TV to the smart terminal; And

(e) displaying the augmented reality contents on the display module of the smart terminal by matching the augmented reality contents with the images photographed by the camera. The home shopping product of digital data broadcasting The present invention provides a method for implementing an augmented reality content on a smart terminal.

In a preferred embodiment, the step (a) comprises: the service server transmitting a watermark or QR code including identification information of the home shopping contents to a VOD area of the digital TV, a TV application area having independent functions and independent areas, The method comprising the steps of:

(C) capturing the watermark or QR code by the smart terminal; And transmitting the photographed watermark or QR code to the augmented reality server,

The step (d) may include searching for a home shopping content matching the watermark or the QR code, And providing the presenting contents to the smart terminal corresponding to the retrieved home shopping contents.

In a preferred embodiment, the step (c) includes: extracting a product image from a screen of the digital TV taken by the smart terminal;

Extracting identification information of a home shopping content being broadcast from the extracted product image; And

And transmitting identification information of the extracted home shopping contents to the augmented reality server.

In a preferred embodiment, the step (d) comprises: receiving, by the augmented reality server, a screen of a digital TV photographed by the smart terminal and extracting a product image from a screen of the digital TV;

Extracting identification information of a home shopping content being broadcasted from the product image extracted by the augmented reality server; And

And the augmented reality server transmitting the augmented reality contents corresponding to the identification information of the home shopping contents to the smart terminal.

In a preferred embodiment, the step (d) includes: receiving the identification information of the smart terminal from the augmented reality server;

Transmitting the identification information of the smart terminal to the service server by the augmented reality server;

Extracting identification information of a home shopping content being broadcast on the set top box paired with the identification number of the smart terminal, and transmitting identification information of the extracted home shopping content to the augmented reality server; And

And the augmented reality server transmitting the augmented reality contents corresponding to the identification information of the home shopping contents to the smart terminal.

In a preferred embodiment, the step (d) includes the steps of: providing the augmented reality contents to the set-top box for the home shopping contents being broadcast on the digital TV; And

And transmitting the augmented reality contents to the smart terminal through the set-top box.

According to a second aspect of the present invention, there is provided a digital data broadcasting service server for transmitting home shopping contents of a digital data broadcasting to a set-top box;

A digital TV receiving the home shopping contents from the set-top box and displaying the home shopping contents on the screen;

An augmented reality server for transmitting an augmented reality content corresponding to the home shopping content being broadcasted to the settop box; And

A smart terminal for driving an Augmented Reality (AR) application and receiving an augmented reality content from the set-top box,

Wherein the smart terminal matches the augmented reality contents with an image photographed by a camera and displays the augmented reality contents on a display module. The system for implementing augmented reality contents related to a home shopping product of digital data broadcasting in a digital TV, to provide.

The present invention has the effect of implementing a new concept of N-screen shopping. It is possible to provide a data broadcasting home shopping through a screen of a TV as well as provide a variety of information about a product being broadcast or an associated product through augmented reality on a screen of a smart phone, Can be used to obtain the necessary information.

In addition, according to the present invention, there is an effect that various information regarding a product that is being broadcasted easily and quickly can be received through an augmented reality even if the name of the product being broadcast is not searched or input.

On the other hand, even if the effects are not explicitly mentioned here, the effect described in the following specification, which is expected by the technical features of the present invention, and its potential effects are treated as described in the specification of the present invention.

1 is a diagram showing an embodiment of the system of the present invention.
2 is a diagram showing one embodiment of the method of the present invention.
3 is a diagram showing an embodiment of the operation of the smart terminal of the present invention.
4 and 5 are views showing an embodiment of a method for connecting a smart terminal of the present invention to an augmented reality server.
6 is a diagram illustrating an embodiment in which the smart terminal of the present invention displays various information about a product being broadcasted through an augmented reality.
7 is a diagram illustrating an embodiment of a smart terminal of the present invention displaying information on an associated product of a product being broadcasted through an augmented reality.
* The accompanying drawings illustrate examples of the present invention in order to facilitate understanding of the technical idea of the present invention, and thus the scope of the present invention is not limited thereto.

In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may obscure the subject matter of the present invention.

In the present invention, Augmented Reality (AR) means a technique of overlapping a virtual object in a real world viewed by a user. More specifically, the augmented reality application of the present invention overlaps and displays the augmented reality contents provided by the augmented reality server on the real world screen of the digital TV viewed by the viewer.

In the present invention, the service server transmits a digital data broadcasting home shopping, the augmented reality server stores and transmits the augmented reality contents, and the pairing server stores the pairing information of the smart terminal and the digital TV (or set-top box) Server. In various embodiments, the servers described above may be combined at least two. For example, the service server and the pairing server may be combined, and the function and operation of the pairing server may be integrated with the augmented reality server.

In addition, the smart terminal may include a mobile terminal such as a tablet PC, a wearable device, etc. in addition to a smart phone. In the present invention, the digital TV is a concept including an IP TV or a SMART TV.

In addition, in a preferred embodiment of the present invention, an application (or an augmented reality application) installed in a smart phone may be a shopping application provided by a home shopping company operating a service server.

1 is a diagram showing an embodiment of the system of the present invention.

1, the system 100 of the present invention may include a digital TV 200, a set top box 210, a service server 220, a smart terminal 300, and an augmented reality server 400 .

The service server 220 transmits the data broadcast home shopping broadcast to the set-top box 210. The transmitted home shopping broadcast is transmitted to the viewer through the digital TV 200. The service server 220 may have various databases in managing data broadcast contents and resources. The database may include a VOD content database, a merchandise information database, a user information database, a recommended content database, and a screen content database. Further, a database for recording logs, a database for recording billing and settlement information, and the like may be further included. These databases may be located in a system managed by the service server 220, or may be located in a repository managed by an external server. In particular, the contents related to the augmented reality may be located in the repository managed by the augmented reality server 400. [

In addition, the service server 220 further includes means for managing and processing data, transmitting and receiving data, means for processing communication with an external device or an external system, and means for authentication and security, This may include various hardware devices, server devices, and software. In the case of a series of procedures, such as a process for authenticating subscriber information matching a set-top box ID, a process for determining recommended content, and a process for making a payment transaction in connection with an external financial institution, It can be executed through communication with an external server and devices, if it resides or needs to be transmitted to the outside.

A set-top box 210 is connected to the digital TV 200, and data transmission / reception with the service server 220 is performed via the set-top box 210. When the digital TV 200 is connected to the service server 220 through the set top box 210, the service server 220 receives a unique identifier (ID: SAID) from the set top box 210, And obtains the user information corresponding to the ID of the set-top box 210 by inquiring the external or internal base particle information management system. The set-top box 210 is usually connected to the digital TV 200 through a cable. The set-top box 210 and the service server 220 are connected through a network medium for executing TV broadcasting and data broadcasting. It includes wired and / or wireless networks. Preferably, a cable for data transmission / reception connected to the service server 220 may be connected to the set-top box 210 by entering the house. The remote controller executes a control command for operating the digital TV 200. [

The smart terminal 300 reproduces the augmented reality contents as a terminal held by the viewer. The smart terminal 300 includes a communication module and a display module.

The augmented reality server 400 provides the augmented reality contents to the smart terminal 300. [ There are two embodiments in which the augmented reality server 400 provides the augmented reality contents to the smart terminal 300 using a certain route.

In one embodiment, the augmented reality server 400 directly provides the augmented reality content to the smart terminal 300. For this, the augmented reality server 400 may be connected to the smart terminal 300 through a wireless network. The connection with the smart terminal 300 may be connected using a mobile communication network used by the smart terminal 300. [ In this embodiment, the data usage fee of the smart terminal 300 may increase when the augmented reality contents are large.

In another embodiment, the augmented reality server 400 provides the augmented reality content via the set-top box 210. Although not shown, the smart terminal 300 is connected to the augmented reality server 400 through a wireless network (for example, Wi-Fi or Bluetooth) provided by the set-top box 210. The data broadcasting provided by the service server 200 is connected to the set-top box 210 through a wired network capable of bidirectional communication. Accordingly, after transmitting the augmented reality contents to the set top box 210 through the wired network, the set top box 210 transmits the augmented reality contents to the smart terminal 300 wirelessly. In this case, the smart terminal 300 is economical since it does not have to pay a separate data usage fee.

2 is a diagram showing one embodiment of the method of the present invention.

As shown in FIG. 2, the service server transmits the home shopping content of the digital data broadcasting to the set-top box (S1000). The digital TV of the viewer receives the home shopping content from the set-top box and displays it on the screen.

The smart terminal of the viewer drives an Augmented Reality (AR) application and takes a picture of the digital TV using the camera of the smart terminal. The smart terminal recognizes the product identification information from the image, watermark, or QR code included in the home shopping content being broadcast (S1100). The product identification information includes information that can identify a product such as a product name and a product code.

The smart terminal transmits identification information of the recognized product to the augmented reality server to request the augmented reality curating (S1200).

The augmented reality server receives the identification information of the product, and provides the augmented reality content for the home shopping content being broadcast on the digital TV to the smart terminal to start the augmented reality curating (S1300). The augmented reality is realized by the smart terminal matching the augmented reality contents with the images taken from the camera and displaying them on the display module of the smart terminal.

Hereinafter, various embodiments of how a smart terminal recognizes a home shopping content being broadcast on a digital TV according to the present invention will be described.

<Recognition using watermark or QR code>

The service server displays a watermark or QR code including identification information of the home shopping contents. The area where the watermark or the QR code is displayed may be any one of a VOD area of the digital TV, a TV application area having independent functions and independent areas, and an event area. The screen interface provided by the data broadcasting home shopping of the present invention may include a TV application area and an event area having a plurality of independent functions and independent areas as well as a VOD area in which home shopping broadcast contents are reproduced. In the TV application area, contents such as a recommendation of a real-time popular channel or a customized current product can be transmitted, and an eventful content such as a surprise discount can be transmitted to the event area.

The smart terminal captures a watermark or QR code, and transmits the watermark or QR code to the augmented reality server.

The augmented reality server searches for a home shopping content matched with a watermark or a QR code, and provides an EV content corresponding to the retrieved home shopping content to the smart terminal.

According to this embodiment, since the identification information of the home shopping contents is automatically inputted by the watermark or the QR code even if the merchandise currently being broadcasted is not manually inputted, the augmented reality The service can be used.

<Product recognition using images>

The smart terminal captures the screen of the digital TV. The product image is extracted from the screen of the digital TV taken by the smart terminal.

In a preferred embodiment, identification information of a home shopping content being broadcast is extracted from a product image directly extracted by the smart terminal. To this end, the smart terminal can access the product image database stored in the service server. In addition, the application of the smart terminal can incorporate an image matching algorithm capable of comparing and searching similar images.

When the identification information is extracted, identification information of the home shopping contents extracted by the smart terminal is transmitted to the augmented reality server.

In another embodiment, the smart terminal simply captures an image and delivers it to the augmented reality server, and the augmented reality server can directly extract the identification information of the home shopping content.

In this embodiment, the augmented reality server receives the screen of the digital TV taken by the smart terminal and extracts the product image from the screen of the digital TV. The augmented reality server extracts identification information of the home shopping contents being broadcast from the extracted product images. To this end, the augmented reality server can rescue the product image database for the broadcast product transmitted by the service server. In addition, the augmented reality server can be equipped with an image matching algorithm capable of comparing and searching similar images.

When the augmented reality server extracts the identification information, the augmented reality content about the extracted identification information is transmitted to the smart terminal.

[Example using pairing]

Another embodiment of the present invention may include a pairing server. The pairing server may store the pairing information for matching the smart terminal and the set-top box, and utilize it variously.

First, the pairing server stores the pairing information of the set top box paired with the smart terminal. During watching the home shopping channel through the digital TV, the smart terminal drives the augmented reality application to access the augmented reality server. The augmented reality server accesses the pairing server and inquires the set top box paired with the smart terminal. The augmented reality server accesses the service server and inquires the home shopping contents being played back on the set-top box. In this way, the augmented reality server can inquire itself by the smart terminal connected to the augmented reality server even if the augmented reality server is not informed of the product currently being viewed. When the inquiry is completed, the augmented reality server provides a virtual reality (VR) content for the home shopping content to the smart terminal. According to this embodiment, it is very convenient that a smart terminal can experience augmented reality contents simply by opening a smart terminal and accessing a specific application while watching a digital TV. This is because it is not necessary to scan and transmit a watermark or a QR code.

In the present invention, a pairing server is a server that interconnects a plurality of devices. For example, a digital TV and a smart terminal are linked. For this, the pairing server can store the pairing information of the corresponding device. The pairing information may include identification information of the device. The identification information is a concept that includes an address to identify and connect to the device. In a preferred embodiment, the identification information may include an IP address, a MAC address, and a set top box ID of the device. For example, the identification information of the smartphone may include login account information as application subscription information, unique token information (unique ID) of the smartphone itself, mobile phone number, and the like. However, in this embodiment, the pairing server may be integrated into the service server or the augmented reality server.

[Execution scenario of the present invention]

3 to 7, an embodiment of a scenario in which the present invention is implemented in a smartphone will be described.

As shown in FIG. 3, the smart terminal drives the augmented reality application (S2000). As shown in FIG. 4, the augmented reality application drives a camera of a smart terminal to shoot a broadcast screen of a digital TV (S2100).

The smart terminal recognizes the goods from the home shopping contents (S2200). The home shopping content 310 shown in FIG. 5 is a home shopping content 310 displayed on a smart terminal, not a digital TV. As shown in FIG. 5, when a home shopping content 310 is photographed, the smart terminal analyzes it and recognizes a product (311). There may be a coupon 313, a stand illumination 315, and a curtain 317 in the recognized product.

When the product image is recognized from the broadcast screen and the identification information of the product is recognized from the product image, the augmented reality content matching the identification information of the product is provided to the smart terminal (S2300).

In a preferred embodiment, the augmented reality can be largely serviced in two categories. One to implement additional images or additional information, and another to provide associated merchandise shopping.

FIG. 6 illustrates providing additional images and additional information to an augmented reality. FIG. 6 shows the augmented reality contents overlapped with the home shopping contents 310 broadcasted on the digital TV. The augmented reality contents may include an additional image 320 and an additional audio 321. [ The additional image 320 may include a recommendation image of an entertainer, an image of a behind-the-story, a description image of a method of using a product, and the like.

In addition, the augmented reality contents may include additional information 323. The additional information may be used to allow the viewer to change the color of the sofa 313 being broadcast, to show the detailed information of the sofa 313, to place the payment process on one side of the screen for the purchase progress, 313 may be rotated 360 degrees.

Figure 7 illustrates that the augmented reality application of the present invention provides an associated merchandise shopping interface.

The augmented reality contents 331, 333, and 335 for recommending an associated product are displayed on the home shopping contents 310 broadcast on the digital TV. In addition to the sofa being broadcast in the main, curtains, stand lights, and to recommend a related product to be able to shop the table. The user can touch the icons such as the curtain view, the stand view, and the table view to request detailed information of the corresponding item.

Meanwhile, the present invention may provide a service for recommending an associated product. For example, even if a product is not displayed on the home shopping contents 310, it is possible to display a cushion associated with a sofa or an augmented reality content recommending a product such as a leather cleaner. At this time, a curator recommending a product may be displayed as an additional image. Also, if there is pairing information, the customized affiliate product may be recommended to the user using the conventional shopping history database.

The scope of protection of the present invention is not limited to the description and the expression of the embodiments explicitly described in the foregoing. It is again to be understood that the present invention is not limited by the modifications or substitutions that are obvious to those skilled in the art.

Claims (7)

(a) transmitting a home shopping content of digital data broadcasting to a set-top box by a service server;
(b) receiving a home shopping content from the set-top box and displaying the digital home TV on a screen;
(c) the smart terminal of the viewer drives an Augmented Reality (AR) application and takes a picture of the digital TV using the camera of the smart terminal;
(d) providing the augmented reality server with the augmented reality contents for the home shopping contents being broadcast on the digital TV to the smart terminal; And
(e) displaying the augmented reality contents on the display module of the smart terminal by matching the augmented reality contents with the images photographed by the camera,
Wherein the step (d) comprises: receiving the identification information of the smart terminal from the augmented reality server;
Transmitting the identification information of the smart terminal to the service server by the augmented reality server;
Extracting the identification information of the home shopping content being broadcast on the set-top box paired with the identification number of the smart terminal, and transmitting identification information of the extracted home shopping content to the augmented reality server; And
Further comprising the step of the augmented reality server transmitting the augmented reality contents corresponding to the identification information of the home shopping contents to the smart terminal. The augmented reality contents related to the home shopping product of the digital data broadcasting, A method for implementing in a smart terminal.

delete delete delete delete delete delete

Images