KR20210017128A - System and method for providing shopping information using augmented reality - Google Patents

Abstract

The present invention relates to a shopping information providing system using augmented reality (AR) to provide live shopping information in AR and shopping information providing method thereof. According to the present invention, the shopping information providing system comprises: a broadcast receiver displaying received shopping broadcast on a screen; a mobile terminal scanning the screen of the broadcast receiver with a camera and extracting product information from the scanned screen of the broadcast receiver; and an AR providing server matching the extracted product information and acquired customer information to analyze correlation, inquiring customer-recommended product information based on the analyzed correlation, and processing the inquired customer-recommended product information to be implemented in AR. The mobile terminal activates the camera to scan a shopping broadcast screen of the broadcast receiver when receiving a product information search request, extracts product information from the scanned shopping broadcast screen, transmits the extracted information to the AR providing server, and displays the customer-recommended product information in AR around the screen scanned through the camera of the broadcast receiver when receiving the customer-recommended product information processed in AR from the AR providing server.

Description

A system for providing shopping information using augmented reality and a method for providing shopping information thereof {SYSTEM AND METHOD FOR PROVIDING SHOPPING INFORMATION USING AUGMENTED REALITY}

The present invention relates to a system for providing shopping information, and more particularly, to a system for providing shopping information using augmented reality capable of providing shopping information currently being broadcast as augmented reality, and a method for providing shopping information thereof.

In general, home shopping refers to a service provided through a broadcasting network or a communication network in order to purchase various products or services.

In the case of using a broadcasting network, home shopping may be implemented using a method in which a home shopping service provider transmits home shopping content for a product to a viewer's receiver through a predetermined channel, and the viewer purchases the product while viewing the home shopping content.

The broadcasting method used for home shopping may be digital broadcasting or analog broadcasting, and the receiver may include devices such as a television, a set-top box, a personal computer, and a mobile communication terminal having a computing function that receive and provide home shopping content. .

Viewers can purchase products broadcast through home shopping content using a payment interface displayed on the receiver or a method such as a phone order.

However, in such a home shopping service, since home shopping contents are collectively transmitted to a plurality of customers according to a broadcast schedule, there is a limitation in providing information, and it is difficult to provide efficient information that recommends only information desired by customers.

As described above, in the existing home shopping service, it is difficult to express all types of products or additional information broadcasted in a specific time period within the broadcast time, and there is a problem in that a separate search must be performed to obtain additional information.

In addition, if the product being broadcast does not suit the taste of the customer, there is also a problem that it is difficult to purchase.

Accordingly, in the future, there is a need to develop a shopping information providing system using augmented reality that can provide various information on products using augmented reality and provide a service capable of providing customer recommended products and experiencing products.

The present invention matches product information extracted from a shopping broadcast screen with customer information acquired in advance, analyzes the correlation, and implements customer recommended product information as augmented reality, thereby providing a high level of satisfaction, fun, and a new experience to customers. It is intended to provide a shopping information providing system using reality and a method of providing shopping information thereof.

In addition, the present invention provides a shopping information providing system using augmented reality that can improve reliability by securing and updating highly reliable information using a database incorporating big data of broadcasters and a method of providing shopping information thereof. I want to.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. I will be able to.

In order to solve the above technical problem, a system for providing shopping information using augmented reality according to an embodiment of the present invention includes a broadcast receiver that displays a received shopping broadcast on a screen, and scans a screen of the broadcast receiver with a camera. A mobile terminal that extracts product information from the screen of the scanned broadcast receiver, and analyzes the correlation by matching the extracted product information with the previously acquired customer information, and searches for customer recommended product information based on the analyzed correlation, and retrieves the customer recommended product information The mobile terminal includes an augmented reality providing server for processing to be implemented as augmented reality, and the mobile terminal activates a camera when a product information search request is received to scan a shopping broadcast screen of the broadcast receiver, and product information from the scanned shopping broadcast screen Is extracted and transmitted to the augmented reality providing server, and when customer recommendation product information processed into augmented reality is received from the augmented reality providing server, the customer recommended product information is displayed in augmented reality around the screen of the broadcast receiver scanned through the camera. I can.

In addition, the shopping information providing method of the shopping information providing system using augmented reality according to an embodiment of the present invention is a shopping information providing method of a shopping information providing system including a mobile terminal and an augmented reality providing server that are communicated with each other, The mobile terminal scans the shopping broadcast screen of the broadcast receiver through a camera, extracts product information from the scanned shopping broadcast screen and transmits it to the augmented reality providing server, product information extracted by the augmented reality providing server and customer information previously acquired Analyzing the relationship with the customer recommended product information, processing and processing the searched customer recommended product information to be implemented in augmented reality, and transmitting the processed customer recommended product information to the mobile terminal, and the mobile terminal to augmented reality. Receiving the processed customer recommended product information and displaying the customer recommended product information in augmented reality around the screen of the broadcast receiver scanned through the camera.

In addition, the mobile terminal of the system for providing shopping information using augmented reality according to an embodiment of the present invention includes a communication unit for communication with an augmented reality providing server, a camera unit for scanning a shopping broadcast screen of a broadcast receiver, and a camera unit. A display unit for displaying customer-recommended product information in augmented reality around a screen of the scanned broadcast receiver, and a communication unit, a camera unit, and a control unit for controlling the display unit, and the control unit activates the camera unit when a product information search request is received. Control the camera unit to scan the shopping broadcast screen of the broadcast receiver, extract product information from the scanned shopping broadcast screen, control the communication unit to transmit the extracted product information to the augmented reality providing server, and augmented reality from the augmented reality providing server Upon receiving the customer recommended product information processed by the camera unit, the display unit may be controlled to display the customer recommended product information in augmented reality around the screen of the broadcast receiver scanned through the camera unit.

In addition, the method of providing shopping information of a mobile terminal according to an embodiment of the present invention includes the steps of scanning a screen of a broadcast receiver displaying a shopping broadcast by activating a camera when a product information search request is received, and the scanned shopping broadcast screen Extracting product information from the augmented reality providing server, transmitting the extracted product information to an augmented reality providing server, and receiving customer recommendation product information processed into augmented reality from the augmented reality providing server, the screen of the broadcast receiver scanned through the camera It may include displaying customer-recommended product information in augmented reality.

In addition, the augmented reality providing server of the shopping information providing system using augmented reality according to an embodiment of the present invention includes a communication unit that is communicatively connected to a mobile terminal, an augmented reality processing unit that implements customer recommended product information as augmented reality, And a control unit for controlling a communication unit and an augmented reality processing unit, wherein the control unit matches the received product information with the previously acquired customer information when product information extracted from the screen of the broadcast receiver displaying shopping broadcast is received from the mobile terminal. Analyzes the correlation, searches for customer recommended product information based on the analyzed association, controls the augmented reality processing unit to implement the searched customer recommended product information as augmented reality, and transmits the processed customer recommended product information to the mobile terminal. You can control the communication unit to do so.

In addition, a method of providing shopping information of an augmented reality providing server according to an embodiment of the present invention includes receiving product information extracted from a screen of a broadcast receiver displaying a shopping broadcast from a mobile terminal, and receiving the received product information and Matching the acquired customer information to analyze the correlation, querying the customer recommended product information based on the analyzed correlation, and processing the searched customer recommended product information to be displayed in augmented reality on the screen of the mobile terminal And, it may include the step of transmitting the customer recommended product information processed in augmented reality to the mobile terminal.

Meanwhile, an embodiment of the present invention may further provide a computer-readable program for executing any one of the shopping information providing methods and a recording medium on which the program is recorded.

The system for providing shopping information using augmented reality and the method for providing shopping information thereof according to at least one embodiment of the present invention configured as described above, analyzes correlation by matching product information extracted from a shopping broadcast screen with customer information obtained in advance. Thus, by implementing customer-recommended product information in augmented reality, it provides an effect that can give customers high satisfaction, fun, and a new experience.

In addition, the present invention provides an effect of improving reliability by securing and updating highly reliable information using a database incorporating big data of broadcasters.

As such, the present invention can increase satisfaction by recommending product information that fits the customer's disposition, and reduce additional customer actions by providing information that is difficult to express in home shopping broadcasting (real-time sex comments, product web reviews) in augmented reality. I can.

In addition, according to the present invention, it is possible to increase customer fun and satisfaction by providing a service in a form capable of experiencing an augmented reality, and to increase reliability by analyzing customer preferences by linking big data of several separate home shopping companies.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.

1 is a schematic diagram illustrating a system for providing shopping information according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating the mobile terminal of FIG. 1.
3 is a block diagram illustrating an augmented reality providing server of FIG. 1.
FIG. 4 is a block diagram illustrating the database of FIG. 1.
5 to 7 are diagrams for explaining a process of providing shopping information implemented in augmented reality.
8 to 11 are flowcharts illustrating a method of providing shopping information according to an embodiment of the present invention.

Hereinafter, an apparatus and various methods to which embodiments of the present invention are applied will be described in more detail with reference to the drawings. The suffixes "module" and "unit" for components used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have meanings or roles that are distinguished from each other by themselves.

In the above, even if all the constituent elements constituting the embodiments of the present invention have been described as being combined into one or operating in combination, the present invention is not necessarily limited to these embodiments. That is, within the scope of the object of the present invention, all of the constituent elements may be selectively combined and operated in one or more. In addition, although all of the components may be implemented as one independent hardware, a program module that performs some or all functions combined in one or more hardware by selectively combining some or all of the components. It may be implemented as a computer program having Codes and code segments constituting the computer program may be easily inferred by those skilled in the art of the present invention. Such a computer program is stored in a computer-readable storage medium, and is read and executed by a computer, thereby implementing an embodiment of the present invention. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, and a carrier wave medium.

In the description of the embodiment, in the case of being described as being formed in "top (top) or bottom (bottom)", "before (front) or after (back)" of each component, "top (top) or bottom (Below)” and “before (front) or after (back)” include both components formed by direct contact with each other or one or more other components disposed between the two components.

In addition, the terms such as "include", "consist of" or "have" described above mean that the corresponding component may be embedded unless otherwise stated, excluding other components Rather, it should be interpreted as being able to further include other components. All terms, including technical or scientific terms, unless otherwise defined, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms generally used, such as terms defined in the dictionary, should be interpreted as being consistent with the meaning in the context of the related technology, and are not interpreted as ideal or excessively formal meanings unless explicitly defined in the present invention.

In addition, in describing the constituent elements of the present invention, terms such as first, second, A, B, (a), (b) may be used. These terms are only used to distinguish the component from other components, and the nature, order, or order of the component is not limited by the term. When a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected or connected to that other component, but another component between each component It should be understood that elements may be “connected”, “coupled” or “connected”.

In addition, in describing the present invention, if it is determined that the subject matter of the present invention may be unnecessarily obscured as matters apparent to those skilled in the art with respect to related known technologies, detailed descriptions thereof will be omitted.

1 is a schematic diagram illustrating a system for providing shopping information according to an embodiment of the present invention.

As shown in FIG. 1, the shopping information providing system of the present invention scans a broadcast receiver 100 displaying a received shopping broadcast on a screen and a screen 110 of the broadcast receiver 100 with a camera 220 Mobile terminal 200 that extracts product information from the screen 110 of the scanned broadcast receiver 100, and analyzes the correlation by matching the extracted product information with previously acquired customer information, and recommends customer product information based on the analyzed correlation It may include an augmented reality providing server 300 that processes and processes the customer recommended product information that is inquired and inquired into augmented reality.

In addition, the shopping information providing system of the present invention may further include a database 400 for storing customer information.

Here, the broadcast receiver 100 is, for example, a fixed type such as a network TV (network TV), a hybrid broadcast broadband TV (HBBTV), a smart TV, an Internet Protocol TV (IPTV), a personal computer (PC), etc. Mobile devices such as a standing device and a personal digital assistant (PDA), a smart phone, a smart watch, a tablet PC, and a notebook may all be included. have.

In addition, the mobile terminal 200 described in the present specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation system, Slate PC, tablet PC, ultrabook, wearable device (e.g., smartwatch, smart glass), head mounted display (HMD) )), etc.

However, it will be readily apparent to those skilled in the art that the configuration according to the embodiment described in the present specification may also be applied to fixed terminals such as digital TVs, desktop computers, and digital signage, except when applicable only to mobile terminals. will be.

In addition, the database 400 may be an individual database of shopping broadcasters or an integrated database in which customer information collected from databases for each shopping broadcaster is integrated and stored, but is not limited thereto.

In addition, when a product information search request is received, the mobile terminal 200 activates the camera 220 to scan the shopping broadcast screen 110 of the broadcast receiver 100, and product information from the scanned shopping broadcast screen 110 Is extracted and transmitted to the augmented reality providing server 300, and upon receiving customer recommendation product information processed into augmented reality from the augmented reality providing server 300, the screen of the broadcast receiver 100 scanned through the camera 220 (110) Customer-recommended product information may be displayed in augmented reality.

Here, when activating the camera 220, the mobile terminal 200 may automatically activate the camera 220 when a user input requesting product information search is received.

As an example, the user input may be a user input of touching a product information search icon displayed on the display of the mobile terminal 200.

As another example, the user input may be a user input for uttering a product information search command by voice.

As another example, the user input may be a user input for recognizing a gesture corresponding to a product information search command.

And, the mobile terminal 200, when scanning the shopping broadcast screen 110 of the broadcast receiver 100, a preview image of the screen 110 of the broadcast receiver 100 scanned through the camera 220 ) Can be displayed on the display.

Subsequently, when scanning the shopping broadcast screen 110 of the broadcast receiver 100, the mobile terminal 200 may control the camera 220 to scan the entire screen area of the broadcast receiver 100.

Next, when extracting product information from the scanned shopping broadcast screen 110, the mobile terminal 200 recognizes the screen area from the broadcast receiver 100 scanned through the camera 220, and Product information currently being broadcast on shopping can be extracted from.

Here, the extracted product information may include at least one of a product image, a product name, and a product logo displayed on the screen of the broadcast receiver, but is not limited thereto.

In addition, when extracting product information from the scanned shopping broadcast screen, the mobile terminal 200 may further extract information on a shopping broadcaster currently being broadcasted for shopping within the recognized screen area.

Here, the extracted shopping broadcaster information may include at least one of a shopping broadcast channel logo, a shopping broadcaster logo, and a shopping broadcaster name displayed on the screen of the broadcast receiver, but is not limited thereto.

In addition, when displaying customer recommended product information in augmented reality, the mobile terminal 200 may arrange and display customer recommended product information according to a preset position value in the augmented reality providing server 300.

In some cases, when displaying customer recommended product information in augmented reality, the mobile terminal 200 is positioned at least one of the top, bottom, left, and right with respect to the screen of the broadcast receiver that scans the customer recommended product information through the camera 220. It can also be displayed.

This is because there is an empty space around the screen of the broadcast receiver, so that the user can easily recognize product information.

For example, when displaying customer-recommended product information in an augmented reality, the mobile terminal 200 displays information about the customer-recommended product, and if the information on the customer-recommended product includes an augmented reality icon and an augmented reality image of the customer-recommended product. The augmented reality icon and the augmented reality image can be displayed without overlapping each other.

The reason is that if the information viewing augmented reality icon and the augmented reality image are overlapped with each other, the user cannot easily recognize product information, and thus reliability and interest in the information may decrease.

As another example, when displaying customer recommended product information in augmented reality, the mobile terminal 200 displays the information view augmented reality icon through the camera when the customer recommended product information includes the information view augmented reality icon on the customer recommended product. It can be displayed without overlapping on the screen of the scanned broadcast receiver.

This is because if the information viewing augmented reality icon and the screen of the broadcast receiver overlap each other, the user cannot easily recognize the product information, and thus reliability and interest in the information may decrease.

Here, the information view augmented reality icon may include a detail information view first touch button, a size view second touch button, and an immediate order third touch button, which is only an example and is not limited thereto.

Next, when displaying customer recommended product information in augmented reality, the mobile terminal 200 displays first customer recommended product information with a large amount of information and second customer recommended product information with a small amount of information on the screen of the broadcast receiver. It can be displayed in different locations around.

The reason is that placing the first customer recommended product information with a large amount of information in a large space area and placing the second customer recommended product information with a small amount of information in a narrow space area allows the user to easily recognize it and make the product information arrangement space efficient. Because it can be used as.

As an example, the mobile terminal 200 may display first customer recommended product information with a large amount of information to be located on the left or right side of the screen of the broadcast receiver, and the second customer recommended product information with a small amount of information may be displayed by the broadcast receiver. It can be displayed at the top or bottom of the screen.

The reason is that the first customer recommended product information with a large amount of information is placed on the left or right side of the screen having a large space area, and the second customer recommended product information with a small amount of information is disposed above or below the screen having a narrow space area. This is because the user can easily recognize and use the product information arrangement space efficiently.

Subsequently, when displaying customer recommended product information in augmented reality, when a part of the body is scanned through the camera, the mobile terminal 200 may fit and display the customer recommended product displayed as augmented reality.

The reason is that, by indirectly experiencing customer-recommended products through augmented reality, users can increase fun and satisfaction, induce purchase, and increase customer satisfaction and purchasing power by recommending products that match customer preferences. .

For example, when a shoe that is a customer recommended product is implemented in virtual reality, when the user's foot is scanned through a camera, the selected shoe is fitted to the user's foot, so that the user may indirectly experience wearing the shoe in virtual reality.

In addition, when displaying customer recommended product information in augmented reality, the mobile terminal 200 acquires real-time customer reaction information for the product on the shopping broadcast screen in addition to the customer recommended product information from the augmented reality providing server 300. Response information can also be displayed in augmented reality.

Here, the real-time customer response information may include real-time comment and review information on the product, but is not limited thereto.

As described above, the reason for displaying real-time customer response information in augmented reality is that information that is difficult to express in home shopping broadcasts is provided in virtual reality, thereby reducing additional actions of customers such as an additional search action for obtaining additional information.

Meanwhile, when the augmented reality providing server 300 receives product information extracted from the screen 110 of the broadcast receiver 100 displaying a shopping broadcast from the mobile terminal 200, the received product information and the previously acquired customer information Matching and analyzing the correlation, querying the customer recommended product information based on the analyzed correlation, processing the searched customer recommended product information to be implemented in augmented reality, and processing the processed customer recommended product information into the mobile terminal 200 Can be transferred to.

Here, the augmented reality providing server 300 classifies the product category based on the received product information when analyzing the association by matching the received product information with the previously acquired customer information, and the database 400 based on the user login information. ), and by matching customer information and product categories, it is possible to analyze the relationship between customer shopping tendencies and corresponding products.

For example, the product information received from the mobile terminal 200 may include at least one of a product image, a product name, and a product logo displayed on the screen of the broadcast receiver 100, but is not limited thereto.

That is, when classifying product categories, the augmented reality providing server 300 may classify product categories from product information including product images, product names, and product logos displayed on the screen of the broadcast receiver 100.

For example, when a product image is a sports shoe and a product logo is Nike, the augmented reality providing server 300 may classify a product category of a shopping broadcast as a shoe.

In addition, the augmented reality providing server 300 receives shopping broadcaster information and user login information from the mobile terminal 200 when obtaining customer information, and a database 400 for obtaining customer information based on shopping broadcaster information Is selected, and corresponding customer information may be obtained from the database 400 selected based on user login information.

As an example, the shopping broadcaster information received from the mobile terminal 200 may include at least one of a shopping broadcast channel logo, a shopping broadcaster logo, and a shopping broadcaster name displayed in the screen 110 of the broadcast receiver 100. , But is not limited thereto.

In addition, the database 400 may be an individual database of a shopping broadcasting company server or an integrated database in which customer information collected from a plurality of shopping broadcasting company server-specific databases is integrated and stored, but is not limited thereto.

That is, when selecting a database, the augmented reality providing server 300 acquires shopping broadcaster information including a shopping broadcast channel logo, a shopping broadcaster logo, and a shopping broadcaster name displayed on the screen of the broadcast receiver 100, and A database for acquiring customer information may be selected based on the shopping broadcaster information.

For example, when the shopping broadcaster logo is an LG home shopping broadcaster, the augmented reality providing server 300 may access the LG home shopping broadcaster's database to obtain customer information from the LG home shopping broadcaster's database.

As another example, the augmented reality providing server 300 may obtain customer information from its own integrated database when there is an integrated database in which customer information collected from a plurality of shopping broadcaster server databases is integrated and stored.

Here, the customer information may include product purchase history information of a corresponding customer, the number of clicks on a specific product, a shopping broadcast viewing time, and wish list information of the customer based on the user login information.

Next, the augmented reality providing server 300, when inquiring about customer recommended product information, selects a main item related to the product based on the relationship between the customer shopping tendency and the product currently being broadcasted, and the similarity with the selected item Other sub-items above the reference value are additionally selected, and customer recommended product information for the selected main item and sub-item can be inquired.

Here, the selected main item includes a specific product of a specific brand, and the selected sub-item is the same product of another brand for a specific product, another product related to a specific product, a product with high similarity to a specific product, It may include an accessory product of a specific product.

For example, the augmented reality providing server 300, if the category of product information currently being broadcast is shoes, and the customer shopping tendency prefers sports products based on customer information, the customer shopping tendency who prefers sports products and the current shopping broadcast Based on the relationship with the existing Nike shoe-related products, a sports shoe product among Nike shoes can be selected as the main item.

In addition, the augmented reality providing server 300 may additionally select a third-party brand Adidas sneakers related to the main item Nike sneakers, Nike sneakers having various functions, and insoles of Nike sneakers as sub-items.

Accordingly, the augmented reality providing server 300 may inquire customer recommended product information for the selected main item and sub-item.

Here, the customer-recommended product information may include an image and detailed information on the customer-recommended product extracted from the main item and the sub-item based on the customer shopping tendency.

Next, when the augmented reality providing server 300 transmits the processed customer recommended product information to the mobile terminal 200, when acquiring the customer's product purchase information and product inquiry information, the customer's shopping tendency based on these information May be analyzed and the analyzed shopping propensity-related update information may be transmitted to the database 400.

Here, the database 400 may be an individual database of a shopping broadcasting company server or an integrated database in which customer information collected from a plurality of shopping broadcasting company server-specific databases is integrated and stored, but is not limited thereto.

Accordingly, according to the present invention, it is possible to obtain product purchase information and product inquiry information of the customer to analyze the shopping tendency of the customer, and update the customer information stored in the database based on the analyzed shopping tendency.

As described above, the present invention secures highly reliable information using a database incorporating big data of broadcasters and updates customer information from time to time, thereby improving reliability.

In the past, information that can be provided within the broadcast time is limited, but the present invention provides additional information through augmented reality to provide fun to customers and indirectly experience wearing products, thereby enhancing customer satisfaction and purchase induction. I can.

In addition, in the current home shopping broadcast, there are various products, but only unilateral information delivery is provided, but the product suitable for personal preference is not provided, whereas in the present invention, personal preference information is identified through big data analysis, Customer satisfaction can be improved by recommending products or broadcasts that match the tendency.

In addition, when using the current home shopping, the customer does not use only a specific home shopping but moves several home shopping companies and selects the best product that suits them. However, the present invention provides a product by comprehensively analyzing the big data of several home shopping companies. Customer discomfort can be minimized.

FIG. 2 is a block diagram illustrating the mobile terminal of FIG. 1.

As shown in FIG. 2, the mobile terminal 200 scans from a communication unit 250 communicating with an augmented reality providing server, a camera unit 260 for scanning a shopping broadcast screen of a broadcast receiver, and a camera unit 260. A display unit 270 for displaying customer-recommended product information in augmented reality around the screen of a broadcast receiver, and a control unit 280 for controlling the communication unit 250, the camera unit 260, and the display unit 270. I can.

Here, the communication unit 250 may be communicatively connected with the augmented reality providing server, and may transmit/receive digital data such as content by connecting to a network by wire or wirelessly.

For example, the communication unit 250 may include a wireless Internet module, a short-range communication module, and a location information module.

Here, the wireless Internet module refers to a module for wireless Internet access, and may be built-in or external to the mobile terminal 200.

The wireless Internet module is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

Examples of wireless Internet technologies include WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc. Data is transmitted and received according to at least one wireless Internet technology within a range including Internet technologies not listed in.

From the point of view that wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc. is made through a mobile communication network, a wireless Internet module that performs wireless Internet access through a mobile communication network is It can also be understood as a kind of module.

The short range communication module is for short range communication, and includes Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and NFC (Near NFC). Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless Universal Serial Bus (USB) technologies may be used to support short-range communication.

The short-range communication module may support wireless communication between the mobile terminal 200 and the augmented reality TV server through wireless area networks.

The local area wireless communication networks may be wireless personal area networks (Wireless Personal Area Networks).

The location information module is a module for obtaining the location (or current location) of the mobile terminal 200, and representative examples thereof include a GPS (Global Positioning System) module or a WiFi (Wireless Fidelity) module.

Meanwhile, when a product information search request is received, the control unit 280 controls the camera unit 260 to scan the shopping broadcast screen of the broadcast receiver by activating the camera unit 260, and retrieves product information from the scanned shopping broadcast screen. Controls the communication unit 250 to transmit the extracted product information to the augmented reality providing server, and when receiving customer recommendation product information processed into augmented reality from the augmented reality providing server, the broadcast scanned through the camera unit 260 The display unit 270 may be controlled to display customer recommended product information in augmented reality around the screen of the receiver.

Here, when activating the camera unit 260, the controller 280 may automatically activate the camera unit 260 when a user input requesting product information search is received.

For example, the user input may be a user input of touching a product information search icon displayed on the display of the mobile terminal 200.

As another example, the user input may be a user input for uttering a product information search command by voice.

As another example, the user input may be a user input for recognizing a gesture corresponding to a product information search command.

And, when scanning the shopping broadcast screen of the broadcast receiver, the control unit 280 controls the display unit 270 to display the screen of the broadcast receiver scanned through the camera unit 260 as a preview image. I can.

Subsequently, when scanning the shopping broadcast screen of the broadcast receiver, the controller 280 may control the camera unit 260 to scan the entire screen area of the broadcast receiver.

Next, when extracting product information from the scanned shopping broadcast screen, the control unit 280 recognizes the screen area from the broadcast receiver scanned through the camera unit 260, and the product currently being broadcasted for shopping within the recognized screen area. Information can be extracted.

Here, the extracted product information may include at least one of a product image, a product name, and a product logo displayed on the screen of the broadcast receiver, but is not limited thereto.

In addition, when extracting product information from the scanned shopping broadcast screen, the controller 280 may further extract information on a shopping broadcaster currently being broadcasted for shopping within the recognized screen area.

Here, the extracted shopping broadcaster information may include at least one of a shopping broadcast channel logo, a shopping broadcaster logo, and a shopping broadcaster name displayed on the screen of the broadcast receiver, but is not limited thereto.

In addition, when displaying the customer recommended product information in augmented reality, the controller 280 may arrange and display the customer recommended product information according to a preset position value in the augmented reality providing server.

In some cases, when displaying customer recommended product information in augmented reality, the controller 280 is positioned at least one of the top, bottom, left, and right with respect to the screen of the broadcast receiver scanned through the camera unit 260 for the customer recommended product information. It can also be displayed.

This is because there is an empty space around the screen of the broadcast receiver, so that the user can easily recognize product information.

For example, when displaying customer-recommended product information in augmented reality, the controller 280 may augment viewing information if the customer-recommended product information includes an augmented reality icon and an augmented reality image of the customer-recommended product. The reality icon and the augmented reality image can be displayed without overlapping each other.

The reason is that if the information viewing augmented reality icon and the augmented reality image are overlapped with each other, the user cannot easily recognize product information, and thus reliability and interest in the information may decrease.

As another example, when displaying customer-recommended product information in augmented reality, the control unit 280, when the customer-recommended product information includes an augmented reality icon for viewing information on a customer-recommended product, the information viewing augmented reality icon is scanned through the camera. It can be displayed without overlapping on the screen of one broadcast receiver.

This is because if the information viewing augmented reality icon and the screen of the broadcast receiver overlap each other, the user cannot easily recognize the product information, and thus reliability and interest in the information may decrease.

Here, the information view augmented reality icon may include a detail information view first touch button, a size view second touch button, and an immediate order third touch button, which is only an example and is not limited thereto.

Next, when displaying the customer recommended product information in augmented reality, the controller 280 transmits the first customer recommended product information with a large amount of information and the second customer recommended product information with a small amount of information among the customer recommended product information around the screen of the broadcast receiver. It can be displayed in different positions.

The reason is that placing the first customer recommended product information with a large amount of information in a large space area and placing the second customer recommended product information with a small amount of information in a narrow space area allows the user to easily recognize it and make the product information arrangement space efficient. Because it can be used as.

As an example, the controller 280 may display first customer recommended product information with a large amount of information to be located on the left or right side of the screen of the broadcast receiver, and display second customer recommended product information with a small amount of information on the screen of the broadcast receiver. It can be displayed to be located on the upper or lower side of the surroundings.

The reason is that the first customer recommended product information with a large amount of information is placed on the left or right side of the screen having a large space area, and the second customer recommended product information with a small amount of information is disposed above or below the screen having a narrow space area. This is because the user can easily recognize and use the product information arrangement space efficiently.

Then, when displaying customer recommended product information in augmented reality, when a part of the body is scanned through the camera unit 260, the controller 280 may fit and display the customer recommended product displayed in augmented reality. have.

The reason is that, by indirectly experiencing customer-recommended products through augmented reality, users can increase fun and satisfaction, induce purchase, and increase customer satisfaction and purchasing power by recommending products that match customer preferences. .

For example, when a shoe that is a customer recommended product is implemented in virtual reality, when the user's foot is scanned through a camera, the selected shoe is fitted to the user's foot, so that the user may indirectly experience wearing the shoe in virtual reality.

In addition, when displaying customer recommended product information in augmented reality, the control unit 280 augments real-time customer response information by acquiring real-time customer response information for the product on the shopping broadcast screen in addition to the customer recommended product information from the augmented reality providing server. You can also mark it as reality.

Here, the real-time customer response information may include real-time comment and review information on the product, but is not limited thereto.

As described above, the reason for displaying real-time customer response information in augmented reality is that information that is difficult to express in home shopping broadcasts is provided in virtual reality, thereby reducing additional actions of customers such as an additional search action for obtaining additional information.

FIG. 3 is a block diagram illustrating the augmented reality providing server of FIG. 1, and FIG. 4 is a block diagram illustrating the database of FIG. 1.

As shown in FIG. 3, the augmented reality providing server 300 includes a communication unit 310 that is communicatively connected to a mobile terminal, an augmented reality processing unit 320 that implements customer recommended product information as augmented reality, and a communication unit ( 310) and a control unit 330 that controls the augmented reality processing unit 320.

Here, the communication unit 310 may be communicatively connected with a mobile terminal and a database, and may transmit/receive digital data such as content by connecting to a network by wire or wirelessly.

For example, the communication unit 310 may include a wireless Internet module, a short-range communication module, and a location information module.

Here, the wireless Internet module refers to a module for wireless Internet access, and may be built-in or external to the augmented reality TV server 300.

The wireless Internet module is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

Examples of wireless Internet technologies include WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc. Data is transmitted and received according to at least one wireless Internet technology within a range including Internet technologies not listed in.

From the point of view that wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc. is made through a mobile communication network, a wireless Internet module that performs wireless Internet access through a mobile communication network is It can also be understood as a kind of module.

The short range communication module is for short range communication, and includes Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and NFC (Near NFC). Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless Universal Serial Bus (USB) technologies may be used to support short-range communication.

The short-range communication module may support wireless communication between the augmented reality providing server 300 and a mobile terminal or between the augmented reality providing server 300 and a database through wireless area networks.

The local area wireless communication networks may be wireless personal area networks (Wireless Personal Area Networks).

The location information module is a module for acquiring the location (or current location) of the augmented reality providing server 300, and representative examples thereof include a GPS (Global Positioning System) module or a WiFi (Wireless Fidelity) module.

On the other hand, the control unit 330, when receiving product information extracted from a screen of a broadcast receiver displaying a shopping broadcast from a mobile terminal, matches the received product information with previously acquired customer information, analyzes the correlation, and makes the analyzed correlation. Based on the inquiry of customer recommended product information, controlling the augmented reality processing unit 320 to implement the searched customer recommended product information into augmented reality, and transmitting the processed customer recommended product information to the mobile terminal. Can be controlled.

Here, the control unit 330 classifies the product category based on the received product information when analyzing the association by matching the received product information with the previously acquired customer information, and the customer from the database 400 based on the user login information. By acquiring information and matching customer information and product categories, it is possible to analyze the relationship between customer shopping tendencies and corresponding products.

For example, the product information received from the mobile terminal may include at least one of a product image, a product name, and a product logo displayed on the screen of a broadcast receiver, but is not limited thereto.

That is, when classifying a product category, the controller 330 may classify the product category from product information including product images, product names, product logos, and the like displayed on the screen of the broadcast receiver.

For example, when the product image is sneakers and the product logo is Nike, the augmented reality providing server may classify the product category of the shopping broadcast as shoes.

And, when acquiring customer information, the control unit 330 receives shopping broadcaster information and user login information from the mobile terminal, selects a database 400 for acquiring customer information based on the shopping broadcaster information, and user login Corresponding customer information may be obtained from the database 400 selected based on the information.

For example, the shopping broadcaster information received from the mobile terminal may include at least one of a shopping broadcast channel logo, a shopping broadcaster logo, and a shopping broadcaster name displayed on the screen of the broadcast receiver, but is not limited thereto.

In addition, the database 400 may be an individual database of a shopping broadcasting company server or an integrated database in which customer information collected from a plurality of shopping broadcasting company server-specific databases is integrated and stored, but is not limited thereto.

For example, as shown in FIG. 4, the database 400 includes an individual database of a shopping broadcasting company server including a database 410 of a home shopping company A, a database 420 of a home shopping company B, and a plurality of shopping broadcasting companies. It may include an integrated database 430 in which customer information collected from each server database is integrated and stored.

That is, when selecting the database 400, the control unit 330 acquires shopping broadcasting company information including a shopping broadcasting channel logo, a shopping broadcasting company logo, and a shopping broadcasting company name displayed on the screen of the broadcasting receiver, and the corresponding shopping broadcasting company A database 400 for acquiring customer information may be selected based on the information.

For example, when the shopping broadcaster logo is an LG home shopping broadcaster, the controller 330 may access the LG home shopping broadcaster's database and obtain customer information from the LG home shopping broadcaster's database.

As another example, the controller 330 may obtain customer information from its own integrated database 430 when there is an integrated database 430 in which customer information collected from a plurality of shopping broadcaster server databases is integrated and stored. .

Here, the customer information may include product purchase history information of a corresponding customer, the number of clicks on a specific product, a shopping broadcast viewing time, and wish list information of the customer based on the user login information.

Next, the controller 330 selects a main item related to the product based on the relationship between the customer shopping tendency and the product currently being broadcasted when searching for customer recommended product information, and the similarity with the selected item is not less than a reference value. Sub-items are additionally selected, and customer-recommended product information for the selected main item and sub-item can be inquired.

Here, the selected main item includes a specific product of a specific brand, and the selected sub-item is the same product of another brand for a specific product, another product related to a specific product, a product with high similarity to a specific product, It may include an accessory product of a specific product.

For example, if the category of the product information currently being broadcast is shoes, and the customer shopping tendency prefers sports products based on customer information, the control unit 330 may control the shopping tendency of customers who prefer sports products and Nike shoes currently being broadcast Based on the association with related products, a sports shoe product among Nike shoes can be selected as the main item.

In addition, the controller 330 may additionally select a third-party brand Adidas sneakers related to the main item Nike sneakers, Nike sneakers having various functions, and insoles of Nike sneakers as sub-items.

Accordingly, the control unit 330 may inquire customer recommended product information for the selected main item and sub-item.

Here, the customer-recommended product information may include an image and detailed information on the customer-recommended product extracted from the main item and the sub-item based on the customer shopping tendency.

Next, the control unit 330, when transmitting the processed customer recommended product information to the mobile terminal, when acquiring the product purchase information and product inquiry information of the customer, analyzes the shopping tendency of the customer based on these information, and analyzes it. Update information related to shopping propensity may be transmitted to the database 400.

Here, the database 400 may be individual databases 410 and 420 of a shopping broadcasting company server or an integrated database 430 in which customer information collected from databases for each shopping broadcasting company server is integrated and stored, but is not limited thereto. .

Accordingly, according to the present invention, it is possible to obtain product purchase information and product inquiry information of the customer to analyze the shopping tendency of the customer, and update the customer information stored in the database based on the analyzed shopping tendency.

5 to 7 are diagrams for explaining a process of providing shopping information implemented in augmented reality.

As shown in FIG. 5, when a user input for selecting the augmented reality product information search icon 240 displayed on the display screen 210 is received, the mobile terminal 200 activates the camera 220 to display The screen 210 can be switched to a camera mode to display an external image.

Here, the mobile terminal 200 may automatically activate the camera 220 when a user input requesting product information search is received.

As an example, the user input may be a user input of touching the augmented reality product information information search icon 240 displayed on the display screen 210 of the mobile terminal 200, as shown in FIG. 5, but the product information search command is voiced. It may be a user input that speaks, or a user input that recognizes a gesture corresponding to a product information search command.

Next, as shown in FIG. 6, the mobile terminal 200 may scan the screen 110 of the broadcast receiver 100 displaying a shopping broadcast through the camera 222.

Here, when the mobile terminal 200 scans the shopping broadcast screen 110 of the broadcast receiver 100, the screen 110 of the broadcast receiver 100 scanned through the camera 220 is viewed as a preview image. ) 120 can be displayed on the display screen 210.

For example, when scanning the shopping broadcast screen 110 of the broadcast receiver 100, the mobile terminal 200 may control the camera 220 to scan the entire screen area of the broadcast receiver 100.

Next, the mobile terminal 200 may extract product information 130 from the scanned shopping broadcast screen.

For example, when extracting product information 130, the mobile terminal 200 recognizes a screen area from the broadcast receiver 100 scanned through the camera 220, and is currently shopping broadcast within the recognized screen area. Product information 130 may be extracted.

Here, the extracted product information 130 may include at least one of a product image 132, a product name 134, and a product logo 136 displayed on the screen of the broadcast receiver, but is not limited thereto.

In addition, when extracting the product information 130, the mobile terminal 200 may further extract the shopping broadcaster information 140 currently being broadcasted for shopping within the recognized screen area.

Here, the extracted shopping broadcaster information 140 may include at least one of a shopping broadcast channel logo, a shopping broadcaster logo, and a shopping broadcaster name displayed on the screen of the broadcast receiver 100, but is not limited thereto.

Next, the mobile terminal 200 may transmit the extracted product information 130 to the augmented reality providing server.

Here, when transmitting the extracted product information 130 to the augmented reality providing server, the mobile terminal 200 may transmit shopping broadcaster information and user login information to the augmented reality providing server together.

Next, the augmented reality providing server, when receiving product information extracted from the screen of the broadcast receiver displaying the shopping broadcast from the mobile terminal, matches the received product information with the previously acquired customer information, analyzes the correlation, and makes the analyzed correlation. Based on customer recommendation product information, it is possible to inquire about customer recommendation product information, implement the inquiry customer recommendation product information as augmented reality, and transmit the processed customer recommendation product information to the mobile terminal.

And, as shown in Figure 7, the mobile terminal 200, upon receiving the customer recommendation product information processed into augmented reality from the augmented reality providing server, the screen of the broadcast receiver 100 scanned through the camera 220 Customer-recommended product information 230 may be displayed on the display screen 210 in augmented reality.

Here, when displaying the customer recommended product information 230 as augmented reality, the mobile terminal 200 may arrange and display the customer recommended product information 230 according to a preset position value in the augmented reality providing server.

In some cases, when displaying the customer recommended product information 230 in augmented reality, the mobile terminal 200 scans the customer recommended product information 230 through the camera 220 on the screen of the broadcast receiver 100 ( 110) may be displayed in at least one of the top, bottom, left, and right.

This is because there is an empty space around the screen of the broadcast receiver, so that the user can easily recognize product information.

For example, when the mobile terminal 200 displays the customer recommended product information 230 in augmented reality, the customer recommended product information 230 displays information on the customer recommended product. The augmented reality icon 234 and the customer recommended product If the augmented reality image 232 of is included, the information view augmented reality icon 234 and the augmented reality image 232 may be displayed so that they do not overlap each other.

This is because if the information viewing augmented reality icon 234 and the augmented reality image 232 overlap each other, the user cannot easily recognize product information, and thus reliability and interest in the information may decrease.

As another example, when the mobile terminal 200 displays the customer recommended product information 230 in augmented reality, if the customer recommended product information 230 includes the information viewing augmented reality icon 234 on the customer recommended product The information view augmented reality icon 234 may be displayed without overlapping on the screen 110 of the broadcast receiver scanned through the camera 220.

This is because if the information viewing augmented reality icon 234 and the screen 110 of the broadcast receiver overlap each other, the user cannot easily recognize product information, and thus reliability and interest in the information may be degraded.

Here, the information view augmented reality icon 234 includes a detailed information view first touch button 234-1, a size view second touch button 234-2, and an immediate order third touch button 234-3. However, this is only an example and is not limited thereto.

Next, when displaying the customer recommended product information 230 in augmented reality, the mobile terminal 200 broadcasts first customer recommended product information with a large amount of information and second customer recommended product information with a small amount of information among customer recommended product information. It can be displayed in different locations around the receiver's screen.

The reason is that placing the first customer recommended product information with a large amount of information in a large space area and placing the second customer recommended product information with a small amount of information in a narrow space area allows the user to easily recognize it and make the product information arrangement space efficient. Because it can be used as.

As an example, the mobile terminal 200 may display first customer recommended product information with a large amount of information to be located on the left or right side of the screen of the broadcast receiver, and the second customer recommended product information with a small amount of information may be displayed by the broadcast receiver. It can be displayed at the top or bottom of the screen.

The reason is that the first customer recommended product information with a large amount of information is placed on the left or right side of the screen having a large space area, and the second customer recommended product information with a small amount of information is disposed above or below the screen having a narrow space area. This is because the user can easily recognize and use the product information arrangement space efficiently.

Next, when displaying the customer recommended product information 230 in augmented reality, the mobile terminal 200 fits the customer recommended product displayed in augmented reality when a part of the body is scanned through the camera 220 to the body part. Can be displayed.

The reason is that, by indirectly experiencing customer-recommended products through augmented reality, users can increase fun and satisfaction, induce purchase, and increase customer satisfaction and purchasing power by recommending products that match customer preferences. .

For example, when a shoe that is a customer recommended product is implemented in virtual reality, when the user's foot is scanned through a camera, the selected shoe is fitted to the user's foot, so that the user may indirectly experience wearing the shoe in virtual reality.

In addition, when displaying the customer recommended product information 230 in augmented reality, the mobile terminal 200 obtains real-time customer reaction information for the product on the shopping broadcast screen in addition to the customer recommended product information from the augmented reality providing server. Response information can also be displayed in augmented reality.

Here, the real-time customer response information may include real-time comment and review information on the product, but is not limited thereto.

As described above, the reason for displaying real-time customer response information in augmented reality is that information that is difficult to express in home shopping broadcasts is provided in virtual reality, thereby reducing additional actions of customers such as an additional search action for obtaining additional information.

8 to 11 are flowcharts illustrating a method of providing shopping information according to an embodiment of the present invention.

As shown in FIG. 8, in a shopping information providing system including a mobile terminal communicating with each other and an augmented reality providing server, first, the mobile terminal scans a shopping broadcast screen of a broadcast receiver through a camera and scans the shopping Product information may be extracted from the broadcast screen and transmitted to the augmented reality providing server (S100).

In addition, the augmented reality providing server analyzes the relationship between the extracted product information and the previously acquired customer information, searches for customer recommended product information, processes the searched customer recommended product information to be implemented as augmented reality, and processes the processed Customer recommended product information may be transmitted to the mobile terminal (S200).

Next, the mobile terminal may receive customer recommendation product information processed with augmented reality and display the customer recommended product information in augmented reality around the screen of the broadcast receiver scanned through the camera (S300).

In some cases, the augmented reality providing server, when acquiring the customer's product purchase information and product inquiry information, analyzes the customer's shopping tendency based on these information, and can update the customer information stored in the database based on the analyzed shopping tendency. have.

As shown in FIG. 9, the step (S100) of extracting product information and transmitting it to the augmented reality providing server will be described in more detail as follows.

First, the mobile terminal receives a product information search request (S110).

Then, the mobile terminal activates the camera when a product information search request is received (S120).

Here, the mobile terminal may automatically activate the camera when a user input requesting product information search is received.

As an example, the user input may be a user input of touching an augmented reality product information information search icon displayed on the display screen of the mobile terminal, but may be a user input of uttering a product information search command with a voice, or corresponding to a product information search command. It may be a user input that recognizes a gesture.

Subsequently, the mobile terminal scans the screen of the broadcast receiver displaying the shopping broadcast through the activated camera (S130).

Here, when scanning the shopping broadcast screen of the broadcast receiver, the mobile terminal may display the screen of the broadcast receiver scanned through the camera on the display screen as a preview image.

For example, when scanning a shopping broadcast screen of a broadcast receiver, the mobile terminal may control the camera to scan the entire screen area of the broadcast receiver.

Next, the mobile terminal extracts product information from the scanned shopping broadcast screen (S140).

As an example, when extracting product information, the mobile terminal may recognize a screen area from a broadcast receiver scanned through a camera, and extract product information currently being broadcasted for shopping within the recognized screen area.

Here, the extracted product information may include at least one of a product image, a product name, and a product logo displayed on the screen of the broadcast receiver, but is not limited thereto.

In addition, when extracting product information, the mobile terminal may further extract information on a shopping broadcaster currently broadcasting shopping within the recognized screen area.

Here, the extracted shopping broadcaster information may include at least one of a shopping broadcast channel logo, a shopping broadcaster logo, and a shopping broadcaster name displayed on the screen of the broadcast receiver, but is not limited thereto.

Then, the mobile terminal transmits the extracted product information to the augmented reality providing server (S150).

Here, when transmitting the extracted product information to the augmented reality providing server, the mobile terminal may transmit shopping broadcaster information and user login information to the augmented reality providing server together.

In addition, as shown in FIG. 10, the step (S200) of processing the inquired customer recommended product information into augmented reality and transmitting it to the mobile terminal will be described in more detail as follows.

First, the augmented reality providing server receives product information extracted from a screen of a broadcast receiver displaying a shopping broadcast from a mobile terminal (S210).

Then, the augmented reality providing server analyzes the correlation by matching the received product information with the previously acquired customer information (S220).

Here, the augmented reality providing server classifies product categories based on the received product information and obtains customer information from the database based on the user login information when analyzing the correlation by matching the received product information with the previously acquired customer information. And, by matching customer information and product categories, it is possible to analyze the relationship between customer shopping tendencies and corresponding products.

For example, the product information received from the mobile terminal may include at least one of a product image, a product name, and a product logo displayed on the screen of a broadcast receiver, but is not limited thereto.

In addition, when obtaining customer information, the augmented reality providing server receives shopping broadcaster information and user login information from the mobile terminal, selects a database for obtaining customer information based on shopping broadcaster information, and Corresponding customer information can be obtained from the selected database.

For example, the shopping broadcaster information received from the mobile terminal may include at least one of a shopping broadcast channel logo, a shopping broadcaster logo, and a shopping broadcaster name displayed on the screen of the broadcast receiver, but is not limited thereto.

Subsequently, the augmented reality providing server inquires the customer recommended product information based on the analyzed correlation (S230).

Here, the augmented reality providing server selects the main item related to the product based on the relationship between the customer shopping tendency and the product currently being broadcasted when searching for customer recommended product information, and the similarity with the selected item is not less than a reference value. Sub-items are additionally selected, and customer-recommended product information for the selected main item and sub-item can be inquired.

Here, the selected main item includes a specific product of a specific brand, and the selected sub-item is the same product of another brand for a specific product, another product related to a specific product, a product with high similarity to a specific product, It may include an accessory product of a specific product.

Next, the augmented reality providing server processes the inquired customer recommendation product information to be displayed as augmented reality on the screen of the mobile terminal (S240).

Then, the augmented reality providing server transmits the customer recommendation product information processed into the augmented reality to the mobile terminal (S250).

Here, the augmented reality providing server, when transmitting the processed customer recommended product information to the mobile terminal, when acquiring the customer's product purchase information and product inquiry information, analyzes and analyzes the customer's shopping tendency based on these information. Update information related to shopping tendencies may be transmitted to a database.

Here, the database may be an individual database of a shopping broadcasting company server or an integrated database in which customer information collected from a plurality of shopping broadcasting company server-specific databases is integrated and stored, but is not limited thereto.

Accordingly, according to the present invention, it is possible to obtain product purchase information and product inquiry information of the customer to analyze the shopping tendency of the customer, and update the customer information stored in the database based on the analyzed shopping tendency.

In addition, as shown in FIG. 11, the step (S300) of displaying customer recommended product information in augmented reality will be described in more detail as follows.

First, the mobile terminal receives customer recommendation product information processed into augmented reality from the augmented reality providing server (S310).

Then, the mobile terminal determines a location to display the customer recommended product information when the customer recommended product information processed in augmented reality is received (S320).

Here, the mobile terminal may determine a location to display customer recommended product information according to a preset location value in the augmented reality providing server.

In some cases, the mobile terminal may determine as a default to display customer recommended product information in at least one of up, down, left, and right on the screen of the broadcast receiver scanned through the camera.

This is because there is an empty space around the screen of the broadcast receiver, so that the user can easily recognize product information.

Next, the mobile terminal displays customer recommended product information in augmented reality around the screen of the broadcast receiver scanned through the camera based on the determined location (S330).

Here, when the mobile terminal displays the customer recommended product information in augmented reality, if the customer recommended product information includes an information view augmented reality icon and an augmented reality image of the customer recommended product, the information view augmented reality icon and Augmented reality images can be displayed without overlapping each other.

The reason is that if the information viewing augmented reality icon and the augmented reality image are overlapped with each other, the user cannot easily recognize product information, and thus reliability and interest in the information may decrease.

As another example, when the mobile terminal displays customer recommended product information in augmented reality, if the customer recommended product information includes an information view augmented reality icon on the customer recommended product, the information view augmented reality icon is scanned through a camera. It can be displayed without overlapping on the receiver's screen.

This is because if the information viewing augmented reality icon and the screen of the broadcast receiver overlap each other, the user cannot easily recognize the product information, and thus reliability and interest in the information may decrease.

In some cases, when displaying customer-recommended product information in augmented reality, the mobile terminal transmits first customer-recommended product information with a large amount of information and second customer-recommended product information with a small amount of information among the customer recommended product information around the screen of the broadcast receiver. It can be displayed in different positions.

The reason is that placing the first customer recommended product information with a large amount of information in a large space area and placing the second customer recommended product information with a small amount of information in a narrow space area allows the user to easily recognize it and make the product information arrangement space efficient. Because it can be used as.

As another case, when displaying customer recommended product information in augmented reality, when a part of the body is scanned through a camera, the mobile terminal may fit and display the customer recommended product displayed in augmented reality.

The reason is that, by indirectly experiencing customer-recommended products through augmented reality, users can increase fun and satisfaction, induce purchase, and increase customer satisfaction and purchasing power by recommending products that match customer preferences. .

In addition, when displaying customer-recommended product information in augmented reality, the mobile terminal acquires real-time customer response information for the product on the shopping broadcast screen in addition to the customer-recommended product information from the augmented reality providing server. It can also be displayed.

Here, the real-time customer response information may include real-time comment and review information on the product, but is not limited thereto.

As described above, the reason for displaying real-time customer response information in augmented reality is that information that is difficult to express in home shopping broadcasts is provided in virtual reality, thereby reducing additional actions of customers such as an additional search action for obtaining additional information.

As described above, the present invention matches product information extracted from a shopping broadcast screen with customer information obtained in advance, analyzes the correlation, and implements customer recommended product information as augmented reality, thereby providing customers with high satisfaction, fun, and a new experience. I can.

In addition, the present invention can improve reliability by securing and updating highly reliable information using a database incorporating big data of broadcasters.

In addition, the present invention can increase satisfaction by recommending product information suitable for the customer's disposition, and reduce additional actions of customers by providing information that is difficult to express in home shopping broadcasts (real-time sex comments, product web reviews) in augmented reality. I can.

In addition, according to the present invention, by providing a service in a form capable of experiencing an augmented reality, it is possible to increase the enjoyment and satisfaction of customers, and it is possible to increase reliability by analyzing customer preferences by linking big data of several separate home shopping companies.

Various embodiments described herein may be implemented in a recording medium that can be read by a computer or a similar device using, for example, software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing a function. It may be implemented by the control unit 140.

According to a software implementation, embodiments such as procedures or functions may be implemented together with separate software modules that perform at least one function or operation. The software code can be implemented by a software application written in an appropriate programming language. In addition, the software code may be stored in a memory and loaded and executed by the controller 140.

In the above embodiment, the information exposure prevention image providing apparatus includes one or more application-specific integrated circuits (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), and field program. It can be realized by a possible gate array (FPGA), controller, microcontroller, microprocessor or other electronic component.

The method according to the above-described embodiment may be produced as a program to be executed on a computer and stored in a computer-readable recording medium. Examples of computer-readable recording media include ROM, RAM, CD-ROM, and magnetic tape. , Floppy disks, optical data storage devices, etc. are included.

The computer-readable recording medium is distributed over a computer system connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes and code segments for implementing the above-described method can be easily inferred by programmers in the technical field to which the embodiment belongs.

It is obvious to those skilled in the art that the present invention can be embodied in other specific forms without departing from the spirit and essential features of the present invention.

Therefore, the detailed description above should not be construed as restrictive in all respects and should be considered as illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: broadcast receiver
200: mobile terminal
300: augmented reality providing server
400: database

Claims (45)

A broadcast receiver that displays the received shopping broadcast on a screen;
A mobile terminal that scans the screen of the broadcast receiver with a camera and extracts product information from the scanned screen of the broadcast receiver; And,
Augmented reality in which the extracted product information and previously acquired customer information are matched to analyze the relationship, and based on the analyzed relationship, customer recommended product information is inquired, and the searched customer recommended product information is processed to be implemented in augmented reality. Including the provision server,
The mobile terminal,
When a product information search request is received, a camera is activated to scan a shopping broadcast screen of the broadcast receiver, product information is extracted from the scanned shopping broadcast screen and transmitted to the augmented reality providing server, and the augmented reality providing server A system for providing shopping information using augmented reality, characterized in that when receiving customer recommended product information processed with augmented reality, the customer recommended product information is displayed in augmented reality around a screen of a broadcast receiver scanned through the camera. The method of claim 1,
Further comprising a database for storing the customer information,
The augmented reality providing server,
A system for providing shopping information using augmented reality, characterized in that, by acquiring product purchase information and product inquiry information of the customer, analyzing the shopping tendency of the customer, and updating customer information stored in the database based on the analyzed shopping tendency. The method of claim 2, wherein the database,
A shopping information providing system using augmented reality, characterized in that it is an individual database of shopping broadcasters or an integrated database in which customer information collected from databases for each shopping broadcaster is integrated and stored. In the shopping information providing method of a shopping information providing system comprising a mobile terminal and an augmented reality providing server that are communicated with each other,
Scanning, by the mobile terminal, a shopping broadcast screen of a broadcast receiver through a camera, extracting product information from the scanned shopping broadcast screen, and transmitting it to the augmented reality providing server;
The augmented reality providing server analyzes the relationship between the extracted product information and the previously acquired customer information to search for customer recommended product information, and processes the searched customer recommended product information to be implemented in augmented reality, and the processing Transmitting the processed customer recommended product information to the mobile terminal; And,
And displaying, by the mobile terminal, the customer recommended product information processed with the augmented reality, and displaying the customer recommended product information as augmented reality around a screen of the broadcast receiver scanned through the camera. A method of providing shopping information in a shopping information providing system. The method of claim 4,
The augmented reality providing server, analyzing the shopping tendency of the customer by obtaining product purchase information and product inquiry information of the customer, and updating the customer information stored in the database based on the analyzed shopping tendency A method of providing shopping information in a shopping information providing system. In the mobile terminal,
A communication unit communicating with the augmented reality providing server;
A camera unit for scanning a shopping broadcast screen of the broadcast receiver;
A display unit for displaying customer recommended product information in augmented reality around a screen of the broadcast receiver scanned from the camera unit; And,
And a control unit for controlling the communication unit, the camera unit, and the display unit,
The control unit,
When a product information search request is received, the camera unit is activated to scan the shopping broadcast screen of the broadcast receiver, and product information is extracted from the scanned shopping broadcast screen, and the extracted product information is used in the augmented reality providing server. When receiving the customer recommendation product information processed into the augmented reality from the augmented reality providing server and controlling the communication unit to transmit to the augmented reality providing server, the customer recommended product information is transferred to the augmented reality around the screen of the broadcast receiver scanned through the camera unit. And controlling the display unit to display. The method of claim 6, wherein the control unit,
And automatically activating the camera unit when a user input for requesting product information search is received when activating the camera unit. The method of claim 6, wherein the control unit,
When scanning the shopping broadcast screen of the broadcast receiver, the screen of the broadcast receiver scanned through the camera unit is displayed on a display as a preview image. The method of claim 6, wherein the control unit,
And when scanning the shopping broadcast screen of the broadcast receiver, controlling the camera to scan the entire screen area of the broadcast receiver. The method of claim 6, wherein the control unit,
When extracting product information from the scanned shopping broadcast screen, a screen area is recognized from a broadcast receiver scanned through the camera unit, and product information currently being broadcasted for shopping is extracted from the recognized screen area. terminal. The method of claim 10, wherein the extracted product information,
And at least one of a product image, a product name, and a product logo displayed on the screen of the broadcast receiver. The method of claim 10, wherein the control unit,
And when product information is extracted from the scanned shopping broadcast screen, information on a shopping broadcaster currently being broadcasted for shopping within the recognized screen area is further extracted. The method of claim 12, wherein the extracted shopping broadcaster information,
And at least one of a shopping broadcast channel logo, a shopping broadcaster logo, and a shopping broadcaster name displayed on the screen of the broadcast receiver. The method of claim 6, wherein the control unit,
And when displaying the customer recommended product information in augmented reality, the augmented reality providing server controls the display unit to arrange and display the customer recommended product information according to a preset position value. The method of claim 6, wherein the control unit,
When displaying the customer recommended product information in augmented reality, the display unit is controlled to display the customer recommended product information at least one of top, bottom, left, and right with respect to the screen of the broadcast receiver scanned through the camera unit. Mobile terminal. The method of claim 6, wherein the control unit,
When displaying the customer-recommended product information in augmented reality, if the customer-recommended product information includes an augmented reality icon for viewing information on the customer-recommended product and an augmented reality image of the customer-recommended product, viewing the information augmented reality icon and augmented And controlling the display unit to display real images not overlapping each other. The method of claim 6, wherein the control unit,
When the customer recommended product information is displayed in augmented reality, if the customer recommended product information includes an information view augmented reality icon on the customer recommended product, the information view augmented reality icon is a screen of a broadcast receiver scanned through the camera unit And controlling the display unit so as not to overlap with each other. The method of claim 6, wherein the control unit,
When displaying the customer-recommended product information in augmented reality, the first customer-recommended product information with a large amount of information and the second customer-recommended product information with a small amount of information among the customer recommended product information are placed at different locations around the screen of the broadcast receiver. And controlling the display unit to display. The method of claim 6, wherein the control unit,
When displaying the customer recommended product information in augmented reality, when a part of the body is scanned through the camera unit, the display unit is controlled to display the customer recommended product displayed in the augmented reality by fitting to the body part. Mobile terminal. The method of claim 6, wherein the control unit,
When displaying the customer-recommended product information in augmented reality, when real-time customer response information for the product on the shopping broadcast screen is obtained in addition to the customer-recommended product information from the augmented reality providing server, the real-time customer response information is displayed in augmented reality. The mobile terminal, characterized in that to control the display unit so as to. The method of claim 20, wherein the real-time customer response information,
A mobile terminal comprising real-time comment and review information on the product. In the method of providing shopping information of a mobile terminal that is communicatively connected to an augmented reality providing server,
Scanning a screen of a broadcast receiver displaying a shopping broadcast by activating a camera when a product information search request is received;
Extracting product information from the scanned shopping broadcast screen;
Transmitting the extracted product information to the augmented reality providing server; And,
When receiving the customer recommended product information processed into the augmented reality from the augmented reality providing server, displaying the customer recommended product information in augmented reality around a screen of a broadcast receiver scanned through the camera. A method of providing shopping information of a mobile terminal. The method of claim 22, wherein activating the camera comprises:
And automatically activating the camera when a user input requesting the product information search is received. The method of claim 22, wherein the scanning of the shopping broadcast screen of the broadcast receiver comprises:
A method of providing shopping information of a mobile terminal, comprising scanning an entire screen area of the broadcast receiver. The method of claim 22, wherein extracting product information from the scanned shopping broadcast screen comprises:
Recognizing a screen area from a broadcast receiver scanned through the camera, and extracting product information currently being broadcasted for shopping within the recognized screen area. The method of claim 25, wherein the extracted product information,
And at least one of a product image, a product name, and a product logo displayed on the screen of the broadcast receiver. The method of claim 22, wherein extracting product information from the scanned shopping broadcast screen comprises:
And further extracting information on a shopping broadcaster currently being broadcasted for shopping within the recognized screen area. The method of claim 27, wherein the extracted shopping broadcaster information,
And at least one of a shopping broadcasting channel logo, a shopping broadcasting company logo, and a shopping broadcasting company name displayed on the screen of the broadcasting receiver. The method of claim 22, wherein the displaying of the customer recommended product information in augmented reality comprises:
The method of providing shopping information of a mobile terminal, characterized in that the augmented reality providing server arranges and displays the customer recommended product information according to a preset location value. The method of claim 22, wherein the displaying of the customer recommended product information in augmented reality comprises:
The method of providing shopping information of a mobile terminal, wherein the customer recommended product information is displayed in at least one of top, bottom, left, and right with respect to a screen of a broadcast receiver scanned through the camera. The method of claim 22, wherein the displaying of the customer recommended product information in augmented reality comprises:
If the customer recommended product information includes an information view augmented reality icon for the customer recommended product and an augmented reality image of the customer recommended product, the information view augmented reality icon and the augmented reality image are displayed so that they do not overlap with each other. A method of providing shopping information in a mobile terminal. The method of claim 22, wherein the displaying of the customer recommended product information in augmented reality comprises:
When the customer recommended product information includes an information viewing augmented reality icon for the customer recommended product, the information viewing augmented reality icon is displayed so as not to overlap the screen of the broadcast receiver scanned through the camera. How to provide shopping information. The method of claim 22, wherein the displaying of the customer recommended product information in augmented reality comprises:
A method of providing shopping information of a mobile terminal, characterized in that, among the customer recommended product information, first customer recommended product information with a large amount of information and second customer recommended product information with a small amount of information are displayed at different locations around the screen of the broadcast receiver. . The method of claim 22, wherein the displaying of the customer recommended product information in augmented reality comprises:
When a part of the body is scanned through the camera, the customer recommended product displayed in the augmented reality is fitted to and displayed on the part of the body. The method of claim 22, wherein the displaying of the customer recommended product information in augmented reality comprises:
The method of providing shopping information of a mobile terminal, comprising displaying the real-time customer response information in augmented reality when real-time customer response information for a product on the shopping broadcast screen is obtained in addition to the customer recommendation product information from the augmented reality providing server. In the augmented reality providing server,
A communication unit communicatively connected to the mobile terminal;
An augmented reality processing unit that implements customer recommended product information into augmented reality; And,
And a control unit for controlling the communication unit and the augmented reality processing unit,
The control unit,
When product information extracted from the screen of a broadcast receiver displaying a shopping broadcast is received from the mobile terminal, the received product information is matched with previously acquired customer information to analyze the correlation, and customer recommended product information based on the analyzed correlation And controlling the augmented reality processing unit so that the inquired customer recommendation product information is implemented as augmented reality, and controlling the communication unit to transmit the processed customer recommendation product information to the mobile terminal. Augmented Reality Providing Server. The method of claim 36, wherein the control unit,
When analyzing the correlation by matching the received product information and previously acquired customer information, classifying product categories based on the received product information, obtaining customer information from a database based on user login information, and An augmented reality providing server, characterized in that by matching product categories and analyzing a relationship between a customer shopping tendency and a corresponding product. The method of claim 37, wherein the database,
An augmented reality providing server, characterized in that it is an individual database of a shopping broadcasting company server or an integrated database in which customer information collected from a database of a plurality of shopping broadcasting company servers is integrated and stored. The method of claim 36, wherein the control unit,
When retrieving the customer recommended product information, a main item related to the product is selected based on the relationship between the customer shopping tendency and the product currently being broadcasted, and other sub-items having a similarity with the selected item greater than or equal to a reference value are additionally selected. And augmented reality providing server, characterized in that for inquiring customer recommended product information for the selected main item and sub-item. The method of claim 36, wherein the control unit,
When transmitting the processed customer recommendation product information to the mobile terminal, the product purchase information and product inquiry information of the customer are obtained to analyze the shopping tendency of the customer, and the analyzed shopping tendency related update information is transmitted to the database. Augmented reality providing server, characterized in that controlling the communication unit. In the method of providing shopping information of an augmented reality providing server that is communicatively connected to a mobile terminal,
Receiving product information extracted from a screen of a broadcast receiver displaying a shopping broadcast from the mobile terminal;
Analyzing the correlation by matching the received product information with previously acquired customer information;
Inquiring customer recommended product information based on the analyzed correlation;
Processing the inquired customer recommendation product information to be displayed in augmented reality on the screen of the mobile terminal; And,
And transmitting the customer recommendation product information processed with the augmented reality to the mobile terminal. The method of claim 41, wherein analyzing the association by matching the received product information with previously acquired customer information,
Categorizing product categories based on the extracted product information, obtaining customer information from a database based on user login information, and analyzing a relationship between customer shopping tendency and the product by matching the customer information and product category. A method of providing shopping information in an augmented reality providing server. The method of claim 41, wherein the step of inquiring the customer recommended product information,
Based on the relationship between the customer's shopping tendency and the product currently being broadcasted, the product-related main item is selected, other sub-items having a similarity with the selected item equal to or greater than the reference value are additionally selected, and the selected main item and sub-item are Shopping information providing method of an augmented reality providing server, characterized in that inquiring about customer recommendation product information. The method of claim 41, wherein transmitting the processed customer recommended product information to the mobile terminal,
A method of providing shopping information of an augmented reality providing server, characterized in that, by acquiring product purchase information and product inquiry information of a customer, analyzing the shopping tendency of the customer, and transmitting the analyzed update information related to the shopping tendency to a database. A computer-readable recording medium storing a program for executing the method according to any one of claims 4 to 5, 22 to 35, and 41 to 44.

Images