WO2017122909A1

WO2017122909A1 - Terminal and control method therefor

Info

Publication number: WO2017122909A1
Application number: PCT/KR2016/011641
Authority: WO
Inventors: 김영덕
Original assignee: 주식회사 팝스라인
Priority date: 2016-01-14
Filing date: 2016-10-17
Publication date: 2017-07-20

Abstract

A terminal and a control method therefor are disclosed. That is, according to the present invention, objects provided to an offline store or a street store can be photographed, and product and service information, additional information, and the like related to the photographed objects are provided through an augmented reality function so as to transfer a product provided offline and additional information related to a store while providing the visual scene encountered in a real department store, shopping mall, street, and the like, thereby improving use convenience, enabling various kinds of information to be provided to a user, and enabling the same product assortments and shopping information as those of a superstore to be provided to the user of a purchasing terminal even though the user visits a small offline store.

Description

Terminal and its control method

The present invention relates to a terminal and a control method thereof, and in particular, a terminal for photographing an object provided in an offline store or a street store, and providing goods and service information and additional information related to the photographed object through an augmented reality function; It relates to a control method thereof.

In general, a mobile terminal is a device that performs a Global Positioning System (GPS) function and a call function (or a call function) and provides a result of the execution to a user.

Such mobile terminals are capable of transmitting and receiving more information in a shorter time as generations and generations are added, and various functions are added, but there is a lack of user interface (UI) development for user-centered convenience.

It is an object of the present invention to provide a terminal and a control method for photographing an object provided in an offline store or a street store and providing goods and service information, additional information, and the like related to the photographed object through an augmented reality function. .

Another object of the present invention, in addition to providing product information and / or service information related to the object photographed at the purchase terminal is a hand-carry product indicating that the product can be directly received or shipped, or offline stores or street stores It provides a terminal and a control method for providing additional information through the augmented reality function whether it is a shipping product that is not handled or is a product that is sold out of stock due to exhaustion of sales.

Yet another object of the present invention is one related to the category within the foreground view when any one of a plurality of categories displayed on one side of the foreground view is selected in the foreground view displayed through virtual reality or augmented reality. A terminal and a control method thereof are further provided to display additional information corresponding to the above object on the foreground view and to switch to a related 3D panoramic image when the object is selected.

Yet another object of the present invention is one related to the category within the foreground view when any one of a plurality of categories displayed on one side of the foreground view is selected in the foreground view displayed through virtual reality or augmented reality. Additionally displaying additional information corresponding to the above object on the foreground view, and if the additional information is selected, repeatedly converting the image information related to the object into an image of interest for each additional display on one side of the foreground view It is to provide a terminal and a control method thereof.

Another object of the present invention is to shoot a panoramic image of the 2D form, the terminal recognizes the object in the photographed panorama image, and inserts an additional object of the 3D form associated with the recognized object to generate an augmented reality image and It is to provide a control method thereof.

Another object of the present invention is to provide a terminal for displaying real-time additional information related to the additional object and a control method thereof when an additional object of 3D type inserted in the augmented reality image is selected.

Still another object of the present invention is to provide a terminal for providing a real-time panoramic image related to the selected image and a control method thereof when any one of a plurality of images is selected.

Another object of the present invention is to detect a preset event for a place name and / or additional information corresponding to an object included in a panoramic image displayed through virtual reality or augmented reality. A terminal and a control method thereof for additionally displaying additional information in a specific region in the panoramic image.

The control method of the augmented reality-based shopping information providing system to which a purchase terminal is applied according to an embodiment of the present invention is a control method of the augmented reality-based shopping information providing system to which a purchase terminal is applied, by a purchase terminal to an offline store or a street store. Photographing the provided object; Transmitting, by the purchasing terminal, image information including the object and identification information of the purchasing terminal; Recognizing, by a server, an object included in the image information transmitted from the purchase terminal; Confirming, by the server, a code related to the recognized object among a plurality of pre-stored code for each object; Identifying, by the server, one or more goods and service information corresponding to the identified code among previously stored goods and service information for each code; Identifying, by the server, additional information applicable to the identified one or more goods and service information; Arranging, by the server, the one or more goods and service information in any one of a predetermined discount rate, an accumulation rate, a sales amount, a popularity, and a brand; And transmitting, by the server, a predetermined number of goods and service information among the sorted one or more goods and service information and the identified additional information to the purchase terminal, wherein the object is the offline store. Or among the point of purchase (POP), leaflet, banner advertisement, board advertising, poster advertising, billing, sign and price tag provided in the street store And at least one of the products and services provided by the offline store or the street store, and the additional information indicates that the corresponding product is a product that can be directly received or delivered among the at least one product and service information to be checked. Information indicating whether or not it is a hand carry product and not sold or sold in offline or street stores It includes any one of the information indicating whether or not the delivery product indicating that the product is out of stock due to exhaustion, the identification information of the purchase terminal, MDN (Mobile Directory Number), Mobile IP, Mobile MAC, Subscriber identity module: Subscriber identification module) may include at least one of the unique card information and the serial number.

Displaying, by the purchasing terminal, one or more goods and service information and additional information transmitted from the server on the photographed image information by applying an augmented reality function as an example related to the present invention; When a product corresponding to the hand carry product is selected from the one or more products and service information displayed on the purchase terminal, by linking with the purchase terminal, the server, and the payment server, the product corresponding to the selected hand carry product Performing a payment function for the payment amount; Receiving, by a store server, member information transmitted from the server, information on the hand carry goods, and identification information of the purchase terminal; And performing, by the store server, an authentication procedure for the user of the purchase terminal visited to the store, and providing the hand carry product to the user of the purchase terminal.

Displaying, by the purchasing terminal, one or more goods and service information and additional information transmitted from the server on the photographed image information by applying an augmented reality function as an example related to the present invention; When a product corresponding to a delivery product is selected from one or more products and service information displayed on the purchase terminal, a payment amount of the product corresponding to the selected delivery product by interworking with the purchase terminal, the server, and the payment server Performing a payment function for the; Receiving, by a store server, member information transmitted from the server, information on the hand carry goods, and identification information of the purchase terminal; And delivering, by the store server, the shipping goods to a shipping destination based on the shipping destination information included in the member information.

In the augmented reality-based shopping information providing system to which a purchase terminal is applied according to an embodiment of the present invention, in the augmented reality-based shopping information providing system to which a purchase terminal is applied, an object provided in an offline store or a street store is photographed and includes the object. A purchase terminal for transmitting the image information and identification information of the purchase terminal; And recognizing an object included in the image information, checking a code related to the recognized object among a plurality of pre-stored code for each object, and one or more goods corresponding to the identified code among pre-stored goods and service information. And confirming the service information, confirming additional information applicable to the identified one or more goods and service information, and providing one or more goods and service information corresponding to the identified code and the confirmed additional information to the purchase terminal. And a server for transmitting, wherein the object includes at least one of a POP, a flyer, a banner advertisement, a billboard advertisement, a poster advertisement, a signage, and a price tag provided in the offline store or the street store, and the offline store or the street. Products and services provided by the store are excluded, and the additional information is Information indicating whether the product is a hand carry product indicating that the product is directly received or delivered, and information indicating that the product is not sold in an offline store or a street store or is out of stock due to exhausted sales. It includes any one of the information indicating whether or not the delivery product, the identification information of the purchase terminal, may include at least one of MDN, mobile IP, mobile MAC, Sim card unique information and serial number.

As an example related to the present invention, the purchase terminal may display one or more goods and service information and additional information transmitted from the server on the photographed image information by applying an augmented reality function.

As an example related to the present invention, when a product corresponding to a hand carry product is selected from one or more products and service information displayed on the purchase terminal, the selected hand is linked with the purchase terminal, the server, and the payment server. Performing a payment function for a payment amount of a product corresponding to a carry product, receiving member information transmitted from the server, information about the hand carry product, and identification information of the purchase terminal, and visiting the store; After performing an authentication procedure for a user, the store may further include a store server for providing the hand carry goods to the user of the purchase terminal.

As an example related to the present invention, when a product corresponding to a delivery product is selected from one or more products and service information displayed on the purchase terminal, the selected delivery product by interworking with the purchase terminal, the server, and the payment server Performs a payment function for a payment amount of a product corresponding to the terminal, receives member information transmitted from the server, information about the delivery product, and identification information of the purchase terminal, and is based on the delivery address information included in the member information; It may further include a store server that delivers the delivery goods to a shipping destination.

A control method of a terminal according to an embodiment of the present invention, the control method of the terminal, comprising: displaying a screen of any one type of the aerial view, street view, road view and foreground view by the display unit; Identifying, by the controller, one or more objects related to the selected category from among a plurality of objects included in the screen when any one category among a plurality of categories included in the screen displayed on the display unit is selected; Confirming, by the controller, outline information of the identified object in the screen; And displaying, by the display unit, place names corresponding to the at least one identified object and additional information related to the at least one identified object on one side of the screen, respectively.

In accordance with another aspect of the present invention, a method of controlling a terminal includes: displaying an augmented reality image through a display unit; When the specific additional object included in the augmented reality image is selected, confirming additional information of any one of real-time additional information, related video, and coupon information related to the selected additional object through a control unit; And controlling, by the control unit, the display unit to display the identified additional information on a specific area within the augmented reality image of the display unit, wherein the augmented reality image is included in the image in a 2D image. It may be a state that includes an additional object of the three-dimensional form associated with the object.

A control method of a terminal according to an embodiment of the present invention, the method of controlling a terminal, comprising: photographing a two-dimensional image through a camera; Displaying the photographed image through a display unit; Recognizing one or more objects included in the captured image through a control unit; Identifying, via the control unit, an additional object having a three-dimensional shape related to the recognized object; Generating an augmented reality image by inserting the identified additional object in the 3D form into the photographed 2D form image through the control unit; And displaying the generated augmented reality image through the display unit.

A control method of a terminal according to an embodiment of the present invention, the method of controlling a terminal, comprising: receiving, by a communication unit, one or more images provided from a server; Displaying, by a display unit, the one or more images received; Confirming, by a controller, whether an app for displaying an augmented reality image related to the selected image is installed in a terminal including the communication unit when a specific image is selected from one or more images displayed on the display unit; And installing the app on the terminal by interworking with the server when the app for displaying an augmented reality image related to the selected image is not installed on the terminal as a result of the checking. It may include.

A terminal according to an embodiment of the present invention, the terminal, the display unit for displaying a screen of any one type of aerial view, street view, road view and foreground view; And when any one of a plurality of categories included in a screen displayed on the display unit is selected, identify one or more objects related to the selected category among a plurality of objects included in the screen, and identify the one or more objects in the screen. And a control unit which checks the outline information of the identified object and controls the display unit to display name information corresponding to the identified one or more objects and additional information related to the identified one or more objects, respectively, on one side of the screen. have.

According to an embodiment of the present invention, a terminal includes: a camera for photographing an image having a dimensional form; A display unit which displays the captured image; And recognize one or more objects included in the photographed image, identify additional objects of three-dimensional shape related to the recognized object, and identify the additional objects of the identified three-dimensional shape in the photographed two-dimensional image. The control unit may include a controller to generate an augmented reality image by inserting the control unit and to control the display unit to display the generated augmented reality image.

A terminal according to an embodiment of the present invention, the terminal, a display unit for displaying an augmented reality image; And when a specific additional object included in the augmented reality image is selected, confirms additional information of any one of real-time additional information, related video, and coupon information related to the selected additional object, and checks a specific area in the augmented reality image of the display unit. And a controller configured to control the display unit to display the identified additional information, wherein the augmented reality image is a state in which an additional object having a 3D shape related to an object included in the image is included in a 2D image. Can be.

According to an embodiment of the present invention, a terminal includes: a communication unit configured to receive one or more images provided from a server; A display unit displaying the received one or more images; And checking whether an app for displaying an augmented reality image related to the selected image is installed in a terminal including the communication unit when a specific image is selected from one or more images displayed on the display unit. When the app for displaying the augmented reality image associated with the selected image is not installed in the terminal, it may include a control unit for installing the app on the terminal by interworking with the server.

The present invention captures an object provided in an offline store or a street store, and provides goods and service information, additional information, and the like related to the photographed object through an augmented reality function, so as to view a view encountered in an actual department store, a shopping mall, or a street. It can be provided as it is, while providing additional information related to products or stores that are provided offline, improving convenience for use, and providing a variety of information to users, even if a user of a purchase terminal visits a small offline store Assortment and shopping information can be provided the same.

In addition, the present invention, in addition to providing product information and / or service information related to the object photographed at the purchase terminal in addition to the hand carry goods indicating that the product can be directly received or shipped, or in the offline store or street store Augmented reality provides additional information on whether the product is a product that is not handled or is out of stock due to exhaustion of sales. Through augmented reality, it is possible to further check the product information of domestic and international offline stores to expand the choice of buyers. In addition, there is an effect that can improve the reliability between buyers and sellers when purchasing directly from overseas.

In addition, the present invention, when any one of a plurality of categories displayed on one side of the foreground view is selected in the foreground view displayed through the virtual reality or augmented reality, one or more objects related to the category in the foreground view Additional information corresponding to the display is further displayed on the foreground view, and when the object is selected, by switching to a related 3D panoramic image, it is easy to switch the screen to the screen desired by the user in virtual reality or augmented reality. There is an effect that can improve the convenience.

In addition, the present invention, when any one of a plurality of categories displayed on one side of the foreground view is selected in the foreground view displayed through the virtual reality or augmented reality, one or more objects related to the category in the foreground view And additionally display additional information corresponding to the on the foreground view, and when the additional information is selected, additionally display image information related to the object on one side of the foreground view, thereby providing a user interface for user convenience. In addition, the user's convenience can be improved by providing additional information related to an object of interest to the user on the current screen.

In addition, the present invention captures a panorama image of the 2D form, recognizes the object in the photographed panorama image, and inserts an additional object of the 3D form associated with the recognized object to generate an augmented reality image, There is an effect that can facilitate the provision of the associated additional information.

In addition, when the additional object of the 3D shape inserted in the augmented reality image is selected, the present invention displays a real-time additional information related to the additional object, thereby providing a user interface for user convenience, and the user is interested in the current screen. The user's convenience can be improved by providing additional information related to the visible object.

In addition, when any one of a plurality of images is selected, the present invention provides a real-time panoramic image related to the selected image, thereby increasing the effect of promoting the goods and / / services related to the image, and the goods and / services of interest There is an effect that can easily check the real-time information on.

In addition, the present invention, if a preset event for the name and / or additional information corresponding to the object included in the panoramic image displayed through the virtual reality or augmented reality is detected, additional information related to the name and / or additional information Is displayed in a specific area within the panoramic image, thereby providing a user interface for user convenience and improving user convenience by providing additional information related to an object of interest to the user on the current screen. have.

1 is a block diagram showing the configuration of a shopping information providing system to which a short horse is applied according to an embodiment of the present invention.

2 is a view showing an example of an exhibition product according to an embodiment of the present invention.

3 is a diagram illustrating an example of a point of purchase (POP) according to an embodiment of the present invention.

4 is a view showing an example of a shear according to an embodiment of the present invention.

5 is a diagram illustrating an example of a banner advertisement according to an embodiment of the present invention.

6 is a diagram illustrating an example of a poster advertisement according to an embodiment of the present invention.

7 is a view showing an example of a signboard according to an embodiment of the present invention.

8 is a flowchart illustrating a control method of the augmented reality-based shopping information providing system to which the terminal according to the first embodiment of the present invention is applied.

9 to 10 are diagrams illustrating a screen of a purchase terminal according to the first embodiment of the present invention.

11 is a block diagram illustrating a configuration of a guide apparatus to which a terminal is applied according to another embodiment of the present invention.

12 is a block diagram showing the configuration of a terminal according to an embodiment of the present invention.

FIG. 13 is a flowchart illustrating a guide method for repeatedly switching to an image of interest for each purpose to which a terminal is applied according to a second embodiment of the present invention.

14 to 18 illustrate screens of a terminal according to a second embodiment of the present invention.

19 is a flowchart illustrating a control method of a terminal according to the third embodiment of the present invention.

20 to 23 are diagrams illustrating a screen of a terminal according to the third embodiment of the present invention.

24 is A flowchart illustrating a control method of a terminal according to the fourth embodiment of the present invention.

25 to 31 illustrate screens of a terminal according to a fourth embodiment of the present invention.

32 is a flowchart illustrating a method of controlling a terminal according to the fifth embodiment of the present invention.

33 to 40 are diagrams illustrating a screen of a terminal according to the fifth embodiment of the present invention.

It should be noted that the technical terms used in the present invention are merely used to describe specific embodiments, and are not intended to limit the present invention. In addition, the technical terms used in the present invention should be interpreted as meanings generally understood by those skilled in the art unless the present invention has a special meaning defined in the present invention, and is excessively comprehensive. It should not be interpreted in the sense of or in the sense of being excessively reduced. In addition, when a technical term used in the present invention is an incorrect technical term that does not accurately express the spirit of the present invention, it should be replaced with a technical term that can be understood by those skilled in the art. In addition, the general terms used in the present invention should be interpreted as defined in the dictionary or according to the context before and after, and should not be interpreted in an excessively reduced sense.

Also, the singular forms used in the present invention include plural forms unless the context clearly indicates otherwise. Terms such as “consisting of” or “comprising” in the present invention should not be construed as necessarily including all of the various components or steps described in the present invention, and some of the components or some steps may not be included. It should be construed that it may further include, or further include, additional components or steps.

In addition, terms including ordinal numbers such as first and second used in the present invention may be used to describe components, but the components should not be limited by the terms. The terms are used only to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, and the same or similar components will be given the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted.

In addition, in describing the present invention, when it is determined that the detailed description of the related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easily understanding the spirit of the present invention and should not be construed as limiting the spirit of the present invention by the accompanying drawings.

1 is a block diagram illustrating a configuration of a shopping information providing system 10 to which a terminal is applied according to an exemplary embodiment of the present invention.

As shown in FIG. 1, the shopping information providing system 10 to which the terminal is applied includes a purchase terminal 100, a server 200, a payment server 300, a sales terminal 400, and a store server 500. . Not all components of the shopping information providing system 10 shown in FIG. 1 are essential components, and the shopping information providing system 10 may be implemented by more components than those shown in FIG. 1. The shopping information providing system 10 may also be implemented by fewer components.

The purchase terminal 100 and / or the sales terminal 400 may be a smart phone, a portable terminal, a mobile terminal, a personal digital assistant (PDA), a PMP ( Portable Multimedia Player Terminals, Telematics Terminals, Navigation Terminals, Personal Computers, Notebook Computers, Slate PCs, Tablet PCs, Ultrabooks, Wearable Devices (Wearable device, such as smart watch, smart glass, head mounted display (HMD), etc.), Wibro terminal, IPTV (Internet Protocol Television) terminal, smart TV, It can be applied to various terminals such as a digital broadcasting terminal, an AVN terminal, an audio / video system, a flexible terminal, and the like. Here, the sales terminal 400 and / or the store server 500 may be a point of sales (POS) terminal provided in the store.

In addition, the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500, respectively, through any wire or wireless communication network or any component or A communication unit (not shown) may be included to communicate with at least one external terminal. Here, the wireless Internet technologies include Wireless LAN (WLAN), Wireless Broadband (Wibro), WiMAX (World Interoperability for Microwave Access: Wimax), HSDPA (High Speed Downlink Packet Access), IEEE 802.16, Long Term Evolution. Long Term Evolution (LTE), a Wireless Mobile Broadband Service (WMBS), and the like. In addition, short-range communication technologies include Bluetooth, Wi-Fi, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Adjacent. Near Field Communication (NFC), Ultrasonic Communication (Ultra Sound Communication (USC)), Visible Light Communication (VLC), Bluetooth Low Energy (BLE), and the like may be included. In addition, the wired communication technology may include power line communication (PLC), USB communication, Ethernet, serial communication, optical / coaxial cable, and the like.

In addition, the communication unit may mutually transmit information with an arbitrary terminal through a universal serial bus (USB).

In addition, the communication unit may be a technical standard or a communication method for mobile communication (for example, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV-DO ( Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (WCDMA), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced), the base station, the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500 and the like Send and receive signals.

In addition, the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500 may each include a storage (not shown). Here, the storage unit stores various user interfaces (UIs), graphical user interfaces (GUIs), and the like.

In addition, the storage unit stores data and programs necessary for operating the purchase terminal 100, the server 200, the payment server 300, the sale terminal 400 and the store server 500, respectively. .

That is, the storage unit includes a plurality of application programs (application programs or applications) that are driven by the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400, and the store server 500. (application)), data for the operation of the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500, instructions may be stored. . At least some of these applications may be downloaded from an external server via wireless communication. In addition, at least some of these applications are basic functions of the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400, and the store server 500 (eg, a telephone). Incoming, outgoing function, message reception, outgoing function) from the time of shipment from the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500 May exist in On the other hand, the application is stored in the storage unit, and are installed in the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500, respectively, To perform an operation (or function) of each of the purchase terminal 100, the server 200, the payment server 300, the sale terminal 400, and the store server 500 by a controller (not shown). Can be driven.

The storage unit may further include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory), Magnetic Memory, Magnetic Disk, Optical Disk, Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read -Only Memory) may include at least one storage medium. In addition, the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500 is a web storage for performing a storage function on the Internet (internet) or operate in connection with the web storage.

In addition, the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500 may each include a display (not shown). The display unit may include a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT LCD), an organic light emitting diode (OLED), and a flexible display. ), A 3D display, an e-ink display, and a light emitting diode (LED). In addition, the display unit may be a touch screen.

In addition, the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500 may each include a voice output unit (not shown). Here, the voice output unit outputs voice information included in a signal processed by a predetermined signal under the control of the controller. Here, the voice output unit may include a receiver, a speaker, a buzzer, and the like.

The voice output unit may output a guide voice generated by the controller.

In addition, the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500 may each include a control unit (not shown). Here, the controller executes overall control functions of the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400, and the store server 500, respectively.

In addition, the control unit of the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400 and the store server 500 by using the program and data stored in the storage unit. Perform overall control. The controller may include a RAM, a ROM, a CPU, a GPU, a bus, and the RAM, a ROM, a CPU, a GPU, and the like may be connected to each other through a bus. The CPU may access the storage unit to perform booting using an operating system stored in the storage unit, and perform various operations using various programs, contents, data, and the like stored in the storage unit.

The purchase terminal 100 communicates with the server 200, the payment server 300, the sale terminal 400, the store server 500, and the like.

In addition, the purchase terminal 100 performs a member registration procedure for the user of the purchase terminal 100 by interworking with the server 200.

In addition, when performing the membership registration process, the purchase terminal 100 must complete an authentication function through an identity verification means (for example, a mobile phone, a credit card, an i-pin, etc.) and a membership registration procedure for the server 200. Can be completed normally.

In addition, when an app preinstalled in the purchase terminal 100 is executed, the purchase terminal 100 displays an app execution result screen. Here, the app execution result screen is a member information item (or menu / button) for receiving or transmitting member information, a shooting item for capturing an image including a specific object, a setting item for setting a function of an app, and a purchase. My page items for managing the purchase history information for the goods or services purchased through the terminal 100, and the like. In this case, the purchase terminal 100 communicates with the server 200 associated with the app in the background state.

In addition, when a member information item is selected from an app execution result screen displayed on the purchase terminal 100, the purchase terminal 100 receives (or receives) member information corresponding to the selected member information item. Display the member information screen.

In addition, the purchase terminal 100 receives member information according to a user input (or user touch). Here, the member information includes a name, an address, a language used, shipping information, a type of shopping object (eg, an avatar type, an emoticon type, etc.).

In addition, the purchase terminal 100 transmits the received member information, identification information of the purchase terminal 100, and the like to the server 200. In this case, the identification information of the purchase terminal 100 includes a mobile directory number (MDN), a mobile IP, a mobile MAC, a subscriber identity module (SIM) card unique information, a serial number, and the like.

In addition, the purchase terminal 100 photographs an object provided (or installed / located) in an offline store or a street store (or a street store). Here, the object is a discount event code (for example, display goods, POP, flyers, banner ads, board advertising, poster advertising, signboards, etc.) provided in each store located in the offline or street Code (including price tags, etc.). At this time, the object in the present invention may be in a state excluding the products and services provided by each store.

At this time, as shown in Figure 2, the display products (Display products) represents a non-sale goods for display exhibited for the sale of goods to customers (for example, clothes on a mannequin, bags that stand out on the display stand, etc. ).

In addition, as illustrated in FIG. 3, the point of purchase (POP) represents an advertisement that provides information on goods and / or services on the spot to customers who visit the store.

In addition, as shown in Figure 4, the leaflet (or leaflet / leaflet / leaflet advertising) represents the form of a simple printed matter for distribution for advertising or propaganda purposes.

In addition, as shown in FIG. 5, the banner advertisement represents an advertisement installed at a specific location of a store to provide information on a specific product and / or service.

In addition, as shown in Figure 6, the poster (billing) represents the paper produced and pasted for promotional purposes, and is produced by considering letters, figures, sentences, etc. to attract attention because it is appealing to the time.

In addition, as shown in FIG. 7, the sign indicates a plate displaying a company name, a store name, a product name, a service name, and the like associated with a store.

That is, the purchase terminal 100 captures an image including an object provided in the offline store or a street store through a photographing unit (not shown) provided in the purchase terminal 100. In this case, the purchase terminal 100 may capture an image including an object included in the offline store or a street store through a shooting menu included in an app execution result screen displayed on the purchase terminal 100.

In addition, the purchase terminal 100 includes a GPS receiver (not shown). Here, the GPS receiver receives the GPS signal transmitted from the satellite and generates the location data of the purchase terminal 100 in real time based on the longitude coordinates and latitude coordinates included in the received GPS signal. Here, the generated location data is defined as the current location (or current location data) of the purchase terminal 100. Here, the position information may be received through Wi-Fi or Wi-Bro communication as well as the GPS receiver.

In addition, the signal received through the GPS receiver is a standard of the wireless network for wireless LAN, including wireless LAN and some infrared communication proposed by the Institute of Electrical and Electronics Engineers (IEEE), Bluetooth, UWB, Zigbee 802.15, the standard for wireless personal area networks (PANs), and wireless metropolitan area networks (MANs) including broadband wireless access (FWA) and broadband wireless access (BWA) The location information of the terminal by using a wireless communication method such as 802.16, which is a standard standard for the standard), and 802.20, which is a standard standard for the mobile Internet, for a wireless mobile broadband (MBWA) including WiBro, WiMAX, etc. In addition, the signal received through the GPS receiver is a wireless LAN and work proposed by the Institute of Electrical and Electronics Engineers (IEEE). 802.11, the standard for wireless networks for wireless LANs, including infrared communications, 802.15, the standard for wireless personal area networks (PANs), including Bluetooth, UWB, ZigBee, and fixed wireless access. : Wireless Standards for Wireless Metropolitan Area Network (MAN) including FWA, etc., 802.16 which is a standard for Broadband Wireless Access (BWA), and Wireless MAN (Mobile Broadband Wireless Access (MBWA) including WiBro, WiMAX, etc.) The location information of the terminal may be provided to the purchase terminal 100 using a wireless communication method such as 802.20, which is a standard for the mobile Internet. The terminal 100 may be configured to be provided to the terminal 100.

As such, the purchase terminal 100 may check the location information of the purchase terminal 100 (or the location information of the point where the object is photographed).

In addition, the purchase terminal 100 transmits the photographed image information, the location information of the confirmed purchase terminal 100, identification information of the purchase terminal 100, and the like to the server 200. Here, the identification information of the purchase terminal 100 includes MDN, Mobile IP, Mobile MAC, Sim (Subscriber Identification Module) card unique information, serial number and the like.

In addition, the purchase terminal 100 receives one or more goods and service information related to the corresponding object included in the image information transmitted from the server 200 in response to the transmitted image information and the like, additional information, and the like. Here, the product and service information may be a product name (or service name), a product unique code (or service unique code), a product image (or service basic image), product description information (or service description information), and experience with a product / service. It includes information, evaluation information on goods / services, appointment information, counseling information, and medical time information. Further, the additional information may include information indicating that the product is a hand carry product indicating that the product can be directly received or delivered, information indicating that the product is a shipping product that is not handled in an offline store or a street store or is out of stock due to exhausted sales. Include.

In addition, the purchase terminal 100 displays the received at least one product and service information, the additional information, etc. on the previously photographed image information displayed on the purchase terminal 100 by applying an augmented reality function. In this case, when the purchase terminal 100 receives a plurality of goods and service information arranged in any one of a discount rate order, earning rate order, sales order, popularity order and brand order transmitted from the server 200, the purchase terminal 100 may display a plurality of goods and services information arranged in any one of the received discount rate, earning rate, sales volume, popularity, and brand. In this case, the purchase terminal 100 may apply a corresponding augmented reality function by a variety of augmented reality functions known in advance.

In addition, when receiving a plurality of unaligned goods and services information, the purchase terminal 100 receives the plurality of goods and services information received in the order of the predetermined discount rate, earning rate, sales volume, popularity order and brand order One or more of the sorted goods and services may be displayed by applying an augmented reality function. In this case, the purchase terminal 100 may display only a predetermined number of the plurality of sorted goods and service information.

In addition to the one or more goods and service information related to the corresponding object included in the image information, and the additional information, the purchase terminal 100 may include one or more related to the corresponding object included in the image information transmitted from the server 200. Additional additional information including coupon, gift certificates, service reservation information, etc. applicable to similar group goods and service information, the one or more goods and service information and / or the one or more similar group goods and service information, and the like. Information and / or a plurality of 3D panoramic image information related to the one or more similar group goods and service information may be received together with the one or more goods and service information and additional information. Here, the 3D panoramic image information may be included in the image information of the actual offline situation, the store, display products, point of purchase (POP), leaflet, banner banner included in the image information of the offline situation. One or more codes corresponding to objects including advertisements, board advertisements, poster advertisements, billings, signs, price tags, etc. may be added. In addition, the object is defined as a state excluding the products and services provided by each store included in the image information of the offline situation. In addition, the code corresponding to the object is in the form of a barcode, QR code, string code, and the like, and includes one or more product information, one or more service information, and the like related to the object. In addition, the 3D panoramic image information is the latest information updated in real time by the linkage between the purchase terminal 100 and the server 200, it is possible to maintain the latest 3D panoramic image information.

In addition, the purchase terminal 100, the received one or more goods and service information, one or more similar group goods and service information, the additional information, on the previously photographed image information displayed on the purchase terminal 100 Additional additional information including coupons, gift certificates, service reservation information, etc. applicable to one or more goods and service information and / or the one or more similar group goods and service information may be displayed by applying an augmented reality function, and the corresponding purchase terminal ( A plurality of 3D panoramic image information related to the one or more goods and service information and / or the one or more similar group goods and service information is displayed on one side of the previously photographed image information displayed on 100). In this case, the 3D panoramic image information, the additional information, and the additional additional information displayed on the purchase terminal 100 may be the latest information updated through the background modeling in the server 200.

That is, when the 3D panoramic image information is updated, the purchase terminal 100 receives the latest updated 3D panoramic image information from the server 200 and replaces the received 3D panoramic image information previously displayed. The latest 3D panoramic image information is changed and displayed.

As such, the purchase terminal 100 may display the latest 3D panoramic image information in real time by real-time interworking with the server 200.

In addition, the purchase terminal 100 displays the 3D panoramic image information while moving the corresponding shopping object in the 3D panoramic image information according to a user input based on the type of the shopping object in the member information.

In addition, specific product and service information may be included among one or more product and service information (and / or one or more similar group product and service information) displayed through the augmented reality function on the previously photographed image information displayed on the corresponding purchase terminal 100. When selected, the purchase terminal 100 receives 3D panoramic image information corresponding to the selected specific product and service information from the server 200, and then 3D panorama corresponding to the provided specific product and service information. Image information may also be displayed.

In addition, the purchase terminal 100 is based on at least one specific product and service information selected from one or more goods and service information displayed on the purchase terminal 100, the server 200, payment server 300 and The payment function is performed by interworking with the sales terminal 400.

In this case, the purchase terminal 100 is the server based on at least one product and service information displayed on the purchase terminal 100 and at least one specific product and service information selected from one or more similar group products and service information The payment function may be performed by interworking with the 200, the payment server 300, and the sales terminal 400.

Here, when a specific product corresponding to a specific product and service information selected by the purchase terminal 100 is a specific product that is a predetermined delivery product and corresponds to a foreign direct purchase, the server 200 makes the purchase. By interlocking with the terminal 100, the payment server 300, and the sales terminal 400, a payment function for an amount including a shipping fee, a tariff, etc. may be performed on a specific product corresponding to the delivery product. have.

In addition, the purchase terminal 100 transmits member information (or member information previously stored in the purchase terminal 100) according to a user input, identification information of the purchase terminal 100, and the like to the server 200. Here, the member information includes a name, an address, a language used, shipping information, a type of shopping object (eg, an avatar type, an emoticon type, etc.).

In this case, for a specific product corresponding to the specific product and the service information selected by the purchase terminal 100, the import customizable goods (for example, meat, goods, weapons, gas, etc.), which are preset (or registered) of the specific product , About, etc.), the purchase terminal 100 receives information (or payment function failure information) indicating that the payment function for the specific product and service information transmitted from the server 200 cannot be performed. Receive. Here, the information indicating that the payment function cannot be performed includes information indicating that a specific product corresponding to the specific product and service information corresponds to a product that cannot be imported.

In addition, the purchase terminal 100 displays information indicating that it is impossible to perform a payment function for the received specific product and service information.

In addition, a specific product corresponding to the specific product and the service information selected in the purchase terminal 100 is a preset (or registered) import customs clearance (including meat, goods, weapons, gas, medicine, etc.) If not applicable, the purchase terminal 100 is linked to the server 200, payment server 300 and sales terminal 400 by the specific product and / or specific service corresponding to the specific product and service information Perform a payment function for the payment amount corresponding to. In this case, when the payment function is performed, the purchase terminal 100 applies a preset discount amount to the normal price amount corresponding to the specific product and service information to perform a payment function for the amount to which the discount amount is applied from the normal price amount.

In addition, after the payment function is normally performed, the purchase terminal 100 receives a payment function execution result (or payment information) provided from the server 200 or the payment server 300. Here, the result of performing the payment function (or payment information) includes product and service information, payment date and time information, order information for each product / service, and the like.

In addition, the purchase terminal 100 displays a result of performing the received payment function.

As such, the purchase terminal 100 may search for goods and / or services, purchase goods and / or services, or receive the purchased goods without actually visiting an offline store, so that goods or The choice of services can be improved / expanded.

That is, when searching for and shopping in department stores or Dongdaemun shopping malls in South Korea through 3D panoramic images from China, the United States, etc., the shopping information of the same offline stores is collected, and the price of the interested product is compared and purchased (or paid). Since the delivery function can be performed by one stop shopping, the user's convenience can be improved.

In addition, it is possible to reduce the cost and time required to create an excessive product design page of the online shopping mall.

In addition, the purchase terminal 100 may be provided with the 3D panoramic image, goods and service information, etc. to which the corresponding language provided from the server 200 is applied according to the language used.

In addition, after the payment function is normally performed, the purchase terminal 100 receives a specific product or a specific service corresponding to the paid at least one specific product and service information provided from the server 200.

In addition, after the payment function is normally performed, when a predetermined shipping product is included among at least one specific product and service information of which payment is completed, the purchase terminal 100 corresponds to the delivery address information included in the corresponding member information. The delivery product is provided from the store server 500 of the store having the stock of the delivery product among the stores closest to the delivery address.

In addition, after the payment function is normally performed, when the preset hand carry product is included among the at least one specific product and service information, the purchase terminal 100 corresponds to the delivery address information included in the corresponding member information. Guidance message including guide information necessary for on-site receipt of a hand carry product provided from a store server 500 of a store having the stock of the delivery product among the stores closest to the delivery address, an identification code such as a barcode or a QR code form, etc. Receive

In addition, the purchase terminal 100 displays a guide message including the received guide information, an identification code such as a barcode or QR code form.

In addition, when a user of the purchase terminal 100 directly visits a store that provides a specific product or a specific service, the store server 500 identifies the barcode or QR code form stored in the purchase terminal 100. After authenticating (or recognizing) a code, the hand carry product may be provided to a user of the purchase terminal 100.

In addition, when a payment for issuing a multi-product discount coupon is performed after the payment function is normally performed, the purchase terminal 100 is provided to the specific product or the specific service issued (or transmitted / provided) from the server 200. Receive a corresponding multi-product discount coupon. Here, the multi-product discount coupon may include a normal price amount, a discount amount, an expiration date, a plurality of product and service item codes related to the specific product and service information (or the information related to the specific product and service information). A plurality of goods and service information / a plurality of goods and service information in the same category as the specific goods and service information), a unique number (or a unique number of a multi-product discount coupon), and the like. In addition, the multi-product discount coupon is a state in which a plurality of products and service item codes related to the specific product and service information are grouped into one code, and are generated in a barcode form, a QR code form, a string form, and the like. In addition, the plurality of products and service information may be a product name (or service name), a product unique code (or service unique code), a product image (or service basic image), product description information (or service description information), and product / service. Experience information about the product, the evaluation information on the product / service, the location information of the store that sells (or provides the service), the phone number of the store, and the like. In this case, the multi-product discount coupon in the form of a string may be generated by a combination of at least one of numbers, letters, special characters, and symbols. Here, the product item code corresponds to a product unique code (or service unique code) in the product information. In addition, the experience information on the product / service is a terminal information of the experience information created in the terminal of the member registered in advance in the server 200, the terminal purchased a multi-product discount coupon related to the product / service provided by the server 200 Experience information that is created.

In addition, the purchase terminal 100 receives a free discount coupon, a discount coupon, a gift certificate of a specific shopping mall, a gift certificate of a conventional market, etc., which are additionally provided from the server 200 according to the purchase of the multi-product discount coupon.

In addition, the purchase terminal 100 stores and displays the received multi-product discount coupon.

In addition, the purchase terminal 100 stores and displays the received free discount coupon, discount coupon, gift certificate of a specific shopping mall, gift certificate of a conventional market, and the like.

As such, when the multi-product discount coupon issuance is included in the delivery information, the purchase terminal 100 sets a preset sales quantity range (or a sales / payment amount range) for the multi-product discount coupon (for example, within 100 units). Alternatively, the multi-product discount coupon may be purchased within 20 million won).

In addition, as described above, by limiting the quantity or amount of the multi-product discount coupon that can be purchased in the purchase terminal 100, illegal transactions can be prevented.

In addition, the purchase terminal 100 receives a customs clearance sent from the server 200. The customs clearance list may include a product name for the at least one specific product, a unique identification information of the product, an HS code, a shipping fee, taxable fare information (including customs duties, VAT, special consumption tax, standard excess fee, education tax, special fishing village tax, state tax, etc.). And the like.

In addition, the purchase terminal 100 displays the received clearance list.

In addition, the purchase terminal 100, the at least one specific product and service, experience information, evaluation information, from the purchase of the specific product or the specific service to the arrival (or coupon issuance), the purchase terminal 100 The identification information and the like are transmitted to the server 200.

In addition, when the current location information of the purchase terminal 100 is located in an area (or an area within a specific country) where the multi-product discount coupon stored in the purchase terminal 100 can be used, the purchase terminal 100 corresponds to the corresponding location information. Displays one or more store information (or store location information) for using the multi-product discount coupon stored in the purchase terminal 100.

In addition, when specific store information is selected from one or more store information that may use the multi-product discount coupon stored in the corresponding purchase terminal 100 displayed on the purchase terminal 100, the purchase terminal 100 may include the purchase terminal ( Based on the current location information of the 100 and the selected specific store information (or location information of the specific store), the road guide information is generated from the current location of the purchase terminal 100 to the store corresponding to the specific store information. In this case, the purchase terminal 100 may identify (or generate) adjacent one or more recommended information within a predetermined distance from a path on the generated road guide information by interworking with the server 200. Here, the recommendation information may be store information for using the multi-product discount coupon. In this case, the road guide information may be generated in the form of 3D panoramic image information by interworking with the server 200.

In addition, the purchase terminal 100 displays the generated road guide information, the generated one or more recommended information, and the like.

As such, the purchase terminal 100 utilizes a 3D panoramic image, an avatar (or an emoticon), and guides the user to follow the avatar when visiting an offline store in virtual reality in consideration of the user's movement of the purchase terminal 100. By providing the information, it can induce interest and improve user convenience.

In addition, in the case where a sales terminal 400 is provided with a particular product and / or service in the store, and the user wants to use the multi-product discount coupon stored in the purchase terminal 100, the purchase terminal 100 is the server ( 200), the interworking between the payment server 300 and the sales terminal 400 performs a use process (or payment process) function for the multi-product discount coupon stored in the purchase terminal 100.

In addition, when the usage process for the multi-product discount coupon stored in the purchase terminal 100 is completed, the purchase terminal 100 is prohibited from using (or reusing) the multi-product discount coupon stored in the purchase terminal 100. Dispose of as much as possible.

In addition, after the payment function is normally performed, when there is a balance of the corresponding gift certificate, the purchase terminal 100 includes a multi-product discount coupon including an available amount (or balance information) transmitted from the server 200. Or outputs a multi-product discount coupon including a usable amount (or balance information) transmitted from the server 200 through printed materials such as a receipt provided from the sales terminal 400. In this case, the available amount of money may be the amount of money in a state of subtracting the amount of payment for performing the payment function from the amount of money available immediately before (or before).

In addition, after the payment function is normally performed, the purchase terminal 100 receives a preset point and / or a discount coupon transmitted from (or issued to) the sales terminal 400.

In addition, the purchase terminal 100 displays and stores the received preset point and / or discount coupon.

In addition, the purchase terminal 100 is based on the validity period included in the multi-product discount coupon, the corresponding validity period (for example, one month from the validity period, 15 days from the validity period, 7 days from the validity period, 3 days from the expiration date, including 1 day from the expiration date, and the amount (or balance) remaining in the gift certificate information indicates that the balance of the multi-product discount coupon remains, the expiration date, etc. Information to be included is displayed for each of the predetermined periods. In this case, information indicating that the balance of the multi-product discount coupon remains, information including an expiration date, etc. may be provided from the server 200.

In addition, the purchase terminal 100 receives experience information on a specific product / service, evaluation information on a specific product / service, and the like according to a user input of the corresponding purchase terminal 100.

In this case, the purchase terminal 100 is configured to receive the experience information, evaluation information, etc. within a predetermined number for a specific product / service, to prevent the provision of a large amount of experience information or evaluation information by the same user, the corresponding experience The reliability of information, evaluation information, etc. can be improved.

In addition, the purchase terminal 100 transmits experience information on the received product / service, evaluation information on the product / service, identification information of the purchase terminal 100, and the like to the server 200.

The server 200 communicates with the purchase terminal 100, the payment server 300, the sale terminal 400, the store server 500, and the like.

In addition, the server 200 performs a member registration procedure for a user of the purchase terminal 100 by interworking with the purchase terminal 100.

In addition, the server 200 performs a membership registration procedure for the user of the sales terminal 400 by interworking with the sales terminal 400.

In addition, the server 200 is one or more stores on the offline or street stores, goods in the store, exhibition products, POP, flyers, banner advertisements, bulletin board through a plurality of cameras (not shown) included in the server 200 The first image and the second image including the advertisement, the poster advertisement, the signboard, the price tag, and the like are photographed. Here, the store includes a department store, a shopping mall, an outlet, a restaurant, a hospital, a building (or a building), and the like. At this time, the server 200 checks the location information (including GPS location information, etc.) of the point at which the first image and the second image are photographed, and checks the checked location information with the first image. Matching (or linking) with the second image and storing the same.

In addition, the server 200 obtains an overlapped region in which the first image and the second image overlap with respect to the photographed first image and the second image based on the pixel orientation.

In addition, the server 200 generates a preliminary panoramic image by synthesizing the first image and the second image based on the obtained overlapped region. In this case, the server 200 may generate a preliminary panoramic image by synthesizing the first and second images photographed by various known synthesis methods. In this case, the generated preliminary panoramic image includes (or linked / matched) location information about a first object, a product, a point (or a coordinate) included in the preliminary panoramic image.

That is, the preliminary panoramic image is not only an image but also a corresponding preliminary search for (or checking) a 3D panoramic image corresponding to the location information (for example, the location information of the purchase terminal 100, etc.) of the terminal in the future. It is configured to include location information on the first object, goods, points, etc. included in the panoramic image.

In an embodiment of the present invention, the first and second images are synthesized based on the obtained overlapping area, and thus the preliminary panoramic image is generated. However, the present invention is not limited thereto. The preliminary panoramic image may be generated by synthesizing the captured first image and the second image by various known synthesis methods.

In this case, the server 200 is based on the first image and the second image photographed through a plurality of cameras (for example, a plurality of cameras configured for a special device for photographing a 3D panoramic image, a special vehicle, etc.). The preliminary panoramic image may be generated.

In addition, the server 200 recognizes one or more first objects included in the preliminary panoramic image, respectively. Here, the first object may be each store located on the offline store, an exhibition product, a POP, a leaflet, a banner advertisement, a bulletin board advertisement, a poster advertisement, a signboard, a price list which is located on the offline store or located in each store. And the like. At this time, the first object in the present invention may be in a state excluding the products and services provided by each store. In addition, the exhibition product, POP, leaflet, banner advertising, bulletin board advertising, poster advertising, signage, etc. may represent a discount event code, the price tag may represent a product code.

That is, the server 200 recognizes each of one or more stores included in the preliminary panoramic image, or recognizes POP, flyer, banner advertisement, bulletin board advertisement, poster advertisement, signboard, price list, etc. included in the preliminary panoramic image, respectively. do.

In addition, the server 200 adds (or inserts) a first code corresponding to each of the recognized one or more first objects to a partial area on the corresponding first object in the preliminary panoramic image, or advances from the first object. A 3D panoramic image is generated by adding (or inserting) adjacently within a set distance. Here, the first code is in the form of a bar code, a QR code, a string code, and the like, and includes one or more product information and one or more service information related to the recognized first object. In addition, the string code may be configured by a combination of letters, numbers, and symbols for each language. In this case, the product information includes a product name, a product unique code, a product image, product description information, experience information on the product, evaluation information on the product, location information of a store selling the corresponding product, a phone number of the store, and the like. The service information may include a service name, a service unique code, a service basic image, service description information, experience information on a service, evaluation information on a service, location information of a store that provides the service, a phone number of a store, and reservation information. Includes consultation information, medical time information, and more. In addition, the generated 3D panoramic image includes (or linked / matched) location information about a first object, a product, a point (or a coordinate) included in the 3D panoramic image.

In addition, the color representing the first code may be represented by various colors in addition to black. In this case, when the first code is a barcode, at least one color of a quiet zone and a pattern (or bar) may be set to various colors. In addition, when the first code is a QR code, a finding pattern, a timing pattern, an alignment pattern, a quiet zone, and a data area (including an error correction code area) constituting the QR code. At least one color of) may be set to various colors. In addition, when the first code is a string code, the color of the string may be set to various colors.

In addition, the color representing the first code may be represented by black or other various colors, and the background color of the first code may be represented by various colors besides white. For example, when the first code is a QR code, at least one color of a binding pattern, a timing pattern, an alignment pattern, and a data area constituting the QR code is displayed in yellow, and the background color (or QR) of the QR code is displayed. You can configure it by setting the quiet zone of the code to red.

As described above, the color or background color of the first code may be set to various colors, and the barcode may be easily identified in the 3D panoramic image.

In addition, the server 200 may generate the 3D panoramic image for each language in consideration of language-specific characteristics.

The 3D panoramic image generated according to an embodiment of the present invention includes (or linked / matches) location information on the first object, product, point (or coordinate) included in the 3D panoramic image.

That is, the 3D panoramic image is not only an image but also a corresponding 3D panoramic image so that a 3D panoramic image corresponding to an arbitrary terminal (for example, location information of the purchase terminal 100, etc.) can be searched (or confirmed) later. It is configured to include location information about the first object, goods, branches, and the like contained in the.

Accordingly, when the arbitrary terminal requests a 3D panoramic image corresponding to the location information, the server 200 includes a plurality of 3D panoramic image information including location information of one or more objects, goods, and points stored in advance. The 3D panoramic image information corresponding to the specific location information requested from the arbitrary terminal may be checked and the checked 3D panoramic image information may be provided to the corresponding terminal.

For example, the server 200 may include an object corresponding to 123-45 Yeoksam-ro, Gangnam-gu, Seoul, which is location information of the arbitrary terminal, from 3D panoramic image information linked to a plurality of objects and / or product location information. You can check the included 3D panoramic image information.

In addition, when there are no objects and / or goods corresponding to 123-45, Yeoksam-ro, Gangnam-gu, Seoul, which is the location information of the terminal, among 3D panoramic image information linked with the location information of the plurality of objects and / or goods, The server 200 may check 3D panoramic image information including objects and / or merchandise similar to location information of the terminal (including 123-44 or 123-46 or Yeoksam-ro, Gangnam-gu, Seoul, etc.). .

In addition, when the plurality of 3D panoramic image information is matched with GPS information configured at a predetermined interval, the server 200 is based on the location information of the arbitrary terminal and the GPS information configured at the predetermined interval based on the arbitrary One piece of 3D panoramic image information corresponding to (or similar / related to) location information of the terminal may be checked.

In addition, a method of checking 3D panoramic image information corresponding to specific location information among a plurality of 3D panoramic image information stored in advance in the server 200 may apply techniques utilized in a map search service.

As described above, the 3D panoramic image information generated by the present invention is linked to the position information, and among the plurality of 3D panoramic image information previously stored in the server 200 with respect to specific position information transmitted from any terminal. The 3D panoramic image information corresponding to the specific location information may be checked and the checked 3D panoramic image information may be provided to the corresponding terminal that has transmitted the specific location information.

In addition, the server 200 stores the recognized object (or unique identification information of the object) in association with the generated code.

In this case, the server 200 extends the outline of the generated first code to the outline of the recognized first object. That is, the server 200 matches and stores the coordinate value of the outline of the generated first code with the coordinate value of the outline of the recognized first object.

As such, since the outline of the first code is extended to the outline of the first object corresponding to the first code, the first code may be selected even when the first object is selected later.

In addition, the server 200 repeatedly performs the preceding steps at a predetermined period (including 10 seconds, 1 minute, etc.) to generate a new 3D panoramic image.

That is, the server 200 generates a new preliminary panoramic image by synthesizing the images taken in real time through the plurality of cameras at a predetermined period (including 10 seconds, 1 minute, etc.), and generates the new Recognizing one or more first objects included in the preliminary panoramic image, and generating a new 3D panoramic image by adding a first code corresponding to the recognized one or more first objects in the new preliminary panoramic image.

In this case, when a person is recognized in the generated new preliminary panoramic image, the server 200 acquires (or extracts) an outline of the person in the new preliminary panoramic image, and the new preliminary panoramic image. The new 3D panorama is removed by removing a recognized person and adding a first code corresponding to the recognized one or more first objects and an outline of the acquired (or extracted) person in the new preliminary panoramic image from which the person has been removed. Create an image. Here, the server 200 adds the contour of the acquired person to the position of the person in the new preliminary panoramic image.

As such, when a person is included in the generated new preliminary panoramic image, the server 200 maintains only the outline of the person with respect to the person included in the new preliminary panoramic image for privacy protection. The internal area may be processed to be an empty space or to be replaced with a screen of a previous preliminary panoramic image generated in a previous step.

In addition, the server 200 checks a difference between the generated new 3D panoramic image and the previously generated 3D panoramic image (or the generated new 3D panoramic image information and the previously generated 3D panoramic image information).

In addition, the server 200 maintains the latest 3D panoramic image by compensating the identified difference in real time to the previously generated 3D panoramic image.

That is, when the generated new 3D panoramic image and the previously generated 3D panoramic image are the same (or there is no difference), the server 200 may include one or more first objects included in the previously generated 3D panoramic image and And / or determines that there is no change in the state of the product, and maintains the 3D panoramic image generated earlier as the latest 3D panoramic image.

In addition, when there is a difference between the generated new 3D panoramic image and the previously generated 3D panoramic image, the server 200 compensates the changed portion (or the identified difference) to the previously generated 3D panoramic image. (Or reflecting), a 3D panoramic image compensated for the difference between the 3D panoramic image to be compared is generated as a new new 3D panoramic image.

In addition, when a person is included in the new 3D panoramic image (or a person is recognized in the new 3D panoramic image), the server 200 is included in the new 3D panoramic image in the previously generated 3D panoramic image. By adding a contour of a person, a 3D panoramic image including the contour-shaped person may be generated as the latest new 3D panoramic image.

As such, when a person is included in the generated new 3D panoramic image, the server 200 maintains only the outline of the person with respect to the person included in the new 3D panoramic image for privacy protection. The internal area may be processed to be an empty space or to be replaced with a screen of a previous 3D panoramic image generated in a previous step.

In addition, when a person is included in the newly generated 3D panoramic image, the server 200 contours the person included in the new 3D panoramic image, and the contoured person internal region is immediately generated in the previous step. As the screen of the 3D panoramic image is displayed, it is possible to prevent an object, a product, etc. in the 3D panoramic image from being hidden and hidden by the person.

In this case, the server 200 may compensate only the difference in the previously generated 3D panoramic image, and replace the generated new 3D panoramic image with the latest new 3D panoramic image instead of the previously generated 3D panoramic image. You may.

As such, the server 200 may maintain the latest 3D panoramic image in real time through background modeling.

In addition, the server 200 receives the image information transmitted from the purchase terminal 100, identification information of the purchase terminal 100, and the like. In this case, the server 200 may also receive location information and the like of the purchase terminal 100 transmitted from the purchase terminal 100.

In addition, the server 200 analyzes the received image information and recognizes an object included in the image information. Here, the object is a discount event code (for example, exhibition products, POPs, flyers, banner advertisements, billboard advertisements, poster advertisements, signs, etc.) provided in each store located in the offline or street, product code (for example For example, a price tag). At this time, the object in the present invention may be in a state excluding the products and services provided by each store. In addition, the identification information of the purchase terminal 100 includes MDN, Mobile IP, Mobile MAC, Sim (Subscriber Identification Module) card unique information, serial number and the like.

In addition, the server 200 checks a code corresponding to the recognized object among a plurality of object-specific codes (or codes for unique identification information of a plurality of objects) stored in advance.

In addition, the server 200 confirms one or more goods and service information corresponding to the identified code (or unique identification information of the code) among a plurality of previously stored product information and service information. At this time, the server 200 is a hand carry goods indicating that the product is directly received or delivered from among the one or more goods and service information (or one or more goods and service information corresponding to the identified code) to be confirmed or Check the additional information on whether the item is a shipping product that is not handled by an offline store or a street store or is sold out due to exhausted sales. Here, the product and service information may be a product name (or service name), a product unique code (or service unique code), a product image (or service basic image), product description information (or service description information), and experience with a product / service. It includes information, evaluation information on goods / services, appointment information, counseling information, and medical time information. In addition, the product information, such as red ginseng, ginseng, cosmetics, home appliances, fashion clothing, sundries, precious metals, dietary supplements, baby products, laver, kimchi, confectionery, food, golf, skiing, performances, sports events, etc. Covers the field. In addition, the service information includes areas such as jjimjilbang, hot spring, rental car, rental, beauty massage, health, plastic surgery, services, performances.

In addition, the server 200 confirms one or more similar group goods and service information related to the identified code among the pre-stored goods and service information for each code.

In addition, the server 200 confirms additional additional information including coupons, gift certificates, service reservation information, etc. applicable to the identified one or more goods and service information, the one or more similar group products and service information, and the like.

In addition, the server 200 checks 3D panoramic image information including the code among one or more previously stored objects, a code corresponding to the object, and a plurality of 3D panoramic image information including location information of the object. In this case, when there is a plurality of 3D panoramic image information including the code, the server 200 reflects the position information of the purchase terminal 100 previously received, corresponding to the one or more previously stored objects, the object. Among the plurality of 3D panoramic image information including a code and location information of the object, based on the code and location information of the purchase terminal 100 (or corresponding to the code and location information of the purchase terminal 100) 3D panoramic image information can also be checked.

In addition, the server 200 transmits to the purchase terminal 100 one or more goods and service information corresponding to the identified code, additional information indicating whether the identified hand carry goods or delivery goods.

In addition, the server 200 may include the identified 3D panoramic image information, at least one product and service information corresponding to the identified code, the at least one similar group product and service information, and the identified hand carry product or delivery. Additional information indicating whether or not the product, additional additional information, etc. may be transmitted to the purchase terminal 100. Here, the additional additional information includes coupons, gift certificates, service reservation information, etc. applicable to the one or more product and service information and / or the one or more similar group product and service information.

In this case, when the one or more goods and service information (or the one or more similar group goods and service information) are plural, the server 200 may provide a plurality of goods and service information (or a plurality of similar group goods and service information). The purchase terminal is arranged in any one of a predetermined discount rate order, earning rate order, sales amount order, popularity order and brand order, and only the preset number of the plurality of sorted goods and service information (or a plurality of similar group goods and service information) is preset. May transmit to 100.

As such, even when visiting a small offline store or a street store, an assortment of goods and shopping information of a large store can be collected in the same way, and a product of interest that a user of the purchase terminal 100 is interested in is sold out in one store. In this case, it may be purchased as a delivery product at another store where inventory exists.

In an embodiment of the present invention, the server 200 provides a 3D panoramic image information providing function related to an object included in the image information photographed by the purchasing terminal 100 in a one-to-one manner with the purchasing terminal 100, one or more products. And performing service information providing function, one or more similar group goods and service information (or one or more similar group goods and similar group service information), additional information providing function, additional additional information providing function, etc. The 3D panoramic image information providing function and the one or more product and service information providing functions by interworking with a friend terminal (not shown) invited by the purchasing terminal 100 and the purchasing terminal 100. Providing the one or more similar group products and service information (or one or more similar group products and similar group service information); The additional information providing function and the additional additional information providing function may be performed.

That is, the purchase terminal 100 transmits information about one or more friend terminals (including, for example, identification information of the friend terminal) and identification information of the purchase terminal 100 to the server 200. Thereafter, the server 200 transmits 3D panoramic image information related to a second object included in the image information photographed by the purchase terminal 100 to the purchase terminal 100 and the one or more friend terminals, the one or more products, and the like. Service information, the one or more similar group products and service information (or one or more similar group products and similar group service information) capability, the additional information, the additional additional information, and the like. In this case, the 3D panoramic image information displayed on the purchase terminal 100 and the one or more friend terminals are interlocked (or synchronized) so that the same 3D panoramic image information is displayed on the purchase terminal 100 and the one or more friend terminals. Different 3D panoramic image information may be displayed on the purchase terminal 100 and the one or more friend terminals, respectively.

As such, the server 200, the purchase terminal 100, and one or more friend terminals interoperate with each other to provide 3D panoramic image information related to the second object included in the image information photographed by the purchase terminal 100, One or more goods and service information providing functions, one or more similar group goods and service information (or one or more similar group goods and similar group service information) providing functions, additional information providing functions, additional additional information providing functions, and the like may be performed.

In addition, the purchase terminal 100 and one or more friend terminals may perform a text message service function, a voice service function, etc. by the relay function of the server 200.

In addition, when at least one specific product and service information is selected from among one or more goods and service information transmitted to the purchase terminal 100 and one or more similar group goods and service information, the server 200 may purchase the terminal. In operation 100, the payment server 300 performs a payment function for a payment amount corresponding to the selected at least one specific product and service information by interworking with the payment server 300 and the sales terminal 400.

That is, the server 200 applies a preset discount amount to a normal price amount corresponding to the at least one specific product and service information to perform a payment function for the amount to which the discount amount is applied from the normal price amount.

In this case, with respect to a specific product corresponding to the specific product and service information selected by the purchase terminal 100, when the specific product is a preset delivery product and a product corresponding to an overseas direct purchase, the server 200 purchases the purchase. By interlocking with the terminal 100, the payment server 300 and the sales terminal 400, the amount including the shipping fee, customs, etc. for the specific product corresponding to the delivery product (for example, the normal price amount In addition to the amount to which the discount is applied, a payment function for the amount of the shipping fee, the duty, etc. for the delivery may be added).

In addition, for a specific product corresponding to the specific product and service information selected by the purchase terminal 100, the import customizable goods (for example, meat, goods, weapons, gas, etc.), which the specific product is preset (or registered) , The drug, etc.), the server 200 transmits information (or payment function failure information) indicating that the payment function cannot be performed on the specific product and service information to the purchase terminal 100. . Here, the information indicating that the payment function cannot be performed includes information indicating that a specific product corresponding to the specific product and service information corresponds to a product that cannot be imported.

In addition, if the specific product corresponds to a preset (or registered) import customs clearance (including meat, goods, weapons, gas, drugs, etc.), the server 200 ends the payment function. .

In addition, after the payment function is normally completed, the server 200 provides the payment terminal 100 with the result of performing the payment function (or payment information). Here, the result of performing the payment function (or payment information) includes at least one product and service information, payment date and time information, price information per order product / service, and the like.

In addition, after the payment function is normally performed, if a predetermined shipping product is included among at least one specific product and service information of which payment has been completed, the server 200 receives member information transmitted from the purchase terminal 100. Based on the plurality of stores (or store information) stored in advance, the store having the stock of the delivery product among the stores closest to the delivery address corresponding to the delivery address information included in the member information is identified.

In addition, the server 200 transmits the member information, the information about the delivery product, the identification information of the purchase terminal 100, and the like to the confirmed store server 500 of the store.

In addition, after the payment function is normally performed, if the preset multi-product discount coupon issuance is included among the at least one payment-completed specific product and service information, the server 200 corresponds to the specific product or the specific service. A multi-product discount coupon is issued (or transmitted) to the purchase terminal 100.

Here, the multi-product discount coupon is generated by the server 200.

That is, after the payment function is normally performed, the server 200 may include a normal price amount, a discount amount, an expiration date, and a plurality of product and service item codes related to the specific product and service information. Or a plurality of product and service information related to the specific product and service information / a plurality of product and service information in the same category as the specific product and service information), a unique number (or a unique number of a multi-product discount coupon), and the like. Generate a multi-product discount coupon (or a multi-service discount coupon / multi-product and service discount coupon). Here, the multi-product discount coupon is a state in which a plurality of products and service item codes related to the specific product and service information are grouped into one code, and are generated in a barcode form, a QR code form, a string form, and the like. In addition, the plurality of products and service information may be a product name (or service name), a product unique code (or service unique code), a product image (or service basic image), product description information (or service description information), and product / service. Experience information about the product, the evaluation information on the product / service, the location information of the store that sells (or provides the service), the phone number of the store, and the like. In this case, the multi-product discount coupon in the form of a string may be generated by a combination of at least one of numbers, letters, special characters, and symbols. Here, the product item code corresponds to a product unique code (or service unique code) in the product information. In addition, the experience information on the product / service is a terminal information of the experience information created in the terminal of the member registered in advance in the server 200, the terminal purchased a multi-product discount coupon related to the product / service provided by the server 200 Experience information that is created.

In addition, the server 200 issues the generated multi-product discount coupon to the purchase terminal 100.

In addition, the server 200 may set in advance the quantity (or sales / payment amount) of the multi-product discount coupon that can be purchased from one purchase terminal 100.

That is, the server 200 corresponds to a specific product and service in one purchase terminal 100 within a preset selling quantity range (or selling / payment amount range) (for example, within 100 or less than 20 million won). It can be configured to purchase a multi-product discount coupon.

As such, the server 200 may limit the volume purchase of a particular product and service, thereby preventing fraudulent transactions.

In addition, after the payment function is normally performed, the server 200 purchases a free discount coupon, a discount coupon, a gift certificate of a specific shopping mall, a gift certificate of a conventional market, and the like, which are additionally provided according to the purchase of the multi-product discount coupon. Issuance (or transmission) to the terminal 100.

In addition, in the case where a sales terminal 400 is provided with a specific product and / or service in the store, and the user wants to use the multi-product discount coupon stored in the purchase terminal 100, the server 200 may be the purchase terminal ( 100), the interworking between the payment server 300 and the sales terminal 400 performs a use process (or payment processing) function for the multi-product discount coupon stored in the purchase terminal 100.

In addition, the server 200 processes (or converts / changes) a usage completion state for the corresponding multi-product discount coupon transmitted (or issued) to the purchase terminal 100.

In addition, after the payment function is normally performed, if there is a balance of the gift certificate, the server 200 transmits a multi-product discount coupon including an available amount (or balance information) to the purchase terminal 100. do. In this case, the available amount of money may be the amount of money in a state of subtracting the amount of payment for performing the payment function from the amount of money available immediately before (or before).

In addition, after the payment function is normally performed, if there is a balance of the corresponding gift certificate, the server 200 provides the purchase terminal 400 with information including an available amount (or balance information). May transmit to 100.

In addition, after the payment function is normally performed, the server 200 transmits a preset point and / or a discount coupon to the purchase terminal 100.

In addition, after the payment function is normally performed, the server 200 may transmit a preset point and / or a discount coupon to the purchase terminal 100 through the sale terminal 400.

In addition, according to the use of the multi-product discount coupon, a fee may be distributed between the server 200, the payment server 300, and the sales terminal 400.

In addition, the server 200 confirms (or generates) adjacent one or more recommended information within a predetermined distance from a path on the road guide information generated by the purchase terminal 100 by interworking with the purchase terminal 100. can do. Here, the recommendation information may be store information for using the multi-product discount coupon. In this case, the server 200 generates the road guide information based on the location information of the purchase terminal 100 and the store information selected by the purchase terminal 100 by interworking with the purchase terminal 100. It may be.

In addition, the server 200 transmits the confirmed one or more recommended information, the generated road guide information, and the like to the purchase terminal 100.

In addition, the validity period included in the multi-product discount coupon issued to the purchase terminal 100 is a predetermined period (for example, one month from the validity period, 15 days from the validity period, 7 days from the validity period, 3 days from the validity period) If the amount of money (or balance) remains within the gift certificate information, the server 200 displays information indicating that the balance of the multi-product discount coupon remains, validity period, etc. Sends the information including the purchase to the terminal (100).

In addition, the server 200 receives experience information on a specific product / service, evaluation information on a specific product / service, and the like transmitted from the purchase terminal 100.

In addition, the server 200 stores the received experience information on the specific product / service, evaluation information on the specific product / service, etc. in association with (or matching) the specific product / service.

As such, the server 200 sets (or specifies) delivery information (or delivery method) set (or designated) by the purchase terminal 100 for a specific product or a specific service corresponding to the at least one specific product and service information that has been paid. According to the specific product or a specific service can be provided to the purchase terminal (100).

In addition, the server 200 generates a clearance list for at least one specific product corresponding to the at least one specific product and service information on which the payment function is performed. The customs clearance list may include a product name for the at least one specific product, a unique identification information of the product, a Harmonized Commodity Description and Coding System (HS) code, a shipping fee, taxable fare information (tariff, VAT, special consumption tax, standard excess amount, education tax) , Special fishing tax, state tax, etc.).

In addition, the server 200 transmits the generated customs clearance list to the purchase terminal 100.

In addition, the server 200 may include experience information, evaluation information, identification information of the purchase terminal 100, etc., from purchase of the specific product or the specific service transmitted from the purchase terminal 100 to arrival (or coupon issuance). Receive

In addition, the server 200 matches and stores the received experience information, evaluation information, etc. with a corresponding specific product or specific service.

The payment server 300 communicates with the purchase terminal 100, the server 200, the sale terminal 400, the store server 500, and the like.

In addition, the payment server 300 is a payment for the specific goods and service information selected by the purchase terminal 100 by interworking with the purchase terminal 100, the server 200, the sales terminal 400, and the like. Perform the function. At this time, when the payment function is performed, the payment server 300 performs a payment function for the amount to which the discount amount is applied from the normal price amount by applying a preset discount amount to the normal price amount corresponding to the specific product and service information.

In addition, after the payment function is normally performed, the payment server 300 provides the purchase terminal 100 with a result of performing the payment function (or payment information). Here, the result of performing the payment function (or payment information) includes product and service information, payment date and time information, order information for each product / service, and the like.

In addition, in the case where a sales terminal 400 is provided with a specific product and / or service in the store, and the user wants to use the multi-product discount coupon stored in the purchase terminal 100, the payment server 300 is the purchase terminal In operation 100, the server 200 and the sales terminal 400 perform a use process (or payment process) function for the multi-product discount coupon stored in the purchase terminal 100.

The sales terminal 400 may be a point of sales (POS) terminal provided in a store.

In addition, the sales terminal 400 is provided (or installed) in any store.

In addition, the sales terminal 400 performs a membership registration procedure for the user of the sales terminal 400 by interworking with the server 200.

In addition, when performing the membership registration process, the sales terminal 400 must complete an authentication function through an identity verification means (for example, a mobile phone, a credit card, an i-pin, etc.) and a membership registration process for the server 200. Can be completed normally.

In addition, the sales terminal 400 communicates with the purchase terminal 100, the server 200, the payment server 300, the store server 500, and the like.

In addition, the sale terminal 400 is linked to the purchase terminal 100, the server 200, the payment server 300, etc., payment for the specific goods and service information selected in the purchase terminal 100 Perform the function.

In addition, the sales terminal 400 is the payment function performance result (or payment information) according to the payment function performance transmitted from the server 200 or the payment server 300, multi-issued from the server 200 Receive a sale quantity of a product discount coupon, and the like.

In addition, the sales terminal 400 may perform a product inventory management function in a store associated with the sales terminal 400 based on a result of the received payment function, the sales quantity of the multi-product discount coupon, and the like.

In addition, the sales terminal 400 applies a customer relation management (CRM) marketing based on the result of performing the received payment function, the sales quantity of the multi-product discount coupon, and the like, and the corresponding sales terminal 400. It can carry out marketing functions for products or services provided by stores.

As such, the sales terminal 400 may be a visitor visitor who visits a corresponding area where a store related to the sales terminal 400 is located for a short period of time without selling a product or service by itself, without a separate leaflet promotion or marketing activity. Through the multi-product discount coupons sold, it is possible to identify the expected sales and the number of customers to visit, so that inventory can be secured and personnel can be allocated, thereby improving management efficiency.

In addition, as described above, by replacing the excessively consumed advertising and promotional design page with the 3D panoramic image, the sales terminal 400 can significantly reduce the excessive advertising costs can reduce the cost.

In addition, in the case where a sales terminal 400 is provided with a specific product and / or service in the store, and the user wants to use the multi-product discount coupon stored in the purchase terminal 100, the sales terminal 400 is a bar code form, Recognize the multi-product discount coupon (or the unique number of the multi-product discount coupon) stored in the purchase terminal 100 in the form of a QR code.

That is, a service of a user of the purchase terminal 100 having the multi-product discount coupon visits a store equipped with the sales terminal 400 to purchase one or more products sold at the store, or provides a service at the store. If the user wants to perform the payment function through the multi-product discount coupon in order to pay for the service after receiving the service, the sales terminal 400 is stored (or issued) the multi-stored in the purchase terminal 100 Recognize the unique number of the product discount coupon (or the information of the corresponding multi-product discount coupon). In this case, the sales terminal 400 may recognize the corresponding multi-product discount coupon through a recognition unit (eg, a barcode recognition unit, a QR code recognition unit, etc.) provided in the sales terminal 400.

In addition, the sales terminal 400 is interlocked between the purchase terminal 100, the server 200 and the payment server 300 based on the recognized multi-product discount coupon (or unique number of the multi-product discount coupon). By using, the multi-product discount coupon stored in the purchase terminal 100 performs the use processing (or payment processing) function.

In addition, the sales terminal 400 processes (or converts / changes) a usage completion state for the corresponding multi-product discount coupon transmitted (or issued) to the purchase terminal 100.

In addition, after the use process (or payment function) for the corresponding multi-product discount coupon is completed, the sales terminal 400 displays the result of the use process for the multi-product discount coupon. Here, the result of using the multi-product discount coupon includes a store name, a store location, a store contact information, product and service information, payment date and time information, order information / service price information, and the like.

In addition, after the use process (or payment function) for the corresponding multi-product discount coupon is completed, the sales terminal 400 outputs the result of the use process for the multi-product discount coupon through a printout such as a receipt.

In addition, after the use processing (or payment function) for the corresponding multi-product discount coupon is completed, if there is a balance of the multi-product discount coupon, the sales terminal 400 is the multi-product transmitted from the server 200. Receive information including available amount (or balance information / updated amount available) in the discount coupon.

In addition, the sales terminal 400 displays information including the amount of money available in the received multi-product discount coupon, or outputs it through printed matters such as the receipt.

In addition, after the use process (or payment function) for the corresponding multi-product discount coupon is completed, the sales terminal 400 transmits (or issues) a preset point and / or discount coupon to the purchase terminal 100. . Here, the points and / or discount coupons may be included in the result of the use process for the multi-product discount coupon or included in the issued receipt.

The store server 500 communicates with the purchase terminal 100, the server 200, the payment server 300, the sales terminal 400, and the like.

In addition, the store server 500 may be the sales terminal 400 provided in a specific store.

In addition, the store server 500 receives the member information transmitted from the server 200, information about the delivery product, identification information of the purchase terminal 100, and the like.

In addition, the store server 500 delivers the delivery product to the corresponding shipping destination based on the shipping destination information included in the member information.

In addition, in the case of a hand carry product (or an on-site receipt product), the store server 500 reads the guidance message including guide information necessary for on-site receipt of the hand carry product, an identification code such as a barcode or QR code form, and the like. Send to the purchase terminal 100. Thereafter, when the user of the purchase terminal 100 directly visits the store providing the corresponding hand carry product, the store server 500 receives the identification code such as the barcode or QR code form stored in the purchase terminal 100. After authentication (or recognition), the hand carry product may be provided to the user of the purchase terminal 100.

As described above, the object provided in the offline store or the street store may be photographed, and product and service information and additional information related to the photographed object may be provided through an augmented reality function.

In addition, as described above, in addition to providing product information and / or service information related to the object photographed at the purchase terminal, whether the product is a hand carry product indicating that the product can be directly received or delivered, or at an offline store or a street store. The augmented reality function may provide additional information about whether the product is a shipment product that is not handled or is a product that is out of stock due to exhaustion of sales.

Hereinafter, a method of controlling an augmented reality based shopping information providing system to which a terminal is applied according to the present invention will be described in detail with reference to FIGS. 1 to 10.

First, the purchase terminal 100 photographs an object provided (or installed / placed) in an offline store or a street store (or street store). Here, the object is a discount event code (for example, exhibition products, POPs, flyers, banner advertisements, billboard advertisements, poster advertisements, signs, etc.) provided in each store located in the offline or street, product code (for example For example, a price tag). At this time, the object in the present invention may be in a state excluding the products and services provided by each store.

For example, the purchase terminal 100 photographs a soy product code (eg, soy sauce 100 liter product code: 20110033698, etc.) provided in an offline store by user control (or user manipulation).

As another example, the user of the purchase terminal 100 photographs the first orthopedic signage located in the street (S810).

Thereafter, the purchase terminal 100 transmits the captured image information, identification information of the purchase terminal 100, and the like to the server 200. Here, the identification information of the purchase terminal 100 includes MDN, Mobile IP, Mobile MAC, Sim (Subscriber Identification Module) card unique information, serial number and the like.

For example, the purchase terminal 100 transmits the photographed first image information (eg, image information including a product code of the soy sauce), identification information of the purchase terminal 100, and the like to the server 200. send.

In this case, the purchase terminal 100 confirms the location information of the corresponding purchase terminal 100 (or the location information of the point where the second object is photographed), and photographs the location information of the checked purchase terminal 100. Image information, identification information of the purchase terminal 100 may be transmitted to the server 200.

As another example, the purchase terminal 100 confirms the location information of the purchase terminal 100 through the GPS receiver. In addition, the purchase terminal 100 is the second image information (for example, the image information including the first orthopedic signage), the location information of the purchase terminal 100, the purchase terminal 100 The identification information and the like are transmitted to the server 200 (S820).

Thereafter, the server 200 receives the image information transmitted from the purchase terminal 100, identification information of the purchase terminal 100, and the like. In this case, the server 200 may also receive location information and the like of the purchase terminal 100 transmitted from the purchase terminal 100.

In addition, the server 200 analyzes the received image information and recognizes an object included in the image information.

As an example, the server 200 recognizes a product code (eg, 20110033698) of the soy sauce included in the received first image information.

As another example, the server 200 recognizes the first orthopedic signboard included in the received second image information (S830).

Thereafter, the server 200 checks a code corresponding to the recognized object among a plurality of object-specific codes stored in advance.

In addition, the server 200 confirms one or more goods and service information corresponding to the identified code among a plurality of previously stored goods information and service information. At this time, the server 200 is a hand carry goods indicating that the product is directly received or delivered from among the one or more goods and service information (or one or more goods and service information corresponding to the identified code) to be confirmed or Check the additional information on whether the item is a shipping product that is not handled by an offline store or a street store or is sold out due to exhausted sales. Here, the product and service information may be a product name (or service name), a product unique code (or service unique code), a product image (or service basic image), product description information (or service description information), and experience with a product / service. It includes information, evaluation information on goods / services, appointment information, counseling information, and medical time information.

For example, the server 200 checks the eleventh bar code corresponding to the recognized soy product code (for example, 20110033698) among prestored object codes (or codes for unique identification information of a plurality of objects). do. Further, the server 200 may store product information on 100 liters of soy sauce of ABC food related to the checked eleventh bar code, product information on 100 liters of soy sauce of BCD food, and CDE among previously stored code and goods information. Check the product information for 100 liters of soy sauce in food and the product information for 100 liters of soy sauce in DEF food. In addition, the server 200, the product information for 100 liters of soy sauce of ABC foods, the product information for 100 liters of soy sauce of BCD foods, the product for 100 liters of soy sauce of CDE foods associated with the identified eleventh bar code Information on 100 liters of soy sauce in ABC food, a hand-carry product indicating that it can be directly received or delivered from the product information on 100 liters of soy sauce in DEF food, product information on 100 liters of soy sauce in BCD food, and Check the product information for 100 liters of soy sauce in CDE food, and check the product information for 100 liters of soy sauce in DEF food, which is a delivery product indicating that the product is out of stock due to exhaustion of sales.

As another example, the server 200 checks a twelfth preset QR code corresponding to the recognized signboard among the pre-stored object-specific codes (or codes for unique identification information of a plurality of objects). In this case, the server 200 reflects the location information of the purchase terminal 100 and is adjacent to the location information of the purchase terminal 100 among the pre-stored object-specific codes (or object-specific codes including location information). The twelfth preset QR code corresponding to the signboard may be checked. In addition, the server 200 is a musculoskeletal ultrasound examination which is a plurality of services (or a plurality of medical subjects) provided by the first orthopedic surgeon associated with the checked 12th QR code among the pre-stored product and service information for each code, Check your body, X-ray diagnostics, pain clinic, and nerve block. In addition, the server 200 checks the reservation information, consultation information, medical time information, etc. for a plurality of services provided by the checked first orthopedic surgery together (S840).

Thereafter, the server 200 transmits one or more goods and service information corresponding to the checked code, additional information indicating whether the checked hand carry goods or delivery goods are provided to the purchase terminal 100.

In this case, when the at least one product and service information is plural, the server 200 sorts the plurality of goods and service information into any one of a predetermined discount rate, a saving rate, a sales volume, a popularity, and a brand. Only a preset number of the plurality of sorted goods and service information may be transmitted to the purchasing terminal 100.

For example, the server 200 may provide product information for 100 liters of soy sauce of ABC foods, product information for 100 liters of soy sauce for BCD foods, and 100 liters of soy sauce for CDE foods. Product information, product information for 100 liters of soy sauce of DEF foods, and additional information indicating that it is a hand carry product (for example, product information for 100 liters of soy sauce for ABC foods, product information for 100 liters of soy sauce for BCD foods) , Information about 100 liters of soy sauce of CDE foods, etc., and additional information indicating that it is a shipping product (for example, product information about 100 liters of soy sauce of DEF foods) are transmitted to the purchase terminal 100. do.

As another example, the server 200 may be a musculoskeletal ultrasound test, a body shape diagnosis test, an X-ray diagnosis which is a plurality of services (or a plurality of medical subjects) provided by a first orthopedic surgeon in relation to the identified 12th QR code. A test, a pain clinic, a nerve block, and the like are transmitted to the purchase terminal 100 (S850).

Thereafter, the purchase terminal 100 receives one or more goods and service information related to the corresponding object included in the image information transmitted from the server 200, additional information, and the like.

In addition, the purchase terminal 100 displays the received at least one product and service information, the additional information, etc. on the previously photographed image information displayed on the purchase terminal 100 by applying an augmented reality function. In this case, when the purchase terminal 100 receives a plurality of goods and service information arranged in any one of a discount rate order, earning rate order, sales order, popularity order and brand order transmitted from the server 200, the purchase terminal 100 may display a plurality of goods and services information arranged in any one of the received discount rate, earning rate, sales volume, popularity, and brand.

For example, as shown in FIG. 9, the purchase terminal 100 relates to a product code of soy sauce included in the first image information transmitted from the server 200 in response to the transmitted first image information. Product information on 100 liters of soy sauce in ABC food, product information on 100 liters of soy sauce in BCD food, product information on 100 liters of soy sauce in CDE food, and product information on 100 liters of soy sauce in DEF food (920) And the like are displayed on the first image information 910 by applying an augmented reality function. In addition, as illustrated in FIG. 9, the purchase terminal 100 may have additional information 930 for each product information among a plurality of product information 920 displayed on the first image information to which the augmented reality function is applied. ) Is displayed on one side of the product information by applying the augmented reality function. Here, the additional information 930 for each product information indicates additional information indicating that it is a hand carry product (for example, product information for 100 liters of soy sauce of ABC food, product information for 100 liters of soy sauce of BCD food, CDE food Product information for 100 liters of soy sauce), additional information indicating that the product is a delivery product (for example, product information for 100 liters of soy sauce of DEF food). In this case, the purchase terminal 100 may display an icon / emoticon indicating a preset hand carry product, an icon / emoticon indicating a preset delivery product, and the like on one side of the corresponding product information.

As another example, as shown in FIG. 10, the purchase terminal 100 includes a first included in the second image information 1010 transmitted from the server 200 in response to the transmitted second image information. Musculoskeletal ultrasound, body diagnosis, X-ray diagnosis, pain clinic, neuroblocking, etc., a plurality of services (or a plurality of medical subjects) provided by the first orthopedic surgeon related to the sign of orthopedic surgery In operation S860, the display is applied.

Thereafter, the purchase terminal 100 is based on the at least one specific product and service information selected from one or more goods and service information displayed on the purchase terminal 100, the server 200, payment server 300 and The payment function is performed by interworking with the sales terminal 400.

In this case, with respect to a specific product corresponding to the specific product and service information selected by the purchase terminal 100, when the specific product is a preset delivery product and a product corresponding to an overseas direct purchase, the server 200 purchases the purchase. By interlocking with the terminal 100, the payment server 300, and the sales terminal 400, a payment function for an amount including a shipping fee, a tariff, etc. may be performed on a specific product corresponding to the delivery product. have.

In this case, for a specific product corresponding to the specific product and the service information selected by the purchase terminal 100, the import customizable goods (for example, meat, goods, weapons, gas, etc.), which are preset (or registered) of the specific product , The drug, etc.), the server 200 transmits information (or payment function failure information) indicating that the payment function cannot be performed on the specific product and service information to the purchase terminal 100. . Here, the information indicating that the payment function cannot be performed includes information indicating that a specific product corresponding to the specific product and service information corresponds to a product that cannot be imported.

In addition, the purchase terminal 100 receives information indicating that the payment function for the specific product and service information transmitted from the server 200 is not possible, and the payment for the received specific product and service information Information indicating that the function cannot be performed is displayed.

In addition, a specific product corresponding to the specific product and the service information selected in the purchase terminal 100 is a preset (or registered) import customs clearance (including meat, goods, weapons, gas, medicine, etc.) If not applicable, the server 200 applies a preset discount amount to a normal price amount corresponding to the at least one specific product and service information to perform a payment function for the amount to which the discount amount is applied from the normal price amount.

In addition, after the payment function is normally completed, the server 200 (or the payment server 300) provides the payment terminal performance result (or payment information) to the purchase terminal 100. Here, the result of performing the payment function (or payment information) includes at least one product and service information, payment date and time information, price information per order product / service, and the like.

In addition, the purchase terminal 100 receives a payment function execution result transmitted from the server 200 (or the payment server 300), and displays the received payment function execution result.

For example, when product information on 100 liters of soy sauce of BCD food is selected from the product information related to the soy sauce shown in FIG. 9, the purchase terminal 100 is the server 200 and the payment server 300. ) And a payment amount (or an eleventh coupon (eg, 15% discount coupon) preset to the normal price amount) corresponding to the product information for 100 liters of the soy sauce, by interworking with the sales terminal 400 and the like. To perform a payment function for the amount to which the discount amount is applied. In addition, after the payment function is normally completed, the server 200 transmits a result of performing the payment function to the purchase terminal 100. In addition, the purchase terminal 100 stores and displays a result of performing a payment function transmitted from the server 200.

As another example, when product information on 100 liters of soy sauce of DEF food corresponding to a product sold out of the offline store is selected among the product information related to the soy sauce shown in FIG. 9, the purchase The terminal 100 is linked with the server 200, the payment server 300 and the sales terminal 400, the payment amount corresponding to the product information for 100 liters of soy sauce of the DEF food (or the The payment function for the 12th coupon (for example, a discount amount corresponding to 10% discount coupon) is applied to the normal price amount. In addition, after the payment function is normally completed, the server 200 transmits a result of performing the payment function to the purchase terminal 100. In addition, the purchase terminal 100 stores and displays a result of performing a payment function transmitted from the server 200.

As another example, when nerve block surgery is selected from among information 1010 about a plurality of services provided by the first orthopedic surgeon illustrated in FIG. 10, the purchase terminal 100 may include the server 200 and the payment. By interworking with the server 300 and the sales terminal 400, a payment function (or a reservation function for the nerve block surgery) for the payment amount corresponding to the nerve block surgery is performed. In addition, after the payment function (or reservation function) is normally completed, the server 200 transmits the payment function execution result (or the reservation function execution result) to the purchase terminal 100. In addition, the purchase terminal 100 stores and displays a payment function execution result (or a reservation function execution result) transmitted from the server 200 (S870).

Subsequently, after the payment function is normally performed, when a predetermined shipping product is included among the at least one specific product and service information that has been paid, the server 200 receives member information transmitted from the purchase terminal 100. Based on the plurality of stores (or store information) stored in advance, the store having the stock of the delivery product among the stores closest to the delivery address corresponding to the delivery address information included in the member information is identified.

For example, when the payment function for the product information for 100 liters of the soy sauce of the DEF food corresponding to the product sold out in the offline store is normally performed, the server 200 stores the station of the DEF food. Product information for 100 liters of soy sauce of the DEF food among a plurality of stores previously registered in the server 200 in order to deliver (or deliver) product information for 100 liters of soy sauce to the user of the purchase terminal 100. The shop 101 is identified as the store closest to the delivery address information (for example, 123-456 Yeoksam-ro, Gangnam-gu, Seoul, Korea) that has a stock of 100 liters of soy sauce of DEF food. In addition, the server 200 transmits the member information, product information for 100 liters of soy sauce of the DEF food, identification information of the purchase terminal 100, etc. to the confirmed store server 500 of the 101st store. do.

In addition, after the payment function is normally performed, in the case where a preset hand carry product is included among at least one specific product and service information of which payment is completed, the server 200 transmits the member information transmitted from the purchase terminal 100. From among a plurality of stores (or store information) stored in advance based on the, the store having the stock of the hand carry goods among the stores closest to the delivery address corresponding to the delivery address information included in the member information.

In addition, the server 200 transmits the member information, the hand carry product information, the identification information of the purchase terminal 100, etc. to the store server 500 of the checked store (S880).

Thereafter, the store server 500 receives the member information transmitted from the server 200, information about the delivery product, identification information of the purchase terminal 100, and the like.

For example, the store server 500 of the 101-th store may store the DEF foods as a delivery address of the corresponding delivery information based on the delivery information (for example, 123-456 Yeoksam-ro, Gangnam-gu, Seoul, Korea). Deliver 100 liters of soy sauce (including courier delivery, for example).

In addition, the store server 500 performs an authentication procedure for the user of the purchase terminal 100 visited to the store, and then provides (or delivers) the corresponding hand carry product to the user of the purchase terminal 100. (S890).

Thereafter, the purchase terminal 100 includes experience information, evaluation information, and purchase information of the at least one specific product and service from the purchase of the specific product or the specific service to the arrival (or the issue of the coupon). The identification information and the like are transmitted to the server 200.

In addition, the server 200 receives experience information, evaluation information, identification information, etc. of the specific product or the specific service transmitted from the purchase terminal 100.

In addition, the server 200 matches and stores the received experience information, evaluation information, and the like with a specific product or a specific service (S900).

11 is a block diagram illustrating a configuration of a guide apparatus 20 to which a terminal is applied according to another embodiment of the present invention.

As shown in FIG. 11, the guide apparatus 20 includes a terminal 600 and a server 700. Not all components of the guide apparatus 20 shown in FIG. 11 are essential components, and the guide apparatus 20 may be implemented by more components than those shown in FIG. 11, and fewer components. Guide device 20 may also be implemented by.

The terminal 600 is a smart phone, a portable terminal, a mobile terminal, and a personal digital assistant, like the previous purchase terminal 100 or the sales terminal 400. PDAs, Portable Multimedia Player (PMP) Terminals, Telematics Terminals, Navigation Terminals, Personal Computers, Notebook Computers, Slate PCs, Tablet PCs, Tablet PCs, Ultrabooks ultrabook), wearable device (e.g., watch type terminal (Smartwatch), glass type glass (Smart Glass), HMD (Head Mounted Display), etc.), Wibro terminal, IPTV (Internet Protocol Television) It can be applied to various terminals such as a terminal, a smart TV, a digital broadcasting terminal, an audio video navigation (AVN) terminal, an audio / video (A / V) system, a flexible terminal, and the like.

As shown in FIG. 12, the terminal 600 includes a communication unit 610, a camera 620, a storage unit 630, a display unit 640, a voice output unit 650, and a controller 660. Not all components of the terminal 600 illustrated in FIG. 12 are essential components, and the terminal 600 may be implemented by more components than those illustrated in FIG. 12, or by fewer components. The terminal 600 may be implemented.

The communication unit 610 communicates with any component inside or any at least one terminal outside via a wired / wireless communication network. In this case, the external arbitrary terminal may include the server 700 and the like. Here, the wireless Internet technologies include a wireless LAN (WLAN), a digital living network alliance (DLNA), a wireless broadband (Wibro), a WiMAX (World Interoperability for Microwave Access: Wimax), and an HSDPA (High Speed Downlink Packet Access). ), HSUPA (High Speed Uplink Packet Access), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), etc. The communication unit 610 transmits and receives data according to at least one wireless Internet technology in a range including the Internet technologies not listed above. In addition, near field communication technologies include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Near Field Communication (NFC). Ultrasound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, and the like may be included. In addition, the wired communication technology may include power line communication (PLC), USB communication, Ethernet, serial communication, serial communication, optical / coaxial cable, and the like.

In addition, the communication unit 610 may mutually transmit information with an arbitrary terminal through a universal serial bus (USB).

In addition, the communication unit 610 is a technical standard or communication method for mobile communication (for example, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (EV), EV) Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (DO), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced) and the like to transmit and receive a radio signal with the base station, the server 700 and the like on a mobile communication network.

In addition, the communication unit 610 may control the current location of the terminal 600 transmitted from the server 700 or the search word transmitted from the terminal 600 to the server 700 under the control of the controller 660. Receive the corresponding aerial view (or sky view / road view / foreground view).

In addition, the communication unit 610 transmits the augmented reality image, additional information, etc. generated in the terminal 600 to the server 700 under the control of the control unit 660.

In addition, the communication unit 610 may control the current location of the terminal 600 transmitted from the server 700 or the keyword transmitted to the server 700 to the terminal 600 under the control of the controller 660. Or augmented reality image, additional information, etc. corresponding to the search term). Here, the additional information includes real time additional information (or detailed information), a related video, a coupon (or coupon information), and the like. In this case, the related video (or promotional video) is a video related to a specific product and / or service, a video related to the use of the product (for example, a video on how to use cosmetics, a video about wearing clothes, a toy assembly method Video, how to use the tool, video about how to cook, etc.), and may include a text and / or voice message.

In addition, the communication unit 610 receives one or more images provided from the server 700 under the control of the control unit 660. Here, the image may be an image captured (or obtained) from an augmented reality image in which an additional object is added (or inserted) to an image captured by the 360 degree camera 620 provided in a specific store. In addition, the image may include a state in which at least one object and additional objects related to the object are included. Here, the object may include one or more stores offline, products in the stores, POPs, flyers, banner advertisements, bulletin board advertisements, poster advertisements, signboards, price lists, and the like. In addition, the additional object may be configured in a three-dimensional form and may be an object for providing real-time additional information (or detailed information), a related video, a coupon (or coupon information), and the like related to the object. In this case, the additional object on the captured image may be displayed in a two-dimensional form.

The camera 620 processes an image frame such as a still image or a video obtained by an image sensor (camera module or camera) in a video call mode, a shooting mode, a video conference mode, and the like. That is, corresponding image data obtained by the image sensor is encoded / decoded according to each standard according to a codec. The processed image frame may be displayed on the display unit 640 under the control of the controller 660. For example, the camera 620 photographs an object (or a subject) (a user image or a video conference person), and outputs a video signal corresponding to the captured image (subject image).

In addition, the camera 620 is an image (or two-dimensional image) including one or more stores offline, merchandise (or exhibition items), POPs, flyers, banner advertisements, bulletin board advertisements, poster advertisements, signs, price lists, etc. (Panorama Video). Here, the store includes a department store, a shopping mall, an outlet, a restaurant, a hospital, a building (or a building), and the like. In this case, the camera 620 may be a 360 degree camera for capturing a two-dimensional panorama image.

In addition, the image frame processed by the camera 620 may be stored in the storage unit 630 or transmitted to an external terminal (including other terminals, the server 700, etc.) through the communication unit 610. have.

The storage unit 630 stores various user interfaces (UIs), graphical user interfaces (GUIs), and the like.

In addition, the storage unit 630 stores data and programs necessary for the terminal 600 to operate.

That is, the storage unit 630 may store a plurality of application programs or applications, data for operating the terminal 600, and instructions that are driven by the terminal 600. At least some of these applications may be downloaded from an external server via wireless communication. In addition, at least some of these application programs may exist on the terminal 600 from the time of shipment for basic functions of the terminal 600 (for example, incoming / outgoing / outgoing function / message reception / outgoing function). Meanwhile, an application program may be stored in the storage unit 630 and installed in the terminal 600, and may be driven by the controller 660 to perform an operation (or function) of the terminal 600.

In addition, the storage unit 630 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD). Memory, etc.), magnetic memory, magnetic disks, optical disks, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), It may include at least one storage medium of the PROM (Programmable Read-Only Memory). In addition, the terminal 600 may operate a web storage that performs a storage function of the storage 630 on the Internet, or may operate in connection with the web storage.

In addition, the storage unit 630 is an aerial view (or sky) corresponding to the current location of the terminal 600 received through the communication unit 610 or the search word transmitted from the terminal 600 to the server 700. Save view / load view / foreground view).

In addition, the storage unit 630 is augmented to correspond to a current location of the terminal 600 received through the communication unit 610 or a keyword (or search term) transmitted to the server 700 to the terminal 600. Stores realistic footage, additional information, and more.

In addition, the storage unit 630 stores one or more images received through the communication unit 610.

In addition, the storage unit 630 is a specific app (or application) (for example, a shopping app) for displaying (or outputting) an augmented reality image provided from the server 700 under the control of the controller 660. Save, or install).

The display unit 640 may display various contents such as various menu screens using a user interface and / or a graphic user interface stored in the storage unit 630 under the control of the controller 660. Here, the content displayed on the display unit 640 includes various text or image data (including various information data) and a menu screen including data such as icons, list menus, combo boxes, and the like. In addition, the display unit 640 may be a touch screen.

In addition, the display unit 640 may be a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display. The display device may include at least one of a flexible display, a 3D display, an e-ink display, and a light emitting diode (LED).

In addition, the display unit 640 is controlled by the control unit 660, the aerial view corresponding to the current location of the terminal 600 or the search word transmitted from the terminal 600 to the server 700 ( Or sky view / road view / foreground view).

In addition, the display unit 640 displays an image captured by the camera 620 under the control of the controller 660.

In addition, the display unit 640 displays an augmented reality image generated by the control of the controller 660.

In addition, the display unit 640 is a keyword transmitted to the server 700 to the current location of the terminal 600 or the terminal 600 received through the communication unit 610 under the control of the control unit 660. The augmented reality image, additional information, etc. corresponding to the (or search term) are displayed.

In addition, the display unit 640 displays one or more images received through the communication unit 610 under the control of the controller 660.

In addition, the display unit 640 controls the augmented reality image provided from the server 700 through a specific app (for example, a shopping app, etc.) installed in the terminal 600 under the control of the controller 660. Display.

The voice output unit 650 outputs voice information included in a signal processed by the controller 660 by a predetermined signal. Here, the voice output unit 650 may include a receiver, a speaker, a buzzer, and the like.

In addition, the voice output unit 650 outputs the guide voice generated by the controller 660.

In addition, the voice output unit 650 is an aerial view corresponding to the current position of the terminal 600 received by the controller 660 or a search word transmitted from the terminal 600 to the server 700 ( Or voice information corresponding to a sky view / road view / foreground view).

In addition, the voice output unit 650 is a keyword transmitted by the controller 660 to the server 700 to the current location of the terminal 600 or the terminal 600 received through the communication unit 610. The audio information corresponding to the augmented reality image, the additional information, or the like corresponding to the search word is output.

In addition, the voice output unit 650 outputs voice information corresponding to the augmented reality image provided from the server 700 by the controller 660.

The controller 660 executes an overall control function of the terminal 600.

In addition, the controller 660 executes an overall control function of the terminal 600 by using the program and data stored in the storage 630. The controller 660 may include a RAM, a ROM, a CPU, a GPU, a bus, and the RAM, a ROM, a CPU, a GPU, and the like may be connected to each other through a bus. The CPU may access the storage unit 630 to perform booting using the O / S stored in the storage unit 630, and various operations using various programs, contents, data, etc. stored in the storage unit 630. Can be performed.

In addition, the controller 660 performs a member registration procedure with respect to the user of the terminal 600 to the server 700 by interworking with the server 700.

In addition, when performing the membership registration process, the controller 660 must complete an authentication function through an identity verification means (for example, a mobile phone, a credit card, an i-pin, an email, etc.) and a membership registration process for the server 700. Can be completed normally.

In addition, by the membership registration process, the control unit 660 provides user information such as birth date, male / female information, telephone number, email, password, name, address, etc. to the server 700 through the communication unit 610. do.

In addition, when an app preinstalled in the terminal 600 is executed, the controller 660 displays an app execution result screen on the display unit 640. Here, the app execution result screen includes a member information item (or menu / button) for receiving or transmitting member information, an image information item for providing an image or a 3D panoramic image, a setting item for setting an app function, and a terminal. My page items for managing the purchase history information for the goods or services purchased through the 600 and the like. In this case, the terminal 600 communicates with the server 700 related to the app in the background state.

In addition, when a member information item is selected from an app execution result screen displayed on the display unit 640, the controller 660 receives member information for receiving (or receiving) member information corresponding to the selected member information item. The screen is displayed on the display unit 640.

In addition, the controller 660 receives member information according to a user input (or user touch). Here, the member information includes a name, an address, a language used, shipping information, a type of shopping object (eg, an avatar type, an emoticon type, etc.).

In addition, the controller 660 transmits the received member information, identification information of the terminal 600, etc. to the server 700 through the communication unit 610. Here, the identification information of the terminal 600 includes a mobile directory number (MDN), a mobile IP, a mobile MAC, a subscriber identity module (SIM) card unique information, a serial number, and the like.

In addition, when an image information item is selected from an app execution result screen displayed on the display unit 640, the controller 660 may provide an aerial view (or street view / road view / foreground view) provided from the server 700. Is received through the communication unit 610. In this case, the controller 660 receives a search word according to a user input, transmits the received search word to the server 700 through the communication unit 610, and transmits the search word from the server 700 in response to the transmission. The aerial view (or street view / road view / foreground view) may be received through the communication unit 610. In this case, the aerial view, the street view, the road view, and the foreground view may be implemented in virtual reality or may be an image (or image) previously photographed through a camera.

The aerial view described in the present invention represents an image captured by the camera 620 provided in the aircraft, the street view (or road view / foreground view) is the camera 620 or a dedicated camera ( The image photographed through the image may be displayed.

In addition, when checking the image of the area where the terminal 600 is located through the camera 620, the controller 660 is an image (for example, street view (or road view) confirmed through the camera 620) / View) is displayed on the display unit 640 (or view finder).

In addition, the controller 660 may be configured to determine the current location of the terminal 600 and information about an object in the displayed street view (or road view / foreground view) (for example, the street view (or road view / foreground view)). Augmented reality information related to the corresponding Street View is checked based on a signboard included therein).

In addition, the controller 660 may apply the identified augmented reality information to the street view (or road view / foreground view) to display an image to which the augmented reality information is applied to an image previously identified through the camera 620. Displayed on the display unit 640.

As described above, in the embodiment according to the present invention, the aerial view, the street view, the road view, the foreground view, etc. may be provided in the form of a real image, a virtual reality, an augmented reality, or the like.

In addition, when any one (or at least one) category is selected from a plurality of categories displayed on one side of the aerial view, street view, road view, foreground view, etc. displayed on the display unit 640, the controller 660 is selected. Identifies one or more objects associated with the selected category from among a plurality of objects included in the aerial view (or street view / road view / foreground view).

In addition, the controller 660 checks information on an outline (or an area occupied by an outline / object) of the object in the aerial view (or street view / road view / foreground view) with respect to the identified object. In this case, the aerial view, the street view, the road view, and the foreground view may be implemented in virtual reality or may be an image (or image) captured by a camera. In addition, the street view, the road view, the foreground view, etc. may be implemented in augmented reality through the camera 620 and the display unit 640.

As such, the controller 660 may be configured to select from among a plurality of categories displayed on one side of a corresponding screen for an aerial view, a street view, a road view, and a foreground view provided in a form of a real image, virtual reality, augmented reality, and the like. You can identify one or more objects associated with a category.

In addition, the controller 660 identifies a place name (or building name / name) corresponding to the identified one or more objects (or one or more objects related to the identified category).

In addition, the controller 660 identifies additional information related to the identified one or more objects (or one or more objects related to the identified category). In this case, the additional information related to the object may be information provided in real time from the server 700 in response to a request for additional information related to the corresponding object of the terminal 600.

In addition, the controller 660 may include a place name (or building name) corresponding to the one or more identified objects on one side of the display unit 640 on which the aerial view (or street view / road view / foreground view) is displayed. Additional information regarding one or more identified objects (or one or more objects related to the identified category) is displayed in the form of a speech bubble or the like.

That is, the controller 660 displays a name corresponding to the identified object and additional information related to the identified object adjacent to the identified object in the aerial view (or street view / road view / foreground view). . In this case, the identified one or more objects displayed on the display unit 640, names of places corresponding to the objects, and additional information related to the objects may include identification of corresponding objects, names of places corresponding to the objects, additional information related to the objects, and the like. To facilitate this, the effect of the light (or specific color) turning off gradually from the maximum state (or lighting state) to the maximum state (or lighting state) and then repeatedly changing from the off state to the maximum state (or lighting state) for the object at a preset time interval. An effect, a neon sign effect, and an effect of changing the color of the light to different colors at preset cycles may be applied.

In addition, when a specific object is selected from one or more objects (or one or more objects for which name / additional information is displayed) to which the preset effect displayed on the display unit 640 is applied, the controller 660 may select the selected specific object. The 3D panoramic image related to the display is displayed on the display unit 640. Here, the 3D panoramic image includes at least one additional object, and the additional object has a maximum (or lit state) of light (or a specific color) with respect to the corresponding object at a predetermined period so as to easily identify the additional object. It may be applied to an effect that is gradually turned off and then repeatedly turned off to gradually turn up (or turned on), a flickering light effect, a neon sign effect, or a light color changes to different colors at predetermined intervals. have. In this case, the additional object may include a point of purchase (POP), a leaflet, a banner advertisement, a poster advertisement, a signboard, a price tag, and the like.

In addition, when another specific object is selected within the 3D panoramic image including at least one additional object displayed on the display unit 640, the controller 660 may be configured to display another 3D panoramic image related to the selected other specific object. It may be displayed on the display unit 640.

As such, when an object to which a preset effect is applied is selected in the 3D panoramic image related to the previously selected category displayed on the current screen of the terminal 600, the controller 660 is configured as an object of the user of the terminal 600. It can be repeatedly switched to the image of interest (or 3D panorama image) according to.

In addition, as described above, the controller 660 corresponds to the selected object whenever an object to which a specific effect is applied is selected from among the 3D panoramic images displayed on the display unit 640 to change the screen for the user's purpose. The screen displayed on the display unit 640 may be switched to an image.

In addition, through the additional image providing process according to the purpose of the user of the terminal 600, the controller 660 is connected to the server 700, the purchase function related to the function selected by the user of the terminal 600 ( Or a payment function).

In addition, when a preset first event for a specific place name and / or specific additional information is detected from among place names corresponding to one or more objects displayed on the display unit 640 and / or one or more additional information relating to one or more objects, The controller 660 may display the aerial view (or the additional name corresponding to the specific name and / or specific additional information (or additional information corresponding to an object related to the name or additional information) on which the name or additional information is displayed. It is displayed on the other side of street view / road view / foreground view. Here, the additional information may be a promotional image corresponding to a specific object related to the specific place name or specific additional information, or a 3D panoramic image related to the specific object (for example, a 3D panorama displayed by screen switching when the object is selected in the previous process). Video, etc.). In this case, when the state in which the predetermined first event is selected (or touched / clicked) in the specific place name and / or specific additional information is maintained, the terminal 600 for the specific place name and / or specific additional information is maintained. And when a touch gesture by the user is detected. Here, the touch zester by the user of the terminal 600 includes a tap, a touch and hold, a double tap, a drag, a flick, a drag and drop, a pinch, a swipe, and the like.

A "tap" is an action in which a user touches a screen (including an object, place name, additional information, etc.) using a finger or a touch tool (e.g., an electronic pen) and immediately lifts it off the screen without moving. Indicates.

"Touch & hold" means a threshold time (e.g., after a user touches a screen (e.g., an object, place name, additional information, etc.) using a finger or a touch tool (e.g., an electronic pen). 2 seconds) or more to hold the touch input. That is, this means that the time difference between the touch-in time point and the touch-out time point is greater than or equal to the threshold time (for example, 2 seconds). In order to allow the user to recognize whether the touch input is a tap or a touch and hold, the feedback signal may be provided visually, acoustically, or tactilely when the touch input is maintained for a threshold time or more. The threshold time may vary depending on implementation.

"Double tap" refers to an operation in which a user touches the screen twice (including an object, place name, additional information, etc.) using a finger or a touch stylus.

A "drag" is a method whereby a user touches a finger or a touch tool on a screen (e.g., an object, place name, additional information, etc.) and then moves the finger or touch tool to another location on the screen while keeping the touch. It means action. An object is moved or a panning operation to be described later is performed due to the dragging operation.

"Flick" means that a user touches a finger or a touch tool on the screen (e.g., an object, place name, additional information, etc.) and then uses the finger or touch tool to select a threshold speed (e.g., 100 pixel / s ) To drag the mouse. The flick may be distinguished from the drag (or panning) based on whether the movement speed of the finger or the touch tool is greater than or equal to the threshold speed (eg, 100 pixel / s).

"Drag and drop" refers to an operation in which a user drags an object (including an object, a place name, additional information, etc.) to a predetermined position on the screen and releases it using a finger or a touch tool.

“Pinch” refers to an operation in which a user moves two fingers in different directions while touching two fingers on a screen (eg, including an object, a place name, additional information, and the like).

"Swipe" is an operation of moving a certain distance in the horizontal or vertical direction while touching an object (including an object, place name, additional information, etc.) on the screen with a finger or a touch tool. The oblique motion may not be recognized as a swipe event.

In addition, when the function corresponding to the first event is terminated, the control unit 660 is the one on the side where the image (or the aerial view / street view / road view / foreground view) before the first event is displayed. And a place name corresponding to the at least one object, at least one additional information related to the at least one object, and the like.

That is, when the state of selecting the specific place name and / or the specific additional information is released, the controller 660 displays an image (or the aerial view / street view / road view / foreground view) before detecting the first event. A display name corresponding to the one or more objects, one or more additional information related to the one or more objects) is displayed on the display unit 640.

In addition, when the preset time has elapsed after the touch gesture detection by the user of the terminal 600, the controller 660 is the image (or the aerial view / street view / road view / foreground) before the first event detection The display unit 640 displays a place name corresponding to the one or more objects, an image in which one or more additional information related to the one or more objects is displayed on one side of the view.

In addition, the controller 660 recognizes one or more objects in the captured image. Here, the object includes one or more stores offline, products in the store, a POP, a flyer, a banner advertisement, a bulletin board advertisement, a poster advertisement, a signboard, a price tag, and the like.

In addition, the controller 660 inserts (or adds / includes) an additional object (for example, an object of 3D type) related to the recognized object to generate an augmented reality image in the captured image. Herein, the additional object may be configured in a three-dimensional form and may be an object for providing real-time additional information (or detailed information), a related video, a coupon, and the like related to the recognized object.

That is, the controller 660 identifies the additional object of three-dimensional form related to the recognized object.

In addition, the controller 660 generates an augmented reality image by inserting an additional object of a three-dimensional form related to the recognized object into the photographed image of a two-dimensional form. Here, the augmented reality image may be composed of the photographed image in a two-dimensional form and an additional object related to the recognized object in the three-dimensional form.

In this case, the recognized object and / or the additional object inserted into the image have a maximum state of light (or a specific color) with respect to the object and / or the additional object at predetermined intervals so as to easily identify the object and / or the additional object. The light may be gradually turned off and the light may be repeatedly changed from the light off state to the maximum state, the light flickering effect, the neon sign effect, and the color of the light may be changed to different colors at predetermined cycles.

As described above, the controller 660 causes the user's interest and / or interest to the two-dimensional image captured by the camera 620, and provides three-dimensional image in the two-dimensional image in order to provide additional information. A new augmented reality image may be generated by inserting (or adding) additional objects of a shape.

In addition, when generating an augmented reality image, various effects may be applied to a corresponding object, an additional object, and the like, and may be configured to facilitate identification in the augmented reality image.

In addition, the controller 660 displays a specific augmented reality image on the display unit 640. Here, the augmented reality image displayed on the display unit 640 corresponds to a keyword (or search term) requested by the user of the terminal 600, and is an augmented reality image previously stored in the storage unit 630, or in real time. It may be an augmented reality image transmitted from the server 700 and received through the communication unit 610.

In addition, when an additional object of a specific three-dimensional form in the augmented reality image displayed on the display unit 640 is selected, the controller 660 may be associated with the selected additional object of the three-dimensional form in a specific region of the augmented reality image. The additional information is displayed on the display unit 640. In this case, the additional information (or additional information related to the additional object in the 3D form) may be information transmitted from the server 700 in real time through the communication unit 610.

That is, when an additional object of a specific 3D shape is selected in the augmented reality image displayed on the display unit 640, the controller 660 includes real-time additional information, a related video, a coupon, etc. related to the selected additional object. Check the additional information related to the additional object.

In addition, the controller 660 may include the identified additional information (for example, real-time additional information related to the additional object, a related video, a coupon, etc.) in a specific area (or popup form) in the augmented reality image of the display unit 640. ) Is displayed.

In this case, the controller 660 may display the checked additional information in the form of a speech balloon in another specific region in the augmented reality image of the display unit 640.

In addition, the controller 660 displays additional information related to the additional object adjacent to the selected additional object (or an object corresponding to the additional object). In this case, the selected additional object (for example, an additional object related to a product) displayed on the display unit 640, an object corresponding to the additional object (for example, a product corresponding to the additional object), and the additional object The additional information (for example, real-time additional information related to the additional object, a related video, a coupon, etc.) may be set at a predetermined period so as to easily identify the corresponding object, the additional object corresponding to the object, and additional information related to the additional object. The effect of the light (or specific color) turning off gradually from the maximum state (or lit state) and from the off state to the maximum state (or lit state) repeatedly for that object and / or additional objects; flashing effect , A neon sign effect, and an effect in which the color of the light changes to different colors at predetermined cycles may be applied. This effect may also be maintained for a preset time.

In addition, the controller 660 is applicable even when a second preset event is detected in addition to the case where an additional object of a specific 3D shape is selected (or touched / clicked) in the augmented reality image displayed on the display unit 640. In response to the second event, the display unit 640 may display additional information related to the additional object including real-time additional information related to the additional object, a related video, a coupon, and the like.

In this case, the second preset event detects a touch gesture by the user of the terminal 600 with respect to the additional object when the state of selecting (or touching / clicking) the additional object of the 3D form is maintained. And the like. Here, the touch zester by the user of the terminal 600 includes a tap, a touch and hold, a double tap, a drag, a flick, a drag and drop, a pinch, a swipe, and the like.

In addition, when the function corresponding to the second event is terminated, the controller 660 returns to the image before the second event detection (or the augmented reality image).

That is, when the state in which the additional object of the 3D form is selected is released, the controller 660 displays the image before detecting the second event (or the augmented reality image) on the display unit 640.

In addition, when the preset time elapses after the touch gesture is detected by the user of the terminal 600, the controller 660 displays the image (or the augmented reality image) before the second event is detected by the display unit 640. Mark on.

In addition, the controller 660 receives one or more images provided from the server 700 through the communication unit 610. Here, the image may be an image captured (or obtained) from an augmented reality image in which an additional object is added (or inserted) to an image captured by the 360 degree camera 620 provided in a specific store. In addition, the image may include a state in which at least one object and additional objects related to the object are included. Here, the object may include one or more stores offline, products in the stores, POPs, flyers, banner advertisements, bulletin board advertisements, poster advertisements, signboards, price lists, and the like. In addition, the additional object may be configured in a three-dimensional form and may be an object for providing real-time additional information (or detailed information), a related video, a coupon (or coupon information), and the like related to the object. In this case, the additional object on the captured image may be displayed in a two-dimensional form.

In addition, the controller 660 displays the received one or more images on the display unit 640.

In addition, when a specific image is selected (or clicked) among one or more images displayed on the display unit 640, the controller 660 may include an augmented reality image (or a virtual reality image / or image related to the selected (or clicked) image). It is checked whether an app for displaying a 3D panoramic image is installed in the terminal 600.

As a result of the check, when the app for displaying an augmented reality image related to the selected (or clicked) image is not installed in the terminal 600, the controller 660 is connected to the server 700. By installing the app on the terminal 600.

In addition, when an app installed in the terminal 600 is executed, the controller 660 displays an app execution result screen on the display unit 640. Here, the app execution result screen includes a member information item (or menu / button) for receiving or transmitting member information, an image information item for providing an image or a 3D panoramic image, a setting item for setting an app function, and a terminal. My page items for managing the purchase history information for the goods or services purchased through the 600 and the like. In this case, the terminal 600 communicates with the server 700 related to the app in the background state.

In addition, the controller 660 transmits the received member information, identification information of the terminal 600, etc. to the server 700 through the communication unit 610. Here, the identification information of the purchase terminal 600 includes a mobile directory number (MDN), a mobile IP, a mobile MAC, a subscriber identity module (SIM) card unique information, a serial number, and the like.

In addition, when the check result, an app for displaying an augmented reality image associated with the selected (or clicked) image is installed in the terminal 600 (or the app through the previous installation step, the app is the terminal 600). In the installed state), the controller 660 displays a real-time augmented reality image related to the selected image provided from the server 700 on the display unit 640. Here, the augmented reality image is a two-dimensional image captured by a 360-degree camera provided in the particular store, an additional object (for example, three-dimensional) of the three-dimensional form associated with the object recognized in the photographed image Type object) may be inserted (or added). In this case, the object includes one or more stores offline, products in the store, POP, flyer, banner advertisement, bulletin board advertisement, poster advertisement, signage, price list, and the like. In addition, the additional object may be configured in a three-dimensional form and may be an object for providing real-time additional information (or detailed information), a related video, a coupon (or coupon information), and the like related to the object.

Further, the object and / or the additional object is gradually turned off and turned off again in the maximum state of the light (or a specific color) with respect to the object and / or the additional object at predetermined intervals to facilitate identification of the object and / or the additional object. The effect may be a state in which the effect of repeatedly changing from a state to a maximum state gradually, a light flickering effect, a neon sign effect, and an effect of changing the color of the light to different colors at predetermined intervals may be applied.

In addition, the additional object may be rotated (or moved) in the left / right / up / down direction under the control of the user of the terminal 600. Match (or rotate) equally to movement.

As such, even though the screen in the augmented reality image rotates in a left / right / up / down direction, the additional object included in the augmented reality image is matched with the corresponding rotation and rotated together to be displayed on the display unit 640. The additional object may be displayed (or exposed) to facilitate confirmation of the user.

That is, when an additional object of a specific three-dimensional form in the augmented reality image displayed on the display unit 640 is selected, the controller 660 may perform real-time additional information, related video, coupon (or coupon information) related to the selected additional object. See additional information about this additional object, including).

In addition, the controller 660 may be further configured to identify the additional information (for example, real-time additional information related to the additional object, related video, coupon information, etc.) in a specific region (or pop-up form) in the augmented reality image of the display unit 640. Inclusive). At this time, the additional information is a light effect (or a specific color) is gradually turned off in the maximum state and the light is gradually changed from the unlit state to the maximum state for a predetermined period so that the additional information can be easily checked. The flickering effect, the neon sign effect, the effect of changing the color of the light to different colors at a predetermined cycle may be applied.

In addition, one or more objects included in the augmented reality image displayed on the display unit 640, an additional object related to the corresponding object, and any one of a specific object, a specific additional object, and specific additional information among the additional information related to the additional object. When the third preset event is detected, the controller 660 displays (or overlaps and displays) additional image information related to any one of the specific object, the specific additional object, and the specific additional information on one side of the augmented reality image. do. The additional image information may include a promotion image corresponding to any one of the specific object, the specific additional object, and the specific additional information, and a 3D panoramic image (or a virtual reality image / augmented reality image) related to the specific object. In this case, when the state in which the selected third event is selected (or touched / clicked) among the specific object, the specific additional object, and the specific additional information is maintained, the specific object, the specific additional object, and the specific additional information are maintained. It includes a case where a touch gesture by the user of the terminal 600 is detected for any one of the. Here, the touch gesture by the user of the terminal 600 includes a tap, a touch and hold, a double tap, a drag, a flick, a drag and drop, a pinch, a swipe, and the like.

In addition, when the function corresponding to the third event is terminated, the controller 660 returns to the augmented reality image that is an image before the third event detection or the augmented reality image in which additional information is displayed.

That is, when the state in which one of the specific object, the specific additional object, and the specific additional information is selected is released, the controller 660 may display the augmented reality image or the additional information, which is an image before the third event is detected. Return to the image.

In addition, when the preset time elapses after the touch gesture is detected by the user of the terminal 600, the controller 660 may display the augmented reality image or the additional information, which is an image before the third event detection. Return to the image.

The server 700 communicates with the terminal 600 and the like.

In addition, the server 700 performs a member registration procedure for the user of the terminal 600 by interworking with the terminal 600.

In addition, as described above, the server 700 may add a first code corresponding to each of the first objects to a partial region on the corresponding first object in the preliminary panoramic image, or may be adjacent to a predetermined distance from the first object. Can be added to create a 3D panoramic image.

In addition, the server 700 may generate the 3D panoramic image by the following process.

In addition, the server 700 may include a sales advertisement, an enlarged image, a video (or a promotional video), a coupon (eg, a discount coupon, a free coupon, etc.), a donation, a comment, respectively, corresponding to the recognized one or more first objects. An object for providing information, and / or another additional object for screen switching, is added (or inserted) to a part of the area on the first object in the preliminary panoramic image or adjacently within a preset distance from the first object. Add (or insert) to create a 3D panoramic image.

That is, the server 700 adds (or inserts) an object related to the sales advertisement corresponding to the recognized one or more first objects to a partial area on the corresponding first object in the preliminary panoramic image, or from the first object. A 3D panoramic image is generated by adding (or inserting) adjacently within a preset distance. Here, the sale advertisement includes a POP, a flyer, a banner advertisement, a poster advertisement, a signboard, a price tag, and the like.

In addition, the server 700 may add (or insert) an object for providing an enlarged image corresponding to the recognized one or more first objects to a partial region on the first object in the preliminary panoramic image, or the first object. A 3D panoramic image is generated by adding (or inserting) adjacently within a preset distance from the object. Here, the enlarged image may be an internal picture (or an additional image related to the product) of a product such as a refrigerator or a washing machine, or an enlarged picture of a fresh food. In the case of a plurality of enlarged images, the enlarged image may be arranged in a slide order.

In addition, the server 700 may add (or insert) an object for providing a video (or a promotional video) corresponding to the recognized one or more first objects to a partial region on the first object in the preliminary panoramic image, or Alternatively, the 3D panorama image may be generated by adding (or inserting) adjacently within a preset distance from the first object. Here, the video (or promotional video) is a video related to the product and / or service, a video related to the use of the product (for example, a video on how to use cosmetics, a video about wearing clothes, a video about how to assemble a toy , Video about how to use the tool, video about how to cook, etc.), and may include text and / or voice messages.

In addition, the server 700 may add (or insert) an object for providing a coupon corresponding to the recognized one or more first objects to a partial area on the first object in the preliminary panoramic image, or the first object. A 3D panoramic image is generated by adding (or inserting) adjacently within a preset distance from. Here, the coupon is a coupon to which a discount is applied when purchasing the corresponding product, and in the case of a plurality of coupons, the coupon is arranged in order by a slide method and may be downloaded to the terminal 600 according to a user selection.

In addition, the server 700 may add an object for providing donation information provided when purchasing goods and / or services related to the recognized one or more first objects to a partial region on the first object in the preliminary panoramic image ( Or insert (or insert) adjacently within a predetermined distance from the first object to generate a 3D panoramic image. Here, the donation displays detailed guide information on the donation information applied when the corresponding product is purchased, and in a plurality of cases, the donation may be arranged in a slide manner.

In addition, the server 700 adds (or inserts) an object for providing comment information on goods and / or services related to the recognized one or more first objects to a partial region on the corresponding first object in the preliminary panoramic image. 3D panorama image is generated by adding (or inserting) adjacently within a predetermined distance from the first object. In this case, the comment includes information related to the corresponding product and / or service registered by another terminal that purchased the corresponding product and / or service (eg, a purchase testimonial, a user review, etc.).

In this case, the object and / or other additional object added to the preliminary panoramic image information gradually increases in the maximum state of the light (or a specific color) with respect to the corresponding object at a predetermined period so as to easily identify the corresponding object (or the other additional object). It may be a state in which the effect of repeatedly changing from the off state to the maximum state again, the light flickering effect, the neon sign effect, and the color of the light changes to different colors at predetermined cycles may be applied.

As described above, various effects may be applied to the object added to the preliminary panoramic image information and / or other additional objects, and may be configured to facilitate identification in the 3D panoramic image information.

In addition, the generated 3D panoramic image includes (or linked / matched) location information about a first object, a product, a point (or a coordinate) included in the 3D panoramic image.

In addition, the server 700 receives a search word transmitted from the terminal 600, identification information of the terminal 600, and the like.

Also, the server 700 checks a 3D panoramic image or image corresponding to the search word among a plurality of 3D panoramic images or images.

In addition, the server 700 transmits a 3D panoramic image or image corresponding to the identified search word to the terminal 600.

In addition, when an object included in the 3D panoramic image provided to the terminal 600 is selected, the server 700 adds an additional 3D panorama related to the object based on the information about the corresponding object transmitted from the terminal 600. Provide an image to the terminal 600.

As such, when an object to which a preset effect is applied is selected in the 3D panoramic image related to the previously selected category displayed on the current screen of the terminal 600, the server 700 requests the terminal 600. The terminal 600 may repeatedly provide an image related to the object selected in the terminal 600 (or the interest image / 3D panoramic image according to the purpose of the user of the terminal 600) to the terminal 600.

In addition, through the additional image providing process according to the purpose of the user of the terminal 600, the server 700 is linked to the terminal 600 and the payment server (not shown) by the user of the terminal 600 The purchase function (or payment function) related to the selected function can be performed. In this case, the server 700 may perform the purchase function by a simple payment function based on a (personal identification number) registered by the terminal 600. Here, the PIN number may be a PIN number corresponding to a payment card type associated with the user of the terminal 600.

In addition, the server 700 may perform an augmented reality image generation function performed in the terminal 600.

An augmented reality image generated according to an embodiment of the present invention includes location information for an object, an additional object, a point (or coordinates / coordinates corresponding to the object and / or the additional object) included in the augmented reality image, or the like (or Interlocking / matching).

That is, the augmented reality image is included in the augmented reality image so as to search (or verify) the augmented reality image corresponding to an arbitrary terminal (including the location information of the terminal 600, etc.) in the future. Configure to include location information about objects, additional objects, points, and so on.

In addition, the server 700 receives a keyword (or search term) transmitted from the terminal 600, identification information of the terminal 600, and the like.

In addition, the server 700 checks an augmented reality image corresponding to the keyword among a plurality of augmented reality images.

In addition, the server 700 transmits an augmented reality image corresponding to the identified keyword to the terminal 600.

In addition, when the additional object included in the augmented reality image provided to the terminal 600 is selected, the server 700 is associated with the additional object based on the information on the additional object transmitted from the terminal 600. Provide additional information to the terminal 600. In this case, the additional information (or additional information related to the 3D type additional object) includes real time additional information related to the additional object, a related video, a coupon (or coupon information), and the like.

In addition, the server 700 is connected to the terminal 600, the terminal 600 to output an app (or a dedicated app / shopping app) for outputting the augmented reality image provided by the server 700, etc. Install on.

In addition, the server 700 manages an image captured by a 360 degree camera provided in a specific store.

In this case, the 360-degree camera is one or more stores of a particular store (or offline) that the 360-degree camera is installed, merchandise (or exhibited goods), POP, flyers, banner advertising, bulletin board advertising, poster advertising, signage, An image (or a 2D image / panorama image) including a price tag is photographed. Here, the store includes a department store, a shopping mall, an outlet, a restaurant, a hospital, a building (or a building), and the like. In this case, the 360-degree camera can take a panoramic image of a two-dimensional form.

In addition, the image frame processed by the 360-degree camera may be stored in a memory (not shown) included in the 360-degree camera itself, or may be provided to the server 700, the terminal 600, or the like through wired / wireless communication. have.

In addition, the server 700 recognizes one or more objects in an image captured by a 360 degree camera provided in the specific store. Here, the object includes one or more stores offline, products in the store, a POP, a flyer, a banner advertisement, a bulletin board advertisement, a poster advertisement, a signboard, a price tag, and the like.

In addition, the server 700 generates an augmented reality image by inserting (or adding / incorporating) an additional object (for example, an object having a three-dimensional shape) related to the recognized object in the photographed image. Herein, the additional object may be configured in a three-dimensional form and may be an object for providing real-time additional information (or detailed information), a related video, a coupon, and the like related to the recognized object.

That is, the server 700 identifies the additional object of the three-dimensional form related to the recognized object.

In addition, the server 700 generates an augmented reality image by inserting an additional object of a three-dimensional form related to the recognized object in the photographed image of a two-dimensional form. Here, the augmented reality image may be composed of the photographed image in a two-dimensional form and an additional object related to the recognized object in the three-dimensional form.

As such, the server 700 may cause the user's interest and / or interest in the two-dimensional image captured by the 360 degree camera provided in the specific store and provide additional information. A new augmented reality image may be generated by inserting (or adding) an additional object having a 3D shape in the image.

In addition, the server 700 captures (or acquires) an arbitrary image (or screen) among the generated augmented reality image information. Here, the image may be an image captured (or obtained) from an augmented reality image in which an additional object is added (or inserted) to an image captured by the 360 degree camera 620 provided in a specific store. In addition, the image may include a state in which at least one object and additional objects related to the object are included. Here, the object may include one or more stores offline, products in the stores, POPs, flyers, banner advertisements, bulletin board advertisements, poster advertisements, signboards, price lists, and the like. In addition, the additional object may be configured in a three-dimensional form and may be an object for providing real-time additional information (or detailed information), a related video, a coupon (or coupon information), and the like related to the object. In this case, the additional object on the captured image may be displayed in a two-dimensional form.

In addition, the server 700 transmits the captured (or acquired) image to the plurality of terminals 600, respectively.

In addition, when an app installation request for displaying an augmented reality image related to the captured image is received in response to the transmission, the server 700 may be linked to the terminal 600 corresponding to the app installation request. As a result, an app for displaying the corresponding augmented reality image is installed in the terminal 600.

In addition, when a request for transmitting an augmented reality image related to the captured image is received in response to the transmission, the server 700 transmits an augmented reality image related to the captured image among a plurality of augmented reality images. Send to the terminal 600 corresponding to the request.

In addition, one or more objects included in the augmented reality image provided to the terminal 600, an additional object related to the object, in advance of any one of a specific object, a specific additional object and specific additional information from the additional information related to the additional object. When the set third event is detected, the server 700 receives a request for additional image information related to any one of the specific object, the specific additional object, and the specific additional information transmitted from the terminal 600.

In addition, the server 700 may be associated with any one of the specific object, the specific additional object, and the specific additional information in response to the request for the additional image information related to any one of the specific object, the specific additional object, and the specific additional information. Additional image information is transmitted to the terminal 600. The additional image information may include a promotion image corresponding to any one of the specific object, the specific additional object, and the specific additional information, and a 3D panoramic image (or a virtual reality image / augmented reality image) related to the specific object. In this case, when the state in which the selected third event is selected (or touched / clicked) among the specific object, the specific additional object, and the specific additional information is maintained, the specific object, the specific additional object, and the specific additional information are maintained. It includes a case where a touch gesture by the user of the terminal 600 is detected for any one of the. Here, the touch gesture by the user of the terminal 600 includes a tap, a touch and hold, a double tap, a drag, a flick, a drag and drop, a pinch, a swipe, and the like.

As such, when any one category among a plurality of categories displayed on one side of the foreground view is selected in the foreground view displayed through virtual reality or augmented reality, one or more objects related to the category in the foreground view are corresponding. The additional information may be additionally displayed on the foreground view, and when the object is selected, the additional information may be converted into a related 3D panoramic image.

In addition, when any one category among a plurality of categories displayed on one side of the foreground view is selected in the foreground view displayed through virtual reality or augmented reality, one or more objects related to the category in the foreground view. Additional information corresponding to may be additionally displayed on the foreground view, and when the additional information is selected, image information related to the corresponding object may be additionally displayed on one side of the foreground view.

In addition, when an additional object (or an additional object to which a preset effect is applied) is selected within the augmented reality image displayed on the current screen of the terminal 600 as described above, the server 700 of the terminal 600 is selected. By request, additional information such as real-time additional information, a related video, a coupon (or coupon information) related to the additional object selected by the terminal 600 may be provided to the terminal 600 in real time.

In addition, as described above, an augmented reality image may be generated by photographing a 2D panoramic image, recognizing an object in the photographed panoramic image, and inserting an additional object of 3D type related to the recognized object.

In addition, when the additional object of 3D shape inserted in the augmented reality image is selected as described above, real time additional information related to the additional object may be displayed.

In addition, when any one of a plurality of images is selected as described above, a real-time panoramic image related to the selected image may be provided.

In addition, when a preset third event for a place name and / or additional information corresponding to an object included in the panoramic image displayed through virtual reality or augmented reality is detected as described above, the third place event related to the place name and / or additional information may be detected. The additional information may be additionally displayed in a specific area within the panoramic image.

Hereinafter, a method of controlling a terminal according to the present invention will be described in detail with reference to FIGS. 11 to 40.

First, when any one (or at least one) category is selected from among a plurality of categories displayed on one side such as an aerial view, a street view, a road view, and a foreground view displayed on the terminal 600, the terminal 600 Identify one or more objects related to the selected category from among a plurality of objects included in the aerial view (or street view / road view / foreground view). In addition, the terminal 600 checks the information on the outline (or the area occupied by the outline / object) of the object in the aerial view (or street view / road view / foreground view) with respect to the identified object. In this case, the aerial view, the street view, the road view, and the foreground view may be implemented in virtual reality or may be an image (or image) captured by a camera. In addition, the street view, the road view, the foreground view, etc. may be implemented in augmented reality through the camera 620 and the display unit 640 provided in the terminal 600.

That is, in the case of an image or an image in virtual reality, the terminal 600 receives an aerial view (or street view / road view / foreground view) provided from the server 700 after accessing the server 700. The received aerial view (or street view / road view / foreground view) is displayed. Thereafter, when any one (or at least one) category is selected from a plurality of categories displayed on one side of the aerial view (or street view / road view / foreground view) displayed on the terminal 600, the terminal 600 is selected. ) May identify one or more objects related to the selected category from among a plurality of objects included in the aerial view (or street view / road view / foreground view).

In addition, in the case of augmented reality, the terminal 600 is a street view (or road view / foreground view) through a camera 620 and a display unit 640 (or a view finder) included in the terminal 600. ) (Or displays an image including one or more objects (eg, buildings, signs, etc.) through the camera 620 and the display unit 640). Then, the current location of the terminal 600, information about the object in the displayed street view (or road view / foreground view) (for example, including a signage included in the street view (or road view / foreground view)) Augmented reality information related to the corresponding street view is confirmed, and the identified augmented reality information is applied to the street view (or road view / foreground view). Subsequently, when any one (or at least one) category is selected from among a plurality of categories displayed on one side of the street view (or road view / foreground view) to which the augmented reality function displayed on the terminal 600 is applied, the terminal is selected. 600 may identify one or more objects related to the selected category from among a plurality of objects included in the street view (or road view / foreground view).

As described above, the embodiment according to the present invention selects among a plurality of categories displayed on one side of the screen for the aerial view, street view, road view, foreground view, etc. provided in the form of real image, virtual reality, augmented reality, etc. You can identify one or more objects related to a particular category.

In addition, the terminal 600 identifies a place name (or building name / name) corresponding to the at least one identified object (or at least one object related to the identified category).

In addition, the terminal 600 confirms additional information related to the identified one or more objects (or one or more objects related to the identified category). In this case, the additional information related to the object may be information provided in real time from the server 700 in response to a request for additional information related to the corresponding object of the terminal 600.

For example, the terminal 600 accesses the server 700 through a web site associated with an app or a specific server 700 pre-installed in the terminal 600. Thereafter, the terminal 600 receives a first aerial view provided from the server 700. Here, the first aerial view may be an image corresponding to a search word (for example, Yeoksam-dong Cathedral) transmitted from the terminal 600 to the server 700. Thereafter, as shown in FIG. 14, the terminal 600 displays the received first aerial view 1400. Here, one side of the first aerial view 1400 may display a plurality of preset categories (including shopping, hotels, tourist attractions, performances, sports, coupon books, etc.) 1411 to 1416. Thereafter, when the hotel 1412 is selected from a plurality of categories displayed on one side of the first aerial view 1400 illustrated in FIG. 14, the terminal 600 is included in the first aerial view 1400. Among a plurality of objects (e.g., buildings, roads, bridges, marinas, trees, etc.), one or more objects (e.g., objects included in the first aerial view 1400) related to the hotel 1412, the selected category. Check the hotels ABC Hotel and DEF Hotel). Thereafter, the terminal 600 identifies names (for example, ABC hotels) for the ABC hotels and the names for the DEF hotels (for example, DEF hotels). Thereafter, the terminal 600 includes additional information regarding the one or more identified objects (eg, ABC hotel and DEF hotel corresponding to a hotel among objects included in the first aerial view 1400). First additional information relating to the ABC hotel, and second additional information relating to the DEF hotel. Here, the first additional information includes reservation status information of the ABC hotel and the like, and the second additional information includes reservation status information and the like of the DEF hotel.

As another example, as shown in FIG. 15, the terminal 600 is located in an area (eg, Jamsil) where the corresponding terminal 600 is located through the camera 620 and the display unit 640 provided in the terminal 600. The Han River Park Helicopter) displays an image (or a first street view) 1500 of the riverbank. Subsequently, the augmented reality information related to the corresponding image is checked based on the current location of the terminal 600 and the information about the object included in the checked image. Thereafter, the terminal 600 applies the identified augmented reality information to the image. Thereafter, as illustrated in FIG. 16, the terminal 600 displays the first street view 1600 to which the augmented reality information is applied. Here, a plurality of categories (eg, including shopping, a hotel, a tourist spot, a coupon book, etc.) 1611 ˜ 1614 corresponding to the augmented reality information may be displayed on one side of the first street view 1600. Thereafter, when the coupon book 1614 is selected from the plurality of categories displayed on one side of the first street view 1600 illustrated in FIG. 16, the terminal 600 is included in the first street view 1600. One or more objects (eg, included in the first street view 1600) related to the selected coupon book 1614 among a plurality of objects (eg, including buildings, roads, bridges, marinas, trees, etc.) The first mart and the second shopping mall). Thereafter, the terminal 600 identifies names of the first marts (eg, GHI marts) and names of the second shopping malls (eg, JKH department stores). Thereafter, the terminal 600 includes additional information (eg, a first mart) related to the identified one or more objects (eg, a first mart and a second shopping mall among the objects included in the first street view 1600). Third additional information related to the second additional information related to the second shopping mall). Here, the third additional information includes discount coupon information and sale information that can be used in the first mart, and the fourth additional information includes discount coupon information and the like that can be used in the second shopping mall (S1310).

Thereafter, the terminal 600 displays a place name (or a building name) corresponding to the at least one identified object and the at least one identified object on one side where the aerial view (or street view / road view / foreground view) is displayed. Or one or more objects related to the identified category).

That is, the terminal 600 displays a name corresponding to the identified object and additional information related to the identified object adjacent to the identified object in the aerial view (or street view / road view / foreground view). . In this case, the identified one or more objects displayed on the terminal 600, names of places corresponding to the objects, and additional information related to the objects may include identification of the corresponding objects, names of places corresponding to the objects, and additional information related to the objects. To facilitate this, the effect of the light (or specific color) turning off gradually from the maximum state (or lighting state) to the maximum state (or lighting state) and then repeatedly changing from the off state to the maximum state (or lighting state) for the object at a preset time interval. An effect, a neon sign effect, and an effect of changing the color of the light to different colors at preset cycles may be applied.

For example, as shown in FIG. 17, the terminal 600 corresponds to a place name corresponding to an ABC hotel 1711, which is an object identified in relation to the hotel, which is the selected category in the first aerial view 1400. For example, the ABC hotel) 1712 and the first additional information (for example, the total number of rooms that are reservation status information of the ABC hotel: 300, the number of available rooms: 150) (1713), and the hotels in the selected category; The name corresponding to the DEF hotel 1721 that is the associated object (for example, the DEF hotel) 1722 and the second additional information (for example, the total number of rooms that are reservation status information of the DEF hotel: 1000, reservations) The number of possible rooms: 550) (1723) is displayed respectively. At this time, each of the

objects

1711 and 1721 corresponding to the ABC hotel and the DEF hotel displayed on the terminal 600 gradually turns off the light in the maximum state and turns off the light again in the maximum state at predetermined intervals. It may be a state in which a changing effect is applied (shown as a state in which a pattern is applied to the ABC hotel and the DEF hotel in FIG. 17) (S1320).

Subsequently, when a specific object is selected from one or more objects (or one or more objects for which name / additional information is displayed) to which the preset effect displayed on the terminal 600 is applied, the terminal 600 selects the selected specific object. Displays a 3D panoramic image associated with the Here, the 3D panoramic image includes at least one additional object, and the additional object has a maximum (or lit state) of light (or a specific color) with respect to the corresponding object at a predetermined period so as to easily identify the additional object. It may be applied to an effect that is gradually turned off and then repeatedly turned off to gradually turn up (or turned on), a flickering light effect, a neon sign effect, or a light color changes to different colors at predetermined intervals. have. In this case, the additional object may include a point of purchase (POP), a leaflet, a banner advertisement, a poster advertisement, a signboard, a price tag, and the like.

In addition, when another specific object is selected within the 3D panoramic image including at least one additional object displayed on the terminal 600, the terminal 600 displays another 3D panoramic image related to the selected other specific object. can do.

As such, when an object to which a preset effect is applied is selected in the 3D panoramic image related to the previously selected category displayed on the current screen of the terminal 600, the terminal 600 is the object of the user of the terminal 600. It can be repeatedly switched to the image of interest (or 3D panorama image) according to.

For example, as shown in FIG. 17, the light is gradually turned off in the maximum state at a predetermined cycle displayed on the terminal 600, and the light is repeatedly changed from the light off state to the sleep state in the maximum state. When the DEF hotel 1721 is selected from the objects ABC Hotel 1711 and DEF Hotel 1721, the terminal 600 displays a 3D panoramic image related to the selected DEF Hotel. In this case, the 3D panoramic image related to the DEF hotel may be an image representing the appearance (or the frontal view) of the DEF hotel, and the sign of the DEF hotel included in the 3D panoramic image related to the DEF hotel lights at a predetermined cycle. In this maximum state, the light is gradually turned off, and the light is turned off again. In addition, when the corresponding sign included in the 3D panoramic image related to the DEF hotel is selected, the terminal 600 corresponds to the 3D panorama corresponding to the floor guide information inside the hotel related to the selected sign (or the sign indicating the DEF hotel). Display the video. In addition, when a specific floor is selected from the 3D panoramic images corresponding to the floor guide information, the terminal 600 may display a 3D panoramic image related to the selected specific floor.

Through the additional image providing process according to the purpose of the user of the terminal 600, the terminal 600 is a reservation function for a specific room selected by the user of the terminal 600 by interworking with the server 700. , Payment function, and the like.

As described above, the terminal 600 converts the image to the image corresponding to the selected object whenever an object to which a specific effect is applied is selected from among the 3D panoramic images displayed on the terminal 600 in order to change the screen for the user's purpose. Can be switched (S1330).

In addition, when a preset first event for a specific place name and / or specific additional information is detected from among place names corresponding to one or more objects displayed on the terminal 600 and / or one or more additional information related to one or more objects, The terminal 600 may display additional information related to the specific name and / or specific additional information (or additional information corresponding to an object related to the name or additional information), and the aerial view (or the name or additional information) is displayed. It is displayed on the other side of street view / road view / foreground view. Here, the additional information may be a promotional image corresponding to a specific object related to the specific place name or specific additional information, or a 3D panoramic image related to the specific object (for example, a 3D panorama displayed by screen switching when the object is selected in the previous process). Video, etc.). In this case, when the state in which the predetermined first event is selected (or touched / clicked) in the specific place name and / or specific additional information is maintained, the terminal 600 for the specific place name and / or specific additional information is maintained. And when a touch gesture by the user is detected. Here, the touch zester by the user of the terminal 600 includes a tap, a touch and hold, a double tap, a drag, a flick, a drag and drop, a pinch, a swipe, and the like.

For example, as shown in FIG. 17, the light is gradually turned off in the maximum state at a predetermined cycle displayed on the terminal 600, and the light is repeatedly changed from the light off state to the sleep state in the maximum state. While the touch state of the first additional information 1713 related to the ABC hotel is maintained among the ABC hotel and the DEF hotel, as illustrated in FIG. 18, the terminal 600 may store the first additional information 1713. On the other side of the displayed aerial view, a promotional image (or ABC hotel promotional image) 1800 for the ABC hotel corresponding to the first additional information related to the first additional information is displayed.

As another example, as shown in FIG. 17, the light is gradually turned off in the maximum state at a predetermined cycle displayed on the terminal 600, and the light is repeatedly changed from the light off state to the sleep state at the maximum. When a swipe of the second additional information 1723 related to the DEF hotel is detected among the ABC hotel and the DEF hotel of the terminal, the terminal 600 of the aerial view in which the second additional information 1723 is displayed. On the other side, the 3D panoramic image related to the DEF hotel corresponding to the second additional information related to the second additional information is displayed for a predetermined time (for example, 5 seconds, 10 seconds, etc.) (S1340).

Thereafter, when the function corresponding to the first event is terminated, the terminal 600 displays the one before the image (or the aerial view / street view / road view / foreground view) before the first event is detected. And a place name corresponding to the at least one object, at least one additional information related to the at least one object, and the like.

That is, when the state of selecting the specific place name and / or the specific additional information is released, the terminal 600 displays an image (or the aerial view / street view / road view / foreground view) before detecting the first event. And an image corresponding to one or more objects, one or more additional information related to the one or more objects, etc. are displayed.

In addition, when the preset time elapses after the touch gesture detection by the user of the terminal 600, the terminal 600 is the image (or the aerial view / street view / road view / foreground) before the first event detection And an image in which a place name corresponding to the one or more objects, one or more additional information related to the one or more objects, etc. are displayed on one side where the view is displayed.

For example, when the touch state of the first additional information related to the ABC hotel is released, the terminal 600 stops displaying the promotional image 1800 of the ABC hotel of FIG. ABC hotels and DEF hotels (1711, 1721), names associated with them (1712, 1722), with the effect that the light is gradually turned off from the maximum state and re-lit from the off state to the maximum state at 17 preset cycles. The display returns to the screen displaying the associated first additional information and the second

additional information

1713 and 1723.

As another example, when the preset time (for example, 5 seconds) elapses after the swipe detection for the second additional information 1723, the terminal 600 displays a 3D panoramic image related to the DEF hotel. The ABC hotel and the

DEF hotels

1711 and 1721 in which the light is gradually turned off from the maximum state and the light is repeatedly changed from the off state to the maximum state repeatedly in the preset cycle of FIG. The screen returns to the screen displaying the associated

names

1712 and 1722, the associated first additional information and the second

additional information

1713 and 1723, and the like (S1350).

19 is a flowchart illustrating a control method of a terminal according to the first embodiment of the present invention.

First, the camera 620 captures an image (or two-dimensional image / panorama image) including one or more stores offline, merchandise in the store, a POP, a flyer, a banner advertisement, a bulletin board advertisement, a poster advertisement, a signage, a price tag, and the like. do. Here, the store includes a department store, a shopping mall, an outlet, a restaurant, a hospital, a building (or a building), and the like. In this case, the camera 620 may be a 360 degree camera for capturing a two-dimensional panorama image.

For example, the camera 620 captures an eleventh image including a hat and glasses, which are exhibition items in a large shopping mall. In addition, as illustrated in FIG. 20, the display unit 640 displays the photographed eleventh image 2000.

As another example, the camera 620 captures a twelfth image including a plurality of stores on-line. In addition, as illustrated in FIG. 21, the display unit 640 displays the photographed twelfth image 2100 (S1910).

Thereafter, the controller 660 recognizes an object in the captured image. Here, the object includes one or more stores offline, products in the store, a POP, a flyer, a banner advertisement, a bulletin board advertisement, a poster advertisement, a signboard, a price tag, and the like.

For example, the controller 660 recognizes hats and glasses included in the captured eleventh image, respectively.

As another example, the controller 660 recognizes a plurality of stores (eg, ABC hotels, DEF hotels, etc.) included in the twelfth image (S1920).

Thereafter, the controller 660 inserts (or adds) an additional object (for example, an object of 3D type) related to the recognized object to generate an augmented reality image in the captured image. Here, the additional object may be configured in a three-dimensional form and may be an object for providing real-time additional information (or detailed information), a related video, a coupon (or coupon information), etc. related to the recognized object.

In addition, the controller 660 generates an augmented reality image by inserting an additional object of a three-dimensional form associated with the recognized object, adjacent to the recognized object in the photographed image of a two-dimensional form. do. Here, the augmented reality image may be composed of the photographed image in a two-dimensional form and an additional object related to the recognized object in the three-dimensional form.

For example, the controller 660 may include a twenty-first embodiment for displaying real time additional information (or product detail information), a related video, a coupon (or coupon information), etc. related to the hat 2210 included in the eleventh image. The 22nd additional object 2240 for identifying the additional object 2220 and representing real time additional information, a related video, a coupon (or coupon information), etc. related to the glasses 2230 included in the eleventh captured image, is displayed. Check it. In addition, as illustrated in FIG. 22, the controller 660 inserts the identified twenty-first additional object 2220 and the twenty-second additional object 2240 into the eleventh image 2000 to insert a first augmented reality image. (2250). In this case, the first augmented reality image may be a state formed of the eleventh image of the 2D form, the 21st additional object of the 3D form, and the 22nd additional object. In addition, the hat in the first augmented reality image is a state in which the light is gradually turned off from the maximum state at a predetermined cycle and the light is repeatedly changed from the unlit state to the maximum state repeatedly (in the first embodiment in FIG. 22). And a pattern is applied to the hat in the augmented reality image.

As another example, the controller 660 may include real-time reservation status information (or real-time additional information), a related video (or promotional image), a coupon (or coupon information) related to the ABC hotel 2310 included in the twelfth image. Check the thirty-first additional object 2320 to display the real-time reservation status information (or real-time additional information) related to the DEF hotel 2330 included in the captured twelfth image, the related video (or the promotional video). Check for a thirty-second additional object 2340 to indicate " In addition, as illustrated in FIG. 23, the control unit 660 inserts the identified thirty-first additional object 2320 and the thirty-second additional object 2340 into the twelfth image 2100 to insert a second augmented reality image. (2350). In this case, the second augmented reality image may be in a state formed of a twelfth image having a two-dimensional form, a thirty first additional object having a three-dimensional form, and a thirty-second additional object. In addition, the ABC hotel and the DEF hotel in the second augmented reality image have a state in which the light is gradually turned off from the maximum state at a predetermined cycle and the light is repeatedly changed from the off state to the maximum state repeatedly (see FIG. 13). The image may be displayed in a state in which a pattern is applied to the ABC hotel and the DEF hotel in the second augmented reality image (S1930).

First, the display unit 640 displays a specific augmented reality image under the control of the controller 660. In this case, the augmented reality image corresponds to a keyword (or search term) requested by the user of the terminal 600, and is stored in the terminal 600 in advance or transmitted from any server 700 in real time. It may be.

For example, as illustrated in FIG. 25, the display unit 640 displays the first augmented reality image 2250 previously stored in the storage unit 630 under the control of the controller 660. The first augmented reality image may include an image of a hat and glasses having a two-dimensional shape, a twenty-first shape having a three-dimensional shape to represent real time additional information, a related video, a coupon (or coupon information), and the like related to the hat. The object may include a twenty-second additional object in a three-dimensional form for representing real time additional information related to the glasses, a related video, a coupon (or coupon information), and the like. In addition, the hat in the first augmented reality image is a state in which the light is gradually turned off from the maximum state at a predetermined cycle and the light is repeatedly changed from the unlit state to the maximum state repeatedly (in the first embodiment in FIG. 25). And a pattern is applied to the hat in the augmented reality image.

As another example, the communication unit 610 receives a second augmented reality image transmitted in real time from the server 700 in relation to a place name (eg, reverse three-way range) that is a specific keyword under the control of the controller 660. . In addition, as illustrated in FIG. 26, the display unit 640 displays the second augmented reality image 2350 received through the communication unit 610 under the control of the controller 660. The second augmented reality image may include an image including a plurality of offline stores (eg, ABC hotel, DEF hotel, etc.) in a two-dimensional form, reservation status information related to the ABC hotel, and video related to the ABC hotel. (Or ABC hotel promotion video), a thirty-first additional object in a three-dimensional form for representing a coupon (or coupon information), reservation status information related to the DEF hotel, and video related to the DEF hotel (or DEF hotel promotion video) , The thirty-second additional object in a three-dimensional form for representing a coupon (or coupon information) and the like. In addition, the ABC hotel and the DEF hotel in the second augmented reality image have a state in which the light is gradually turned off from the maximum state at a predetermined cycle and the light is repeatedly changed from the off state to the maximum state repeatedly (Fig. 26). The image may be displayed in a state where a pattern is applied to the ABC hotel and the DEF hotel in the second augmented reality image (S2410).

Subsequently, when an additional object of a specific 3D shape in the augmented reality image displayed on the display unit 640 is selected, the controller 660 is associated with the selected 3D type of additional object in a specific area of the augmented reality image. The additional information is displayed on the display unit 640. In this case, the additional information (or additional information related to the additional object in the 3D form) may be information transmitted from the server 700 in real time through the communication unit 610.

In addition, the controller 660 may be further configured to identify the additional information (for example, real-time additional information related to the additional object, related video, coupon information, etc.) in a specific region (or pop-up form) in the augmented reality image of the display unit 640. Inclusive).

For example, when the real-time additional information object (or real-time additional information) 2221 included in the twenty-first additional object related to the hat in the first augmented reality image illustrated in FIG. 25 is selected, the controller 660 may select the The first request signal for the real-time additional information related to the hat is transmitted to the server 700 through the communication unit 610. Thereafter, the control unit 660 includes the communication unit 610 in real time additional information related to the hat (for example, product detailed information about the hat) transmitted from the server 700 in response to the first request signal. Receive through. Thereafter, as illustrated in FIG. 27, the controller 660 may include the received real-time additional information (eg, product detail information about the hat, etc.) 2710 in the first augmented reality image. Displayed in the first area.

As another example, when the related video object 2222 included in the twenty-first additional object related to the hat in the first augmented reality image shown in FIG. 25 is selected, the control unit 660 is connected to the related video related to the hat. The second request signal for the transmission is transmitted to the server 700 through the communication unit 610. Thereafter, the controller 660 receives, through the communication unit 610, a video related to the hat (for example, a video related to the corresponding hat) transmitted from the server 700 in response to the second request signal. . Subsequently, as shown in FIG. 28, the controller 660 displays the received video related to the hat (eg, the video related to the corresponding hat) 2810 in the second area in the first augmented reality image. do.

As another example, when the coupon 2223 included in the twenty-first additional object related to the hat in the first augmented reality image shown in FIG. 25 is selected, as shown in FIG. 29, the controller 660 Coupon information 2910 corresponding to the coupon associated with the hat is displayed in a third area of the first augmented reality image.

As another example, when the reservation status information 2321 included in the thirty-first additional object related to the ABC hotel in the second augmented reality image shown in FIG. 26 is selected, the controller 660 is connected to the ABC hotel. The third request signal for real time reservation status information is transmitted to the server 700 through the communication unit 610. Thereafter, the controller 660 is configured to provide real-time reservation status information related to the ABC hotel transmitted from the server 700 in response to the third request signal (for example, the total number of rooms which are reservation status information of the ABC hotel: 300). , Number of available rooms: 150) is received through the communication unit 610. Then, as shown in Figure 30, the control unit 660 is real-time reservation status information related to the received ABC hotel (for example, the total number of rooms that are the reservation status information of the ABC hotel: 300, the number of available rooms) 3010 are displayed in the fourth area in the second augmented reality image.

As another example, when the related video included in the thirty-first additional object related to the ABC hotel in the second augmented reality image shown in FIG. 26 is selected, the controller 660 is configured to perform the related video related to the ABC hotel. The fourth request signal is transmitted to the server 700 through the communication unit 610. Thereafter, the control unit 660 transmits the communication unit 610 to the promotion image (or ABC hotel promotion image) for the real-time ABC hotel associated with the ABC hotel transmitted from the server 700 in response to the fourth request signal. Receive through. Subsequently, as shown in FIG. 31, the controller 660 displays the received promotional image (or ABC hotel promotional image) 3110 of the real-time ABC hotel related to the received ABC hotel in the second augmented reality image. Displayed in five areas (S2420).

First, the communication unit 610 receives one or more images provided from the server 700. Here, the image may be an image captured (or obtained) from an augmented reality image in which an additional object is added (or inserted) to an image captured by the 360 degree camera 620 provided in a specific store. In addition, the image may include a state in which at least one object and additional objects related to the object are included. Here, the object may include one or more stores offline, products in the stores, POPs, flyers, banner advertisements, bulletin board advertisements, poster advertisements, signboards, price lists, and the like. In addition, the additional object may be configured in a three-dimensional form and may be an object for providing real-time additional information (or detailed information), a related video, a coupon (or coupon information), and the like related to the object. In this case, the additional object on the captured image may be displayed in a two-dimensional form.

As an example, the communication unit 610 receives a first image transmitted from the server 700. In addition, as shown in FIG. 33, the controller 660 displays the received first image 3300 on the display unit 640. Here, the first image 3300 is captured from a first augmented reality image in which an image captured in real time through a 360 degree camera installed at a furniture corner in an AAA department store includes additional objects related to one or more objects included in the image. It may be an image. In addition, the first image 3300 may include the first furniture 3310 and the second furniture 3320 included in the captured image in real time with the augmented reality function applied to the captured image, and the first image. The first additional object 3330 and the second additional object 3340 associated with the furniture 3310 and the second furniture 3320. In this case, an augmented reality function (or a virtual reality function) is applied to the first additional object 3330 and the second additional object 3340, so that the first furniture 3310 and the second additional object 3340 are applied. The first additional object 3330 and the second additional object 3340 for representing real-time additional information (or product detail information) related to the furniture 3320, related videos, coupons (or coupon information), etc. are included. Can be. In this case, the first additional object 3330 and the second additional object 3340 may be a two-dimensional form or a three-dimensional form to which an augmented reality function (or a virtual reality function) is applied according to image capture (S3210).

Subsequently, when a specific image is selected (or clicked) from one or more images displayed on the display unit 640, the controller 660 may perform an augmented reality image (or virtual reality image / or image related to the selected (or clicked) image). It is checked whether an app for displaying a 3D panoramic image is installed in the terminal 600.

For example, when the first image displayed on the display unit 640 is selected, the controller 660 is a shopping app for displaying an augmented reality image related to a furniture corner in the AAA department store provided from the server 700. It is checked whether the state is installed in the terminal 600 (S3220).

For example, when the shopping app is not installed in the terminal 600, the controller 660 communicates with the server 700 through the communication unit 610, and in conjunction with the server 700 By installing the shopping app on the terminal 600 (S3230).

In addition, the additional object may be rotated (or moved) in the left / right / up / down direction under the control of the user of the terminal 600. It can match the movement equally.

For example, when an app for displaying an augmented reality image related to the selected (or clicked) image is installed in the terminal 600, as shown in FIG. 34, the controller 660. Receives a real-time eleventh augmented reality image related to the first image provided from the server 700 through the communication unit 610. In addition, the controller 660 displays the received real-time eleventh augmented reality image (or a real-time eleventh augmented reality image related to the first image) 3400 through a shopping app installed in the terminal 600. Here, the eleventh augmented reality image 3400 may include the first furniture 3410 and the second furniture 3420 included in the eleventh image of the two-dimensional image captured in real time, and the first furniture (the three-dimensional image). The first additional object 3430 and the second additional object 3440 related to the 3410 and the second furniture 3420 may be included. In addition, the first furniture 3410 and the second furniture 3420 in the eleventh augmented reality image 3400 may be repeatedly turned off from the maximum state to the maximum state in the preset state, and then repeatedly from the off state to the maximum state. The changing effect may be applied (shown as a pattern applied to the first furniture 3410 and the second furniture 3420 in the eleventh augmented reality image 3400 on FIG. 34).

In addition, when the eleventh augmented reality image 3400 illustrated in FIG. 34 is rotated from right to left by a sliding function of the user of the terminal 600, as illustrated in FIG. 35, the controller 660. Displays a twelfth augmented reality image 3500 on the display unit 640. In this case, the first additional object 3430 and the second additional object 3440 associated with the first furniture 3410 and the second furniture 3420 are matched with the rotation, and the user easily corresponds to the rotation.

Additional objects

3430 and 3440 may be displayed to check (S3240).

For example, the real-time additional information object (or real-time additional information) 3431 included in the first additional object 3430 related to the first furniture 3410 in the eleventh augmented reality image 3400 illustrated in FIG. 34 is When selected, the controller 660 transmits a first request signal for real time additional information related to the first furniture 3410 to the server 700 through the communication unit 610. Thereafter, the controller 660 includes real-time additional information (for example, product detailed information about the first furniture) related to the first furniture 3410 transmitted from the server 700 in response to the first request signal. ) Is received through the communication unit 610. Then, as shown in FIG. 36, the controller 660 may provide the real time additional information (eg, product detail information about the first furniture) 3410 related to the received first furniture 3410. The first area is displayed in the eleventh augmented reality image 3400.

As another example, when the related video object 3432 included in the first additional object 3430 related to the first furniture 3410 in the eleventh augmented reality image 3400 illustrated in FIG. 34 is selected, the controller 660 transmits a second request signal for a related video related to the first furniture 3410 to the server 700 through the communication unit 610. Thereafter, the controller 660 transmits a related video (eg, a video related to the first furniture) related to the first furniture 3410 transmitted from the server 700 in response to the second request signal. Receive via 610. Thereafter, as shown in FIG. 37, the controller 660 may display the received video related to the first furniture 3410 (for example, the video / storage furniture video related to the first furniture) 3710. The second area is displayed on the second area in the eleventh augmented reality image 3400.

As another example, when the coupon 3333 included in the first additional object 3430 related to the first furniture 3410 in the eleventh augmented reality image 3400 illustrated in FIG. 34 is selected, FIG. As illustrated, the controller 660 displays coupon information 3810 corresponding to the coupon associated with the first furniture 3410 in a third area of the eleventh augmented reality image 3400 (S3250).

For example, as shown in FIG. 34, the light is turned off gradually in the maximum state and the light is repeatedly changed from the light off state to the maximum state in a predetermined cycle displayed on the display unit 640. While the touch state of the second furniture 3420 is maintained in the eleventh augmented reality image 3400 related to the furniture store in the AAA department store, as illustrated in FIG. 39, the controller 660 is configured to include the eleventh augmentation. On the other side of the reality image 3400, a promotional image (or an advertisement image for the second household) 3910 for the second household is displayed on the display unit 640.

As another example, as shown in FIG. 34, when a swipe of the second furniture 3420 displayed on the display unit 640 is detected, the controller 660 may display the eleventh augmented reality image ( On the other side of the 3400, the promotional image related to the second household is displayed on the display unit 640 for a preset time (including 5 seconds, 10 seconds, etc.).

As another example, as shown in FIG. 40, while the touch state of the real-time additional information 3610 related to the first furniture 3410 displayed on the display unit 640 shown in FIG. 36 is maintained, The controller 660 may be connected to the real-time additional information (or the first furniture) on the other side of the eleventh augmented reality image 3400 in which real-time additional information 3610 related to the first furniture 3410 is displayed. The promotional image 4010 for the first household is displayed on the display unit 640 (S3260).

Thereafter, when the function corresponding to the third event is terminated, the controller 660 returns to the augmented reality image that is an image before the third event detection or the augmented reality image in which additional information is displayed.

For example, when the touch state of the second furniture 3420 is released, the controller 660 stops displaying the promotional image 3910 of the second furniture of FIG. 39, and the foregoing FIG. 34. The display returns to a screen displaying a real-time eleventh augmented reality image 3400 related to the first image of the.

As another example, when the preset time (for example, 5 seconds) elapses after the swipe detection of the second furniture 3420, the controller 660 displays the display of a promotional image related to the second furniture. The display returns to the screen displaying the eleventh augmented reality image 3400 related to the first image of FIG. 34.

As another example, when the touch state of the real time additional information 3610 related to the first furniture 3410 is released, the controller 660 may perform the real time additional information (or the first household) of FIG. 40. The display of the promotion image 4010 for the first household in relation to the second household is stopped, and the real-time eleventh augmented reality image in which the real-time additional information 3610 related to the first furniture 3410 of FIG. 36 is displayed together is displayed. The display returns to 3400 (S3270).

As described above, an embodiment of the present invention photographs an object provided in an offline store or a street store, and provides goods and service information, additional information, and the like related to the photographed object through an augmented reality function, thereby realizing a department store. , While providing views that are encountered in shopping malls, streets, etc. while providing additional information related to products or stores offered offline, it can improve the convenience of use, provide various information to users, and purchase small offline stores Even if the user of the visitor can be provided with the same assortment and shopping information of the large store.

In addition, the embodiment of the present invention, as described above, in addition to providing product information and / or service information related to the object photographed at the purchase terminal, in addition to the hand carry goods indicating that the product can be directly received or delivered Through augmented reality, additional information on whether a product is a product that is not handled by an offline store or a street store or sold out due to exhaustion of sales is provided through augmented reality to further check product information of domestic and international offline stores. This can expand buyers' choices and improve reliability between buyers and sellers when buying directly from abroad.

According to an embodiment of the present invention, as described above, when any one of a plurality of categories displayed on one side of the foreground view is selected in the foreground view displayed through the virtual reality or augmented reality, the foreground view Additional information corresponding to one or more objects related to the category is further displayed on the foreground view, and when the object is selected, the additional information is switched to a related 3D panoramic image to a screen desired by a user in virtual reality or augmented reality. The convenience of the user can be improved by facilitating screen switching.

According to an embodiment of the present invention, as described above, when any one of a plurality of categories displayed on one side of the foreground view is selected in the foreground view displayed through the virtual reality or augmented reality, the foreground view Additionally displays additional information corresponding to one or more objects related to the category on the foreground view, and additionally displays image information related to the object on one side of the foreground view when the additional information is selected, A user interface may be provided for the user's convenience, and the user's convenience may be improved by providing additional information related to an object of interest to the user on the current screen.

In addition, as described above, the embodiment of the present invention captures a panorama image in a 2D form, recognizes an object in the captured panorama image, and augments by inserting an additional object in a 3D form related to the recognized object. By generating a real image, it is easy to provide additional information related to a specific object.

In addition, the embodiment of the present invention, as described above, when the additional object of the 3D shape inserted in the augmented reality image is selected, by displaying the real-time additional information associated with the additional object, to provide a user interface for user convenience In addition, the user's convenience can be improved by providing additional information related to an object of interest to the user on the current screen.

In addition, as described above, when an image of any one of the plurality of images is selected, an embodiment of the present invention provides a real-time panoramic image related to the selected image, thereby promoting a promotion effect on goods and / or services related to the image. Increase, and can easily check the real-time information on the goods and services of interest.

According to an embodiment of the present invention, as described above, when a preset third event for a place name and / or additional information corresponding to an object included in a panoramic image displayed through virtual reality or augmented reality is detected, Additional information related to the place name and / or additional information is additionally displayed in a specific area within the panoramic image to provide a user interface for user convenience and to provide additional information related to an object of interest to the user on the current screen. It is possible to improve the convenience of the user.

The above description may be modified and modified by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

The present invention captures an object provided in an offline store or a street store, and provides goods and service information, additional information, and the like related to the photographed object through an augmented reality function, so as to view a view encountered in an actual department store, a shopping mall, or a street. It can be provided as it is, while providing additional information related to products or stores that are provided offline, improving convenience for use, and providing a variety of information to users, even if a user of a purchase terminal visits a small offline store Assortment and shopping information can be provided in the same way, and can be widely used in a panoramic image field, a shopping information provision field, a marketing field, a terminal field, a server field, and the like.

Claims

In the control method of the augmented reality-based shopping information providing system to which the purchase terminal is applied,

Photographing, by a purchase terminal, an object provided in an offline store or a street store;

Transmitting, by the purchasing terminal, image information including the object and identification information of the purchasing terminal;

Recognizing, by a server, an object included in the image information transmitted from the purchase terminal;

Confirming, by the server, a code related to the recognized object among a plurality of pre-stored code for each object;

Identifying, by the server, one or more goods and service information corresponding to the identified code among previously stored goods and service information for each code;

Identifying, by the server, additional information applicable to the identified one or more goods and service information;

Arranging, by the server, the one or more goods and service information in any one of a predetermined discount rate, an accumulation rate, a sales amount, a popularity, and a brand; And

Transmitting, by the server, a predetermined number of goods and services information among the sorted one or more goods and services information and the identified additional information to the purchasing terminal,

The object is

Point of Purchase (POP), leaflet, banner advertisement, board advertisement, poster advertisement, billing, sign, and price list included in the offline store or the street store tag), and excludes goods and services provided by the offline store or the street store,

The additional information,

Information indicating whether the product is a hand carry product indicating that the product is directly received or delivered, and information indicating that the product is not sold in an offline store or a street store or is out of stock due to exhausted sales. Contains any one of the pieces of information that indicate whether the package is a

Identification information of the purchase terminal,

A method of controlling an augmented reality based shopping information providing system to which a purchase terminal including at least one of a mobile directory number (MDN), a mobile IP, a mobile MAC, a subscriber identity module (SIM) card unique information, and a serial number is applied.
The method of claim 1,

Displaying, by the purchasing terminal, one or more goods and service information and additional information transmitted from the server on the photographed image information by applying an augmented reality function;

When a product corresponding to the hand carry product is selected from the one or more products and service information displayed on the purchase terminal, by linking with the purchase terminal, the server, and the payment server, the product corresponding to the selected hand carry product Performing a payment function for the payment amount;

Receiving, by a store server, member information transmitted from the server, information on the hand carry goods, and identification information of the purchase terminal; And

The store server may further include providing the hand carry product to the user of the purchase terminal after performing an authentication procedure for the user of the purchase terminal visited to the store. Control method of a reality-based shopping information providing system.
The method of claim 1,

Displaying, by the purchasing terminal, one or more goods and service information and additional information transmitted from the server on the photographed image information by applying an augmented reality function;

When a product corresponding to a delivery product is selected from one or more products and service information displayed on the purchase terminal, a payment amount of the product corresponding to the selected delivery product by interworking with the purchase terminal, the server, and the payment server Performing a payment function for the;

Receiving, by a store server, member information transmitted from the server, information on the hand carry goods, and identification information of the purchase terminal; And

The method of controlling the augmented reality-based shopping information providing system to which a purchase terminal is applied, further comprising the step of delivering the delivery goods to a delivery address based on the delivery address information included in the member information by the store server.
In the augmented reality-based shopping information providing system applied to a purchase terminal,

A purchase terminal photographing an object provided in an offline store or a street store, and transmitting image information including the object and identification information of the purchase terminal; And

Recognizes an object included in the image information, checks a code related to the recognized object among a plurality of pre-stored code for each object, one or more goods corresponding to the identified code among pre-stored goods and service information per code, and Confirm service information, identify additional information applicable to the identified one or more goods and service information, and transmit one or more goods and service information corresponding to the identified code and the confirmed additional information to the purchase terminal; Server,

The object is

At least one of a POP, a flyer, a banner advertisement, a bulletin board advertisement, a poster advertisement, a signage, and a price tag provided in the offline store or the street store, and excludes goods and services provided by the offline store or the street store. ,

The additional information,

Information indicating whether the product is a hand carry product indicating that the product is directly received or delivered, and information indicating that the product is not sold in an offline store or a street store or is out of stock due to exhausted sales. Contains any one of the pieces of information that indicate whether the package is a

Identification information of the purchase terminal,

Augmented reality-based shopping information providing system applied to a purchase terminal including at least one of MDN, mobile IP, mobile MAC, Sim card unique information and serial number.
The method of claim 4, wherein

The purchase terminal,

Augmented reality-based shopping information providing system with a purchase terminal, characterized in that to display one or more goods and service information and additional information transmitted from the server on the photographed image information by applying an augmented reality function.
The method of claim 5, wherein

When a product corresponding to the hand carry product is selected from the one or more products and service information displayed on the purchase terminal, by linking with the purchase terminal, the server, and the payment server, the product corresponding to the selected hand carry product Perform a payment function on the payment amount,

After receiving the member information transmitted from the server, the information on the hand carry product and identification information of the purchase terminal, and performing an authentication procedure for the user of the purchase terminal visited the store, the hand carry product is purchased An augmented reality-based shopping information providing system to which a purchase terminal is applied, further comprising a store server provided to a user of the terminal.
The method of claim 5, wherein

When a product corresponding to a delivery product is selected from one or more products and service information displayed on the purchase terminal, a payment amount of the product corresponding to the selected delivery product by interworking with the purchase terminal, the server, and the payment server Perform a billing function for,

Further comprising a store server for receiving the member information transmitted from the server, the information on the delivery product and the identification information of the purchase terminal, and delivers the delivery product to the delivery destination based on the delivery address information included in the member information. Augmented reality-based shopping information providing system to which the purchase terminal is applied.
In the control method of the terminal,

Displaying, by the display unit, a screen in any one form of the aerial view, street view, road view, and foreground view;

Identifying, by the controller, one or more objects related to the selected category from among a plurality of objects included in the screen when any one category among a plurality of categories included in the screen displayed on the display unit is selected;

Confirming, by the controller, outline information of the identified object in the screen; And

And displaying, by the display unit, a name corresponding to the at least one identified object and additional information related to the at least one identified object, respectively, on one side of the screen.
In the control method of the terminal,

Displaying an augmented reality image through a display unit;

When the specific additional object included in the augmented reality image is selected, confirming additional information of any one of real-time additional information, related video, and coupon information related to the selected additional object through a control unit; And

Controlling the display unit to display the identified additional information in a specific area within the augmented reality image of the display unit, by the control unit;

The augmented reality video,

The control method of the terminal, characterized in that the additional object of the three-dimensional form associated with the object included in the image in the two-dimensional image.
In the control method of the terminal,

Photographing a two-dimensional image through a camera;

Displaying the photographed image through a display unit;

Recognizing one or more objects included in the captured image through a control unit;

Identifying, via the control unit, an additional object having a three-dimensional shape related to the recognized object;

Generating an augmented reality image by inserting the identified additional object in the 3D form into the photographed 2D form image through the control unit; And

And displaying the generated augmented reality image through the display unit.
In the control method of the terminal,

Receiving, by the communication unit, one or more images provided from the server;

Displaying, by a display unit, the one or more images received;

Confirming, by a controller, whether an app for displaying an augmented reality image related to the selected image is installed in a terminal including the communication unit when a specific image is selected from one or more images displayed on the display unit; And

And when the app for displaying an augmented reality image related to the selected image is not installed in the terminal, by the controller, installing the app on the terminal by interworking with the server. Control method of a terminal comprising.
In the terminal,

A display unit which displays a screen of any one type of the aerial view, the street view, the road view, and the foreground view; And

When any one category among a plurality of categories included in the screen displayed on the display unit is selected, one or more objects related to the selected category are checked among the plurality of objects included in the screen, and the confirmation in the screen is performed. And a control unit which checks the outline information of the identified object and controls the display unit to display name information corresponding to the identified one or more objects and additional information related to the identified one or more objects on one side of the screen.
In the terminal,

A camera for photographing a three-dimensional image;

A display unit which displays the captured image; And

Recognize one or more objects included in the photographed image, identify additional objects of three-dimensional form related to the recognized object, and add the identified three-dimensional additional objects to the photographed two-dimensional image. And a controller configured to insert and generate an augmented reality image and to control the display unit to display the generated augmented reality image.
In the terminal,

A display unit displaying an augmented reality image; And

When a specific additional object included in the augmented reality image is selected, the additional information of any one of real-time additional information, related video, and coupon information related to the selected additional object is checked, and the specific additional object included in the augmented reality image is checked. A control unit for controlling the display unit to display the confirmed additional information;

The augmented reality video,

And a three-dimensional additional object related to an object included in the image in a two-dimensional image.
In the terminal,

Communication unit for receiving one or more images provided from the server;

A display unit displaying the received one or more images; And

When a particular image is selected from the one or more images displayed on the display unit, it is determined whether an app for displaying an augmented reality image related to the selected image is installed in a terminal including the communication unit. And a controller configured to install the app on the terminal by interworking with the server when an app for displaying an augmented reality image related to the selected image is not installed on the terminal.