KR20200134503A - System and method for providing and guiding optimized shoping routes - Google Patents

Abstract

A system for providing and guiding an optimized shopping route according to an embodiment of the present invention is a management system of a large store including fixed surveillance cameras installed by a plurality of areas in which products are arranged and at least one mobile camera providing a full view of a store. The system for providing and guiding an optimized shopping route comprises: a store interior map generation unit generating an entire flat map of the store interior based on image information transmitted from fixed surveillance cameras and/or at least one mobile camera; a user recognition for recognizing a user terminal entering the large store; and a shopping information providing unit providing the recognized user with route information to a store where a product to be purchased by the user is displayed using the generated internal map when the user terminal is recognized by the user recognition unit.

Description

System and method for providing and guiding optimal shopping routes {SYSTEM AND METHOD FOR PROVIDING AND GUIDING OPTIMIZED SHOPING ROUTES}

An embodiment of the present invention relates to a system for providing and guiding an optimal shopping route and a method thereof. In particular, a map inside a store in which changes such as changes in the location of a store in a large shopping store are continuously updated, and is used to optimize the It relates to a system and method for providing and guiding a shopping route.

Consumers who visit large stores, such as large marts or department stores, shop based on the list of items they want to purchase, and in general, consumers visit the large stores and it is likely that the items they mainly purchase are determined according to their shopping tendency. high.

However, the large store has a greater number of times to remodel due to reasons such as changes in the stores, compared to the smaller store, and the location of the store where products are arranged may change each time. Accordingly, even consumers who frequently visit the large-sized stores do not know in advance the location of the product to be purchased due to such fluctuations, and thus often sell off the shopping space to find the location where the product is displayed. Accordingly, there may be a problem that the moving distance and the time required for purchasing a product in a shopping space increase.

An embodiment of the present invention generates an internal map of the large store using image information captured by a plurality of fixed surveillance cameras and a predetermined mobile camera arranged in a large store, and uses this to provide a user entering a large store. It is an object of the present invention to provide an optimal shopping route providing and guiding system and method for guiding the optimum shopping route and various shopping information.

In order to achieve the above object, an optimal shopping route providing and guidance system according to an embodiment of the present invention includes fixed surveillance cameras installed for each of a plurality of areas in which products are arranged, and at least one mobile camera capable of viewing the entire store. What is claimed is: 1. A system for managing a large store, comprising: a store interior map generator for generating an entire flat map of an interior of a store based on image information transmitted from the fixed surveillance cameras and/or at least one or more mobile cameras; A user recognition unit for recognizing a user terminal entering the large store; And a shopping information providing unit that, when the user terminal is recognized by the user recognition unit, provides the recognized user with information on a route to a store where a product to be purchased by the user is displayed using the generated internal map.

A display product information correction unit for correcting display product information by checking whether the final product matches the product displayed at a preset location by using payment information according to the purchase of the product of the user terminal user transmitted from the cash register of the large store. Can include.

The store interior map generator may generate a store interior map by synthesizing first captured image information transmitted from each of the fixed surveillance cameras and second captured image information transmitted from the at least one mobile camera.

The store interior map generator may perform image processing processing by determining a degree of distortion of each of the first captured image information and calculating a perspective value of the first captured image information.

The store interior map generation unit analyzes the first photographed image information transmitted from the fixed surveillance cameras installed in each area, recognizes whether or not the products displayed on the shelves for each area are changed, and the changed product based on this Display information and product classification information corresponding thereto may be generated.

The store interior map generator may update the generated interior map by reflecting display information of the changed product and product classification information corresponding thereto.

When it is determined that the user terminal is a registered member when the user terminal enters the large store, the user recognition unit can recognize the entrance of the user terminal through a short-range communication method based on user information previously stored in the database. have.

If the user terminal is not recognized by the short-range communication method, the database through the user's face, clothing, skin color, and key information captured by an adjacent fixed surveillance camera or a mobile camera when the user holding the user terminal enters. You can recognize the user by searching the customer's appearance information stored in.

When the shopping guide application installed in the user terminal is executed and the generated internal map is displayed on the user terminal, the shopping information providing unit may provide information on a shopping route of the shortest distance using the internal map of the store.

The method for providing and guiding an optimal shopping route according to an embodiment of the present invention includes fixed surveillance cameras installed for each of a plurality of areas in which products are arranged, and at least one mobile camera capable of viewing the entire store. A method for providing and guiding a shopping route provided to a user terminal, the method comprising: generating an entire flat map of a store based on image information transmitted from the fixed surveillance cameras and/or at least one mobile camera; Recognizing a user terminal entering the large store; When the user terminal is recognized, providing the recognized user with route information to a stand in which the product to be purchased by the user is displayed using the generated internal map.

It may further include the step of correcting the displayed product information by checking whether the final purchase product matches the product displayed at a preset location by using payment information according to the product purchase of the user terminal user transmitted from the cash register of the large store. have.

In the generating of the store interior map, the store interior map may be generated by synthesizing first captured image information transmitted from the respective fixed surveillance cameras and second captured image information transmitted from the at least one or more mobile cameras. have.

In the generating of the store interior map, image processing may be performed by determining a degree of distortion of each of the first captured image information and calculating a perspective value of the first captured image information.

In the generating of the store interior map, the first photographed image information transmitted from the fixed surveillance cameras installed for each area is analyzed to recognize whether or not the products displayed on the shelves for each area are changed, and based on this It may further include generating display information of the changed product and product classification information corresponding thereto.

The generating of the store interior map may further include updating the generated interior map by reflecting display information of the changed product and product classification information corresponding thereto.

Recognizing the user terminal may include, when the user terminal enters the large store, if it is determined that the user terminal is a registered member, the user terminal enters the user terminal through a short-range communication method based on user information previously stored in the database. Can be recognized.

If the user terminal is not recognized by the short-range communication method, the database through the user's face, clothing, skin color, and key information captured by an adjacent fixed surveillance camera or a mobile camera when the user holding the user terminal enters. You can recognize the user by searching the customer's appearance information stored in.

In the providing of the shopping route information, when the shopping guide application installed in the user terminal is executed and the generated internal map is displayed on the user terminal, the shopping route information of the shortest distance is provided using the internal map of the store. can do.

According to such an embodiment of the present invention, an internal map of the large store is generated by using image information captured by a plurality of fixed surveillance cameras and a predetermined mobile camera arranged in a large store, and using this, An optimal shopping route and various shopping information can be guided to the entering user.

1 is a block diagram showing the overall configuration of a system for providing and guiding an optimal shopping route according to an embodiment of the present invention.
FIG. 2 is a block diagram showing the configuration of the user terminal shown in FIG. 1;
3 is a block diagram showing the configuration of a store management apparatus according to an embodiment of the present invention.
4 is a block diagram illustrating an embodiment of an internal configuration of the processor shown in FIG. 3.
5 is a diagram schematically illustrating configurations for generating a map inside a store according to an embodiment of the present invention.
6 is a diagram illustrating steps of generating and updating a store interior map in a method for providing and guiding an optimal shopping route according to an embodiment of the present invention.
7 is a diagram illustrating steps of providing shopping information using a map inside a store to a user and correcting display product information in an embodiment of the present invention.

The content described in the technology column that serves as the background of the present invention is only intended to help understanding the background art of the technical idea of the present invention, and therefore it can be understood as the content corresponding to the prior art known to those skilled in the art. none.

In the description below, for purposes of explanation, many specific details are presented to aid in understanding the various embodiments. However, it is obvious that various embodiments may be practiced without these specific details or in one or more equivalent manners. In other instances, well-known structures and devices are shown in block diagrams in order to avoid unnecessarily obscuring the various embodiments.

Each block in the attached block diagram may be executed by computer program instructions (execution engine), and these computer program instructions may be mounted on a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment. The instructions, executed by the processor of other programmable data processing equipment, generate means for performing the functions described in each block of the block diagram.

These computer program instructions can also be stored in computer-usable or computer-readable memory that can be directed to a computer or other programmable data processing equipment to implement a function in a particular way, so that the computer-usable or computer-readable memory It is also possible to produce an article of manufacture in which the instructions stored in the block diagram contain instruction means that perform the functions described in each block of the block diagram.

In addition, since the computer program instructions may be mounted on a computer or other programmable data processing equipment, a series of operation steps are performed on a computer or other programmable data processing equipment to create a process that is executed by a computer, It is also possible for the instructions to perform the possible data processing equipment to provide a function to perform the functions described in each block of the block diagram.

In addition, each block may represent a module, segment, or part of code including one or more executable instructions for executing specified logical functions, and in some alternative embodiments, the functions mentioned in the blocks or steps are ordered. It is also possible to occur outside of.

That is, the two illustrated blocks may actually be executed at the same time, and the blocks may be executed in the reverse order of corresponding functions as needed.

The terms used herein are for the purpose of describing specific embodiments and not for the purpose of limiting. Throughout the specification, when a certain part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary. Unless otherwise defined, terms used herein have the same meaning as commonly understood by those of ordinary skill in the field to which the present invention belongs.

Hereinafter, embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

1 is a block diagram showing the overall configuration of a system for providing and guiding an optimal shopping route according to an embodiment of the present invention.

Referring to FIG. 1, an optimal shopping route providing and guidance system according to an embodiment of the present invention includes a plurality of user terminals 110_1, 110_2,..., 110_n, a network 120, a store management device 130, and It is configured to include a large store 140.

A shopping guide application provided by the store management device 130 may be installed on the user terminals 110, and various shopping information of the large store 140 entering the store may be obtained through this. For example, the shopping guide application may provide information on an optimal route to a product to be purchased by a user by using an internal map of the large store 140. If the shopping guide application is not installed in the user's terminal, the user may download the application from the store management device 130 and install it in the user's terminal.

The user terminals 110 include mobile devices that the user can carry and may be a smart phone, a tablet, a notebook, or the like.

The network 120 may be implemented as a wireless communication network or a wired communication network. In this case, the communication method is not limited, and not only a communication method using a communication network (eg, a mobile communication network, a wireless LAN network, a wired Internet, a broadcasting network) that the network 120 may include, but also a short-range wireless communication between devices may be included.

The store management device 130 generates an interior map of the large store 140, periodically checks whether the interior layout of the large store 140 changes, and updates the interior map, based on this An operation of providing optimal route information to a product to be purchased by the user may be performed to user terminals entering 140 using the updated internal map. In this case, the store management device 130 may be configured with at least one server device.

The large store 140 may be a large shopping mall such as a warehouse type large mart or a department store, and the large store 140 includes a plurality of fixed surveillance cameras (CCTV) 140a, a mobile camera 140b, and a cash register 140c. ) Are placed.

The store management device 130 according to the embodiment of the present invention receives photographed image images from the plurality of fixed surveillance cameras (CCTV) 140a and mobile cameras 140b installed in the large store 140, and uses this You can create an interior map of a large store. In addition, it is possible to perform an operation of correcting the displayed product information by checking whether the product purchased by the user matches the product displayed at a preset location by using the payment information according to the user's product purchase transmitted from the cash register 140c. have.

That is, the store management device 130 includes a store interior map generator (410 in Fig. 4) that generates a map inside the store, a user recognition unit (420 in Fig. 4) that recognizes a user terminal entering the store, and When the displayed product information is changed differently from a preset method, a display product information correction unit (430 in Fig. 4) appropriately corrects the displayed product information, and the recognized user uses the internal map of the store to become the purchase target of the user. It is configured to include a shopping information providing unit (440 in FIG. 4) that provides information on an optimal route to a store where products are displayed, which will be described in more detail later with reference to FIGS. 3 to 7.

2 is a block diagram showing the configuration of the user terminal shown in FIG. 1.

Referring to FIG. 2, each user terminal 110 may include a communication module 111, a memory 112, a display module 113, an audio output module 114, and a control unit 116.

The communication module 111 may communicate and connect with any internal component or at least one external device through the network 120. In this case, an external device may include, for example, the store management apparatus 130 shown in FIG. 1. Here, as the wireless Internet technology, wireless LAN (WLAN), DLNA (Digital Living Network Alliance), Wibro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), HSDPA (High Speed Downlink Packet Access) ), High Speed Uplink Packet Access (HSUPA), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), etc. In addition, the communication module 111 transmits and receives data according to at least one wireless Internet technology in a range including Internet technologies not listed above.

In addition, short-range communication technologies include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Near Field Communication (NFC). , Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, and the like may be included.

The memory 112 stores various user interfaces (UI), graphic user interfaces (GUI), and the like. In addition, the memory 112 stores data and programs necessary for the user terminal 110 to operate. That is, the memory 112 may store a plurality of application programs (hereinafter referred to as “applications”) driven by the user terminal 110, data for operation of the user terminal 110, and commands. That is, at least one or more applications 112a are stored in the memory 112 of the user terminal 110 according to an embodiment of the present invention. At least some of these applications 112a may be downloaded from an external server through wireless communication. An embodiment of the present invention is characterized by including a shopping guide application provided by the store management device 130 shown in FIG.

In addition, the applications 112a are automatic speech recognition (ASR) applications, map applications, media applications (for example, QuickTime, MobileMusic.app, or MobileVideo.app), social networking applications (for example,) For example, it may be implemented as a Facebook, Twitter, etc.), an Internet browsing application, or the like. Meanwhile, the applications 112a may be stored in the memory 112, installed in the user terminal 110, and driven by the controller 116 to perform an operation (or function) of the user terminal 110. .

In addition, the memory 112 is a flash memory type (Flash Memory Type), a hard disk type (Hard Disk Type), a multimedia card micro type (Multimedia Card Micro Type), a card type of memory (for example, SD or XD memory, etc.) ), magnetic memory, magnetic disk, optical disk, RAM (Random Access Memory: RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory: ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM ( It may include at least one storage medium among (Programmable Read-Only Memory).

The display module 113 displays various contents such as various menu screens using a user interface and/or a graphic user interface stored in the memory 112 under the control of the controller 116, and is controlled by the controller 116. The execution result screen of the executed application can be displayed. As an example, when the shopping guide application is executed, the display module 113 may display an updated store interior map and information on an optimal route to a store in which a user's purchase target product using the same is displayed. Here, the content displayed on the display module 113 may include a menu screen indicating various text or image data (including various information data), icons, and list menus. The display module 113 includes a liquid crystal display (LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, and an electronic ink display (e- ink display), LED (Light Emitting Diode), etc.

The audio output module 114 may output voice information included in a signal processed by a predetermined signal under the control of the controller 116. Here, the audio output module 114 may include a receiver, a speaker, and the like.

3 is a block diagram showing the configuration of a store management apparatus according to an embodiment of the present invention.

Referring to FIG. 3, the store management apparatus 130 may be implemented as a server including a memory 132, a database 133, a processor 134, a communication module 136, and an input/output interface 138.

The memory 132 is a computer-readable recording medium and may include a non-destructive large-capacity recording device such as RAM, ROM, and disk drive. In addition, an operating system and at least one program code may be stored in the memory 132. These software components may be loaded from a computer-readable recording medium separate from the memory 132 using a drive mechanism. The recording medium readable by such a separate computer may include a recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In addition, the software components may be loaded into the memory 132 through the communication module 136.

The database 133 may store and maintain information transmitted and received from the user terminals 110 connected to the store management apparatus 130 according to an embodiment of the present invention. For example, user terminals installed with the shopping guide application downloaded from the store management device 130 may be managed as a member of the store management device 130, in which case the database 133 serves as a member information DB Various pieces of information corresponding to the user terminals 110 connected to the store management apparatus 130 may be stored by performing the operation.

In addition, the database 133 may store various configurations installed in the large store 140, for example, information transmitted from a fixed surveillance camera 140a, a mobile camera 140b, and a cash register 140c. In addition, various information such as product display and arrangement information and inventory list information in the large store 140 may be stored.

The processor 134 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to the processor 134 by the memory 132 or the communication module 136. For example, the processor 134 may be configured to execute a command received according to a program code stored in a recording device such as the memory 132. That is, the processor 134 may include a program module that is implemented in software by C, C++, Java, Visual Basic, Visual C, etc. to perform various functions.

The processor 134 of the store management device 130 according to an embodiment of the present invention includes a store interior map generator (410 in FIG. 4) that generates a map inside the store, and a user recognition unit that recognizes a user terminal entering the store. (420 in FIG. 4), a display product information correction unit (430 in FIG. 4) that appropriately corrects when product information displayed on a store is changed from a preset method, and the internal map of the store is provided to the recognized user. It is configured to include a shopping information providing unit (440 in FIG. 4) that provides information on an optimal route to a store in which the product to be purchased by the user is displayed.

Various functions implemented through the processor 134 of the store management apparatus 130 according to an embodiment of the present invention will be described in more detail below with reference to FIGS. 4 to 7.

The communication module 136 may provide a function for the user terminals 110 and the messenger service providing device 130 to communicate with each other through the network 120. For example, control signals, commands, contents, files, etc. provided under the control of the processor 134 of the store management device 130 are transferred to the user terminals 110 through the network 120 through the communication module 136. Can be transmitted.

The input/output interface 138 may serve to provide an interface between an input device implemented by a keyboard or a mouse and an output device such as a display.

FIG. 4 is a block diagram showing an embodiment of an internal configuration of the processor shown in FIG. 3, and FIG. 5 is a diagram schematically illustrating configurations for generating a store interior map according to an embodiment of the present invention.

In addition, FIG. 6 is a diagram showing steps of generating and updating a store interior map in a method for providing and guiding an optimal shopping route according to an embodiment of the present invention, and FIG. 7 is a diagram illustrating the steps of using a store interior map in an embodiment of the present invention. A diagram showing steps of providing shopping information to a user and correcting display product information.

Hereinafter, each functional block of the store management apparatus according to an embodiment of the present invention and a method of providing and guiding an optimal shopping route performed by the store management apparatus according to the present invention will be described in more detail with reference to FIGS. 4 to 7.

First, referring to FIG. 4, the processor 134 of the store management device 130 includes a store interior map generator 410 that generates a map inside the store, and a user recognition unit 420 that recognizes a user terminal entering the store. , When the product information displayed on the shelf is changed differently from a preset method, the display product information correction unit 430 appropriately corrects the product information, and the recognized user uses the internal map of the store to become a purchase target of the user. A shopping information providing unit 440 may be provided to provide information on an optimal route to a store on which products are displayed.

In addition, the processor 134 interlocks with the database 133 storing information provided to the user terminal 110 in identifying the terminal 110 of the user who is a registered member who has accessed the store management device 130 In the operation of the store management device shown in FIGS. 6 and 7, that is, the steps of generating and updating a store interior map among methods of providing and guiding an optimal shopping route, the processor 134 (FIG. 6) And it is possible to control the store management device 130 to provide shopping information using a map inside the store to the user and perform steps (FIG. 7) of correcting the displayed product information.

Here, the processor 134 may be implemented to execute an instruction according to a code of an operating system included in the memory 132 and at least one program code. At this time, the components in the processor 134, that is, the store interior map generation unit 410, the user recognition unit 420, the display product information correction unit 430, and the shopping information providing unit 440 are the store management device ( It may be understood that different functions performed by the processor 134 are classified and expressed by a control command provided by the program code stored in 130).

The store interior map generator 410 generates a store interior map by synthesizing the first captured image information transmitted from the fixed surveillance cameras 140a installed in the store and the second captured image information transmitted from the mobile camera 140b. Perform the action.

Specifically, referring to FIG. 5, the large store 140 may be divided into a plurality of areas in which products are arranged, and fixed surveillance cameras 140a may be installed in each of the areas. In FIG. 5, it is described as an example that it is divided into 9 zones (A to I) and 9 fixed surveillance cameras 140a are installed for each zone, but the embodiment of the present invention is not limited thereto.

The surveillance cameras 140a photograph a corresponding region and generate first photographed image information corresponding thereto. For example, the first surveillance camera installed in the area A may photograph a shelf or a shelf disposed in the area A, and products displayed on the shelf/shelf.

In the embodiment of the present invention, codes (eg, barcodes, QR codes, etc.) may be installed on the shelves or shelves so that the camera can recognize information such as shelves or shelves arranged for each area. As another example, a predetermined type of marking may be applied to the outside of the shelf or the shelf so that the processor can recognize the shelf through the photographed image. Through this processing, the image information captured by the first surveillance camera can be analyzed to distinguish the shelves/shelves arranged in the area A, and products displayed on the shelves/shelves can be distinguished and classified.

Further, the area photographed by the first surveillance camera may cover all areas A shown in FIG. 5. Likewise, the second surveillance camera installed in area B may cover all areas B, in this case, the boundary between area A and area B may correspond to an area photographed by both the first and second surveillance cameras. .

That is, by collecting image information of a plurality of surveillance cameras installed in all the divided areas, a flat map corresponding to the entire area inside the store can be generated, and the internal map includes shelves/shelves arranged for each area. Products displayed on the shelves/shelves may also be distinguished and classified.

In order to generate an entire flat map as the store interior map using images captured by the surveillance cameras installed for each area, the store interior map generation unit 410 according to the embodiment of the present invention includes the first photographing It is possible to determine the degree of distortion of image information and perform image processing to compensate for the distortion. For example, a perspective value of the first captured image information corresponding to each region may be calculated and reflected to generate a flat interior map in the store.

In addition, an embodiment of the present invention further installs a separate mobile camera 140b to generate a more accurate internal map in addition to a plurality of previously installed surveillance cameras in generating the internal map, and 2 It is possible to create a more sophisticated internal map by using the captured image information.

Referring to FIG. 5, it is preferable that the at least one mobile camera 140b is installed at a location that allows a view of the entire store, and for example, this may be implemented as a drone camera. That is, a perspective value of the second captured image information transmitted from the mobile camera 140b may be calculated and additionally reflected to generate a flat interior map in the store.

Therefore, referring to FIGS. 5 and 6, the store interior map generator 410 according to an embodiment of the present invention uses first and second captured image information transmitted from the fixed surveillance cameras and mobile cameras. Create a map inside the store. For example, for more accurate mapping, the entire floor plan picture may be sampled at an accurate location and point using image information transmitted from mobile cameras in addition to the previously installed fixed surveillance cameras.

In addition, by analyzing the first photographed image information transmitted from the fixed surveillance cameras 140a installed in each area, products displayed on the shelves installed in each area can be recognized. For example, the recognition of the product is through various methods such as weight of the product and the degree of separation from the shelf, and color recognition when the transmitted image is sampled, or the product is no longer displayed at the store. Is to be able to recognize.

That is, the store interior map generation unit 410 according to the embodiment of the present invention analyzes the first photographed image information transmitted from the fixed surveillance cameras 140a installed for each area, and displays the information on the shelves for each area. Recognizing whether a product has changed, and based on this, display information of the changed product and product classification information corresponding thereto can be generated, and the information generated through this can be used to update the existing product classification information stored in the database 133. I can.

In addition, the store interior map generator 410 may update the generated interior map by reflecting display information of the changed product and product classification information corresponding thereto.

The user recognition unit 420 performs an operation of recognizing a user terminal entering the large store 140. As an example, when the user terminal 110 downloads a shopping guide application provided by the store management device 130 and is managed as a member subscriber, the store management device when the user terminal 110 enters the store ( 130) may recognize the position of the user terminal 110 through a predetermined network 120, for example, a short-range communication method such as Bluetooth or NFC.

In addition, the processor 134 interlocks with the database 133 storing information provided to the user terminal 110 in identifying the terminal 110 of the user who is a registered member who has accessed the store management device 130 I have to do it.

That is, the user recognition unit 420 according to an embodiment of the present invention is a short-range communication method based on user information previously stored in the database 133 when it is determined that the user terminal is a registered member when the user terminal enters the store. The position of the user terminal 110 may be recognized through Bluetooth or NFC.

As another embodiment, when the user terminal 110 is not recognized through a short-range communication method, etc., the user's face captured by an adjacent fixed surveillance camera or a mobile camera when the user holding the user terminal 110 enters It is also possible to recognize the user by searching for the external appearance information of the customer stored in the database 133 through clothes, skin color, and height.

Referring to FIG. 7, when the user terminal 100 is recognized by the user recognition unit 420, the shopping information providing unit 440 uses the internal map of the store to the recognized user to become a purchase target of the user. It is possible to provide information on the optimal route to the shelves where products are displayed.

In this case, the product to be purchased by the user may be set with reference to the shopping history of the user terminal, and the like, and the shopping history of the user terminal may be stored and managed in the database 133.

As an example, when the user of the user terminal 110 who has installed the shopping guide application provided by the store management device 130 and subscribed as a member enters the large store 140, it is previously stored in the database 133 Based on the user information, the position of the user terminal 110 may be recognized through a short-range communication method such as Bluetooth or NFC, and the shopping guide application of the user terminal 110 may be automatically executed.

When the shopping guide application is executed, the internal map generated and updated by the store internal map generating unit 410 may be displayed on the user terminal 110, and the user can shop more easily and efficiently by using the internal map. You will be able to enjoy it.

In addition, in the case of the user terminal 110 registered as a member, the history of visiting and shopping at the large store is stored in the database 133, and product information mainly purchased by the user can be automatically selected based on this. I can.

Accordingly, when the shopping guide application is executed and the internal map is displayed on the user terminal 110, the shopping information providing unit 440 according to an embodiment of the present invention uses the internal map of the store to determine the purchase target of the user. It is possible to provide information on the optimal route to the shelves where the products are displayed.

When the user completes shopping easily and conveniently by using the shopping information provided by the shopping information providing unit 440 through the shopping guide application, the user can purchase a product.

At this time, when the user finally purchases and pays for the product, the cash register 140c provides the final payment information to the display product information correction unit 430.

Accordingly, the display product information correction unit 430 checks whether the product purchased by the user matches the product displayed at a preset location using the payment information according to the user's product purchase transmitted from the cash register 140c, and displays the product. An operation of correcting product information may be performed. That is, the display product information correction unit 430 may appropriately correct the product information displayed on the shelf when it is changed differently from a preset method.

For example, when the user withdraws the first product from the first store, information on the first product may be checked through updated and stored display product information. More specifically, as described above, the store interior map generating unit 410 analyzes the first photographed image information transmitted from the fixed surveillance cameras 140a installed for each area, and displays the products displayed on the shelves for each area. Recognizing whether there is a change, it is possible to generate display information of the changed product and product classification information corresponding thereto based on this, and the information generated through this may update existing product classification information stored in the database 133 .

However, although it is determined that the user has purchased the first product, the first product does not match the product information stored in the database 133 as a result of checking the payment information according to the user's product purchase transmitted from the actual cash register 140c. If not, the display product information may be corrected through the display product information correction unit 430.

In addition, even if the information on the product withdrawn by the user is not stored in the database 133, the product information correction unit 430 uses the payment information according to the user's product purchase transmitted from the cashier 140c. Display product information can be updated.

As an example, when the location from which the product was withdrawn is stored as each point, and assuming that the product is withdrawn from the stored point, the first user withdraws confectionery from the first point and detergent from the second point and completes the payment. , When the second user withdraws the confectionery at the first point and the tissue paper at the third point and completes the payment, and when the third user withdraws the tissue paper at the third point, the display product information correction unit 430 ) May determine display product information by using the information to determine that confectionery is displayed at the first point and tissue paper is displayed at the third point. However, if the second point is set to have a high probability that the detergent is on display, and then, if the fourth user withdraws the detergent from the second point, it is determined that the detergent is on display at the second point at that time, Information can be confirmed.

On the other hand, when the fourth user withdraws bread other than confectionery at the first point, the product information on display at the determined first point is released, and after that, it is checked whether another user withdraws a certain product at the first point. After that, it is possible to determine the display product information of the first point.

As described above, in the present invention, specific matters such as specific components, etc., and limited embodiments and drawings have been described, but this is provided only to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , If a person of ordinary skill in the field to which the present invention belongs, various modifications and variations are possible from these descriptions.

Therefore, the spirit of the present invention is limited to the described embodiments and should not be defined, and all things that are equivalent or equivalent to the claims as well as the claims to be described later fall within the scope of the spirit of the present invention. .

Claims (18)

In the management system of a large store including fixed surveillance cameras installed for each of a plurality of areas in which products are arranged, and at least one or more mobile cameras capable of viewing the entire store,
A store interior map generation unit that generates an entire plan map of the store interior based on image information transmitted from the fixed surveillance cameras and/or at least one mobile camera;
A user recognition unit for recognizing a user terminal entering the large store;
Optimal including a shopping information providing unit that provides the recognized user with route information to the store where the product to be purchased by the user is displayed by using the generated internal map when the user terminal is recognized by the user recognition unit Shopping route provision and guidance system. The method of claim 1,
A display product information correction unit for correcting display product information by checking whether the final product matches the product displayed at a preset location by using payment information according to the purchase of the product of the user terminal user transmitted from the cash register of the large store. Including the best shopping route provision and guidance system. The method of claim 2,
The store interior map generator provides an optimal shopping route for generating a store interior map by synthesizing first captured image information transmitted from the respective fixed surveillance cameras and second captured image information transmitted from the at least one mobile camera. And guidance system. The method of claim 3,
The store interior map generation unit determines the degree of distortion of each of the first captured image information, calculates a perspective value of the first captured image information, and performs image processing. The method of claim 2,
The store interior map generation unit analyzes the first photographed image information transmitted from the fixed surveillance cameras installed in each area, recognizes whether or not the products displayed on the shelves for each area are changed, and the changed product based on this An optimal shopping route providing and guiding system that generates display information and product classification information corresponding thereto. The method of claim 5,
The store interior map generator, an optimal shopping route providing and guidance system for updating the generated interior map by reflecting display information of the changed product and product classification information corresponding thereto. The method of claim 6,
When the user terminal is determined to be a registered member when the user terminal enters the large store, the user recognition unit is optimal for recognizing the position of the user terminal through a short-range communication method based on user information previously stored in the database. Shopping route provision and guidance system. The method of claim 7,
If the user terminal is not recognized by the short-range communication method, the database through the user's face, clothing, skin color, and key information captured by an adjacent fixed surveillance camera or a mobile camera when the user holding the user terminal enters. An optimal shopping route providing and guidance system that recognizes users by searching for customer appearance information stored in. The method of claim 1,
When the shopping guide application installed in the user terminal is executed and the generated internal map is displayed on the user terminal, the shopping information providing unit provides the shortest shopping route information by using the internal map of the store. Provision and guidance system. In the method of providing and guiding a shopping route provided to a user terminal entering a large store including fixed surveillance cameras installed for a plurality of areas in which products are arranged, and at least one mobile camera capable of viewing the entire store,
Generating a map of the entire interior of the store based on image information transmitted from the fixed surveillance cameras and/or at least one mobile camera;
Recognizing a user terminal entering the large store;
And when the user terminal is recognized, providing the recognized user with route information to a store where the product to be purchased by the user is displayed using the generated internal map. The method of claim 10,
Optimal further comprising the step of correcting the displayed product information by checking whether the final purchased product matches the product displayed at a preset location by using payment information transmitted from the checkout counter of the large store according to the product purchase of the user terminal user How to provide and guide shopping routes. The method of claim 11,
The step of generating the interior map of the store may include synthesizing first captured image information transmitted from each of the fixed surveillance cameras and second captured image information transmitted from the at least one or more mobile cameras to generate a store interior map. How to provide and guide shopping routes The method of claim 12,
In the generating of the store interior map, the optimal shopping route is provided and guided by determining the degree of distortion of each of the first captured image information and calculating a perspective value of the first captured image information to perform image processing. Way. The method of claim 11,
In the generating of the store interior map, the first photographed image information transmitted from the fixed surveillance cameras installed for each area is analyzed to recognize whether or not the products displayed on the shelves for each area are changed, and based on this An optimal shopping route providing and guiding method further comprising the step of generating display information of changed products and product classification information corresponding thereto. The method of claim 14,
The generating of the store interior map further comprises updating the generated interior map by reflecting display information of the changed product and product classification information corresponding thereto. The method of claim 10,
Recognizing the user terminal may include, when the user terminal enters the large store, if it is determined that the user terminal is a registered member, the user terminal enters the user terminal through a short-range communication method based on user information previously stored in the database. How to provide and guide the best shopping route to recognize. The method of claim 16,
If the user terminal is not recognized by the short-range communication method, the database through the user's face, clothing, skin color, and key information captured by an adjacent fixed surveillance camera or a mobile camera when the user holding the user terminal enters. A method of providing and guiding the optimal shopping route to recognize users by searching for customer appearance information stored in The method of claim 10,
In the providing of the shopping route information, when the shopping guide application installed in the user terminal is executed and the generated internal map is displayed on the user terminal, the shopping route information of the shortest distance is provided using the internal map of the store. How to provide and guide the best shopping route.

Images