KR20220145072A - Unmanned store operation system and unmanned store management method - Google Patents

Abstract

The present invention relates to an unmanned store operating system and an unmanned store management method, wherein the unmanned store operating system displays image information for a plurality of unmanned stores and allows a manager to easily respond to a plurality of customers and process in real time according to image information through a terminal. The unmanned store operating system comprises: a first terminal installed at the entrance of each door of a plurality of stores to support control of the door; a second terminal capable of recognizing each door of the plurality of stores and provided to manage the interior of the store; a third terminal provided inside each of the plurality of stores to support inconveniences and inquiries within the store; an operator terminal for an operator who operates the store; a management server connected to the first terminal, the second terminal, and the third terminal, respectively, through a network, and guiding admission customer management in real time in response to image information transmitted through the first terminal, the second terminal, and the third terminal; a display module for displaying image information from the first terminal, the second terminal, and the third terminal through the management server, respectively; and an IP-PBX transmitting voice information and image information from the first terminal, the second terminal, and the third terminal to the management server in real time according to the operation of any one of the first terminal, the second terminal, and the third terminal. Therefore, the present invention can efficiently manage unmanned stores.

Description

Unmanned store operation system and unmanned store management method

The present invention relates to an unmanned store operating system and an unmanned store management method, and in particular, image information for a plurality of stores executing unmanned control is displayed, and the control person in charge according to the image information through the terminal for a plurality of customers in real time It relates to an unmanned store operating system and an unmanned store management method that can easily perform response and processing.

With the development of the Internet and the influence of economic development, various cultures are spreading in society, and stores that provide various products and services are being created. Managers of these stores often decide on the various tasks that need to be performed in the store and hire staff to assist them in doing so.

For example, in a PC room that is widely distributed in each place in the country, a plurality of PCs that store and use necessary programs inside and a worker who manages the entire PC room are essential. Users who want to use the PC room use a PC that is not in use after entering the PC room. Of course, the cost of using the PC is to be paid in arrears.

Users check the programs stored in the PC they have selected, and if the necessary programs are already installed, they can be used as they are. The necessary program from the operator is directly installed on the PC of his/her choice and used.

In addition, the PC itself or the PC room management system placed in the PC room calculates the time the user uses the computer, and after the user ends the use of the PC, the cost is settled and billed, and the user is the worker before leaving the PC room. By paying the cost to them, the economic activity between them is terminated. On the other hand, when a user using the PC room has any complaints about the use, etc., the operator or worker at the counter is notified of this, and the worker takes appropriate measures after confirming the complaint or, in some cases, refunds a certain amount to the user. . Therefore, there was a disadvantage that, when operating the PC room, a worker who is a separate person who manages the PC and the PC room is required.

In addition, POS terminals are used in store management that provides services such as convenience stores, restaurants, PC rooms, Internet karaoke rooms, and DVD rooms. Such a POS terminal is used by a manager, an employee, or a part-time job, and the cost for an employee to use such a terminal increases.

Accordingly, recently, unmanned management of convenience stores, PC rooms, and karaoke rooms is underway. Crime prevention and security are very important factors in the operation of such unmanned stores, and there may be observations through CCTV during the store's open hours, analysis of movements after the closing time, and a reporting process through this.

An example of such a technique is disclosed in Documents 1 to 3 and the like.

For example, in the following Patent Document 1, a 360-degree camera installed to overlap around the main facilities of an unmanned store and photographing the unmanned store in real time, based on real-time shooting data obtained through the 360-degree camera, a user is identified and An integrated server that tracks, detects, tracks, and responds to events occurring in the unmanned store, stores the real-time shooting data obtained from the server, and stores user identification information for the user of the unmanned store Security including a door opening and closing facility, disaster prevention facility and theft bell controlled by the server based on the database, the operator portable terminal receiving the event occurrence signal when the event occurs from the server, and the event occurrence signal when the event occurs Disclosed is a crime prevention and security management system for unmanned store operation including a control device.

In addition, in Patent Document 2 below, at least one camera installed at a specific location in the store, an analysis unit that analyzes images acquired in real time by the camera to analyze whether an abnormal event occurs in the store, and access management of the store and a processor for controlling the analysis unit to be operated when the store is operated in an unattended state by interworking with the system, and when the processor generates an abnormal situation event, a temporary account that can access the processor is stored as a server or security agent Disclosed is a store control device that controls to transmit to a terminal.

On the other hand, in the following Patent Document 3, a PC room terminal for transmitting the photographed face image together with a PC room identifier or terminal identifier, which is fixedly installed in each PC room, respectively, to photograph a customer's face, a face image from the PC room terminal, and Receive a PC room identifier or terminal identifier, extract facial feature points from a face image, identify the extracted facial feature points as a customer identifier, and register the customer identifier together with the face image and PC room identifier or terminal identifier to unattend the PC room. About a PC room unmanned control system including a PC room unattended control server for performing control and a database for storing customer identifiers, face images, and PC room identifiers or terminal identifiers transmitted from the PC room unattended control server into a database has been disclosed.

Republic of Korea Patent Publication No. 10-2234555 (registered on March 25, 2021) Republic of Korea Patent Publication No. 10-2132086 (registered on July 2, 2020) Republic of Korea Patent Publication No. 2020-0104041 (published on March 3, 2020)

In Patent Document 1 as described above, when the user moves out of the angle of view of the 360-degree camera together with the item equipped with the identification tag from the unmanned store, the photographed data captured through the 360-degree camera is converted to theft photographing data corresponding to theft during the event. The server can classify, but there was a problem that it could not manage the admission customers.

In addition, although Patent Document 2 discloses a technology that can operate for store control only when the store is operating in an unattended state by interlocking with the access management system, the technology for the access management system is not disclosed, and many There was no disclosure on the technology to link and manage each of the stores.

On the other hand, the technology disclosed in Patent Document 3 is a technology developed by the present inventor to take a customer's face, extract facial feature points, and identify it as a customer identifier to perform unattended PC room control. Although the structure of guiding customers through the terminal has been disclosed, it is applicable only to specific customers in a specific PC room. There was a problem of increasing the cost for

In addition, since the technology disclosed in the above-mentioned patent document is executed only on a control server, etc., it is not possible to accurately grasp the customer's status, and there is a problem in that it cannot be easily applied to a large number of customers in a plurality of stores.

An object of the present invention was made to solve the above-described problems, and it is to control the door through real-time remote unmanned control, real-time customer response, customer's unauthorized entry and exit management, POS device and kiosk remote control, remote real-time broadcasting, CCTV control and It is to provide an unmanned store operating system and unmanned store management method that can build and display the same system as a worker resident in a store by learning by AI, and respond to customers in real time according to changes in display status.

Another object of the present invention is to change the border color on the display part of the image of the first terminal, the second terminal, and the third terminal displayed on the display module, so that the control person can easily recognize it in real time and execute customer response and processing. It is to provide an unmanned store operating system and an unmanned store management method.

Another object of the present invention is to provide an unmanned store operating system and an unmanned store management method that enable a customer to smoothly enter and exit another store by registering customer information once.

Another object of the present invention is to provide an unmanned store operating system and an unmanned store management method that can easily perform the management of an air conditioner, a broadcasting equipment, and a storage box such as beverages in a store from a remote location.

Another object of the present invention is to provide an unmanned store operating system and an unmanned store management method that can minimize the loss of goods in an article storage box such as beverages in an unmanned store.

In order to achieve the above object, the unmanned store operating system according to the present invention can recognize the first terminal provided to support the control of the doors by being mounted on the entrance of each door of a plurality of stores, and each door of the plurality of stores. There is, a second terminal provided to manage the inside of the store, a third terminal provided inside each of a plurality of stores to support inconveniences and inquiries in the store, an operator terminal for an operator who operates the store, Management that is connected to the first terminal, the second terminal, and the third terminal through a network, respectively, and guides the management of admission customers in real time in response to the image information transmitted through the first terminal, the second terminal, and the third terminal A server, a display module for displaying image information from the first terminal, the second terminal, and the third terminal through the management server, respectively, according to the operation of any one of the first terminal, the second terminal and the third terminal and an Internet Protocol Private Branch Exchange (IP-PBX) for transmitting voice information and image information from the first terminal, the second terminal, and the third terminal to the management server in real time, and the display module includes the IP-PBX It is characterized in that the image information transmitted through the first terminal, the second terminal, and the image information from the third terminal is displayed simultaneously in real time.

In addition, in the unmanned store operating system according to the present invention, it is characterized in that it further comprises a database for storing information about the store and information about the customer.

In addition, in the unmanned store operating system according to the present invention, the first terminal, the second terminal, and the third terminal each include a SIP (Session Initiation Protocol) intercom, and the second terminal includes a detection sensor for detecting a human body, , is interlocked with CCTV installed in the store, and the third terminal is installed in a Point Of Sales (POS) terminal or kiosk provided inside the store.

In addition, in the unmanned store operating system according to the present invention, the management server includes a customer management module that manages the image information of the ID and face part of the customer photographed by the camera of the first terminal, the first terminal, the second terminal, A store management module for managing information on a store equipped with a third terminal, a terminal management module for guiding the management of customers in real time according to transmission and reception information from the first terminal, the second terminal, and the third terminal, the IP- A display management module that manages to simultaneously display the image information from the first terminal, the second terminal and the third terminal transmitted through the PBX on the display module in real time, the customer transmitted from the first terminal or the second terminal and a learning module for learning image information and storing it in the database, and an IoT management module for managing information on IoT devices installed in the store.

In addition, in the unmanned store operating system according to the present invention, the store is characterized in that any one of a PC room, a karaoke room, an indoor golf driving range, and an unmanned store.

Also, in the unmanned store operating system according to the present invention, the display module includes image information of a first terminal transmitted from 10 stores, image information of a second terminal transmitted from 5 stores, and a third terminal transmitted from 5 stores. It is characterized in that the image information is displayed.

In addition, in order to achieve the above object, the unmanned store management method according to the present invention includes (a) a first terminal provided at the entrance of each door of a store, a second terminal provided to manage the inside of the store, and a store in the store. Installing a third terminal 300 supporting complaints and inquiries, providing an IP-PBX for transmitting voice information and image information to a management server in real time, (b) the first terminal in a display module , displaying the image information from the second terminal and the third terminal, (c) changing the border color of the display portion of the image of the first terminal displayed on the display module according to the call signal of the customer in the first terminal, , a guide step of guiding photographing for identification or face recognition of a customer, (d) photographing the ID or face of the customer in the first terminal according to the guidance in step (c), and according to the photographed image information, Determining whether the management server is a normal customer, (e) if determined as a normal customer in the step (d), opening the store door and registering customer information in a database, wherein the display module includes 10 stores It is characterized in that the image information of the first terminal transmitted from the , image information of the second terminal transmitted from five stores, and image information of the third terminal transmitted from the five stores are displayed.

Also, in the unmanned store management method according to the present invention, (f) detecting whether there is an entrance of a customer other than the customer who entered by the step (e) in the second terminal, (g) another step in the step (f) If it is determined that the customer enters and exits, changing the border color of the display portion of the image of the second terminal displayed on the display module, and guiding the abnormal state through the second terminal, (h) in the step (g) It characterized in that it further comprises the step of notifying the abnormal state to the operator terminal.

Further, in the unmanned store management method according to the present invention, the management server extracts a feature vector from the face image of the image information photographed as the customer information in the step (c) and stores the extracted feature vector in a database.

As described above, according to the unmanned store operating system and the unmanned store management method according to the present invention, the image information transmitted through the first terminal, the second terminal, and the third terminal is displayed in real time, and the change state of the image information is displayed. By displaying, the effect that the management of the unmanned store can be easily and efficiently executed is obtained.

In addition, according to the unmanned store operating system and unmanned store management method according to the present invention, by providing a learning module that learns the image information of the customer transmitted from the first terminal or the second terminal and stores it in the database, The effect of being able to solve the problem is obtained.

In addition, according to the unmanned store operating system and the unmanned store management method according to the present invention, by providing image information through the first terminal, the second terminal, and the third terminal transmitted from a plurality of stores in one display module, control The effect is also obtained that the person in charge can easily handle and process customers in real time for a large number of unmanned stores.

In addition, according to the unmanned store operating system and unmanned store management method according to the present invention, the second terminal is provided with a detection sensor for detecting a human body, and by realizing a configuration interlocked with CCTV installed in the store, abnormal customer access in the unmanned store is prevented. At the same time as blocking, the effect of minimizing the loss of goods is also obtained.

1 is a view showing an example of a CNN (Convolutional Neural Network) structure applied to image processing in an unmanned store operating system according to the present invention;
2 is a conceptual diagram of an unmanned store operating system for a PC room as an example to which the present invention is applied;
3 is a configuration diagram of an unmanned store operating system according to the present invention;
4 is a block diagram of the first terminal shown in FIG. 3;
5 is a block diagram of the second terminal shown in FIG. 3;
6 is a block diagram of the management server shown in FIG. 3;
7 is a block diagram of the database shown in FIG. 3;
8 is a view showing a display state in the display module shown in FIG. 3;
9 is a view showing an example of a display change of store information on the first terminal shown in FIG. 8;
10 is a view showing an example of a display change of the store information on the second terminal and the third terminal shown in FIG. 8;
11 is a view for explaining the display of the store status and the first to third terminals shown in FIG. 8;
12 is a flowchart for explaining a process of managing a customer entering a store to which the unmanned store operating system according to the present invention is applied.

The above and other objects and novel features of the present invention will become more apparent from the description of the present specification and accompanying drawings.

As used herein, the term "entry customer" means a customer who enters a store to use a store such as a PC room, karaoke room, indoor golf driving range, and unmanned store, and the "entrance part of the door" is used by the customer to enter the store from the outside. It means a part of the door, and "terminal" refers to a device connected to the management server through wired or wireless or a repeater, and controlled by the management server.

In addition, as used herein, "image information" refers to information obtained by photographing the face of an admission customer with a terminal or photographing information of an identification card that can identify the entering customer.

In addition, the "deep learning module" used herein may be prepared based on a Convolutional Neural Network (CNN) structure, which is a kind of Artificial Neural Networks (ANN), and the artificial neural network is a neural neuron. As a network modeled after the (Neuron) structure, it is a network made to learn patterns or algorithms by imitating the connection structure of synapses between neurons. For example, as shown in FIG. ), hidden layer, and output layer, it can be divided into various types according to the method of each layer. In other words, CNN is configured to process, classify, and output input, feature extraction, and feature unchanged according to position change or distortion.

Next, the concept of the unmanned store operating system according to the present invention will be described with reference to FIG. 2 . 2 is a conceptual diagram of an unmanned store operating system for a PC room as an example to which the present invention is applied.

As shown in Figure 2, when the unmanned store operating system according to the present invention is applied to a PC room, a plurality of PC rooms (PC room #1, PC room #2, ..., PC room #n) network through an Internet Protocol Private Branch Exchange (IP-PBX) or PBX. This IP-PBX is connected to the management server through a communication network, and may be provided to be connected to a plurality of PC rooms, respectively. In addition, the communication network may be a network, a high-speed backbone network of a large-scale communication network capable of large-capacity and long-distance voice and data services, or a next-generation wired or wireless network for providing the Internet or high-speed multimedia services. When the communication network is a mobile communication network, it may be a synchronous mobile communication network or an asynchronous mobile communication network. As an embodiment of the asynchronous mobile communication network, it may be a next-generation communication network such as a Wideband Code Division Multiple Access (WCDMA) method, a 3G LTE network, a 4G network, a 5G network, or other IP-based IP networks.

The above-described IP-PBX may transmit voice information and image information to the management server in real time in response to each terminal in a plurality of PC rooms to be displayed on the display unit of the management PC. One management PC may be configured to manage a plurality of PC rooms so that the management PC can be provided corresponding to a plurality of PC rooms.

Also, the IP-PBX and the management server may be connected to, for example, a manager PC and configured to control a plurality of management PCs from the manager PC. In addition, although it has been described as an exemplary configuration of a PC room in FIG. 2 , it is not limited thereto, and it can be applied to stores such as karaoke rooms, indoor golf driving ranges, and unmanned stores.

Hereinafter, an embodiment according to the present invention will be described with reference to the drawings.

3 is a block diagram of an unmanned store operating system according to the present invention.

As shown in FIG. 3, the unmanned store operating system according to the present invention is a first terminal 100 provided to support door control by being mounted on the entrance portion of each door of a plurality of stores, and each of the plurality of stores A second terminal 200 capable of recognizing an entrance door and provided to manage the interior of the store, a third terminal 300 provided in each of a plurality of stores to support inconveniences and inquiries within the store; The first terminal 100, the second terminal 200, and the third terminal 300 are respectively connected to the first terminal 100, the second terminal 200, and the third terminal 300 through an operator terminal 400 for an operator who operates the store, and the first terminal 100 , the second terminal 200, the management server 500 that guides the customer management in real time in response to the image information transmitted through the third terminal 300, information about the store and store information about the customer database 600, the first terminal 100, the second terminal 200, the display module 700 for displaying image information through the third terminal 300, respectively, the first terminal 100, the first According to the operation of any one of the second terminal 200 and the third terminal 300, voice information and image information from the first terminal 100, the second terminal 200, and the third terminal 300 are displayed in real time. It may include an IP-PBX 800 that transmits to the management server.

In addition, the unmanned store operating system according to the present invention may include a manager PC as shown in FIG. 2 as a manager terminal. In addition, the first terminal 100, the second terminal 200, and the third terminal 300 described above may be provided in one store, and may operate in conjunction with the IP-PBX 800 provided in the store. .

The first terminal 100 includes a Session Initiation Protocol (SIP) intercom having a voice call and video transmission function, and as shown in FIG. 4 , a transceiver 110, a camera 120, a call unit ( 130 ), a speaker 140 , and a microphone 150 may be provided. FIG. 4 is a block diagram of the first terminal shown in FIG. 3 .

The transceiver 110 is provided to transmit a call signal to the management server 500 through the IP-PBX 800 when the customer operates the call unit 130 to enter the store, and the camera 120 ) is provided for identification and face recognition of customers. The speaker 140 and the microphone 150 are provided for a call between the customer and the management server 500 . For example, when the customer wears a mask and operates the call unit 130 , the management server 500 may transmit a guide voice such as “Take off the mask” through the speaker 140 .

The guidance voice from the management server 500 is automatically provided by the management server 500 when feature points such as eyes, nose, and mouth are not photographed in the image information about the customer's face photographed through the camera 120 . can be executed as In the image information photographed by the camera 120 , when it is determined as an adult by the valid admission customer, for example, the customer's ID, the door is automatically opened under the control of the management server 500 . Such determination of an adult by the customer's ID card can be easily realized with a normal character recognition program for the date of birth written on the ID card.

On the other hand, the information about the customer's face part photographed by the camera 120 of the first terminal 100 is a simple picture file like the prior art, and is not registered as a file such as JPEG, PNG, or GIF, but is photographed. A feature vector is extracted from the face image of the image information and stored. That is, since the unmanned store operating system according to the present invention extracts and stores the feature vector from image information, it is possible to solve the problem of the customer's portrait right.

In addition, image information about the customer's face part photographed by the camera 120 of the first terminal 100 is extracted as a feature vector and stored in the database 600 . Therefore, the image information of the customer generated through the first terminal 100 of the PC room, which is the first store in which the unmanned store operating system according to the present invention is provided, is stored in the database 600, and the customer stored in the database 600 is viewed When entering a karaoke store, which is a second store to which the unmanned store operating system according to the invention is applied, the door can be opened only by the customer's confirmation by the camera of the first terminal of the karaoke room. In addition, in the unmanned store operating system according to the present invention, since the image information of the customer in the first store is stored and operated in a vector method, in the second store, even if the customer wears a mask, it is easy to become a customer registered in the first store become recognizable.

The second terminal 200, like the first terminal 100, includes a SIP (Session Initiation Protocol) intercom with voice calls and video transmission functions, and is interlocked with CCTV installed in the store, as shown in FIG. 5 . As described above, a transceiver 110 , a camera 120 , a detection unit 130 , a speaker 140 , and a microphone 150 may be provided. FIG. 5 is a block diagram of the second terminal shown in FIG. 3 .

The sensing unit 160 is not entered through the first terminal 100, but is provided to detect when a customer who enters the store arbitrarily opens the door in the store, or sees one from the outside and hides and sneaks in. 2 It is provided in order to enable confirmation in the management server 500 by the image information photographed through the camera 120 of the terminal 200 . That is, when the first terminal 100 is not operated and the customer enters, when the movement of the human body is detected by the sensing unit 160 , the management server 500 controls the speaker 140 of the second terminal 200 . It is possible to control it through a message such as announcements through it.

In addition, since the second terminal 200 detects illegally withdrawing drinks, snacks, etc. from a storage box such as drinks through real-time monitoring in conjunction with a CCTV installed in the store, it is possible to prevent economic loss in the unmanned store.

The third terminal 300 may also include a Session Initiation Protocol (SIP) intercom having voice call and video transmission functions, and may have the same configuration as that of the first terminal 100 . The third terminal 300 is, for example, installed in a POS (Point Of Sales) terminal or a kiosk provided inside a store, and as a customer unfamiliar with the store, various inconvenience situations such as an elderly customer may occur. In this case, when the call unit bell is pressed, it is provided to solve the problem through 1:1 response with the control center staff provided in the management server 500 .

The operator terminal 400 is a terminal used by an operator who operates a store, and is provided with a communication function for data communication with the management server 500 through a network, for example, a smart phone; Portable terminal, mobile terminal, personal digital assistant (PDA), PMP (Portable Multimedia Player) terminal, personal computer, notebook computer, tablet PC, Various terminals such as flexible terminals can be applied. When it is determined that an emergency situation in the store has occurred, the management server 500 transmits state information in the store photographed by the first terminal 100 , the second terminal 200 , and the third terminal 300 to the operator terminal 400 or it may be transmitted to a control system provided at a police station or the like.

The network establishes wireless communication between each of the first terminal 100 , the second terminal 200 , the third terminal 300 , and the operator terminal 400 and the IP-PBX 800 and the management server 500 . In order to do this, wireless LAN (Wireless LAN: WLAN), DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), etc. can However, the present invention is not limited thereto, and various applications may be carried out through relaying according to the development of wireless communication technology.

In addition, in the above description, the communication relationship between the first terminal 100 , the second terminal 200 , the third terminal 300 , and the IP-PBX 800 and the management server 500 has been separately described. Instead, the first terminal 100 , the second terminal 200 , the third terminal 300 , and the IP-PBX 800 may interwork to maintain a communication state with the management server 500 . That is, the first terminal 100 , the second terminal 200 , and the third terminal 300 maintain a communication state with the management server 500 through the IP-PBX 800 , and the first terminal 100 . , the second terminal 200 . According to the operation of any one of the third terminals 300 , the IP-PBX 800 is linked and maintained in a communication state with the management server 500 .

In addition, in the above description, the first terminal 100 , the second terminal 200 , and the third terminal 300 have been described in a structure in which each operates independently, but the present invention is not limited thereto, It might work. That is, for example, it is also possible to provide a structure in which the second terminal 200 is controlled by the management server 500 in conjunction with a CCTV, air conditioner, broadcasting equipment, storage boxes such as beverages, etc. provided in the store. To this end, the second terminal 200 may be connected for wired/wireless communication with a CCTV, air conditioner, broadcasting equipment, and storage box for items such as beverages provided in the store.

The management server 500 is provided in a normal server structure and connected to a network, such as store use guide information, etc. to the first terminal 100 , the second terminal 200 , the third terminal 300 and the operator terminal 400 . and manages customer image information. The management server 500 is preferably constructed to serve as a kind of web server, web service server, mobile web server or application server. In addition, a web page for providing store state information and the like to the operator terminal 400 may be provided.

The management server 500 will be described with reference to FIG. 6 .

6 is a configuration block diagram of the management server shown in FIG.

As shown in FIG. 6, the management server 500 of the unmanned store operating system according to the present invention includes the first terminal 100, the second terminal 200, and the third terminal ( 300), an operator terminal 400 and a database 300 and a transmission/reception module 510 capable of transmitting or receiving information via wire or wireless, a customer's ID photographed by the camera 120 of the first terminal 100, and A customer management module 520 that manages image information of a face part, a store management module that manages information on stores equipped with the first terminal 100, the second terminal 200, and the third terminal 300 ( 530 ), a terminal management module 540 , and the operator terminal 400 for guiding the customer management in real time according to transmission/reception information from the first terminal 100 , the second terminal 200 , and the third terminal 300 . ), image information from the first terminal 100 , the second terminal 200 , and the third terminal 300 transmitted through the operator management module 550 , the IP-PBX 800 for managing information on a display management module 560 that manages to be displayed on the display module 700 at the same time in real time, learns the image information of the customer transmitted from the first terminal 100 or the second terminal 200, and the database 600 ) may include a learning module 570 for storing the information, and an IoT management module 580 for managing information on IoT devices installed in the store.

The transmission/reception module 510 is composed of a normal transmission/reception module, and the first terminal 100, the second terminal 200, the third terminal 300, and the operator using the network and the IP-PBX 800 It is provided to transmit/receive store management information to and from the terminal 400 and the database 300 by wire or wireless, and is not limited to a specific communication module.

The customer management module 520 manages image information of a customer who enters a store for the first time (a customer entering the first store) photographed by the first terminal 100 and stores it in the database 600 , and stores it in the database 600 , When the customer of the first store entering the store) is stored in the database 600 , it is possible to enter the second store only by checking the image information in the first terminal 100 and the customer information stored in the database 600 . manage to Also, in the customer management module 520 , for example, in the case of an adult-only store, it is determined whether the customer is a normal customer who can enter based on the customer's ID or image information photographed in the first terminal 100 .

The store management module 530 is a type of store, for example, as a type of store, information about the stores of PC rooms, karaoke rooms, indoor golf driving ranges, and unmanned stores, and each store is linked to the unmanned store operating system according to the present invention. It manages information on whether the store has been subscribed to, etc., and the terminal management module 540 provides information about the installation location of the first terminal 100, the second terminal 200, and the third terminal 300 installed in each store. Manage information about information and functions.

The operator management module 550 manages the name, address, phone number, and membership subscription period of the operator of the store using the operator terminal 400 stored in the database 600 , and manages location information where the store is located.

For example, the display management module 560 may display the image information transmitted from the first terminal 100 , the second terminal 200 , and the third terminal 300 on the display module 700 . , image information of the first terminal 100 transmitted from 10 stores to the display module 700 , image information of the second terminal 200 transmitted from 5 stores, and the third terminal 300 transmitted from 5 stores ), the event list at each store, the store status and the status of the first to third terminals, mute, screen capture, information on delay time setting conditions after clicking the door open button, etc. manage

The learning module 570 applies a deep learning module to the image information transmitted from the first terminal 100 , the second terminal 200 or the third terminal 300 to learn and store it in the database 600 . prepared for In the learning module 570 , for example, when image information photographed by the first terminal 100 is an image requested to be registered, a face image is extracted by applying a face recognition model, and a plurality of A feature vector is generated, an array consisting of a plurality of feature vectors for each customer, customer identification information, and customer access permission information including information on stores to which the customer can enter is created, and the generated arrays are merged. You can learn customer image information by creating an array file.

That is, the learning module 570 is a customer's face image extraction, which is configured based on a convolutional neural network (CNN) as shown in FIG. 1 to extract a face image including a face region from an input image. , a first neural network having a first depth is used to generate a feature map of a plurality of input images, and a plurality of first sub-input images including a face region among a plurality of input images are generated based on the feature map. are primary, and a feature map of the plurality of first sub-input images is generated using a second neural network network having a second depth deeper than the first depth, and a feature map generated through the second neural network network Secondary selection of a plurality of second sub-input images including a face region from among the plurality of first sub-input images based on A feature map of the plurality of second sub-input images is generated, and a face image including a face region is selected and learned from among the plurality of second sub-input images based on the feature map generated through the third neural network. can do.

In addition, the learning module 570 applies a plurality of deep learning modules as shown in FIG. 1 to the face image extraction unit for the customer's face to input a learning image, a probability value that the learning image includes a face region, A convolution filter applied to extracting a face image by calculating the coordinates and landmark coordinates of the face region, and feeding back the calculated probability values, coordinates of the face region, and landmark coordinates to the face image extractor according to a back propagation algorithm Learning to update at least one of filter coefficients and weights may be performed.

As described above, in the learning module 570 of the management server 500 according to the present invention, a vector extraction algorithm for extracting a feature vector that enables facial features to be recognized from a customer's face image is applied to increase the facial recognition accuracy. It is possible to reduce the storage capacity for the image information of the customer stored in the database (600).

The IoT management module 580 is provided to remotely control, from the management server 500, items such as CCTVs, air conditioners, broadcasting equipment, and beverages, which are electrical and electronic devices installed in the store in conjunction with the second terminal 200 . do.

In the configuration of the management server 500 described above, the customer management module 520 , the store management module 530 , the terminal management module 540 , the operator management module 550 , the display management module 560 , and the learning module 570 . , although the IoT management module 580 has been described as a separate configuration, it is not limited thereto and may be realized as a series of continuous operations by a program.

As shown in FIG. 6 , the database 600 includes a store information DB 610 that stores information on the type and location of a store, the first terminal 100 , the second terminal 200 , and the third terminal. (300) and the terminal information DB 620 for storing information about the operator terminal 400, the first terminal 100, the second terminal 200, storing guide information through the third terminal 300 Guide information DB 630, customer information DB 640 for storing information about customers, vector information DB 650 for storing vector information about the customer's face image taken in the first terminal 100, to the store IoT information DB 660 that stores information about installed IoT devices, operator information DB 670 that stores information about the operator's phone number, address, etc., stores state information of the store displayed on the display module 700 It may include a state information DB (680). FIG. 7 is a block diagram of the database shown in FIG. 3 .

Such a configuration of the database 600 is only a preferred embodiment, and in developing a specific device, it may be configured in a different structure according to the database construction theory in consideration of the ease and efficiency of access and search.

The display module 700 includes a display unit as shown in FIG. 8 , and may display states of a plurality of stores. FIG. 8 is a view showing a display state in the display module shown in FIG. 3 . Such a display module 700 may be a display unit of the management PC as shown in FIG. 2 .

Meanwhile, in FIG. 8 , the display module 700 includes image information of the first terminal transmitted from 10 stores (entry status information of customers provided at the entrance), and image information of the second terminal transmitted from 5 stores (images in the store). information), image information of the third terminal transmitted from 5 stores (help status information), event list at each store, store status and status display of the first to third terminals, mute, screen capture, door open button Although the structure in which setting buttons such as delay time setting, etc. are displayed after clicking is not limited thereto, the display of image information may be changed according to the state of the store.

In addition, as shown in FIG. 8 for the image information for each terminal, the menus of mute, screen capture, call end, and door open are provided for the store status in which the store information about the first terminal is displayed, and the second Menus of mute, screen capture, call end, and broadcast are prepared for the store status in which store information related to the terminal is displayed, and mute, screen capture, call end, and call menus are provided for the store status in which the store information regarding the third terminal is displayed can be provided. For example, the mute button may be switched to a mute function when clicked, and an icon indicating the current mute status, and the capture button realizes a video capture function for the store when clicked, and the captured image information is stored in the database 600 It can be saved as a file. In addition, it may be provided to close the screen after time elapses by setting a delay time after clicking the door open button. When each of these buttons is clicked, the manager can easily recognize it by changing the color of the border on the display part of the image related to the corresponding store, and it is also possible to display a call-enabled state.

That is, as shown in FIG. 9 , when the first terminal is operated in the store of “entrance (Z Jukjeon)”, display management of image information from the first terminal transmitted through the network and the IP PBX 800 By the module 560, a mute button, a capture button, and a border color change in the display portion of the door open signal can be easily recognized by the administrator. 9 is a diagram illustrating an example of a display change of store information on the first terminal shown in FIG. 8 .

In addition, as shown in FIG. 10 , when the second terminal and the third terminal are operated in the store of "entrance (Z Jukjeon)", the second terminal and the third terminal transmitted through the network and the IP PBX 800 With respect to the image information from the display management module 560, the administrator can easily recognize by changing the border color in the display portion of the broadcast and call, respectively. 10 is a diagram illustrating an example of a display change of store information on the second terminal and the third terminal shown in FIG. 8 .

In FIG. 10, the screen of the store can be kept closed when the call ends is clicked in the image display state of the second terminal and the third terminal, and the delay time setting function, for example, in the image display state (center) of the second terminal When set to 10 seconds, it can be set to automatically close the screen, and even in the image display state (help) of the third terminal, it can be set to a delay time setting function, for example, to automatically close the screen when set to 300 seconds.

In addition, the event list in FIG. 8 shows the contents and time of entry, store, and help as the contents of events occurring in each store, and a search tab is provided to search information about each store, and in the store status, the first The intercom status of the first to third terminals is displayed. That is, in the event list part, the store name, time, and minutes are displayed on the list list, and the call record list can be checked up to now, and the list is maintained without being deleted even when the program is terminated. In addition, such a list can be used as a scroll bar, a list of recent events is displayed at the top, and the image state (central call screen) information by the second terminal is maintained without being recorded. In addition, when a corresponding store in the event list list is double-clicked, it is provided to switch to a call screen, for example, to switch to an entry/exit screen, and a help to a help screen.

On the other hand, as shown in FIG. 11 , the display of the store status and the connection status (intercom status) of the first to third terminals shown in FIG. 8 is a call status indicator lamp corresponding to the first to third terminals. Three BLF (Busy Lamp Field) lamps are provided, green indicates the access status of the first terminal, the store status of the second terminal, and the connection status of the third terminal's help request, and red indicates the disconnected status indicates. By displaying the display states of the first to third terminals on the display module 700 , the manager can easily recognize the connection state in each store. In addition, when the manager double-clicks the store status menu through the manager terminal, access by the first to third terminals, the inside of the store, and the store screen switching according to the request for help can be executed. 11 is a view for explaining the display of the store status and the first to third terminals shown in FIG. 8 .

In addition, the setting button provided in the lower right of FIG. 8 is provided to set conditions for managing the operating states in the first to third terminals in the management server 500, for example, as a store registration setting, IP PBX (800) can be set. That is, the IP PBX 800 that is interlocked with the first terminal, the second terminal, and the third terminal installed in the store for the SIP phone number of the entrance, center, and help to the store to which the unmanned store operating system according to the present invention is applied. You can set the automatic disconnection time setting, return function, and the like in the first terminal, the second terminal, and the third terminal. In addition, automatic call answer (connection) and end time settings (timer 3 to 30 seconds) can be set in the IP PBX (800), and when a call comes in, sound effects and settings, and ringtone settings for the desired sound file can be set in the IP PBX. (800) can be set. In addition, file settings for automatic voice reproduction of central broadcasting can be set in the IP PBX 800 .

The IP PBX 800 may operate in conjunction with each of the first terminal 100, the second terminal 200, and the third terminal 300 installed in each store, and the first terminal 100, the second According to the operation of any one of the terminal 200 and the third terminal 300, the voice information and the image information from the first terminal 100, the second terminal 200, and the third terminal 300 are networked in real time. may be transmitted to the management server 500 through That is, the IP PBX 800 may be provided to transmit an emergency call to the management server 500 . Such a call of the IP PBX 800 may be notified to the manager through an alarm or the like through the manager PC as shown in FIG. 2 .

Next, an example of a method of managing a customer entering a store to which the unmanned store operating system as described above is applied will be described with reference to FIG. 12 .

12 is a flowchart illustrating a process of managing a customer entering a store to which the unmanned store operating system according to the present invention is applied.

First, the installation of the system related to the unmanned store operation according to the present invention will be described.

The first terminal 100 provided at the entrance of each door of the store for a store in which the unmanned store operating system according to the present invention is operated, for example, a PC room, karaoke, indoor golf driving range, or an unmanned store. Install the second terminal 200 provided to manage the inside of the store, and the third terminal 300 that supports inconveniences and inquiries in the store, and the first terminal 100 and the second terminal 200 and an IP-PBX 800 for transmitting voice information and image information from the third terminal 300 to the management server 500 in real time.

Meanwhile, a manager for managing each store is set and registered in the management server 500 through the manager PC shown in FIG. 2 . In such a setting, time settings such as automatic call reception and automatic termination can be executed for voice information through each store, and the store unique number, the first terminal 100, the second terminal 200, and the third in the store The registration setting of the terminal 300 is also executed. In addition, the transmission state of audio information and image information between the IP-PBX 800 and the management server 500 provided for each store is also set.

In addition, as shown in FIG. 8 through the manager PC, the manager displays the number of stores to be displayed on the display module 700 , the first terminal 100 , the second terminal 200 , and the third terminal 300 . Setting conditions, a list of events in each store, the store status and the state of the first to third terminals, mute, screen capture, and the display state of the delay time setting condition after clicking the door open button can be set.

As described above, when the settings of the first terminal 100 , the second terminal 200 , and the third terminal 300 installed in the store and the management server 500 are completed, as shown in FIG. 8 , the display module At 700 , image information by the first terminal 100 , the second terminal 200 , and the third terminal 300 is displayed. When a customer enters a store in which the unmanned store operating system according to the present invention is operated, for example, a PC room, a karaoke room, an indoor golf driving range, or a store such as an unmanned store, the customer enters the first terminal mounted on the entrance of the store When the call unit 130 provided in 100 is pressed (S10), the door of the store remains closed, so the call signal is transmitted to the management server in real time through the transceiver 110, the network and the IP PBX 800 500), and as shown in Fig. 9, in the display means 700, the color of the border is changed in the display portion of the image related to the corresponding store.

That is, according to the call signal of the call unit 130 received through the transmission/reception module 510 of the management server 500 , it is displayed on the display means 700 by the customer management module 520 , and the first terminal 100 . A shooting guide for identification and/or face recognition of the customer is executed through the speaker 140 of the (S20).

According to the guidance in step S20, the camera 120 of the first terminal 100 takes a picture of the ID and/or face of the customer for the corresponding customer (S30), and the management server 500 according to the captured image information ) in the customer management module 520 determines whether it is a normal customer (S40). That is, in the customer management module 520 , if the store the customer wants to enter is an adult-only store, it is determined whether the customer is a normal customer who can enter based on the customer's ID or image information photographed in the first terminal 100 . .

If it is determined as a normal customer in step S40, the store door is opened (S50), and customer information is registered (S60). That is, the customer management module 520 manages the image information of the customer who first enters the store (the first store) photographed by the first terminal 100 and stores it in the customer information DB 640 of the database 600, , when a customer entering another store (the second store) is stored in the customer information DB 640 of the database 600 , the image information in the first terminal 100 and the customer information DB 640 of the database 600 . ) is managed so that entry is possible only by checking the customer information stored in the In the registration of such customer information, a deep learning module is applied to image information in the learning module 570 to extract a face image by applying a face recognition model, a plurality of feature vectors are generated from the extracted face image, and each customer An array consisting of a plurality of feature vectors, customer identification information, and customer access authorization information including information on stores to which the customer can enter is created, and the array file is created by merging the generated arrays to create an image of the customer. The information is learned and stored in the customer information DB 640 of the database 600 .

On the other hand, if the customer is not a normal customer in step S40, an entry prohibition is announced through the speaker 140 of the first terminal 120, and the operation is terminated (S140). However, even though the customer trying to enter is normal, if the customer claims that it is an error in the customer's ID and/or image information, the customer calls the control manager of the management server 500 by requesting a call through the microphone 150 and opens the door. may be open.

In addition, after opening the door in step S50, if the customer arbitrarily opens the door in the store or sees someone from the outside and hides and sneaks in, as shown in FIG. 10, the second terminal 200 In the detection unit 160 of the detection unit 160 of the customer other than the customer who has entered, that is, detects whether there is a normal visitor (S70), the customer who does not go through the registration procedure in the customer management module 520 of the management server 500 It is determined whether or not (S80).

If it is determined in step S80 that the entry of another customer is not in the normal state, it is displayed on the display means 700 by the customer management module 520 of the management server 500 , and the speaker 140 of the second terminal 200 . ) to guide the abnormal state to the customer who entered the store abnormally (S90). Thereafter, as shown in FIG. 10 , the camera 120 and the detection unit 160 of the second terminal 200 continue to detect the abnormal entrant, and the customer management module 520 detects the abnormal entrant. It is determined whether the exit has been made (S100).

The abnormal state in step S100 is notified to the operator terminal 400 by the operator management module 550 (S110). On the other hand, if the abnormal entrant exits by himself in step S100, the process proceeds to step S80 and the above-described process is repeated. In addition, when the abnormal entrant exits the store, the process proceeds to step S10 and the above-described process may be executed.

In addition, when the customer who normally enters in step S80 is a customer who is not familiar with the store, and various inconvenience situations such as an elderly customer occur, as shown in FIG. Sales) When operating the call unit of the third terminal 300 installed in the terminal or kiosk (S120), the store management module 520 may execute the store guidance through the speaker of the third terminal 300 (S130). .

On the other hand, each of the above-described steps is performed by the display module 700 as shown in FIG. 8 , for example, image information of the first terminal 100 transmitted from 10 stores, and the image information transmitted from 5 stores. It may be displayed as image information of the second terminal 200 and image information of the third terminal 300 transmitted from five stores. When such a display module 700 is applied, the control person in charge may actively manage the unattended store from a remote location through the manager PC as shown in FIG. 2 .

The invention made by the present inventors has been described in detail according to the above embodiments, but the present invention is not limited to the above embodiments and various modifications can be made without departing from the gist of the present invention.

By using the unmanned store operating system and the unmanned store management method according to the present invention, it is possible to efficiently manage the unmanned store.

100: first terminal
200: second terminal
300: third terminal
500 : management server
700: display module
800 : IP PBX

Claims (9)

A first terminal installed at the entrance of each door of a plurality of stores to support the control of the door;
A second terminal capable of recognizing each entrance of a plurality of stores and provided to manage the interior of the store;
A third terminal provided inside each of a number of stores to support inconveniences and inquiries within the store;
an operator terminal for an operator who operates the store;
Management that is connected to the first terminal, the second terminal, and the third terminal through a network, respectively, and guides the management of admission customers in real time in response to the image information transmitted through the first terminal, the second terminal, and the third terminal server,
a display module for displaying image information from the first terminal, the second terminal, and the third terminal through the management server, respectively;
IP-PBX that transmits voice information and image information from the first terminal, the second terminal, and the third terminal to the management server in real time according to the operation of any one of the first terminal, the second terminal, and the third terminal (Internet Protocol Private Branch Exchange),
The display module, the unmanned store operating system, characterized in that the image information from the first terminal, the second terminal and the third terminal transmitted through the IP-PBX is displayed simultaneously in real time. In claim 1,
The unmanned store operating system, characterized in that it further comprises a database for storing information about the store and information about the customer. In claim 2,
The first terminal, the second terminal, and the third terminal each include a SIP (Session Initiation Protocol) intercom,
The second terminal includes a sensor for detecting the human body, and is interlocked with CCTV installed in the store,
The third terminal is an unmanned store operating system, characterized in that it is installed in a POS (Point Of Sales) terminal or kiosk provided inside the store. In claim 3,
the management server
A customer management module for managing the image information of the customer's ID and face part photographed by the camera of the first terminal;
a store management module for managing information on stores equipped with the first terminal, the second terminal, and the third terminal;
a terminal management module for guiding entry customer management in real time according to transmission/reception information from the first terminal, the second terminal, and the third terminal;
a display management module that manages to simultaneously display the image information from the first terminal, the second terminal and the third terminal transmitted through the IP-PBX on the display module in real time;
A learning module for learning the image information of the customer transmitted from the first terminal or the second terminal and storing it in the database;
An unmanned store operating system comprising an IoT management module for managing information on IoT devices installed in the store. In claim 1,
The store is an unmanned store operating system, characterized in that any one of a PC room, a karaoke room, an indoor golf driving range, and an unmanned store. in paragraph 3.
In the display module, image information of the first terminal transmitted from 10 stores, image information of the second terminal transmitted from 5 stores, and image information of the third terminal transmitted from 5 stores are displayed on the display module. store operating system. (a) a first terminal provided at the entrance of each entrance of the store, a second terminal provided to manage the interior of the store, and a third terminal 300 to support inconveniences and inquiries in the store, and , providing an IP-PBX for transmitting voice information and image information to a management server in real time;
(b) displaying image information from the first terminal, the second terminal and the third terminal on a display module;
(c) a guide step of changing the border color of the image of the first terminal displayed on the display module according to the call signal of the customer in the first terminal, and guiding photographing for identification or face recognition of the customer;
(d) photographing the ID or face of the customer in the first terminal according to the guidance in step (c), and determining whether the customer is a normal customer in the management server according to the photographed image information;
(e) if it is determined as a normal customer in step (d), opening the store door and registering customer information in a database;
The display module is unmanned, characterized in that the image information of the first terminal transmitted from 10 stores, image information of the second terminal transmitted from 5 stores, and image information of the third terminal transmitted from 5 stores are displayed on the display module How to manage your store. In claim 7,
(f) detecting whether there is an entrance of a customer other than the customer who entered by the step (e) in the second terminal;
(g) when it is determined that another customer enters and exits from the step (f), changing the border color to the display portion of the image of the second terminal displayed on the display module, and guiding the abnormal state through the second terminal;
(h) the unmanned store management method further comprising the step of notifying the operator terminal of the abnormal state in step (g). In claim 8,
The management server extracts a feature vector from a face image of the image information taken as customer information in step (c) and stores the extracted feature vector in a database.

Images