KR102501646B1 - Admission customer management system and management method - Google Patents

Abstract

It relates to an entrance customer management system and management method in which a controller can easily respond and process in real time according to image information through a terminal for a plurality of customers in a plurality of stores executing unmanned control. A first terminal installed at the entrance of each door and provided to support control of the door, a second terminal capable of recognizing each door of a plurality of stores and prepared to manage the inside of the store, and each of a plurality of stores A third terminal provided inside the store to support inconveniences and inquiries in the store, an operator terminal for the operator who operates the store, and connected to the first terminal, the second terminal, and the third terminal through a network, respectively. , In response to the image information transmitted through the first terminal, the second terminal, and the third terminal, a configuration including a management server for guiding customer management in real time is provided, so that the unmanned store can be managed efficiently. .

Description

Admission customer management system and management method

The present invention relates to a customer management system and management method, and in particular, a controller can easily respond and process in real time according to image information through a terminal for a plurality of customers in a plurality of stores executing unmanned control It is about the position customer management system and management method.

With the spread of various cultures in society due to the development of the Internet and the influence of economic development, stores providing various products and services are appearing. Managers of these stores often run their business by determining various tasks to be performed in the store and hiring employees who can help them.

For example, PC rooms that are widely distributed throughout the country require multiple PCs that store and use necessary programs, multiple PCs, and workers who manage the entire PC room. A client who wants to use a PC room enters the PC room and then uses a PC that is not in use. Of course, the cost of using the PC is deferred.

The user checks the programs stored in the PC he/she selects, and if the necessary program is already installed, he/she can use it as it is. Install and use the necessary programs from the operator directly on the PC of your choice.

The PC itself placed in the PC room or the PC room management system calculates the time the user uses the computer, and after the user ends using the PC, the cost is calculated and billed, and the user pays the cost to the worker before leaving the PC room. By paying , economic activities between them are terminated. On the other hand, when a user using a PC room has a complaint about use, etc., the operator or worker at the counter is notified of it, and the worker checks the complaint and takes appropriate action or, in some cases, refunds a certain amount to the user. . Therefore, a separate worker is essential when operating a PC room. That is, there was a disadvantage that a separate person, that is, a worker, who manages the PC and the PC room is required.

Meanwhile, POS terminals are used in store management providing services such as convenience stores, restaurants, PC rooms, Internet karaoke rooms, and DVD rooms. Such POS terminals are used by managers, employees, or part-time workers, etc., and costs for employees to use such terminals increase.

Accordingly, unmanned management is also progressing in convenience stores, PC rooms, and karaoke rooms.

One example of this technique is disclosed in Documents 1 to 3 below.

For example, in Patent Document 1 below, an authentication server for receiving encryption codes and billing information and transmitting authenticated real-time code data, a serial set-top box for receiving a service charging signal and transmitting an authentication code, and the service charging signal NP serve that receives and performs service charging, a terminal that performs a corresponding operation according to a service charging signal from the NP server, receives an authentication code from the authentication server, transmits service charging details and sales information, and the serial set-top box Disclosed is a store management system equipped with a POS terminal that transmits a service charging signal and receives an authentication code, and performs service charging with the NP sub.

In addition, Patent Document 2 below includes a biometric information database in which biometric information of a plurality of Internet users is stored, a plurality of web servers requesting identification of Internet users using the biometric information, and the biometric information database when requested by the web server. A bio server that stores biometric information and determines whether the biometric information stored in the biometric information database matches the biometric information of the Internet user requested by the web server, and a service that stores the history of requests, confirmation, and results of the biometric information Disclosed is a biometric information storage system having a record database and a service communication server installed in each of the bio-server and the web server to interface or relay data transmission between the bio-server and the web server.

On the other hand, in Patent Document 3 below, a PC room terminal is fixedly installed in each PC room to photograph a customer's face and transmits the captured face image together with a PC room identifier or terminal identifier, and a face image and PC room identifier or terminal identifier is received, facial feature points are extracted from the face image, extracted facial feature points are identified as customer identifiers, and the customer identifier is registered together with the face image and PC room identifier or terminal identifier, and PC room unmanned PC room identifiers are registered. A PC room unmanned control system including a PC room unmanned control server for performing control and a database for storing customer identifiers, face images, and PC room identifiers or terminal identifiers transmitted from the PC room unmanned control server into a database has been initiated.

Republic of Korea Patent Publication No. 2009-0001546 (published on January 9, 2009) Republic of Korea Patent Publication No. 2000-0059245 (published on October 5, 2000) Republic of Korea Patent Publication No. 2020-0104041 (published on September 3, 2020)

As described above, Patent Document 1 discloses a technology in which accurate sales management without loss of sales and convenience of service provision through sales authentication using a POS terminal of a store system is disclosed, but the problem of not being able to manage admission customers there was

In addition, Patent Document 2 discloses a configuration for sharing biometric information stored in a database with a plurality of websites and confirming the identity of an Internet user using biometric information stored in a database, but managing a plurality of stores separately. A technology that can do this has not been disclosed.

On the other hand, the technology disclosed in Patent Document 3 is a technology developed by the present inventor to perform unmanned PC room control by photographing the customer's face, extracting facial feature points, and identifying them as a customer identifier. Although a structure for guiding customers through a terminal is disclosed, since it is applicable only to a specific customer in a specific PC room, when applied to a plurality of customers in a plurality of stores, the capacity of the control server must be increased, and the control There was a problem of increasing costs for

In addition, since the technology disclosed in the above-mentioned patent literature is executed only on the control server, etc., it is not only impossible to accurately grasp the customer's status, but also there is a problem that it cannot be easily applied to a plurality of customers in a plurality of stores.

An object of the present invention has been made to solve the above-mentioned problems, and door control through real-time remote unmanned control, real-time customer response, unauthorized entry and exit management of customers, POS machine and kiosk remote control, remote real-time broadcasting, CCTV control and It is to provide a position customer management system and management method that can build and operate the same system as the store where workers reside by learning by AI.

Another object of the present invention is to provide an entrance customer management system and management method in which a controller can easily respond and process in real time according to image information through a terminal.

Another object of the present invention is to provide an entrance customer management system and management method through which a customer can smoothly enter and exit a store by registering customer information once.

Another object of the present invention is to provide an entrance customer management system and management method that can easily perform management of air conditioners, broadcasting equipment, and product lockers such as beverages in a store from a remote location.

Another object of the present invention is to provide an entrance customer management system and management method capable of minimizing loss of items in a locker such as beverages in an unmanned store.

In order to achieve the above object, the entrance customer management system according to the present invention can recognize the first terminal, each door of the plurality of stores, mounted on the entrance part of each door of the plurality of stores and provided to support the control of the door. A second terminal provided to manage the inside of the store, a third terminal provided inside each of the plurality of stores to support inconveniences and inquiries in the store, an operator terminal for the operator who operates the store, Management that is connected to the first terminal, the second terminal, and the third terminal through a network, and guides admission customer management in real time in response to image information transmitted through the first terminal, second terminal, and third terminal Characterized in that it includes a server.

In addition, in the admission customer management system according to the present invention, a display module for displaying image information through the first terminal, the second terminal, and the third terminal, respectively, and a database for storing information about the store and information about customers entering the store It is characterized by further including.

In addition, in the entrance customer management system according to the present invention, the first terminal, the second terminal, and the third terminal each include a SIP (Session Initiation Protocol) intercom, and the second terminal includes a sensor for detecting a human body, , It is linked with CCTV installed in the store, and the third terminal is characterized in that it is installed in a POS (Point Of Sales) terminal or kiosk provided inside the store.

In addition, in the entrance customer management system according to the present invention, the management server includes a customer management module for managing the customer's identification card and image information of the face captured by the camera of the first terminal, the first terminal, the second terminal, A store management module for managing information about a store equipped with a third terminal, a terminal management module for guiding customer management in real time according to transmission/reception information from the first terminal, the second terminal, and the third terminal; It is characterized by including a learning module for learning and storing customer image information transmitted from a terminal or a second terminal in the database, and an IoT management module for managing information on IoT devices installed in the store.

In addition, in the admission customer management system according to the present invention, the store is characterized in that it is any one of a PC room, a karaoke room, an indoor golf driving range, and an unmanned store.

In addition, in the admission customer management system according to the present invention, the display module includes image information of the first terminal transmitted from 10 stores, image information of the second terminal transmitted from 5 stores, and image information of the third terminal transmitted from 5 stores. Characterized in that image information of is displayed.

In addition, in order to achieve the above object, the admission customer management method according to the present invention includes (a) a photographing guide step for recognizing a customer's ID or face according to a customer's call signal in a first terminal, (b) the above step (a) Taking a picture of the customer's ID or face in the first terminal according to the guidance from the first terminal, and determining whether the customer is a normal customer in the management server according to the photographed image information, (c) if the customer is determined to be a normal customer in step (b) Opening the door and registering customer information, (d) detecting whether there is another customer other than the customer who entered and exited in step (c) in the second terminal, (e) step (d) If it is determined that another customer has entered or exited, it is characterized in that it includes a step of guiding an abnormal state through the second terminal, and (f) a step of notifying an operator terminal of the abnormal state in the step (e).

In addition, in the customer management method according to the present invention, it is characterized in that it further comprises the step of supporting complaints and inquiries in the store through the third terminal for the customer who has entered and exited through the step (c).

In the method for managing entrance customers according to the present invention, the management server extracts a feature vector from a face image of image information captured as customer information in step (c) and stores it in a database.

As described above, according to the admission customer management system and management method according to the present invention, a management server that guides admission customer management in real time in response to image information transmitted through the first terminal, the second terminal, and the third terminal is provided. By providing, the effect that management of an unmanned store can be performed efficiently is obtained.

In addition, according to the admission customer management system and management method according to the present invention, by providing a learning module for learning the customer's image information transmitted from the first terminal or the second terminal and storing it in a database, the problem of the customer's portrait right The effect of being able to solve is obtained.

In addition, according to the admission customer management system and management method according to the present invention, by providing a display module for displaying image information through the first terminal, the second terminal, and the third terminal, respectively, the control person in charge is in an unmanned store in real time. The effect that the response and processing of can be easily performed is also obtained.

In addition, according to the admission customer management system and management method according to the present invention, the second terminal has a sensor for detecting a human body and realizes a configuration interlocked with a CCTV installed in the store to block abnormal customer access in an unmanned store. At the same time, the effect of minimizing the loss of goods is also obtained.

1 is a diagram showing an example of a CNN (Convolutional Neural Network) structure applied to image processing in an admission customer management system according to the present invention;
2 is a block diagram of an entrance customer management system according to the present invention;
3 is a block diagram of a first terminal shown in FIG. 2;
4 is a block diagram of a second terminal shown in FIG. 2;
5 is a block diagram of the management server shown in FIG. 2;
6 is a block diagram of the database shown in FIG. 2;
7 is a diagram showing a display state in the display module shown in FIG. 2;
Figure 8 is a flow chart for explaining the process of managing admission customers according to the present invention.

The above and other objects and novel features of the present invention will become more apparent from the description of this specification and accompanying drawings.

As used herein, the term "entry customer" refers to a customer who enters to use a store such as a PC room, karaoke room, indoor golf driving range, or unmanned store, and "entry entrance part" refers to a customer entering the store from the outside. It means the part of the door, and "terminal" means a device that is connected to the management server through wired/wireless or relay, and is controlled by the management server.

In addition, "image information" used in the present application refers to information obtained by photographing a part of a customer's face using a terminal or photographing information of an identification card by which a customer can be identified.

In addition, the "deep learning module" used herein may be prepared based on the structure of a CNN (Convolutional Neural Network), and this CNN is a type of Artificial Neural Networks (ANN), and the artificial neural network is a neural neuron. (Neuron) structure, and it is a network made to learn patterns or algorithms by imitating the synapse connection structure between neurons. For example, as shown in FIG. 1, the input layer (Input Layer) ), hidden layer, and output layer, it can be divided into various types depending on the method of each layer. In other words, CNN is configured to process, classify, and output input, feature extraction, and feature unchanged according to position change or distortion.

Hereinafter, an embodiment according to the present invention will be described according to the drawings.

2 is a block diagram of an entrance customer management system according to the present invention.

As shown in FIG. 2, the entrance customer management system according to the present invention includes a first terminal 100 installed at the entrance of each door of a plurality of stores and provided to support control of the door, and each of the plurality of stores. A second terminal 200 capable of recognizing entrance doors and prepared to manage the inside of a store, a third terminal 300 provided inside each of a plurality of stores to support inconveniences and inquiries in the store, The operator terminal 400 for the operator who operates the store is connected to the first terminal 100, the second terminal 200, and the third terminal 300 through a network, respectively, and the first terminal 100 , Management server 500 for guiding customer management in real time in response to the image information transmitted through the second terminal 200 and the third terminal 300, storing information about the store and information about customers who enter the store and a display module 700 that displays image information through the database 600, the first terminal 100, the second terminal 200, and the third terminal 300, respectively.

The first terminal 100 includes a SIP (Session Initiation Protocol) intercom having voice call and video transmission functions, and as shown in FIG. 3, a transceiver 110, a camera 120, a call unit ( 130), a speaker 140, and a microphone 150. FIG. 3 is a block diagram of a first terminal shown in FIG. 2 .

The transceiver 110 is provided to transmit a call signal to the management server 500 when a customer to enter and exit the store operates the call unit 130, and the camera 120 recognizes the customer's ID and face. is prepared for The speaker 140 and the microphone 150 are provided for communication between the customer and the management server. For example, when a customer wears a mask and activates the calling unit 130, the management server 500 may transmit a guide voice such as "Take off the mask" through the speaker 140.

The guidance voice from the management server 500 is automatically generated by the management server 500 when feature points such as eyes, nose, and mouth are not captured in the image information of the customer's face captured through the camera 120. can be run with When it is determined from the image information photographed by the camera 500 that an effective entrance customer, for example, an adult, the door is automatically opened under the control of the management server 500 .

On the other hand, information on the customer's face photographed by the camera 120 of the first terminal 100 is not registered as a file such as JPEG, PNG, or GIF as a simple picture file, as in the prior art, A feature vector is extracted from the face image of the photographed image information and stored. That is, in the entrance customer management system according to the present invention, feature vectors are extracted and stored from image information, so that the problem of customer's portrait rights can be solved.

In addition, image information on the customer's face photographed by the camera 120 of the first terminal 100 is stored in the database 600 . Therefore, the customer's image information generated through the first terminal 100 of the PC room equipped with the admission customer management system according to the present invention is stored in the database 600, and the karaoke to which the admission customer management system according to the present invention is applied When a customer enters, the door can be opened only by the customer's confirmation through the camera of the first terminal of the karaoke room. In addition, in the admission customer management system according to the present invention, since the image information of the customer at the first store is stored and operated in a vector method, at the second store, even if the customer wears a mask, it is easy to be a registered customer at the first store. become recognizable.

Like the first terminal 100, the second terminal 200 includes a SIP (Session Initiation Protocol) intercom having voice call and video transmission functions, and is interlocked with a CCTV installed in the store, as shown in FIG. As described above, a transceiver 110, a camera 120, a sensing unit 160, a speaker 140, and a microphone 150 may be provided. FIG. 4 is a configuration block diagram of the second terminal shown in FIG. 2 .

The detection unit 160 is provided to detect when an entering customer opens the door at will in the store without entering through the first terminal 100 or when a customer sees a person from the outside and sneaks in after hiding. 2 It is provided to enable the management server 500 to check the image information captured through the camera 120 of the terminal 200. That is, when the first terminal 100 is not operated and a customer enters, and the motion of the human body is detected by the sensing unit 160, the management server 500 switches the speaker 140 of the second terminal 200. It enables control through announcements and announcements at any time.

In addition, the second terminal 200 detects the illegal withdrawal of beverages, snacks, etc. from storage boxes such as beverages through real-time monitoring in conjunction with CCTVs installed in stores, thereby preventing economic loss in unmanned stores.

The third terminal 300 may also include a Session Initiation Protocol (SIP) intercom having voice call and video transmission functions, and may have the same configuration as the first terminal 100 . This third terminal 300 is installed, for example, in a POS (Point Of Sales) terminal or kiosk provided inside the store, and various inconveniences such as customers unfamiliar with the store and elderly customers may occur. In this case, when the caller bell is pressed, the control center staff provided in the management server 500 is prepared to solve the problem in a 1:1 response.

The operator terminal 400 is a terminal used by an operator who operates a store, and has a communication function capable of data communication with the management server 500 through a network, for example, a smart phone, A portable terminal, a mobile terminal, a personal digital assistant (PDA), a portable multimedia player (PMP) terminal, a personal computer, a notebook computer, a tablet PC, Various terminals such as flexible terminals can be applied. When it is determined that an emergency situation in the store has occurred, the management server 500 transmits state information in the store captured by the first terminal 100, the second terminal 200, and the third terminal 300 to an operator terminal. (400).

The network is for establishing wireless communication between each of the first terminal 100, the second terminal 200, the third terminal 300, and the operator terminal 400 and the management server 500, a wireless LAN ( Wireless LAN (WLAN), Digital Living Network Alliance (DLNA), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA) , IEEE 802.16, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), Wireless Mobile Broadband Service (WMBS), and the like can be applied. However, it is not limited thereto, and various applications can be implemented through relaying according to the development of wireless communication technology.

Also, in the above description, the communication relationship between the first terminal 100, the second terminal 200, the third terminal 300, and the management server 500 has been separately described, but is not limited thereto, and the first terminal (100), the second terminal 200 and the third terminal 300 may be interlocked to maintain a communication state with the management server 500.

In addition, in the above description, the first terminal 100, the second terminal 200, and the third terminal 300 have been described as a structure in which each operates independently, but is not limited thereto, and is interlocked with an IoT device installed in a store. it might work That is, for example, a structure in which the second terminal 200 is controlled by the management server 500 may be provided in conjunction with CCTVs, air conditioners, broadcasting equipment, and storage boxes for beverages provided in the store. To this end, the second terminal 200 may be connected to CCTVs, air conditioners, broadcasting equipment, storage boxes for beverages, etc. provided in the store for wired/wireless communication.

The management server 500 is provided in a normal server structure and is connected to a network to provide store use guide information, etc. and manages the customer's image information. This management server 500 is preferably built to serve as a kind of web server, web service server, mobile web server or application server. In addition, a web page for providing store status information and the like to the operator terminal 400 may be provided.

The management server 500 will be described with reference to FIG. 5 .

5 is a configuration block diagram of the management server shown in FIG. 2 .

As shown in FIG. 5, the management server 500 of the entrance customer management system according to the present invention includes the first terminal 100, the second terminal 200, the third terminal 300, and the operator terminal 400 And a transmission/reception module 510 capable of transmitting or receiving information with the database 300 wired or wirelessly, managing the image information of the customer's ID and face part photographed by the camera 120 of the first terminal 100. A customer management module 520, a store management module 530 that manages information about stores in which the first terminal 100, the second terminal 200, and the third terminal 300 are installed, the first terminal ( 100), a terminal management module 540 for guiding customer management in real time according to transmission/reception information from the second terminal 200 and the third terminal 300, and an operator managing information on the operator terminal 400 A management module 550, a display management module 560 that manages image information to be displayed on the display module 700, learns the customer's image information transmitted from the first terminal 100 or the second terminal 200 It may include a learning module 570 for storing in the database 600 and an IoT management module 580 for managing information about IoT devices installed in the store.

The transmission/reception module 510 is composed of a normal transmission/reception module, and the first terminal 100, the second terminal 200, the third terminal 300, the operator terminal 400 and the database ( 300) and wired/wireless transmission/reception of store management information, and is not limited to a specific communication module.

The customer management module 520 manages image information of a customer entering a store for the first time photographed by the first terminal 100 and stores it in a database 600, and a customer entering from another store stores the image information in the database 600. If stored in , it is managed so that only the image information in the first terminal 100 and the customer information stored in the database 600 can be checked. In addition, the customer management module 520 determines whether the customer is a normal customer who can enter, for example, in the case of an adult-only store, based on the customer's identification card or image information captured by the first terminal 100 .

The store management module 530 provides information about the type of store, for example, a PC room, a karaoke room, an indoor golf driving range, and an unmanned store, and each store is interlocked with the admission customer management system according to the present invention. The terminal management module 540 controls information about the installation locations of the first terminal 100, the second terminal 200, and the third terminal 300 installed in each store. Manage information about information and functions.

The operator management module 550 manages the store operator's name, address, phone number and member registration period using the operator terminal 400 stored in the database 600, and manages location information where the store is located.

The display management module 560 determines the conditions under which the image information transmitted from the first terminal 100, the second terminal 200, and the third terminal 300 can be displayed on the display module 700, for example. , Image information of the first terminal 100 transmitted from 10 stores, image information of the second terminal 200 transmitted from 5 stores, and image information transmitted from 5 stores to the display module 700, the third terminal 300 transmitted from 5 stores ) manages information about the conditions under which the image information of ) can be displayed.

The learning module 570 learns by applying a deep learning module to the image information transmitted from the first terminal 100, the second terminal 200, or the third terminal 300 and stores it in the database 600. prepared for In the learning module 570, for example, when the image information photographed by the first terminal 100 is an image requested for registration, a face recognition model is applied to extract a face image, and a plurality of face images are extracted from the extracted face image. A feature vector is created, an array composed of a plurality of feature vectors for each customer, customer identification information, and customer access authority information including information about stores that the customer can enter is created, and the created arrays are merged to An array file can be created to learn customer image information.

That is, the learning module 570 extracts a customer's face image, and is configured based on a Convolutional Neural Network (CNN) as shown in FIG. 1 to extract a face image including a face region from an input image. , A feature map of a plurality of input images is generated using a first neural network having a first depth, and a plurality of first sub-input images including a face region among the plurality of input images based on the feature map are primarily selected, a feature map of the plurality of first sub-input images is generated using a second neural network network having a second depth deeper than the first depth, and the feature map generated through the second neural network network Secondarily selects a plurality of second sub-input images including a face region among the plurality of first sub-input images based on, and uses a third neural network network having a third depth deeper than the second depth A feature map of the plurality of second sub-input images is generated, and a face image including a face region is selected from among the plurality of second sub-input images based on the feature map generated through the third neural network for learning. can do.

In addition, the learning module 570 inputs a learning image by applying a plurality of deep learning modules as shown in FIG. 1 to the facial image extraction unit for the customer's face, and the probability value that the learning image includes the face region; A convolution filter applied to face image extraction by calculating face area coordinates and landmark coordinates, and feeding back the calculated probability values, face area coordinates and landmark coordinates to the face image extraction unit according to a back propagation algorithm. Learning to update at least one of filter coefficients and weights of s may be executed.

As described above, in the learning module 570 of the management server 500 according to the present invention, a vector extraction algorithm for extracting a feature vector capable of recognizing facial features from a customer's face image is applied to increase face recognition accuracy. In addition, the storage capacity of customer image information stored in the database 600 can be reduced.

The IoT management module 580 is provided to remotely control, from the management server 500, electrical and electronic devices such as CCTVs, air conditioners, broadcasting equipment, and storage boxes such as beverages installed in stores in conjunction with the second terminal 200. do.

In the configuration of the above-described management server 500, the customer management module 520, the store management module 530, the terminal management module 540, the operator management module 550, the display management module 560, the learning module 570 Although the IoT management module 580 has been described as a separate configuration, it is not limited thereto and may be realized as a series of continuous operations by a program.

As shown in FIG. 6, the database 600 includes a store information DB 610 for storing information about the type and location of stores, the first terminal 100, the second terminal 200, and the third terminal. 300 and a terminal information DB 620 for storing information about the operator terminal 400, and storing guide information through the first terminal 100, the second terminal 200, and the third terminal 300 A guide information DB 630, a customer information DB 640 for storing customer information, a vector information DB 650 for storing vector information on a customer's face image captured by the first terminal 100, It may include an IoT information DB 660 that stores information about installed IoT devices and an operator information DB 670 that stores information about the operator's phone number, address, and the like. FIG. 6 is a block diagram of the database shown in FIG. 2 .

The configuration of the database 600 is only a preferred embodiment, and in developing a specific device, it may be configured in a different structure according to the database construction theory in consideration of the ease and efficiency of access and search.

The display module 700 includes a display unit as shown in FIG. 7 and can display the status of a plurality of stores. FIG. 7 is a diagram illustrating a display state in the display module shown in FIG. 2 .

Meanwhile, in FIG. 7 , image information of a first terminal transmitted from 10 stores, image information of a second terminal transmitted from 5 stores, and image information of a third terminal transmitted from 5 stores are transmitted to the display module 700 in FIG. Although the display structure is shown, it is not limited to this, and the display of image information can be changed according to the state of the store.

Next, a method for managing admission customers by the admission customer management system as described above will be described with reference to FIG. 8 .

8 is a flowchart illustrating a process of managing admission customers according to the present invention.

First, when a customer enters and exits a store where the entrance customer management system according to the present invention is operated, for example, a PC room, a karaoke room, an indoor golf driving range, or an unmanned store, the customer enters the store through a control panel installed at the entrance of the store door. 1 Press the call unit 130 provided in the terminal 100 (S10). At this time, the store door is kept closed, and the call signal is transmitted to the management server 500 through the transceiver 110 and the network.

According to the call signal of the calling unit 130 received through the transmission/reception module 510 of the management server 500, the customer management module 520 transmits the customer's ID and/or information through the speaker 140 of the first terminal 100. Alternatively, shooting guidance for face recognition is executed (S20).

According to the guidance in step S20, the customer's ID and/or face is photographed by the camera 120 of the first terminal 100 (S30), and the management server 500 according to the photographed image information ) In the customer management module 520, it is determined whether the customer is normal (S40). That is, if the store the customer wants to enter is an adult-only store, the customer management module 520 determines whether the customer is a normal customer who can enter based on the customer's identification card or image information captured by the first terminal 100. .

If it is determined as a normal customer in step S40, the store door is opened (S50) and customer information is registered (S60). That is, the customer management module 520 manages the image information of the customer entering the store for the first time photographed by the first terminal 100, stores it in the customer information DB 640 of the database 600, and enters the store from another store. When a customer who wants to is stored in the customer information DB 640 of the database 600, entry is made only by checking the image information in the first terminal 100 and the customer information stored in the customer information DB 640 of the database 600. manage to be able to In order to register such customer information, a face recognition model is applied by applying a deep learning module to image information in a learning module 570 to extract a face image, generate a plurality of feature vectors from the extracted face image, and generate a plurality of feature vectors for each customer. An array consisting of a plurality of feature vectors, customer identification information, and customer access authority information including information on stores that the customer can access is created for each array, and an array file is created by merging the generated arrays to create an image of the customer. The information is learned and stored in the customer information DB 640 of the database 600.

On the other hand, if the customer is not a normal customer in step S40, an entry prohibition is announced through the speaker 140 of the first terminal 120, and the process ends (S140). However, even though the customer trying to enter is normal, if the customer's ID and/or image information is claimed to be an error, the customer requests a call through the microphone 150 to call the control person in the management server 500 to open the door. may be open.

In addition, after opening the door in the above step S50, if the entering customer opens the door randomly in the store or if he sees one person from the outside and sneaks in, the detector 160 of the second terminal 200 It detects whether there is a customer other than one customer, that is, a normal visitor (S70), and determines whether the customer has not gone through the registration procedure in the customer management module 520 of the management server 500 (S80).

If it is determined in the step S80 that another customer is entering or exiting the store in an abnormal state, the customer management module 520 of the management server 500 uses the speaker 140 of the second terminal 200 to enter the store abnormally. An abnormal state is notified to the customer (S90). Thereafter, the camera 120 and the detection unit 160 of the second terminal 200 continue to detect the abnormal visitor, and the customer management module 520 determines whether the abnormal visitor has left (S100).

The abnormal state in step S100 is notified to the operator terminal 400 by the operator management module 550 (S110). On the other hand, in the case where the abnormal entrant voluntarily exits in step S100, the process proceeds to step S80 and the above-described process is repeated. In addition, when an abnormal visitor who exits enters the store, the process described above may be executed by proceeding to step S10.

In addition, if the customer who normally entered the store in step S80 is not familiar with the store, and various inconveniences occur, such as an elderly customer, a point of sales (POS) terminal provided inside the store or a system installed in a kiosk. 3 When the call unit of the terminal 300 is operated (S120), the store management module 520 may execute store guidance through the speaker of the third terminal 300 (S130).

Meanwhile, each step described above is performed by the display module 700, for example, the image information of the first terminal 100 transmitted from 10 stores and the image information of the second terminal 200 transmitted from 5 stores. , may be displayed as image information of the third terminal 300 transmitted from five stores. When such a display module 700 is applied, a controller may actively manage an unmanned store from a remote location.

Although the invention made by the present inventors has been specifically described according to the above embodiments, the present invention is not limited to the above embodiments and can be changed in various ways without departing from the gist of the invention.

By using the entrance customer management system and management method according to the present invention, it is possible to efficiently manage an unmanned store.

100: first terminal
200: second terminal
300: third terminal
500: management server
700: display module

Claims (9)

A first terminal installed at the entrance of each door of a plurality of stores to support control of the door;
A second terminal capable of recognizing each door of a plurality of stores and provided to manage the inside of the store;
A third terminal provided inside each of a plurality of stores to support inconveniences and inquiries within the store;
An operator terminal for the operator who operates the store;
It is connected to the first terminal, the second terminal, and the third terminal respectively provided in a plurality of stores through a network, and enters in real time in response to image information transmitted through the first terminal, second terminal, and third terminal. management server to guide customer management;
A display module for displaying image information through the first terminal, the second terminal, and the third terminal, respectively;
The second terminal is controlled by the management server in conjunction with CCTVs, air conditioners, broadcasting equipment, and storage boxes provided in the store,
The management server applies a deep learning module to the customer's image information transmitted from the first terminal or the second terminal, learns and stores the learning module in a database, CCTV installed in the store, air conditioner, broadcasting equipment, and a storage box. Admission customer management system comprising an IoT management module for managing information about. In paragraph 1,
The admission customer management system, characterized in that it further comprises a database for storing information about the store and information about the entrance customer. In paragraph 2,
The first terminal, the second terminal, and the third terminal each include a Session Initiation Protocol (SIP) intercom,
The second terminal includes a sensor for detecting a human body and is interlocked with a CCTV installed in the store,
The third terminal is an entrance customer management system, characterized in that installed in a POS (Point Of Sales) terminal or kiosk provided inside the store. In paragraph 3,
The management server
A customer management module for managing the image information of the customer's identification card and face captured by the camera of the first terminal;
A store management module for managing information about stores equipped with the first terminal, the second terminal, and the third terminal;
The admission customer management system comprising a terminal management module for guiding admission customer management in real time according to the transmission and reception information from the first terminal, the second terminal, and the third terminal. In paragraph 1,
The admission customer management system, characterized in that the store is any one of a PC room, a karaoke room, an indoor golf driving range, and an unmanned store. In paragraph 2,
The display module displays image information of the first terminal transmitted from 10 stores, image information of the second terminal transmitted from 5 stores, and image information of the third terminal transmitted from 5 stores. customer management system. (a) a shooting guide step for recognizing a customer's ID or face according to a customer's call signal in the first terminal;
(b) photographing the customer's identification card or face in the first terminal according to the guidance in step (a), and determining whether the customer is a normal customer in the management server according to the photographed image information;
(c) opening the store door and registering customer information if the customer is determined to be a normal customer in step (b);
(d) detecting, in the second terminal, whether a customer other than the customer entered and exited by step (c) is present;
(e) notifying an abnormal state through the second terminal when it is determined that another customer has entered or exited in step (d);
(f) notifying an operator terminal of the abnormal state in step (e);
The display module displays image information through the first terminal, the second terminal, and the third terminal, respectively;
The second terminal is controlled by a management server in conjunction with CCTVs, air conditioners, broadcasting equipment, and storage boxes provided in the store,
The learning module of the management server learns by applying a deep learning module to the customer's image information transmitted from the first terminal or the second terminal and stores it in a database, and CCTVs, air conditioners, broadcasting equipment, and storage boxes installed in the store are stored. The information about the admission customer management method, characterized in that managed by the IoT management module. In paragraph 7,
The admission customer management method further comprising the step of supporting complaints and inquiries in the store through a third terminal for the customer who has entered and exited by the step (c). In paragraph 8,
Wherein the management server extracts a feature vector from a face image of image information captured as customer information in step (c) and stores it in a database.

Images