CN108876504A

CN108876504A - A kind of unmanned selling system and its control method

Info

Publication number: CN108876504A
Application number: CN201710817042.8A
Authority: CN
Inventors: 宋晨
Original assignee: Beijing Megvii Technology Co Ltd; Beijing Maigewei Technology Co Ltd
Current assignee: Beijing Megvii Technology Co Ltd; Beijing Maigewei Technology Co Ltd
Priority date: 2017-09-12
Filing date: 2017-09-12
Publication date: 2018-11-23
Anticipated expiration: 2037-09-12
Also published as: CN108876504B

Abstract

The embodiment provides a kind of unmanned selling system and its control methods.This method includes：In vivo detection and recognition of face are carried out to the facial image of acquisition, determined whether to open the first access control system according to recognition result；The image acquired in demographics region is used for demographics, opens the second access control system when the number is an only people；It identifies article to be paid, obtains payment information；Exit region carries out recognition of face again, opens outlet access control system if recognition of face passes through.The present invention is based on wireless sensor technology and computer vision technique, realize shop nobody sell, reduce human cost, improve shopping experience.

Description

A kind of unmanned selling system and its control method

Technical field

The present invention relates to a kind of artificial intelligence systems, in particular to a kind of to utilize wireless sensor technology, computer vision skill The unmanned selling system and its control method that art is realized.

Background technique

In the prior art, market or shop are typically necessary outfit one or several shopping guides or checkout personnel, even if It is free supermarket, also at least needs a cashier, and the staff in a shop is also only limited to work in this shop. Moreover, generally requiring to be lined up, this just brings big inconvenience to shopping when customer is more.

Although also there is the appearance of vending machine, due to space limitations, vending machine can only often peddle compared with For single commodity, customer can not possess the same experience of going window-shopping, also, customer can not freely select before payment Commodity.

Summary of the invention

The present invention is proposed in view of the above problem.The present invention provides a kind of unmanned selling system and its controlling parties Method.

According to an aspect of the present invention, a kind of unmanned selling system is provided, including, the first image collecting device, second Image collecting device, third image collecting device, face recognition module, the first number statistical module, article payment module, first Access control system, the second access control system and outlet access control system, wherein

First image collecting device, for obtaining facial image；

Face recognition module is based on for carrying out In vivo detection based on the facial image if In vivo detection passes through Face bottom library and the facial image carry out recognition of face, to obtain the first face recognition result, if the recognition of face Pass through, then opens first access control system；

First number statistical module, it is described if be turned off for detecting whether first access control system has been switched off The image information in the second image acquisition device demographics region, the image information based on the demographics region carry out Demographics are to obtain the first statistical result；If first statistical result is more than a people, second gate inhibition is protected Closing is held, if first statistical result is a people, the second access control system is opened；

Article payment module, article to be paid for identification, and obtain payment information；

Access control system is exported, for using the image information of the third image acquisition device exit region, is utilized The face recognition module identifies the facial image in the image information of the exit region, is known with obtaining the second face Not as a result, opening the outlet access control system if the recognition of face passes through.

It illustratively, further include that module is veritified in outlet, the outlet access control system includes third access control system and the 4th Access control system, wherein

The article payment module is also used to, and successful information is paid if obtained, by the article mark to be paid It is denoted as prepaid article；

It veritifies module and, if had turned on, detects outlet core for detecting whether the third access control system opens in outlet Region is tested with the presence or absence of not paying article, if second face recognition result, which passes through and is not detected, does not pay article, Open the 4th access control system.

Illustratively, the system also includes registration modules, for receiving the facial image of user's submission, to the face Image carries out picture quality judgement, if described image Quality estimation is qualification, the facial image is registered to the face Bottom library.

Illustratively, the system also includes the 4th image collecting device and number control module, the number controls mould The image information that block is used to sell region based on the 4th image acquisition device carries out demographics to obtain the second system As a result, if second statistical result is greater than the first predetermined threshold, first access control system remains turned-off meter.

Illustratively, the system also includes image analysis module, for based on the image information for selling region into The humanized analysis of every trade obtains the attribute information of the pedestrian, and, pedestrian is carried out based on the image information for selling region Behavioural analysis, obtains travel track, dwell regions and the residence time in the dwell regions of the pedestrian, described in identification Behavior is taken, tries on and is tried out in the examination of pedestrian.

Illustratively, the article payment module further includes the first RFID reader and deactivation device, described First RFID reader is used to read the RFID tag being placed in article payment region；The deactivation device is used Deactivating operation is carried out in the article is paid the RFID tag for having paid article in region.

Illustratively, it further includes the second RFID reader that module is veritified in the outlet, and second radio frequency identification is read Device is taken to veritify region with the presence or absence of the RFID tag for not carrying out deactivating operation for detecting the outlet.

Illustratively, the system also includes the first gravity detection device and the second gravity detection device,

The first gravity detection device is used to carry out gravity detection in the demographics region to obtain the first weight； The second gravity detection device, which is used to veritify region in the outlet, carries out gravity detection to obtain the second weight；And

The detection outlet, which is veritified region and be whether there is, not to be paid article and includes, if the second weight of target object There is no the successful information of the payment of the target object greater than the first weight, also, in the system, it is determined that the outlet core It tests region presence and does not pay article.

Illustratively, the system also includes consumption to veritify module,

It is right for time corresponding to the first face recognition result according to same target and the second face recognition result institute The time answered determines that first after first time section, and the time according to corresponding to second face recognition result is pre- Section of fixing time determines the second time interval；

It checks with the presence or absence of successful information is paid in the first time section and second time interval, if not In the presence of successful information is paid, then veritify whether consumer behavior occurs in first time section, if there is no consumption to go Then to veritify and passing through；In case of consumer behavior, then payment reminders information is pushed.

Illustratively, the consumption is veritified module and is specifically used for, and the monitor video for selling region is transferred, according to the monitoring Video veritifies whether the object has occurred consumer behavior in first time section.

Illustratively, the outlet access control system includes third RFID reader, and the third radio frequency identification is read Device is used to read the RFID tag data of the article carried positioned at the object of exit region；Module is veritified in the consumption It is specifically used for, the second face recognition result of the data and the object that are read according to the third RFID reader, Veritify whether the object has occurred consumer behavior in first time section.

According to another aspect of the present invention, a kind of control method of unmanned selling system is additionally provided, including:

Obtain the facial image of the first image acquisition device；

In vivo detection is carried out based on the facial image, if In vivo detection passes through, based on face bottom library and described Facial image carries out recognition of face, to obtain the first face recognition result, if the recognition of face passes through, opens first Access control system；And

Detect whether the first access control system has been switched off, if be turned off, the second image acquisition device demographics The image information in region, the image information based on the demographics region carry out demographics to obtain the first statistical result； If first statistical result is more than a people, the second access control system is remained turned-off, if first statistical result When for a people, then the second access control system is opened；

It identifies article to be paid, and obtains payment information；And

The image information for acquiring exit region, identifies the facial image in the image information of the exit region, To obtain the second face recognition result, if recognition of face passes through, outlet access control system is opened.

Illustratively, the method also includes,

Identification article to be paid, and payment information is obtained, it further include paying successful information if obtained, It is prepaid article by the article marking to be paid；And

Whether detection third access control system opens, if had turned on, detects outlet and veritifies region with the presence or absence of not paying Article opens the 4th access control system if second face recognition result, which passes through and is not detected, does not pay article.

Illustratively, the method also includes receiving the facial image that user submits, carry out image to the facial image The facial image is registered to face bottom library if described image Quality estimation is qualification by Quality estimation.

Illustratively, the method also includes the image information in region is sold in acquisition, based on the image for selling region Information carries out demographics to obtain the second statistical result；And

If second statistical result is greater than the first predetermined threshold, the first access control system is remained turned-off.

Illustratively, the method also includes carrying out pedestrian's attributive analysis based on the image information for selling region, obtain Obtain the attribute information of the pedestrian；And

Pedestrian behavior analysis is carried out based on the image information for selling region, the travel track of the pedestrian is obtained, stops It stays region and in the residence time of the dwell regions, identifies that behavior is taken, tries on and tried out in the examination of the pedestrian.

Illustratively, identification article to be paid includes being placed on object using the reading of the first RFID reader Product pay the RFID tag in region；

It is described to include for prepaid article by the article marking to be paid, the article is paid into penetrating in region Frequency identification tag carries out deactivating operation.

Illustratively, it includes using the second radio frequency that region is veritified with the presence or absence of article is not paid in the detection outlet Identification reader detects the outlet and veritifies region with the presence or absence of the RFID tag for not carrying out deactivating operation.

Illustratively, the method also includes,

Gravity detection is carried out in the demographics region to obtain the first weight；

Region, which is veritified, in the outlet carries out gravity detection to obtain the second weight；And

Illustratively, the method also includes,

According to corresponding to time corresponding to the first face recognition result of same target and the second face recognition result Time determines the first pre- timing after first time section, and the time according to corresponding to second face recognition result Between section determine the second time interval；Checking, which whether there is in the first time section and second time interval, pays successfully Information then veritify whether consumer behavior occurs in first time section, if do not had if there is no successful information is paid There is generation consumer behavior, then veritifies and pass through；In case of consumer behavior, then payment reminders information is pushed.

Illustratively, the method also includes,

The monitor video for selling region is transferred, veritify the object according to the monitor video is in first time section It is no that consumer behavior has occurred.

Illustratively, the method also includes,

Read the RFID tag data for being located at the article of object carrying of exit region；It is according to reading Second face recognition result of label data and the object, veritifies whether the object is disappeared in first time section Take behavior.

Unmanned selling system according to an embodiment of the present invention and its control method, utilize wireless sensor technology, computer Vision technique, can be realized shop, convenience store nobody sell, be arranged on megastore, square it is such nobody sell and be System, not only reduces human cost, improves and sell efficiency, and improve the shopping experience of customer.

Detailed description of the invention

The embodiment of the present invention is described in more detail in conjunction with the accompanying drawings, the above and other purposes of the present invention, Feature and advantage will be apparent.Attached drawing is used to provide to further understand the embodiment of the present invention, and constitutes explanation A part of book, is used to explain the present invention together with the embodiment of the present invention, is not construed as limiting the invention.In the accompanying drawings, Identical reference label typically represents same parts or step.

Fig. 1 shows the schematic flow of the control method according to an embodiment of the invention for unmanned selling system Figure；

Fig. 2 shows the schematic block diagrams of unmanned selling system according to an embodiment of the invention；

Fig. 3 shows the vertical view in the schematic unmanned shop that unmanned selling system according to an embodiment of the invention is realized Figure；

Fig. 4 shows bowing for the schematic unmanned shop that unmanned selling system in accordance with another embodiment of the present invention is realized View.

Specific embodiment

In order to enable the object, technical solutions and advantages of the present invention become apparent, root is described in detail below with reference to accompanying drawings According to example embodiments of the present invention.Obviously, described embodiment is only a part of the embodiments of the present invention, rather than this hair Bright whole embodiments, it should be appreciated that the present invention is not limited by example embodiment described herein.Based on described in the present invention The embodiment of the present invention, those skilled in the art's obtained all other embodiment in the case where not making the creative labor It should all fall under the scope of the present invention.

To solve problem as described above, the embodiment of the present invention provides a kind of unmanned selling system.Wherein, using computer Vision technique identifies customer, including recognition of face, Activity recognition, using wireless sensor technology or barcode scanning mode to object Product are identified.The building in unmanned shop may be implemented in unmanned selling system provided in an embodiment of the present invention, without employing salesman, Realize the full self-help shopping of customer.

Firstly, describing the unmanned selling system for realizing the embodiment of the present invention referring to Fig. 2.

As shown in Fig. 2, unmanned selling system 200 include the first image collecting device 201, the second image collecting device 202, Third image collecting device 203, face recognition module 204, the first number statistical module 205, article payment module 206, first Access control system 207, the second access control system 208 and outlet access control system 209.

First image collecting device 201, for obtaining facial image.

Digital camera, common camera, high-definition camera, web camera can be used in first image collecting device 201 Equal image/videos acquire equipment, carry out image acquisition to special scenes.For example, the first image collecting device 201 can be arranged In the top of unmanned shop entrances, set certain angle, enable the first image collecting device 201 collect quality compared with Image or video good and that meet human face detection and recognition requirement.The facial image got can be directly collected Original image, the image being also possible to after being pre-processed through the first image collecting device 201.Second image collecting device 202 and third image collecting device 203 device identical with the first image collecting device 201 can be used, also can be used The device different from the first image collecting device 201.

Face recognition module 204, for carrying out In vivo detection based on the facial image, if In vivo detection passes through, Recognition of face is carried out based on face bottom library and the facial image, to obtain the first face recognition result, if the face It is identified by, then opens first access control system 207.

In vivo detection is carried out for the facial image that the first image collecting device 201 obtains, and facial image here can be with It is a frame or multiple image, is also possible to the video of a series of images frame composition.Biopsy method, which can be used, specifically to be set It is standby to carry out, for example, infrared structure light, judge whether face is living body using the reflection case for the structure light beaten on face, it can also Not have to particular device, the method for only using image and video analysis, for example, the specific instruction of sending (such as shake the head, open one's mouth, point Head etc.), judge whether face is living body by the image comprising face specific action or video of analysis acquisition, in another example, Three-dimensional reconstruction is carried out to face in the image of different angle using two or multiple faces, people is judged by three-dimensional shape information Whether face is living body.

Face bottom library can be a database for saving registration user's facial image, can also save from facial image The facial image feature extracted.It can be the facial image that user uploads when registering by terminal, for example, user It is linked into registration interface by cell phone application or other software program, shoot a human face photo or is selected in existing photo It selects a human face photo to be uploaded, cloud server saves this photo as face bottom library photo after veritifying.

The method that face identification method can be learnt using deep neural network, convolutional neural networks even depth, can also be with Using the feature of the Manual definitions such as LBP, SIFT and the conventional method of pattern classification.It can be then directed to advanced row Face datection Face datection frame carries out recognition of face again.

First access control system 207 may be any type of door, be determined whether to open this according to the result of recognition of face Fan door.It may include an electromagnetic control system, if the result judgement of recognition of face is that face is identified as function, to first Access control system 207 send first instruction so that electromagnetic control system loses magnetism, door can be opened freely, with allow customer into Enter.First access control system 207 can also take other forms, such as revolving door.Second access control system 208, outlet access control system 209 and other access control systems form identical with the first access control system 207 can be used, different forms can also be used.

First number statistical module 205, for detecting whether first access control system 207 has been switched off, if closed It closes, second image collecting device 202 acquires the image information in demographics region, the figure based on the demographics region As information carries out demographics to obtain the first statistical result.

A number is arranged in the first access control system 207 and the second access control system 208 in the occurrence of trailing in order to prevent Statistical regions place the second image collecting device 202 in demographics region, by counting people based on the method for image analysis Pedestrian's quantity in number statistical regions.Start to unite after customer enters demographics region and closes the first access control system 207 Pedestrian's quantity in demographics region is counted, the first statistical result is obtained.

If first statistical result is more than a people, second access control system 208 is remained turned-off, if institute State the first statistical result be a people when, then open second access control system 208.

By the analysis to the first statistical result, may thereby determine that has several individuals in demographics region, if it exceeds One people then refuses to open the second access control system 208.It is alternatively possible to shown using screen or voice mode is prompted, Prompt once only can permit people entrance；If an only people, the second access control system 208 is opened, customer is allowed to enter Shopping area.In this way, after recognition of face opening gate being passed through to avoid a people, and have and entered in the form of trailing The loss of article can be effectively avoided in other unrecognized people.

Article payment module 206, article to be paid, obtains payment information for identification.

After customer enters shopping area, commodity can be freely selected, it, can when customer has selected commodity to need to pay the bill To be paid to scheduled payment region.Article payment module 206 can set one for reading for realizing delivery operation The region for taking merchandise news is placed in the area to commodity, and identification device identifies the commodity in region, and will identification Commodity register out, commodity amount, commodity price are shown that user can prompt to carry out delivery operation according to payment.Example Property, it can be paid using cell phone application or Third-party payment platform.After the completion of payment, article payment module 206 is real-time Ground gets payment information, shows pay successfully on the display apparatus.It optionally, can in order to improve the accuracy that commodity identify The maximum commodity amount that each delivery operation can identify is arranged, for example, most multipair 5 commodity are identified and are propped up every time It pays, the maximum commodity amount that can be identified can freely be set as the case may be.

Access control system 209 is exported, for using the image information of the third image acquisition device exit region, benefit The facial image in the image information of the exit region is identified with the face recognition module, to obtain the second face Recognition result opens the outlet access control system if the recognition of face passes through.

One outlet region is set, outlet access control system 209 is set in exit region, illustratively, exports access control system A sect can be set, two sects also can be set.For a sect is arranged, customer is gone in exit region, Image Acquisition Device starts to acquire facial image, and further carries out recognition of face, if recognition of face success, opens outlet gate inhibition system System, the commodity that customer can carry purchase are walked out.

According to an embodiment of the invention, outlet access control system 209 can also be arranged two sects, i.e. third access control system and the Four access control systems, if the article payment module 206, which is also used to obtain, pays successful information, by the object to be paid Product are labeled as prepaid article；In addition module is veritified in setting outlet, for detecting whether the third access control system opens, such as Fruit has turned on, then detects the outlet and veritify region with the presence or absence of not paying article and if second recognition of face passes through And be not detected and do not pay article, then open the 4th access control system.

Outlet is veritified region and is arranged between third access control system and the 4th access control system, and customer enters outlet core Testing region indicates that the customer has settled accounts completion, is intended to veritify and go out, or the customer not done shopping also is intended to veritify to go out.Customer's warp Third access control system is crossed to enter after the outlet veritifies region, commodity detection device start to detect outlet veritification region be No there are unpaid commodity, it is therefore intended that prevents customer from carrying unpaid commodity and leaves；In addition, placing in place Third image collecting device 203, for the recognition of face to departure personnel.It is not propped up if recognition of face passes through and is not detected The commodity paid then open the 4th access control system, allow customer to go out, shopping process terminates.If recognition of face fail or Region detection is veritified to unpaid commodity in outlet, then is not turned on the 4th access control system, permission customer is by third gate inhibition System be back to shopping area, with carry out commodity return or continue payment etc. operation.It is arranged in addition, veritifying region in the outlet The problem of one emergency device, prevents recognition of face from not passing through, and customer can not go out, if customer is entered by recognition of face Shopping area, but can not be gone out by recognition of face, it can be sought help by device of crying for help, remote administrator can carry out manually It assists.

According to an embodiment of the invention, setting registration module, for receiving the facial image of user's submission, to the face Image carries out picture quality judgement, if described image Quality estimation is qualification, the facial image is registered to the face Bottom library.

Due to the facial image for the registration of face bottom library that customer uploads be it is diversified, it is fuzzy if it is image, Illumination is bad, non-positive face photo etc., and for the bottom library photo of recognition of face, it will cause the reductions of the precision of recognition of face, therefore, It needs to carry out picture quality judgement to the human face photo of upload in registration link.Illustratively, picture quality judgement may include Image size, image fog-level, image irradiation condition, human face posture etc., if picture quality is judged as qualified, by the figure As uploading to face bottom library.In addition, In vivo detection can be carried out in registration process.Optionally, face bottom library can be located at this Ground can also be located at cloud.

According to an embodiment of the invention, further including the 4th image collecting device (attached to be not shown in the figure) and number control mould Block, the image information that the number control module is used to sell region based on the 4th image acquisition device carry out number Statistics is to obtain the second statistical result, if second statistical result is greater than the first predetermined threshold, the first gate inhibition system System remains turned-off.

In order to improve shopping experience, needs to control the shopping number of shopping area, adopted selling region the 4th image of setting Acquisition means obtain the second statistical result, if it exceeds in the number of setting for counting to the real-time number for selling region Limit, such as 20 people, then no longer allow new customer to enter, and keeps the first access control system to close, and number is prompted to expire.4th figure As acquisition device can be a photographic device, or multiple photographic devices, to cover the range of shopping area whole. Optionally, when reaching the number upper limit, sound or visual cues can be carried out, such as show and reached by a display device The number upper limit, or voice prompting is carried out by audio device.

According to an embodiment of the invention, further including image analysis module, for based on the image information for selling region Pedestrian's attributive analysis is carried out, the attribute information of the pedestrian is obtained, and, it is gone based on the image information for selling region People's behavioural analysis obtains travel track, dwell regions and the residence time in the dwell regions of the pedestrian, identifies institute Behavior is taken, tries on and is tried out in the examination for stating pedestrian.

Image or video information based on the 4th image acquisition device are analyzed, based on video structural with And the method for deep learning carries out video analysis, detects pedestrian present in image or video, attribute and behavior to pedestrian It is analyzed, whether the attribute of pedestrian includes but is not limited to the age of pedestrian, gender, clothing and wear glasses, according to pedestrian Attribute and its analysis of behavior can analyze hobby of the particular customer when buying commodity, for example, age bracket is between 20-30 Customer prefer which commodity, customer women is interested in the commodity in which region etc..Pass through the semanteme to video point Analysis, can help shop administrator targetedly to place commodity, and commodity are placed on reasonable position.

According to an embodiment of the invention, the article payment module further includes the first RFID reader and deactivation Device, first RFID reader are used to read the RFID tag being placed in article payment region；It is described to go The RFID tag for having paid article that activation device is used to pay the article in region carries out deactivating operation.

Illustratively, by the way of radio frequency identification (RFID) carry out commodity mark and identification.For example, in each quotient The label of RFID is all posted on product, and the merchandise classification, commodity price, commodity weight, commodity production day are all stored on every kind of label Phase, commodity shelf-life, merchandise resources information etc.；When being paid, article pays the first radio frequency identification in region and reads Device reads the RFID tag being placed in article payment region, optionally, after reading the merchandise news stored on RFID tag, It is shown on the display apparatus, for example, the property of the can choose ground information such as display of commodity serial number, product name, commodity price, to After customer clicks confirmation payment, direct customers carry out delivery operation, for example, hand can be paid or passed through by bank card Machine APP and Third-party payment platform are paid.Alternatively it is also possible to not use display device, only merchandise news is pushed To user terminal, user carries out delivery operation at the terminal.After getting the successful information of payment to article payment module, go The article is paid the RFID tag for having paid article in region and carries out deactivating operation by activation device, for example, will RFID tag carries out deactivating operation, and the RFID tag by deactivating operation cannot be detected by RFID reader.

According to an embodiment of the invention, it further includes the second RFID reader that module 207 is veritified in the outlet, described the Two RFID readers are used to detect the outlet and veritify region with the presence or absence of the radio frequency identification marking for not carrying out deactivating operation Label.

Second RFID reader is veritified region in outlet and is detected, and does not carry out deactivated RFID if detected Label then illustrates that customer carries still unpaid commodity, and goods damage, needs that customer is prompted to return under such circumstances in order to prevent Payment region is paid.

According to an embodiment of the invention, further include the first gravity detection device and the second gravity detection device,

First gravity detection device and the second gravity detection device are used for weight when detecting when customer enters and going out.One As in the case of, if customer does not do shopping, weight when going out should be equal to weight when entering.If body when going out Great weight when entering, then it represents that customer is done shopping, to the customer since the time point for entering the first gate inhibition, The payment information for whether having the customer is inquired in systems, if there is no payment information, it is determined that customer, which carries, not to be paid Commodity.Because RFID tag is typically pasted onto the packaging of commodity above, it can prevent someone from deliberately tearing RFID in this way Label is to escape the detection that region is veritified in outlet.Further, when inquiring the payment information of customer, it can inquire and pay The weight information of commodity, and further check the weight information and whether subtract the difference of the weight entered with the weight gone out and be consistent It closes, if do not met, it is determined that there is non-payment for merchandise.

According to an embodiment of the invention, further include that module is veritified in consumption,

It is right for time corresponding to the first face recognition result according to same target and the second face recognition result institute The time answered determines that first after first time section, and the time according to corresponding to second face recognition result is pre- Section of fixing time determines the second time interval；It checks in the first time section and second time interval with the presence or absence of payment Successful information then veritifies consumer behavior whether occurs in first time section, such as if there is no successful information is paid There is no consumer behaviors for fruit, then veritify and pass through；In case of consumer behavior, then payment reminders information is pushed.

In another example, access control system 209 one sects of setting are exported, customer can not be in quotient after having chosen commodity Paid in shop, illustratively, can by the way of barcode scanning payment, for example, customer, which has chosen commodity, can leave unmanned shop, At this moment, he goes to exit region, can go out after recognition of face, but customer need is complete within the certain time after going out At payment, for example, half an hour is considered as doing shopping if in half an hour, customer completes payment in another place It completes.If all without payment record in the half an hour in the time in unmanned shop and after going out, core of expanding consumption It tests, veritifies whether customer has occurred consumer behavior in this period of time from entering to going out, if without consumer behavior, core It tests and passes through, pay in case of consumer behavior but not, then push payment reminders information to the customer.Further, such as Fruit is more than to pay not yet certain time, then can reduce the credit score of the account, or blacklist is added in the customer, no It is allowed to be again introduced into unmanned shop.

It is specifically used for according to an embodiment of the invention, module is veritified in the consumption, transfers the monitor video for selling region, root Veritify whether the object has occurred consumer behavior in first time section according to the monitor video.

It, can that is, artificial to veritify by the way of artificial, root when whether consumer behavior occurring according to monitor video veritification The customer that consumer behavior occurs and the type of merchandize of consumption are confirmed according to video monitoring；It can also use based on computer vision Algorithm is veritified automatically, that is, identifies the commodity that customer takes in which region, the commodity that the method identification customer based on image analysis takes Type.

According to an embodiment of the invention, the outlet access control system 209 includes third RFID reader, the third RFID reader is used to read the RFID tag data of the article carried positioned at the object of exit region；It is described Consumption is veritified module and is specifically used for, the second people of the data and the object that are read according to the third RFID reader Face recognition result, veritifies whether the object has occurred consumer behavior in first time section.

When veritifying the consumer behavior of customer, the mode of radio frequency identification can also be used.When user goes out by exit region When going, RFID reader just has read the information that the radio frequency identification marking on commodity entrained by customer is signed, although using The mode of RFID identifies commodity, but still may not necessarily pay in shop, props up in the certain time after departure after customer It pays.The commodity that user carries away are identified by the way of RFID, and combine face recognition result when going out, and are veritified specific The consumer behavior of customer.

According to the another aspect of invention, a kind of control method of unmanned selling system is provided, Fig. 1 is shown according to this hair The schematic flow chart of the control method of the unmanned selling system of bright one embodiment.This approach includes the following steps.

Step S110 obtains the facial image of the first image collecting device 201 acquisition.

Digital camera, common camera, high-definition camera, web camera can be used in first image collecting device 201 Equal image/videos acquire equipment, carry out image acquisition to special scenes.For example, the first image collecting device 201 can be arranged In the top of unmanned shop entrances, set certain angle, enable the first image collecting device 201 collect quality compared with Image or video good and that meet human face detection and recognition requirement.The facial image got can be directly collected Original image, the image being also possible to after being pre-processed through the first image collecting device 201.

Step S120 carries out In vivo detection based on the facial image, if In vivo detection passes through, is based on face bottom library And the facial image carries out recognition of face and, if the recognition of face passes through, is opened with obtaining the first face recognition result Open the first access control system 207.

Step S130, whether the first access control system 207 of detection has been switched off, if be turned off, the second image collecting device The image information in 202 acquisition demographics regions, the image information based on the demographics region carry out demographics to obtain To the first statistical result.If first statistical result is more than a people, the second access control system 208 is remained turned-off, such as When first statistical result described in fruit is a people, then the second access control system 208 is opened

By the analysis to the first statistical result, may thereby determine that has several individuals in demographics region, if it exceeds One people then refuses to open the second access control system 208.It is alternatively possible to shown using screen or voice mode is prompted, Prompt once only can permit people entrance；If an only people, the second access control system 208 is opened, customer is allowed to enter Shopping area.In this way, after recognition of face opening gate being passed through to avoid a people, and there is the form trailed and enter The loss of article can be effectively avoided in other unrecognized people.

Step S140 identifies article to be paid, and obtains payment information.

Step 150, the image information for acquiring exit region, to the facial image in the image information of the exit region into Row identification, to obtain the second face recognition result, if recognition of face passes through, opens outlet access control system.

According to an embodiment of the invention, can also include the following steps：

Identification article to be paid, and payment information is obtained, it further include paying successful information if obtained, It is prepaid article by the article marking to be paid；Whether detection third access control system opens, if had turned on, examines It measures mouth and veritifies region with the presence or absence of article is not paid, do not paid if second face recognition result passes through and is not detected Article then opens the 4th access control system.

According to an embodiment of the invention, can also include the following steps, the facial image that user submits is received, to the people Face image carries out picture quality judgement, if described image Quality estimation is qualification, the facial image is registered to the people Face bottom library.

According to an embodiment of the invention, can also include the following steps, the image information in region is sold in acquisition, based on described The image information for selling region carries out demographics to obtain the second statistical result；If second statistical result is greater than first Predetermined threshold, then the first access control system remains turned-off.

According to an embodiment of the invention, can also include the following steps, carried out based on the image information for selling region Pedestrian's attributive analysis obtains the attribute information of the pedestrian；Pedestrian behavior point is carried out based on the image information for selling region Analysis, obtains travel track, dwell regions and the residence time in the dwell regions of the pedestrian, identifies the pedestrian's Behavior is taken, tries on and is tried out in examination.

According to an embodiment of the invention, the step S150 further includes following steps, the first RFID reader is used Read the RFID tag being placed in article payment region；It is described by the article marking to be paid be prepaid object Product include that the article is paid the RFID tag in region and carries out deactivating operation.

According to an embodiment of the invention, the step S160 further includes following steps, the second RFID reader is used It detects the outlet and veritifies region with the presence or absence of the RFID tag for not carrying out deactivating operation.

According to an embodiment of the invention, can also include the following steps,

Gravity detection is carried out in the demographics region to obtain the first weight；Region, which is veritified, in the outlet carries out weight Power is detected to obtain the second weight；The detection outlet, which is veritified region and be whether there is, not to be paid article and includes, if target Second weight of object is greater than the first weight, also, does not have the successful information of the payment of the target object in the system, then It determines that outlet veritification region exists and does not pay article.

Weight when being entered using gravity detection device detection customer and when going out.Under normal circumstances, if customer does not have Shopping, then weight when going out should be equal to weight when entering.If weight when going out is greater than weight when entering, Indicate that customer is done shopping, whether to the customer since the time point for entering the first gate inhibition, inquire has this in systems The payment information of customer, if there is no payment information, it is determined that customer carries unpaid commodity.Because of RFID tag one As be all to be pasted onto the packagings of commodity above, can prevent someone from deliberately tearing RFID tag in this way to escape outlet and veritify region Detection.Further, when inquiring the payment information of customer, the weight information of payment for merchandise can be inquired, goes forward side by side one The difference whether the step verification weight information subtracts the weight entered with the weight gone out is consistent, if do not met, it is determined that In the presence of non-payment for merchandise.

According to an embodiment of the invention, further include following steps,

Fig. 3 shows the schematic diagram for applying the unmanned shop 300 of the unmanned selling system of the present invention, and Fig. 3 illustrates the nothing The top view in people shop 300.

It may be any type of shop using unmanned shop of the invention, for example, the shop being arranged in common StoreFront, Or the shop of Portakabin building.Unmanned shop 300 includes an entrance and one outlet, and entrance and exit can be in office What position setting appropriate.Twice access control system is respectively set in inlet and exit, the first access control system is arranged in inlet 310 and second access control system 320, in exit, the outlet access control system of setting includes third access control system 330 and the 4th gate inhibition System 340.

It is the space of a relative closure between first access control system 310 and the second access control system 320, referred to as number is united Region 340 is counted, after customer enters demographics region 340 by the first access control system 310, passes through computer vision technique Number in the region is counted, if an only people, the second access control system 320 is opened, customer is allowed to pass through second Access control system 320 enters the shopping area in unmanned shop 300, if the number in demographics region 340 has been more than a people, Then explanation has the case where trailing enters, and is not turned on the second access control system 320, and customer is prompted once only to can enter a people.It is logical The occurrence of crossing demographics, not only preventing trailing, and it is convenient for the management in unmanned shop, it is therefore prevented that the generation of goods damage.

It is a relatively closed space between third access control system 330 and the 4th access control system 340, referred to as exports core Test region 350.If customer enters the outlet by third access control system 330 and veritifies region 350, it is determined that the master of the customer Seeing wish is to leave unmanned shop 300 in the future, and the apparatus for checking open detection in region 350 is veritified in outlet at this time, and whether veritify has Unpaid commodity open the 4th access control system 340 if the commodity not paid are not detected, and customer is allowed to go out, until This, the shopping activity of customer is completed.If being veritified in region 350 in outlet and having detected still unpaid commodity, on the one hand, protect The closing of the 4th access control system 340 is held, on the other hand, reminds consumer items payment not in such a way that visual cues or sound sensation prompt Complete, and allow customer by third access control system 330 back to shopping area, customer can continue to complete delivery operation or Unpaid commodity are returned into original place.

Fig. 4 shows the schematic diagram for applying the unmanned shop 400 of the unmanned selling system of the present invention, and Fig. 4 illustrates the nothing The top view in people shop 400.

It may be any type of shop using unmanned shop of the invention, for example, the shop being arranged in common StoreFront, Or the shop of Portakabin building.Unmanned shop 400 includes an entrance and one outlet, and entrance and exit can be in office What position setting appropriate.Twice access control system is set in inlet, the first access control system 410 and the second gate inhibition is arranged in inlet Outlet access control system 430 is arranged in exit in system 420, and outlet access control system only has a sect.

It is the space of a relative closure between first access control system 410 and the second access control system 420, referred to as number is united Region 440 is counted, after customer enters demographics region 440 by the first access control system 410, passes through computer vision technique Number in the region is counted, if an only people, the second access control system 420 is opened, customer is allowed to pass through second Access control system 420 enters the shopping area in unmanned shop 400, if the number in demographics region 440 has been more than a people, Then explanation has the case where trailing enters, and is not turned on the second access control system 420, and customer is prompted once only to can enter a people.It is logical The occurrence of crossing demographics, not only preventing trailing, and it is convenient for the management in unmanned shop, it is therefore prevented that the generation of goods damage.

One outlet region 450 is set near exit, when customer comes into exit region, system thinks that the customer wants It gos out, opens image acquisition device facial image, collected facial image is identified, if be identified by, Outlet access control system 430 is directly opened, customer is allowed to go out.In the present embodiment, after allowing customer to go out within a certain period of time It is paid again, so being only provided with a sect in outlet access control system.

Although describing example embodiment by reference to attached drawing here, it should be understood that above example embodiment are only exemplary , and be not intended to limit the scope of the invention to this.Those of ordinary skill in the art can carry out various changes wherein And modification, it is made without departing from the scope of the present invention and spiritual.All such changes and modifications are intended to be included in appended claims Within required the scope of the present invention.

Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually It is implemented in hardware or software, the specific application and design constraint depending on technical solution.Professional technician Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed The scope of the present invention.

In several embodiments provided herein, it should be understood that disclosed device and method can pass through it Its mode is realized.For example, apparatus embodiments described above are merely indicative, for example, the division of the unit, only Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can be tied Another equipment is closed or is desirably integrated into, or some features can be ignored or not executed.

In the instructions provided here, numerous specific details are set forth.It is to be appreciated, however, that implementation of the invention Example can be practiced without these specific details.In some instances, well known method, structure is not been shown in detail And technology, so as not to obscure the understanding of this specification.

Similarly, it should be understood that in order to simplify the present invention and help to understand one or more of the various inventive aspects, To in the description of exemplary embodiment of the present invention, each feature of the invention be grouped together into sometimes single embodiment, figure, Or in descriptions thereof.However, the method for the invention should not be construed to reflect following intention：It is i.e. claimed The present invention claims features more more than feature expressly recited in each claim.More precisely, such as corresponding power As sharp claim reflects, inventive point is that the spy of all features less than some disclosed single embodiment can be used Sign is to solve corresponding technical problem.Therefore, it then follows thus claims of specific embodiment are expressly incorporated in this specific Embodiment, wherein each, the claims themselves are regarded as separate embodiments of the invention.

It will be understood to those skilled in the art that any combination pair can be used other than mutually exclusive between feature All features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so disclosed any method Or all process or units of equipment are combined.Unless expressly stated otherwise, this specification (is wanted including adjoint right Ask, make a summary and attached drawing) disclosed in each feature can be replaced with an alternative feature that provides the same, equivalent, or similar purpose.

In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments In included certain features rather than other feature, but the combination of the feature of different embodiments mean it is of the invention Within the scope of and form different embodiments.For example, in detail in the claims, embodiment claimed it is one of any Can in any combination mode come using.

Various component embodiments of the invention can be implemented in hardware, or to run on one or more processors Software module realize, or be implemented in a combination thereof.It will be understood by those of skill in the art that can be used in practice Microprocessor or digital signal processor (DSP) are realized in the device according to an embodiment of the present invention for pedestrian detection The some or all functions of some modules.The present invention is also implemented as a part for executing method as described herein Or whole program of device (for example, computer program and computer program product).It is such to realize that program of the invention May be stored on the computer-readable medium, or may be in the form of one or more signals.Such signal can be from Downloading obtains on internet website, is perhaps provided on the carrier signal or is provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and ability Field technique personnel can be designed alternative embodiment without departing from the scope of the appended claims.In the claims, Any reference symbol between parentheses should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not Element or step listed in the claims.Word "a" or "an" located in front of the element does not exclude the presence of multiple such Element.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer real It is existing.In the unit claims listing several devices, several in these devices can be through the same hardware branch To embody.The use of word first, second, and third does not indicate any sequence.These words can be explained and be run after fame Claim.

The above description is merely a specific embodiment or to the explanation of specific embodiment, protection of the invention Range is not limited thereto, and anyone skilled in the art in the technical scope disclosed by the present invention, can be easily Expect change or replacement, should be covered by the protection scope of the present invention.Protection scope of the present invention should be with claim Subject to protection scope.

Claims

1. a kind of unmanned selling system, including,

First image collecting device, the second image collecting device, third image collecting device, face recognition module, the first number Statistical module, article payment module, the first access control system, the second access control system and outlet access control system, wherein

First image collecting device, for obtaining facial image；

Face recognition module, if In vivo detection passes through, is based on face for carrying out In vivo detection based on the facial image Bottom library and the facial image carry out recognition of face, to obtain the first face recognition result, if the recognition of face passes through, Then open first access control system；

First number statistical module, for detecting whether first access control system has been switched off, if be turned off, described second The image information in image acquisition device demographics region, the image information based on the demographics region carry out number Statistics is to obtain the first statistical result；If first statistical result is more than a people, second gate inhibition keeps closing It closes, if first statistical result is a people, the second access control system is opened；

Access control system is exported, for using the image information of the third image acquisition device exit region, using described Face recognition module identifies the facial image in the image information of the exit region, to obtain the second recognition of face knot Fruit opens the outlet access control system if the recognition of face passes through.

2. system according to claim 1 further includes that module is veritified in outlet, the outlet access control system includes third gate inhibition System and the 4th access control system, wherein

The article payment module is also used to, and is paid successful information if obtained, is by the article marking to be paid Prepaid article；

Module is veritified in outlet, for detecting whether the third access control system opens, if had turned on, is detected outlet and is veritified area It, if second face recognition result, which passes through and is not detected, does not pay article, is opened with the presence or absence of article is not paid in domain 4th access control system.

3. system according to claim 2 further includes registration module, for receiving the facial image of user's submission, to institute It states facial image and carries out picture quality judgement, if described image Quality estimation is qualification, the facial image is registered to institute State face bottom library.

4. system according to claim 2 further includes the 4th image collecting device and number control module, the number control The image information that molding block is used to sell region based on the 4th image acquisition device carries out demographics to obtain the Two statistical results, if second statistical result is greater than the first predetermined threshold, first access control system is remained turned-off.

5. system according to claim 4 further includes image analysis module, for based on the image letter for selling region Breath carries out pedestrian's attributive analysis, obtains the attribute information of the pedestrian, and, it is carried out based on the image information for selling region Pedestrian behavior analysis, obtains travel track, dwell regions and the residence time in the dwell regions of the pedestrian, identifies Behavior is taken, tries on and is tried out in the examination of the pedestrian.

6. system according to claim 2, the article payment module further includes the first RFID reader and goes Device is activated, first RFID reader is used to read the RFID tag being placed in article payment region；Institute It states deactivation device and carries out deactivating operation for the article to be paid to the RFID tag for having paid article in region.

7. according to the method described in claim 6, it further includes the second RFID reader that module is veritified in the outlet, described the Two RFID readers are used to detect the outlet and veritify region with the presence or absence of the radio frequency identification marking for not carrying out deactivating operation Label.

8. system according to claim 2, further includes the first gravity detection device and the second gravity detection device,

The first gravity detection device is used to carry out gravity detection in the demographics region to obtain the first weight；It is described Second gravity detection device, which is used to veritify region in the outlet, carries out gravity detection to obtain the second weight；And

The detection outlet, which is veritified region and be whether there is, not to be paid article and includes, if the second weight of target object is greater than First weight, also, there is no the successful information of the payment of the target object in the system, it is determined that veritify area in the outlet Domain, which exists, does not pay article.

9. system according to claim 1 further includes that module is veritified in consumption,

For corresponding to time corresponding to the first face recognition result according to same target and the second face recognition result Time determines the first pre- timing after first time section, and the time according to corresponding to second face recognition result Between section determine the second time interval；

Checking, which whether there is in the first time section and second time interval, pays successful information, if there is no Successful information is paid, then veritifies whether consumer behavior occurs in first time section, if there is no consumer behavior, Veritification passes through；In case of consumer behavior, then payment reminders information is pushed.

10. system according to claim 9,

The consumption is veritified module and is specifically used for, and the monitor video for selling region is transferred, according to monitor video veritification Whether object has occurred consumer behavior in first time section.

11. system according to claim 9,

The outlet access control system includes third RFID reader, and the third RFID reader is located at for reading The RFID tag data for the article that the object of exit region carries；The consumption is veritified module and is specifically used for, according to Second face recognition result of data and the object that the third RFID reader is read, veritifies the object and exists Whether consumer behavior has occurred in first time section.

12. a kind of control method of unmanned selling system, including,

Obtain the facial image of the first image acquisition device；

In vivo detection is carried out based on the facial image, if In vivo detection passes through, is based on face bottom library and the face Image carries out recognition of face, to obtain the first face recognition result, if the recognition of face passes through, opens the first gate inhibition system System；And

Detect whether the first access control system has been switched off, if be turned off, the second image acquisition device demographics region Image information, image information based on the demographics region carries out demographics to obtain the first statistical result；If When first statistical result is more than a people, then the second access control system remains turned-off, if first statistical result is one When people, then the second access control system is opened；

It identifies article to be paid, and obtains payment information；And

The image information for acquiring exit region, identifies the facial image in the image information of the exit region, to obtain It obtains the second face recognition result and opens outlet access control system if recognition of face passes through.

13. according to the method for claim 12,

Identification article to be paid, and payment information is obtained, it further include successful information being paid if obtained, by institute Stating article marking to be paid is prepaid article；And

The method also includes,

Whether detection third access control system opens, if had turned on, detects outlet and veritifies region with the presence or absence of not paying article, If second face recognition result, which passes through and is not detected, does not pay article, the 4th access control system is opened.

14. according to the method for claim 13, further include,

The facial image that user submits is received, picture quality judgement is carried out to the facial image, if described image quality is sentenced Break as qualification, the facial image is registered to face bottom library.

15. according to the method for claim 13, further include,

The image information in region is sold in acquisition, carries out demographics based on the image information for selling region to obtain the second system Count result；And

16. according to the method for claim 15, further include,

Pedestrian's attributive analysis is carried out based on the image information for selling region, obtains the attribute information of the pedestrian；And

Pedestrian behavior analysis is carried out based on the image information for selling region, obtains travel track, the stacking area of the pedestrian Domain and in the residence time of the dwell regions, identifies that behavior is taken, tries on and tried out in the examination of the pedestrian.

17. according to the method for claim 13, identification article to be paid includes being read using the first radio frequency identification Device reads the RFID tag being placed in article payment region；

It is described to include for prepaid article by the article marking to be paid, the article is paid into the radio frequency in region and is known Distinguishing label carries out deactivating operation.

18. according to the method for claim 17, detection outlet veritification region, which whether there is, does not pay article packet It includes,

The outlet, which is detected, using the second RFID reader veritifies region with the presence or absence of the radio frequency for not carrying out deactivating operation Identify label.

19. according to the method for claim 13, further include,

20. according to the method for claim 12, further include,

According to the time corresponding to time corresponding to the first face recognition result of same target and the second face recognition result Determine the first predetermined amount of time after first time section, and the time according to corresponding to second face recognition result Determine the second time interval；It checks in the first time section and second time interval and successfully believes with the presence or absence of payment Breath then veritifies whether consumer behavior occurs in first time section, if do not sent out if there is no successful information is paid Raw consumer behavior, then veritify and pass through；In case of consumer behavior, then payment reminders information is pushed.

21. according to the method for claim 20, further include,

The monitor video for selling region is transferred, veritifies whether the object is sent out in first time section according to the monitor video Consumer behavior is given birth to.

22. according to the method for claim 20, further including

Read the RFID tag data for being located at the article of object carrying of exit region；According to reading the label Second face recognition result of data and the object, veritifies whether the object has occurred consumption row in first time section For.