CN104346883A - Point of sale (POS) device capable of detecting customer - Google Patents

Abstract

The invention discloses a point of sale (POS) device capable of detecting a customer. The device is implemented by the following steps: automatically taking a photo by using a camera loaded in a POS machine; capturing a customer face image from the taken image through a human face detection method; extracting human face features by using a human face feature extraction method, and matching the extracted human face features with member human face features saved in a database by using a human face matching method to finish detection of the customer. The customer is identified while the POS machine is used for executing a sales record through human face detection and authentication process, so that CRM ((Customer Relationship Management) service, payment authentication service and payment record service are provided.

Description

A kind of POS device identifying client

Technical field

The present invention relates to e-commerce field, particularly relate to the safety certification application of ecommerce.

Background technology

1. background

As point-of-sale terminal, market proposes multi-functional requirement to POS (Point-Of-Sales), particularly following two demands:

1. Customer Relationship Management CRM (Custmor Relationship Managemet).

POS as point-of-sale terminal be also realize CRM the most directly hold end, but there is the POS terminal of CRM function at present, need the salesman operating POS to carry out extra input service when busy work, client can be realized and identify.

2. checkout security needs (Security Reqirement of Payment)

Along with popularizing of the non-cash means of payment such as electronic money, credit card, POS terminal must realize the security paid further.Common signature, simple password do not act on substantially for malice crime.And in shop the way of mounting monitor, also only can carry out the process afterwards of payment accident, but lack handle before happening measure.

Patent No. US2011231285 RETAIL MOBILE POINT-OF-SALE (POS) SOFTWARE APPLICATION describes, by the method for bar code recognition client.But this method needs client to provide bar code, or need client to carry to be loaded with the carrier of bar code.

On the other hand, in biometric image recognition technology, much research successfully achieves the algorithm of face recognition, as C. Papageorfiou, M. Oren, and T. Poggio. A general frame-work for object detection. In Internatioanl Conference On Computer Vision, 1998. but these results of study only utilize in camera, or in independent security system, due to process data volume, do not import the case row of public service at present.

2. invention target

There is provided the POS of a kind of automatic identification client, automatically complete client's identification by POS, meanwhile, client is without the need to carrying any article, and salesman without the need to inputting while merchandising.

Summary of the invention

Face authentification device is imported in POS, the biological characteristic of client is collected by face authentification device, and passage is deployed in local or long-range biological identification program completes certification to client, thus the consumer behavior of record client, complete the function needed for CRM simultaneously.

Progressive of the present invention is:

1. the convenience of client---client is without the need to carrying member card or other devices;

2. the convenience of salesman---salesman, without the need in busy sale, inputs any information about client;

3. the security paid---by face recognition, strengthen the safe class paid further, according to the safe class of specifying, signature can be reduced, the safe handling that input password etc. are unnecessary simultaneously;

4. payment processes data volume---adopt the method for parallel processing of cloud computing, face recognition process can not be limited by data volume, complete within the fixed time.

Relative to existing research/creation, present invention omits and manual input user profile or the authenticating step by bar code or other identifications.

Accompanying drawing explanation

Fig. 1 hardware construction figure

Fig. 2 software architecture diagram

Fig. 3 program structure-example 2

Fig. 4 facial recognition programs structural drawing

Fig. 5 face recognition processing flow chart

Fig. 6 face judges treatment scheme (Human face Detection Processing)

The instance graph of Fig. 7 characteristic rectangle

The key diagram of the value of Fig. 8 characteristic rectangle

Tu9Ren face characteristic pattern

Figure 10 face characteristic data structure and search method

Embodiment

The present invention is by the hardware construction shown in figure below:

Fig. 1 hardware construction figure

POS of the present invention (1), comprises with lower module

1. display (2), for usual POS business, shows face recognition result in the present invention

2. camera (3), for taking the biological characteristics such as the face of client

3. mainboard (4), provides the various standard such as bus, USB to connect

4. store (41), point internal memory and external memory are used for save routine and data

5. CPU (42), for various computing

6. preferably can comprise communication module (43), for the communicating of server

Described camera 3, comprises camera lens, imageing sensor CMOS, digital signal processing module.Described digital signal processing module adopts discrete Fourier transform (DFT) (Discrete Fourier transform), described camera takes the image of 384 x 288 more than pixel, can rest image be taken, also can process the animation of more than 15 loyal frame/ seconds.

Other each hardware modules described are the standard module of city dealer, and CPU (42), storage (41) connect communicated data signal by the bus on mainboard (4) and USB, COM etc.

Dispose operating system in described device, and on an operating system application program is installed.Program structure of the present invention is as shown below:

Fig. 2 software architecture diagram

As shown in the figure, software configuration of the present invention, comprises follow procedure

1. POS program (6), described POS program (6) is common knowledge, comprises merchandise control and the billing module of city dealer.

2. facial recognition programs (7), for face recognition process, structure and the process of facial recognition programs describe in detail in following sections.

3. operating system OS (5), described operating system (5), for managing described hardware module, and provides POS program (6), the interface of facial recognition programs (7) and addressing means (51).Described operating system (5), is common knowledge, is generally Windows, Linux, and the operating system platform such as iOS, Android.

4. customer characteristic database (8), described customer characteristic database (8), for being stored in remote server in example, by communication module (43), realizes the communication of data.

Fig. 3 program structure-example 2

As simple utilization, described customer characteristic database (8), also can be deployed in local storage (41), its processing mode is identical with long-range.

Described facial recognition programs (7), structure is as shown below:

Fig. 4 facial recognition programs structural drawing

As shown in the figure, facial recognition programs is by following module composition

1. face judge module (71), judges for the treatment of face, and face judges that treatment scheme describes at following sections.

2. feature extraction module (72), for Feature Extraction, Feature Extraction process describes in following sections.

3. characteristic matching module (73), mates for the face characteristic object preserved with database, and characteristic matching process describes in following sections.Described each module can be deployed in POS this locality, also can need to be deployed in long-range server according to service.

The present invention, in described face recognition module, realizes face recognition process, and the method for described face recognition process is as shown below:

Fig. 5 face recognition processing flow chart

As shown in the figure, face recognition module (7) of the present invention, uses following steps to realize face recognition (Detect):

S1. obtain portrait, and carry out pre-treatment (image preprocessing);

S2. judgement detection is carried out to face (Human face), if image is identifiable design face, then extract the image of people face part, otherwise discarded;

S3. Extraction of face features, obtains face feature data;

S4. according to face feature matching algorithm, customer identification is judged;

Described step S1, specifically by camera (3) shooting image, and carries out simple image procossing, adjusts light and shade, colourity, to illumination, removes unwanted picture information;

Face in described step S2 judges, because needs realize instant identification (RealTime detection), the present invention is by the mechanical treatment algorithm of face judge module (71), and the short time, (RealTime) completed judgement.Described face judges treatment scheme, shown in figure below:

Fig. 6 face judges treatment scheme (Human Face Detection Processing)

As shown in the figure, face recognition module (7) of the present invention, uses following steps to realize face recognition:

S21. integral image (integral image) calculates;

S22. two sorting algorithms, whether judgment of learning is face;

S23. extract the image of face, delete background parts image;

Described integer graphics calculations, all digital images can as bitmap process, and under the prerequisite that resolution is certain, all bitmaps can be certain rectangle by Image Segmentation Methods Based on Features.The feature of each rectangle as shown in Figure 6;

The instance graph of Fig. 7 characteristic rectangle

As shown in the figure, A, B are double square, and C is three-rectangle, and D is 4-rectangle.

Although also can directly process each pixel, characteristic rectangle is utilized to have following two benefits:

1. feature can be carried out direct coding to those reluctant bitmaps and calculate

2. the speed calculated is far away higher than directly using bitmap

The simplest characteristic rectangle is Haar basis functons, be published in paper---C. Papageorfiou, M. Oren, and T. Poggio. A general frame-work for object detection. In Internatioanl Conference On Computer Vision, 1998. in, in the present invention, wherein 3 features are utilized.

1. pair-rectangle, the pixel (pixel) of the value of double square and two rectangular areas is different, and laterally or longitudinally adjacent region has identical size and dimension.

2. three-rectangle, the pixel that three-rectangle may be calculated overall region deducts the pixel of Intermediate grey

3. four-rectangle, the feature of four-rectangle is diagonal angle rectangle,

In processes, adopt the pictorial element of nonlinear discrete, and Haar basis unlike, the pixel that the present invention uses is far above Haar basis mode.Rectangular characteristic can be calculated fast by intermediate variable, and defining this intermediate variable is in the present invention integral image

$\mathrm{ii}(x,y)=\underset{x^{\′}\≤x,y^{\′}\≤y}{\mathrm{\Σ}}i(x^{\′},y^{\′})$

Wherein ii (x, y) is integral image, and i (x, y) is original image (original image)

The key diagram of the value of Fig. 8 characteristic rectangle

In the drawings, D field can regard the reference with other field as, and integral image is pixel and (the Sum of Pixels) of rectangle A at point 1 place (P1), and the value of point 2 (P2) is A+B, being A+C at point 3 (P3), is A+B+C+D at point 4.The summation of D may be calculated 4+1-(2+3)

$\begin{array}{c}s(x,y)=s(x,y-1)+i(x,y)\\ \mathrm{ii}(x,y)=\mathrm{ii}(x-1,y)+s(x,y)\end{array}$

Wherein s (x, y) is pixel and (sum), as s (x ,-1)=0 and ii (-1, y)=0, integral image may be calculated original portrait.As shown in Figure 7, use integral image, any one rectangle and can falling into a trap calculation 4 arrays.Obvious two matrixes and difference can calculate in 8 references.Because the definition of two-rectangular characteristic, two adjacent rectangles can be fallen into a trap calculation with reference to arrays at 6, and the feature of 3-rectangle is 8, and the feature of 4 rectangles is 9.

Characteristic rectangle is the method tentatively judging face, during its image that can also apply to other judges.Also have other determination methods as equal replacement, the present invention does not focus on the intelligence behavior of algorithm, and is the judgement utilizing image.

Whether described two sorting algorithms are face for judgment of learning.In judgement, use following algorithm:

$h_j\left(x\right)=\left\{\begin{array}{c}1...p_j{f}_j\left(x\right)<p_j{\mathrm{\&theta;}}_j\\ 0...\mathrm{otherwise}\end{array}\right\}$

Wherein x is the wicket of the 24x24 pixel of whole image.The flow process of such process is as follows:

1. one group of image is expressed as, (x ₁, y ₁) ..., (x _n, y _n) wherein y _i=0,1 represents positive (Positive) i.e. face respectively, or negative (Negative) is for non-face.

2. initializes weights work as y _i=0,1, wherein m is negative quantity, and l is positive quantity

3. work as t=1 ..., T circular treatment following steps

3.1 standardization (Normalize) weight, , therefore w _tfor probability distribution.

3.2 for each feature j, makes a classification h _i, wherein error characteristic can use w _irepresentative,

3.3 select classification h _iwith minimum error characteristic ε _t

3.4 upgrade weight , work as ε _i=0 represents that classification is correct, ε _i=1 represents contrary,

4. finally obtain $h\left(x\right)=\left\{\begin{array}{c}1...\underset{t=1}{\overset{T}{\mathrm{\&Sigma;}}}{\mathrm{\&alpha;}}_t{h}_t\left(x\right)\&GreaterEqual;\frac{1}{2}\underset{t=1}{\overset{T}{\mathrm{\&Sigma;}}}{\mathrm{\&alpha;}}_t\\ 0...\mathrm{otherwise}\end{array}\right\}$ Wherein

In actual face judges, we adopt face characteristic to be as shown in Figure 8 example, in fact also have other features, but owing to illustrating that length does not illustrate one by one.

Tu9Ren face characteristic pattern

1. eyes are than the cheekbone surrounding dark (f1) below eyes,

2. eyes are than mid-eye part and nose dark (f2)

Above-mentioned algorithm after tested, error rate ten thousand/following, and the computing velocity of process 384x288 pixel picture is below 0.3 second, and above-mentioned test uses Intel Atom Z2760 CPU.

After taking out face image, delete other unnecessary image informations, and face image is submitted to described step S3. Extraction of face features, obtain face feature data.In recent years, the research of many recognitions of face is all directly find out face feature to graphical analysis, but these methods need the information that process is a large amount of and face image is irrelevant, therefore lack instantaneity.The present invention adopts face to judge the step be separated with recognition of face, and the speed of such recognition of face is likely to rise dramatically.Meanwhile, in order to wide scale public service, consider remote human face property data base, database adopts cloud computing to build, and walks abreast and carries out signature search.The present invention calculates on framework in apache Hadoop cloud and tests.

Described step S3 Extraction of face features, obtains face feature data;

The Unidentified face of new shooting, needs to be saved.And obtain SIFT, and after face is identified, record SIFT feature and its 3 regions.Client or other system provide the personal information such as name.All records are stored in , wherein facial characteristics is stored in .Wherein each record is the number percent of SIFT feature in face.

Described feature adopts SIFT (Scale invariant feature transform).Adjust to given size by portrait, and portrait is changed and extracts unmodifiable feature.Changeable feature has such as beard, hair style, spectacle-frame etc.About the process of SIFT.

Described step S4 recognition of face matching process

Human face recognition model, adopt D. Lowe paper Object Recognition from local scal-invariant. In intl. Conf. on Computer Vision, the identification matching process of pages (2): 1150-1157, in order to improve processing speed, use Euclidean distance (Euclidean distance), and meet following 3 condition s _i, then coupling is judged as.

Being provided with a vector is facial image, wherein n is the quantity of the facial image feature of preserving in database.

Condition 1: if for any one i, s _i>t _i

Condition 2: meet while condition 1, Max (s _i)-2nd (s _i)>=2 × 2nd (s _i), wherein Max is maximum SIFT value, and 2nd the 2nd is worth greatly

Condition 3: meet while condition 1 wherein owning the maximum SIFT value of removing

As in do not have to meet the S of 3 conditions _i, then for not log in member.。

Recognition of face tupe

Be defined as follows in a model

F _iestimate that i-th face is the possibility being identified as object

Face characteristic in S image

wherein, P (f _i) be initialized to the 1/n saved as in database in n face.P (f _i| s) be conditional probability function, under the condition of feature s, the probability of coupling, P (f _i) ^tp (f is equaled in example t _i| s) ^t-1, at last example t-1, the information in former frame therefore can be thought of as.P (s|f _i) be the estimated value obtained from category feature vector.

Absolute probability, from whole category feature vector, the percent similarity of middle each face of acquisition .

Relative probability, from often opening face, the acquaintance number percent of the whole faces in relative class vector after all identifying.

wherein tp _iit is the sum that i opens the found SIFT of face.Probability P (the f of all acquaintance number percent _i| s) be all stored in vector P (F).P(F)={P(f ₁|s),P(f ₂|s),…,P(f _n|s)}。Identify client, when the possibility that its face is target exceedes other, identify that it is target MaxP (f _i| s)-2ndP (f _i| s)>=2 × 2ndP (f _i| s), wherein Max is the maximal value of P (F), 2 ^ndit is the maximal value beyond removing maximal value.

The problem of above-mentioned recognition methods is, the face image of each client, all needs and all feature s in database _nrelatively, P (s|f is calculated _i).The process of described computing method and data volume are O (n), and when data volume increases time, the identifying processing time also increases relatively.When particularly processing public service, treatment capacity has tens thousand of, even millions of, and the processing time of linear increase cannot meet service quality.

When process a large number of users, the present invention, for described method, is further advanced by the structure of cloud computing, realizes dispersion and parallel process.The present invention realizes the dispersion treatment of data on apache Hadoop cloud framework.Record uses KVS (Key-Value Store) mode to manage.User record is stored in different physical store at random, adopts parallel processing when retrieval process.When stored number is m, process data and time are function.For shortening retrieval time further, to the data structure of preserved face characteristic data acquisition the following stated, and differentiation index is added to data record.

Figure 10 face characteristic data structure and search method

As shown in the figure,

S41 distinguishes classification according to facial characteristics.The differentiation of described facial characteristics can have many methods, such as judges the age according to facial characteristics, then according to age level as differentiation.Same differentiating method also has sex, skin color etc.In the present invention, adopt the ratio of eyes Distance geometry nose length as differentiation.

S42 preserves facial feature data.Extract facial characteristics s

S43 carries out matching treatment to the data meeting differentiation in each storage.Described matching treatment is according to above-mentioned chapters and sections step, but process utilizes the parallel processing manner of cloud computing, and only mates the data meeting differentiation.

The record of face characteristic, comprises

1. distinguish index (R1)

2. feature i (R2), described in be characterized as class vector data, in actual load (Implement), adopt key value data to store (Key-Value data store).Described feature, has the feature such as eyes distance, eye socket peripheral shape type, wing of nose width, nose length, ears shape type.

The user of application example 1 electronic money follows the trail of

Electronic money comprises the various cashless payment such as debit card, credit card.When process electronic money payment, classic method retains signature, but signature is easily forged, and cannot as evidence of crime.By the preservation of user's face image, as user's method for tracing, traditional signature can be replaced.Processing procedure is as follows:

1. automatically snap user's face image

2. utilize face of the present invention to judge disposal route, process also or obtains face image

3. preserve face image together with use record

This application example is claim---a kind of method of the automatic record customer face image for POS.

Application example 2 member identifies

Peddle in activity in sale, the excellent client usually becoming member should enjoy better service, and is treated with a certain discrimination.But there is no the means that automatically can identify member at present, only to rely on artificial cognition.Adopting means provided by the invention, by POS when settling accounts, automatically can identify member.Concrete process is as follows:

1. automatically snap user's face image

2. utilize step S2 face of the present invention to judge treatment scheme, process also or obtains facial image

3. utilize step S3 of the present invention to carry out Extraction of face features, obtain face feature data

4. utilize step S4 of the present invention according to face feature matching algorithm, judge customer identification

If inquired as logging in member, then carry out member's process.

More than employ specific case to be described specific embodiments of the invention, the explanation of this embodiment just understands method of the present invention and core concept for helping; Simultaneously for one of ordinary skill in the art, according to thought of the present invention, all have change in specific embodiments and applications and change part, such as

1. micro-amendment structure of the present invention, increases/reduces insignificant utility appliance device, and it is intensive or independent of correlation unit;

2. the inching of implementation step order, as not having the process of sequencing to exchange;

Therefore, this description should not be construed as limitation of the present invention, all any amendments done within the spirit and principles in the present invention, replaces, deletes the improvement of additional step on an equal basis, is all included in of the present invention comprising in scope.

Claims (5)

1. automatically identify a client's POS device, it is characterized in that described device comprises following hardware module,

(1) display, for usual POS business, picture is sold in display

(2) camera, for taking the biological characteristics such as the face of client

(3) mainboard, provides the various standard such as bus, USB to connect

(4) store, point internal memory and external memory are used for save routine and data

(5) CPU, for various computing

(6) preferably can comprise communication module, for the communicating of server.

2. device as claimed in claim 1, is characterized in that the routine package disposed in described storage is containing with lower module,

(1) POS program,

(2) face recognition module, in described face recognition module, actual load face recognition disposal route

(3) operating system,

(4) customer characteristic database.

3. customer characteristic database as claimed in claim 2, is further characterized in that, can is preferably deployed in remote server, realizes data communication by communication module.

4., for a method for the face recognition process of the automatic identification client of POS, it is characterized in that described recognition algorithms comprises step

(1) obtain portrait, and carry out pre-treatment;

(2) face is judged, if image is identifiable design face, be then for further processing, otherwise discarded;

(3) Extraction of face features, obtains face feature data;

(4) according to face feature matching algorithm, client is judged.

5., for a method for the automatic record customer face image of POS, it is characterized in that described recognition algorithms comprises face determination methods.