CN107122730A - Free dining room automatic price method - Google Patents

Free dining room automatic price method Download PDF

Info

Publication number
CN107122730A
CN107122730A CN201710272861.9A CN201710272861A CN107122730A CN 107122730 A CN107122730 A CN 107122730A CN 201710272861 A CN201710272861 A CN 201710272861A CN 107122730 A CN107122730 A CN 107122730A
Authority
CN
China
Prior art keywords
vegetable
layer
network
service plate
dining room
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710272861.9A
Other languages
Chinese (zh)
Inventor
朱继
乐金伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Haijiao Network Technology Co Ltd
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN201710272861.9A priority Critical patent/CN107122730A/en
Publication of CN107122730A publication Critical patent/CN107122730A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201—Market modelling; Market analysis; Collecting market data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/12—Hotels or restaurants
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/12—Cash registers electronically operated

Abstract

A kind of free dining room automatic price method based on computer vision technique, the function of the system is divided into service plate induction module, vegetable segmentation module and vegetable identification module, the method that service plate induction module employs infrared induction and static detection cascade, while Quick Acquisition is realized, it is ensured that the fine definition of picture；Vegetable segmentation module employs Area generation network, while the positioning and segmentation of fast accurate is realized, it is ensured that for the anti-interference of circumstance of occlusion；Vegetable identification module employs the twin measurement network of depth, overcomes dependence of the conventional depth learning method for training samples number.Of the invention that dishes need not be changed when dining room is applied directly by image recognition vegetable, deployment is convenient, cost is low.The present invention can acquire specific in detail vegetable picture and consumption information etc., can provide marketing analysis for dining room, provide healthy diet analysis for client, be the important step that wisdom dining room is realized based on big data.

Description

Free dining room automatic price method
Technical field
It is of the invention to split and recognize for free dining room automatic price method, more particularly to intelligent vegetable, belong to computer and regard Feel technology and wisdom dining room technical field.
Background technology
Free fast food is a kind of snack pattern for developing extension on the basis of Chinese fast food.Free dining room can neatly be enumerated The food made in batch, client holds pallet queuing and selects the vegetable liked, and clearing at once are served.This pattern facilitates fast Victory, the free degree is high, and application almost covers including institutes such as industrial zone, factory of enterprise, government offices, school, hospital, armies Some organization units, market scale constantly expands.But current free dining room still relies upon artificial valuation clearing.One side cash register Input computer is to complete valuation clearing one by one by the price of every assorted cold dishes for member's needs, and less efficient and easy error causes peak The long influence dining experience of phase queuing；On the other hand, with expanding economy and the change of population structure, human cost is constantly climbed Rise, the big free dining room of flow of the people generally requires to employ several cashiers to tackle peak period, adds operation cost.
Recently as the proposition and development of deep learning model, artificial intelligence system is achieved in many individual event tasks Breakthrough, or even surmounted the performance of the mankind, and be widely applied in industrial circle acquirement.And the food and drink of manpower-intensive type Service trade still still rests on the complete manually-operated stage.Free dining room is realized using computer vision and artificial intelligence technology Automatic price system, can reduce queuing time, and lifting dining experience reduces human cost, effectively improves the fortune in free dining room Seek efficiency.
At present, there is the automatic price system based on RFID technique of maturation free in some school lunch services etc. on the market Run in formula dining room.But such system is often costly, deployment is complicated, and due in dishes RFID chip be easily damaged, it is real Border application effect is unsatisfactory.By contrast, the pricing system cost based on computer vision technique is relatively low, and deployment is flexible, Shandong Rod is strong., can will be current according to the object of identification by the literature search of existing vision automatic price system discovery Automatic price system based on image recognition is divided into two categories below：
Firstth, dishes is recognized
The dishes that free dining room is used can be distinguish between by color, shape.The profile of dishes is corresponded with dish valency, By recognizing that automatic price can be achieved in the color of dishes, shape.This kind of system includes China Patent Publication No. CN103971471, publication date August in 2014 6 days, patent name：Service plate automatic price method based on colour recognition and its it is System, the system distinguishes price using color sensor by the identification of the color to service plate bottom.China Patent Publication No. CN104463167, publication date on March 25th, 2015, patent name：A kind of dining room automatic settlement method and system, the system are led to Cross the shape for extracting dishes and tone characteristicses progress template matches realize that dishes is recognized.The advantage of this kind of method be it is reliable and stable, But it is limited in that：One side dining room must change the dishes of distinction；On the other hand, dish washing and during serving It is both needed to make a distinction dishes arrangement, adds the workload of employee.
Secondth, vegetable is recognized
This kind of system completes clearing by extracting the feature recognition vegetables such as texture, the color of vegetable in itself.In such as State patent publication No. CN104077842, publication date on October 1st, 2014, patent name：Free dining room based on image recognition is certainly Help payment device and its application method.Using the method for machine learning, off-line learning is carried out to vegetable image, then in clearing Vegetable is identified the corresponding dish valency of acquisition.The advantage for the method that the patent is proposed is can to reduce system without changing dishes Cost, but there is problems with practical application：One is using Hough transform algorithm to carry out service plate detection in the patent, right The integrality of outline of service plate requires higher, it is impossible to tackle the situation (ratio that the part service plate easily occurred in practical application is blocked Such as disc stack, hand is blocked)；Two be to cook vegetable identification, such mould using the disaggregated model based on convolutional neural networks in the patent Shape parameter is complicated, requires very big to data volume, to recognize that a kind of vegetable must gather up to a hundred or even thousands of assorted cold dishes product pictures Enough accuracys rate can be reached by having, once and new vegetable is released in dining room, re -training and more new model need to be resurveyed again, to meal The practical application in the Room brings inconvenience and difficult.
By contrast, although the Automated Clearing House system based on vegetable identification based on the system that dishes is recognized than having more challenge, But it need not change dining room dishes, and the more detailed sales information specific to vegetable can be obtained, it is bigger to dining room meaning, if The problem of stability, accuracy rate and practicality can be solved, then should turn into scheme of more preferably valuating.
The content of the invention
For the above mentioned problem existed based on the automatic price system that vegetable is recognized existing at present, the present invention proposes a kind of New free dining room automatic price system and method.The program is mutually cascaded by infrared inductor with video static detection program Method improve the speed and definition of service plate picture collection；By the precise positioning of Area generation real-time performance ability vegetable and Segmentation, and when service plate is blocked with very strong robustness and anti-interference；By by deep learning and tolerance Habit technology is combined, and using the twin measurement network of depth, improves the accuracy rate and robustness of vegetable identification, is overcome traditional deep Spend dependence of the learning network to sample size so that only need one vegetable sample picture of storage that the dish can be achieved in database The accurate match identification of product, improves flexibility, practicality and the universality of system.
The invention provides a kind of free dining room automatic price method, it comprises the following steps：
1) arrange that the NVIDIA series of high definition camera device, outfit with CUDA computation capabilities is aobvious on cash register table top Card main frame, two-sided display screen, the IC POSs, infrared inductor and cog region, high definition camera device pass through USB and main frame phase Even, the IC POSs are connected by serial ports with main frame；
2) main frame, two-sided display screen, IC card reader power supplys are connected, starts the automatic price program on main frame；
3) when service plate enters cog region, infrared inductor triggers the pricing on main frame by serial communication and started Video acquisition；
4) main frame automatic price program does static detection using frame difference method to the video that camera device is collected, once it is determined that Service plate is static to be put, and triggers image interception；
5) vegetable segmentation is carried out using the vegetable detection technique based on Area generation network；
6) deep learning and metric learning technology are combined, based on each vegetable in the twin measurement Network Recognition service plate of depth；
7) pricing reads the pricing information in local data base, total price is calculated and be shown according to vegetable species
8) dining room client completes to pay by the POS, returns to step 2.
The free dining room automatic price system of the present invention recognizes three main moulds comprising service plate sensing, vegetable segmentation and vegetable Block.
Service plate of the service plate induction module of the present invention on infrared inductor, high definition camera device and host computer is static Detection program is constituted.Infrared inductor comprising infrared sensor and control Emission and receiving of infrared single-chip microcomputer, single-chip microcomputer with Trigger signal is transmitted by serial communication between host computer.Service plate is passed once will trigger induction installation into cog region to main frame Pass rs 232 serial interface signal.Main frame receives corresponding rs 232 serial interface signal, will read the video information of camera collection, starts static detection Program, determine service plate come to a complete stop put well after intercept at once service plate image input vegetable segmentation module.Wherein, static detection task is used Frame difference method is realized, the pixel difference of adjacent two frame of video is gathered by calculating, determines whether service plate comes to a complete stop and puts well.The present invention is proposed Infrared inductor pressure sensitive device traditional compared with the service plate inductive scheme that static detection program is cascaded it is more rapid, Reliably, it can effectively prevent from service plate from not coming to a complete stop when putting well just to trigger too early, improve the quality and definition of collection image.
The vegetable segmentation module of the present invention, employs the dividing method based on Area generation network.The Area generation network Layer is returned by feature extraction layer, foreground classification layer and frame to constitute.Wherein feature extraction layer is linear single by multilayer convolutional layer, amendment First activation primitive layer and pondization are laminated plus constituted.Service plate image obtained after feature extraction layer it is down-sampled after last layer volume Product characteristic pattern.On this feature figure, centered on each location of pixels, generation comprising three kinds of areas, three kinds of aspect ratios it is multiple Couple candidate detection window.Foreground classification layer includes a full articulamentum and a softmax returns output layer, and it is examined with each candidate It is input to survey window character pair graph region, and it is the prospect or background for including vegetable to judge institute formation zone, and exports corresponding Confidence level.Frame, which returns layer, includes the linear regression output layer of a full articulamentum and one 4 dimension, the candidate to being judged as vegetable Detection block carries out further position accurate adjustment so that the position of detection block is more fitted the actual position of vegetable.The present invention is proposed The vegetable dividing method fast accurate based on Area generation network, the global information of vegetable can be considered, compared to tradition The computer vision methods positioned based on dishes shape, it is lower to the integrity demands of profile, to the Shandong of circumstance of occlusion Rod is stronger, can successfully manage disc stack, hand and the interference implementations such as block.
The vegetable identification module of the present invention, deep learning is combined with metric learning, proposes to be based on the twin measurement of depth The vegetable recognition methods of network.Existing deep learning technology do the basic framework classified using convolutional neural networks, it is necessary into Thousand vegetable picture samples up to ten thousand can just train accurately and reliably identification model, and can only the vegetable crossed of identification learning, one New vegetable is released in denier dining room, and the model trained before will fail to be trained, it is necessary to resurvey a large amount of new dish samples, is given The practical application in dining room causes inconvenience and difficult.It is of the invention by deep learning and metric learning for the limitation of the program Technology is combined, it is proposed that the new vegetable identification framework based on twin network.The network by twin binary channels feature extraction layer, Relative measurement layer and contrast loss function layer composition.Wherein feature extraction layer includes the twin of shared network structure and weighting parameter Binary channels network, it using two vegetable pictures simultaneously as input, the convolutional layer cascaded by multilayer, the linear elementary layer of amendment and Pond layer extracts the characteristic vector of every vegetable respectively.Relative measurement layer is calculated using the characteristic vector of two pictures as input Relative Euclidean distance between feature, finally enters contrast loss function layer.Contrast the form such as following formula of loss function：
DW(X1,X2)=| | GW(X1)-GW(X2)||2
Wherein Y=0 represents that two pictures belong to same vegetable, and Y=1 represents that two pictures belong to different vegetables.DWTable This X of sample1And X2Euclidean distance, m is the distance threshold being previously set.Contrast loss letter is minimized by back-propagation algorithm Number, distance is approached between can making same class vegetable feature, and different vegetable characteristic distances are become estranged, can be real by setting corresponding threshold value The comparison of existing vegetable species.Consequently, it is possible to for each new vegetable, the picture input database of a new vegetable only need to be gathered, Accurately identifying for vegetable can be completed.
The implementation of the present invention includes：(being equipped with has CUDA parallel for infrared inductor, high definition camera device, main frame The serial video card of the NVIDIA of computing capability), the IC POSs.Wherein infrared inductor is made up of infrared sensor and single-chip microcomputer, It is connected by serial ports with main frame；High-definition camera harvester is made up of high-definition camera and support, passes through USB and all-in-one phase Even；The double screen of all-in-one is put back-to-back, and a screen is just to dining room client, and another panel is (convenient real to administrative staff in cashier When check and manage dish information, valuation settlement process is interfered without administrative staff)；The IC POSs pass through serial ports and all-in-one phase Even.
Compared with prior art, beneficial effects of the present invention are as follows：
1) present invention can realize the automatic price clearing in dining room, improve efficiency and accuracy rate that dining room pays link, Reduce human cost；
2) present invention can Direct Recognition vegetable, to the dishes of dish-up without particular/special requirement, and do not changed shadow by indoor light Ring, therefore special dishes need not be changed when dining room is applied, without increase LED illumination lamp light filling, deployment is convenient, cost is low；
3) method that the present invention is cascaded using infrared inductor and video static detection program, can be gathered fast and reliablely Clear static service plate picture, prevents service plate from not coming to a complete stop with regard to too early triggering collection；
4), can be quick and very accurate present invention employs the positioning of the dishes based on Area generation network and dividing method Ground Split goes out the picture of each vegetable, to the strong interference immunity of circumstance of occlusion；
5) deep learning is combined by the present invention with metric learning technology, proposes the vegetable based on the twin measurement network of depth Recognition methods, solves dependence of the conventional deep learning art for sample size, it is only necessary to store a vegetable sample picture Accurately identifying for the vegetable can be achieved, so as to flexibly effectively cope with the identification valuation of new vegetable；
6) present invention acquires picture and sales data information of vegetable etc., can carried for dining room while valuation is recognized Sale data analysis, healthy diet report analysis is provided for dining room client, is to make the important of wisdom dining room based on big data Link.
Brief description of the drawings
Fig. 1 is the structural representation of the specific embodiment of the invention；
Fig. 2 splits the Area generation network diagram of module for the present invention applied to vegetable；
Fig. 3 is applied to the twin measurement network diagram of depth of vegetable identification module for the present invention；
Fig. 4 is running software flow chart；
Label in accompanying drawing is respectively：1st, high definition camera device, 2, main frame, 3, the display screen in face of dining room client, 4, face To the display screen of cashier, 5, the IC POSs, 6, infrared inductor, 7, cog region.
Embodiment
Embodiments of the invention are elaborated below in conjunction with the accompanying drawings, the present embodiment is using technical solution of the present invention before Carry, give detailed embodiment and specific operating process, but protection scope of the present invention is not limited to following embodiments.
Embodiment
As shown in Figures 1 to 4, implementation of the invention includes building and software systems for hardware system to embodiments of the invention Build two parts.
Hardware system in the present embodiment is as shown in figure 1, specifically include high definition camera device 1, main frame 2, in face of dining room client Display screen 3, the display screen 4 in face of cashier, the IC POSs 5, infrared inductor 6 and cog region 7；High definition camera device 1 It is made up of, is connected by USB with main frame 2 high-definition camera and support；Main frame 2, the IC POSs 5, cog region 7 are arranged in cash register On table top, cog region 7 is located at the underface of high definition camera device 1；Display screen 3 in face of dining room client, showing in face of cashier Display screen 4 is arranged in the top of main frame 2 and put back-to-back, and the display screen 3 in face of dining room client is just to dining room client, in face of cash register The display screen 4 of member is just to administrative staff in cashier；Infrared inductor 6 is made up of infrared sensor and single-chip microcomputer, is lain in On cash register table top and just to cog region 7, it is connected by serial ports with main frame 2；The IC POSs 5 are connected by serial ports with all-in-one 2 Connect
The installation of hardware system needs to meet following requirement：
First, the installation of high definition camera device：
High-definition camera is arranged on directly over indicator screen, and the visual field is required to cover the scope of cog region.Camera Setting height(from bottom) with gather the vegetable in image whether clearly be with reference to come specifically determine.Can be over the display after installation The demarcation of cog region scope is carried out on visualization interface, calibration result is stored in the database of all-in-one.
Second, the installation of infrared inductor：
Infrared inductor is close to cash register table top and is fixedly mounted on immediately ahead of display, highly less than service plate edge height, The distance of reaction of infrared sensor is adjusted to 10 centimetres, and detection frequency is adjusted to 100 times/second.
3rd, the installation of double screen integration main frame：
Double screen integration main frame is positioned over before cash register table surface cog region, and a screen is just to dining room client, and another panel is to receiving Yin Tainei administrative staff.
Software systems operating procedure is as shown in figure 4, specific as follows in the present embodiment：
1) start initialization, test main frame and the connection of camera device, infrared inductor and the IC POSs, confirm Enter after connection is errorless and wait service plate state；
2) when service plate enters cog region, infrared inductor sends trigger signal, and what main frame reading camera was transmitted regards Frequency signal, by detecting the pixel difference of adjacent two frame, judges whether service plate is static and puts.Put once detecting service plate and having come to a complete stop It is good, service plate picture is intercepted, into step 3；
3) service plate picture is read, the Area generation network trained is inputted, the candidate frame of non-vegetable is filtered, by frame line Property return after fine position, export the particular location of each vegetable in picture, realize that vegetable is split；
4) the vegetable picture being partitioned into is inputted the twin measurement net of depth by the vegetable position exported according to Area generation network The vegetable example deposited in network, with database is compared one by one, and the kind of vegetable is determined according to the output of the relative measurement layer of twin network Class；
5) pricing information in the vegetable species matching database identified, price is included in screen, triggering The POS waits client to pay, and vegetable and consumption information typing all-in-one database, are transferred to step 2 after paying successfully.
The Area generation network model that vegetable segmentation module of the present invention is used needs to determine network parameter by training in advance, Its training method and step are as follows, as shown in Figure 2：
1) the vegetable segmentation information to part service plate picture is manually marked, and is used as the training sample of Area generation network This；
2) the model initialization network weight ginseng of the pre-training on extensive public image data collection is utilized；
3) candidate frame generated on last layer of convolutional layer and the vegetable detection block of demarcation are contrasted, if overlap proportion Vegetable prospect sample is designated as more than 0.7, background sample is designated as if overlap proportion is less than 0.3；
4) the foreground and background sample characteristics obtained in 3 input prospect two is classified layer, passes through back-propagation algorithm iteration Optimize the weights of Area generation network；
5) the prospect sample characteristics obtained in 3 are inputted into the linear regression model (LRM) that frame returns layer, it is same by reverse Propagation algorithm optimizes weights, the actual position that the location parameter fitting for allowing network to export is marked；
6) repeat step 4 and 5, stop repetitive exercise when loss function is not continued to and declined, obtain final network Parameter.
The twin measurement network model of depth that vegetable identification module of the present invention is used also needs to determine network by training in advance Parameter, its training method and step are as follows, as shown in Figure 3：
1) the part vegetable picture that early stage is collected is marked by name of the dish, training sample is constituted two-by-two, wherein two pictures Belong to the positive sample that is designated as of same vegetable, what two pictures belonged to different vegetables is designated as negative sample；
2) the model initialization network weight parameter of the pre-training on extensive common image data set is utilized；
3) every group of sample is sequentially input into the twin measurement network extraction high-level semantics features of depth, passes through relative measurement layer meter The characteristic distance between two photos is calculated, contrast loss function layer is finally entered；
4) it is that positive sample or negative sample calculate corresponding contrast and lost according to institute's input sample, passes through back-propagation algorithm Adjust network weight parameter；
5) repeat step 3 and 4, stop repetitive exercise when loss function is not continued to and declined, obtain final network Parameter.
The above embodiments are merely illustrative of the technical solutions of the present invention, and the present invention is not limited to embodiments above, also There can be many variations.For those of ordinary skill in the art, core technical features of the present invention are not being departed from On the premise of, some improvements and modifications can also be made, these improvements and modifications also should be regarded as protection scope of the present invention.

Claims (8)

1. a kind of free dining room automatic price method, it is characterised in that comprise the following steps：
1) on cash register table top arrange high definition camera device, the main frame with two-sided display screen, the IC POSs, infrared inductor and Cog region, high definition camera device is connected by USB with main frame, and the IC POSs are connected by serial ports with main frame；
2) main frame, two-sided display screen, IC card reader power supplys are connected, starts the automatic price program on main frame；
3) when service plate enters cog region, infrared inductor triggers the pricing on main frame by serial communication and starts video Collection；
4) main frame automatic price program does static detection using frame difference method to the video that camera device is collected, once it is determined that service plate It is static to put, trigger image interception；
5) vegetable segmentation is carried out using the vegetable detection technique based on Area generation network；
6) deep learning and metric learning technology are combined, based on each vegetable in the twin measurement Network Recognition service plate of depth；
7) pricing reads the pricing information in local data base, total price is calculated and be shown according to vegetable species
8) dining room client completes to pay by the POS, returns to step 2.
2. free dining room automatic price method according to claim 1, it is characterised in that the automatic price in the main frame Program is run according to the following steps：
1) initialization, test main frame and the connection of high definition camera device, infrared inductor and the IC POSs, connection are started Enter after errorless and wait service plate state；
2) when service plate enters cog region, service plate induction module is started, control infrared inductor sends trigger signal, and main frame is read The vision signal that camera is transmitted is taken, by detecting the pixel difference of adjacent two frame, judges whether service plate is static and puts.Once detection Come to a complete stop and put well to service plate, service plate picture has been intercepted, into step 3；
3) start vegetable segmentation module, read service plate picture, input area generation network filters the candidate frame of non-vegetable, passed through After frame linear regression fine position, particular location of each vegetable in picture is exported, realizes that vegetable is split；
4) vegetable identification module is started, the vegetable position exported according to Area generation network inputs the vegetable picture being partitioned into deep Twin measurement network is spent, is compared one by one with the vegetable example deposited in database, according to the output of the relative measurement layer of twin network Recognize the species of vegetable；
5) pricing information in the vegetable species matching database identified, price is included in screen, triggering is swiped the card Machine waits client to pay, and vegetable and consumption information typing host data base, are transferred to step 2 after paying successfully.
3. free dining room automatic price method according to claim 2, it is characterised in that the service plate sensing mould in step 2 Service plate static detection program cascade of the block on infrared inductor and main frame is constituted；Infrared inductor includes infrared sensor With the single-chip microcomputer of control Emission and receiving of infrared, trigger signal is transmitted by serial communication between single-chip microcomputer and host computer；Service plate Once will trigger induction installation to main frame transmission rs 232 serial interface signal into cog region, main frame receives corresponding rs 232 serial interface signal, just Can read camera collection video information, start static detection program, determine service plate come to a complete stop put well after intercept service plate figure at once Picture.
4. free dining room automatic price method according to claim 3, it is characterised in that the service plate static detection in main frame Program employs frame difference method, and program can continue to calculate the adjacent two frames pixel difference of collection video, until pixel difference is less than necessarily Threshold value, you can determine whether service plate comes to a complete stop and put well.
5. free dining room automatic price method according to claim 2, it is characterised in that the vegetable splits module, is used Dividing method based on Area generation network, the Area generation network is returned by feature extraction layer, foreground classification layer and frame Layer is constituted；Wherein feature extraction layer is by multilayer convolutional layer, the linear unit activating function layer of amendment and pondization stacking plus constitutes；Service plate Image obtained after feature extraction layer it is down-sampled after last layer of convolution characteristic pattern；On this feature figure, with each pixel Centered on position, multiple couple candidate detection windows of the generation comprising three kinds of areas, three kinds of aspect ratios；Foreground classification layer includes one entirely Articulamentum and a softmax return output layer, and it is using each couple candidate detection window character pair graph region as input, and judgement is given birth to It is the prospect or background for including vegetable into region, and exports corresponding confidence level；Frame, which returns layer, includes a full articulamentum With the linear regression output layer of one 4 dimension, the further position accurate adjustment of couple candidate detection frame progress to being judged as vegetable so that inspection More fit the actual position of vegetable the position for surveying frame.
6. free dining room automatic price method according to claim 2, it is characterised in that the vegetable identification module will be deep Degree study is combined with metric learning, employs the vegetable recognition methods based on the twin measurement network of depth；The network is by twin Binary channels feature extraction layer, relative measurement layer and contrast loss function layer composition；Wherein feature extraction layer includes shared network knot The twin binary channels network of structure and weighting parameter, it is using two vegetable pictures simultaneously as input, the convolution cascaded by multilayer Layer, the linear elementary layer of amendment and pond layer extract the characteristic vector of every vegetable respectively；The spy of the relative measurement pictures of layer two Vector is levied as input, the relative Euclidean distance between feature is calculated, contrast loss function layer is finally entered；Contrast loss function Form such as following formula：
<mrow> <mi>L</mi> <mrow> <mo>(</mo> <mi>W</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mn>1</mn> <mn>2</mn> </mfrac> <mrow> <mo>(</mo> <mn>1</mn> <mo>-</mo> <mi>Y</mi> <mo>)</mo> </mrow> <msup> <msub> <mi>D</mi> <mi>W</mi> </msub> <mn>2</mn> </msup> <mo>+</mo> <mfrac> <mn>1</mn> <mn>2</mn> </mfrac> <mi>Y</mi> <mo>{</mo> <mi>m</mi> <mi>a</mi> <mi>x</mi> <mrow> <mo>(</mo> <mn>0</mn> <mo>,</mo> <mi>m</mi> <mo>-</mo> <msub> <mi>D</mi> <mi>W</mi> </msub> <mo>)</mo> </mrow> <mo>}</mo> </mrow>
DW(X1,X2)=| | GW(X1)-GW(X2)||2
Wherein Y=0 represents that two pictures belong to same vegetable, and Y=1 represents that two pictures belong to different vegetables.DWRepresent sample X1And X2Euclidean distance, m is the distance threshold being previously set.Structural parameters such as Fig. 3 of the one of which concrete form of the network It is shown.The network overcomes strong dependency of the deep learning technology to training samples number, and every kind of vegetable only needs vegetable Picture input database, you can complete the accurate matching identification of vegetable.
7. free dining room automatic price method according to claim 5, it is characterised in that the Area generation network of use Training method and step are as follows：
1) the vegetable segmentation information to part service plate picture is labeled, and is used as the training sample of Area generation network；
2) the model initialization network weight parameter of the pre-training on extensive public image data collection is utilized；
3) candidate frame generated on last layer of convolutional layer and the vegetable detection block of demarcation are contrasted, if overlap proportion is more than 0.7 is designated as vegetable prospect sample, and background sample is designated as if overlap proportion is less than 0.3；
4) the foreground and background sample characteristics obtained in step 3 input prospect two is classified layer, passes through back-propagation algorithm iteration Optimize the weights of Area generation network；
5) the prospect sample characteristics obtained in step 3 are inputted into the linear regression model (LRM) that frame returns layer, it is same by reverse Propagation algorithm optimizes weights, the actual position that the location parameter fitting for allowing network to export is marked；
6) repeat step 4 and 5, stop repetitive exercise when loss function is not continued to and declined, obtain final network parameter；
8. free dining room automatic price method according to claim 6, it is characterised in that it is characterized in that twin using depth The training method and step of raw measurement network are as follows：
1) the part vegetable picture collected is constituted into training sample, wherein two pictures belong to two-by-two by the good classification of name of the dish point Same vegetable is designated as positive sample, and what two pictures belonged to different vegetables is designated as negative sample；
2) the model initialization network weight parameter of the pre-training on extensive common image data set is utilized；
3) every group of sample is sequentially input into the twin measurement network extraction high-level semantics features of depth, two is calculated by relative measurement layer The characteristic distance between photo is opened, contrast loss function layer is finally entered；
4) it is that positive sample or negative sample calculate corresponding contrast and lost according to institute's input sample, is adjusted by back-propagation algorithm Network weight parameter；
5) repeat step 3 and 4, stop repetitive exercise when loss function is not continued to and declined, obtain final network parameter.
CN201710272861.9A 2017-04-24 2017-04-24 Free dining room automatic price method Pending CN107122730A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710272861.9A CN107122730A (en) 2017-04-24 2017-04-24 Free dining room automatic price method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710272861.9A CN107122730A (en) 2017-04-24 2017-04-24 Free dining room automatic price method

Publications (1)

Publication Number Publication Date
CN107122730A true CN107122730A (en) 2017-09-01

Family

ID=59725374

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710272861.9A Pending CN107122730A (en) 2017-04-24 2017-04-24 Free dining room automatic price method

Country Status (1)

Country Link
CN (1) CN107122730A (en)

Cited By (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107844790A (en) * 2017-11-15 2018-03-27 上海捷售智能科技有限公司 A kind of vegetable identification and POS and method based on image recognition
CN107944860A (en) * 2017-11-15 2018-04-20 上海捷售智能科技有限公司 A kind of bakery identification and cash register system and method based on neutral net
CN107992826A (en) * 2017-12-01 2018-05-04 广州优亿信息科技有限公司 A kind of people stream detecting method based on the twin network of depth
CN108364417A (en) * 2018-02-02 2018-08-03 河海大学常州校区 Intelligent checkout system and method based on Hough circle and color euclidean distance method
CN109145906A (en) * 2018-08-31 2019-01-04 北京字节跳动网络技术有限公司 The image of target object determines method, apparatus, equipment and storage medium
CN109165656A (en) * 2018-07-24 2019-01-08 上海常仁信息科技有限公司 A kind of feature information extraction system and method for vegetable image
CN109242017A (en) * 2018-08-30 2019-01-18 杨镇蔚 Intelligent identification Method, device and the equipment of object information
CN109377205A (en) * 2018-12-06 2019-02-22 深圳市淘米科技有限公司 A kind of cafeteria's intelligence settlement system based on depth convolutional network
WO2019062017A1 (en) * 2017-09-27 2019-04-04 缤果可为（北京）科技有限公司 Method, device, and self-service checkout counter for performing product recognition on the basis of neural network
CN109615358A (en) * 2018-11-01 2019-04-12 北京伟景智能科技有限公司 A kind of dining room automatic settlement method and system based on deep learning image recognition
CN109816439A (en) * 2019-01-14 2019-05-28 珠海格力电器股份有限公司 Fruits and vegetables intelligence pricing method, device, storage medium and equipment
CN109830072A (en) * 2019-02-26 2019-05-31 魔珐(上海)信息科技有限公司 The valuation of view-based access control model identification and cash device, control system and method
CN109919803A (en) * 2019-01-31 2019-06-21 秒针信息技术有限公司 A kind of vegetable detection method and device
CN110020603A (en) * 2019-03-08 2019-07-16 华东师范大学 Self-checkout system and method based on convolutional neural networks
CN110059551A (en) * 2019-03-12 2019-07-26 五邑大学 A kind of automatic checkout system of food based on image recognition
CN110069995A (en) * 2019-03-16 2019-07-30 浙江师范大学 A kind of service plate moving state identification method based on deep learning
CN110084642A (en) * 2019-04-19 2019-08-02 哈尔滨新中新电子股份有限公司 Vegetable recognition methods based on target detection and image retrieval
CN110084587A (en) * 2019-02-28 2019-08-02 杭州效准智能科技有限公司 A kind of service plate automatic settlement method based on edge context
CN110415463A (en) * 2019-07-25 2019-11-05 腾讯科技（深圳）有限公司 A kind of method of payment and terminal
CN110519570A (en) * 2019-09-11 2019-11-29 安徽璞华大数据技术有限公司 Data application based on campus monitoring
CN110599257A (en) * 2019-09-19 2019-12-20 福建工程学院 Method and system for calculating total amount of dishes based on image recognition technology
CN110852733A (en) * 2019-10-22 2020-02-28 杭州效准智能科技有限公司 Intelligent catering settlement system based on RFID fusion dish image matching identification
CN110874595A (en) * 2019-10-22 2020-03-10 杭州效准智能科技有限公司 Multi-dish dinner plate intelligent segmentation method based on deep learning
CN110880223A (en) * 2018-09-06 2020-03-13 阿里巴巴集团控股有限公司 Commodity settlement method, server, electronic device, medium, and terminal
CN111047799A (en) * 2019-11-29 2020-04-21 北京三快在线科技有限公司 Cash register device, article identification method and storage medium
CN111080493A (en) * 2018-10-18 2020-04-28 杭州海康威视数字技术股份有限公司 Dish information identification method and device and dish self-service settlement system
CN111143585A (en) * 2019-04-11 2020-05-12 王晓东 Automatic big data sorting method
CN111178274A (en) * 2019-12-30 2020-05-19 秒针信息技术有限公司 Information prompting method and device, computer readable storage medium and electronic device
CN111369524A (en) * 2020-03-02 2020-07-03 北京推想科技有限公司 Nodule recognition model training method, nodule recognition method and device
CN111680603A (en) * 2020-05-28 2020-09-18 浙江师范大学 Dish detection and identification method
WO2020215952A1 (en) * 2019-04-23 2020-10-29 北京京东尚科信息技术有限公司 Object recognition method and system
CN112115906A (en) * 2020-09-25 2020-12-22 广州市派客朴食信息科技有限责任公司 Open dish identification method based on deep learning target detection and metric learning
CN112115901A (en) * 2020-09-25 2020-12-22 广州市派客朴食信息科技有限责任公司 High-accuracy food identification method
CN112115903A (en) * 2020-09-25 2020-12-22 广州市派客朴食信息科技有限责任公司 Method for improving dish identification system identification precision based on deep learning
CN113255596A (en) * 2021-06-29 2021-08-13 中运科技股份有限公司 Intelligent video image analysis system and method based on big data
CN113343003A (en) * 2021-06-30 2021-09-03 上海歆广数据科技有限公司 Dining nutrition construction recording system and method
CN113470284A (en) * 2021-07-08 2021-10-01 广州五谷信息科技有限公司 Multi-screen ordering equipment and using method thereof

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090057401A1 (en) * 2007-08-31 2009-03-05 Drb Systems, Incorporated System and methods for providing prepaid car wash or lube services
US7568618B1 (en) * 2005-03-01 2009-08-04 Gfc Automat, Inc. Automated food service and billing system and method
CN103426240A (en) * 2013-08-01 2013-12-04 浙江大学 Freestyle dining room self-help paying device and using method thereof
CN104077842A (en) * 2014-07-02 2014-10-01 浙江大学 Freestyle restaurant self-service payment device based on image identification and application method of device
CN104463167A (en) * 2014-11-20 2015-03-25 中国科学院上海微系统与信息技术研究所 Automatic restaurant settlement method and system
CN105741139A (en) * 2016-01-31 2016-07-06 上海海角网络科技有限公司 Automatic charging method for self-service restaurant and implementation apparatus thereof
CN105938553A (en) * 2016-04-14 2016-09-14 厦门理工学院 Canteen plate automatic billing method and system
CN106096932A (en) * 2016-06-06 2016-11-09 杭州汇萃智能科技有限公司 The pricing method of vegetable automatic recognition system based on tableware shape

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7568618B1 (en) * 2005-03-01 2009-08-04 Gfc Automat, Inc. Automated food service and billing system and method
US20090057401A1 (en) * 2007-08-31 2009-03-05 Drb Systems, Incorporated System and methods for providing prepaid car wash or lube services
CN103426240A (en) * 2013-08-01 2013-12-04 浙江大学 Freestyle dining room self-help paying device and using method thereof
CN104077842A (en) * 2014-07-02 2014-10-01 浙江大学 Freestyle restaurant self-service payment device based on image identification and application method of device
CN104463167A (en) * 2014-11-20 2015-03-25 中国科学院上海微系统与信息技术研究所 Automatic restaurant settlement method and system
CN105741139A (en) * 2016-01-31 2016-07-06 上海海角网络科技有限公司 Automatic charging method for self-service restaurant and implementation apparatus thereof
CN105938553A (en) * 2016-04-14 2016-09-14 厦门理工学院 Canteen plate automatic billing method and system
CN106096932A (en) * 2016-06-06 2016-11-09 杭州汇萃智能科技有限公司 The pricing method of vegetable automatic recognition system based on tableware shape

Cited By (42)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019062017A1 (en) * 2017-09-27 2019-04-04 缤果可为（北京）科技有限公司 Method, device, and self-service checkout counter for performing product recognition on the basis of neural network
CN107944860A (en) * 2017-11-15 2018-04-20 上海捷售智能科技有限公司 A kind of bakery identification and cash register system and method based on neutral net
CN107844790A (en) * 2017-11-15 2018-03-27 上海捷售智能科技有限公司 A kind of vegetable identification and POS and method based on image recognition
CN107992826A (en) * 2017-12-01 2018-05-04 广州优亿信息科技有限公司 A kind of people stream detecting method based on the twin network of depth
CN108364417A (en) * 2018-02-02 2018-08-03 河海大学常州校区 Intelligent checkout system and method based on Hough circle and color euclidean distance method
CN109165656A (en) * 2018-07-24 2019-01-08 上海常仁信息科技有限公司 A kind of feature information extraction system and method for vegetable image
CN109242017A (en) * 2018-08-30 2019-01-18 杨镇蔚 Intelligent identification Method, device and the equipment of object information
CN109145906A (en) * 2018-08-31 2019-01-04 北京字节跳动网络技术有限公司 The image of target object determines method, apparatus, equipment and storage medium
CN110880223B (en) * 2018-09-06 2022-03-04 阿里巴巴集团控股有限公司 Commodity settlement method, server, electronic device, medium, and terminal
CN110880223A (en) * 2018-09-06 2020-03-13 阿里巴巴集团控股有限公司 Commodity settlement method, server, electronic device, medium, and terminal
CN111080493B (en) * 2018-10-18 2023-09-05 杭州海康威视数字技术股份有限公司 Dish information identification method and device and dish self-service settlement system
CN111080493A (en) * 2018-10-18 2020-04-28 杭州海康威视数字技术股份有限公司 Dish information identification method and device and dish self-service settlement system
CN109615358A (en) * 2018-11-01 2019-04-12 北京伟景智能科技有限公司 A kind of dining room automatic settlement method and system based on deep learning image recognition
CN109615358B (en) * 2018-11-01 2020-11-03 北京伟景智能科技有限公司 Deep learning image recognition-based restaurant automatic settlement method and system
CN109377205A (en) * 2018-12-06 2019-02-22 深圳市淘米科技有限公司 A kind of cafeteria's intelligence settlement system based on depth convolutional network
CN109816439A (en) * 2019-01-14 2019-05-28 珠海格力电器股份有限公司 Fruits and vegetables intelligence pricing method, device, storage medium and equipment
CN109919803A (en) * 2019-01-31 2019-06-21 秒针信息技术有限公司 A kind of vegetable detection method and device
CN109830072A (en) * 2019-02-26 2019-05-31 魔珐(上海)信息科技有限公司 The valuation of view-based access control model identification and cash device, control system and method
CN110084587B (en) * 2019-02-28 2021-08-10 智准宝智能科技(上海)有限公司 Automatic dinner plate settlement method based on edge context
CN110084587A (en) * 2019-02-28 2019-08-02 杭州效准智能科技有限公司 A kind of service plate automatic settlement method based on edge context
CN110020603A (en) * 2019-03-08 2019-07-16 华东师范大学 Self-checkout system and method based on convolutional neural networks
CN110059551A (en) * 2019-03-12 2019-07-26 五邑大学 A kind of automatic checkout system of food based on image recognition
CN110069995A (en) * 2019-03-16 2019-07-30 浙江师范大学 A kind of service plate moving state identification method based on deep learning
CN111143585A (en) * 2019-04-11 2020-05-12 王晓东 Automatic big data sorting method
CN110084642A (en) * 2019-04-19 2019-08-02 哈尔滨新中新电子股份有限公司 Vegetable recognition methods based on target detection and image retrieval
WO2020215952A1 (en) * 2019-04-23 2020-10-29 北京京东尚科信息技术有限公司 Object recognition method and system
CN110415463A (en) * 2019-07-25 2019-11-05 腾讯科技（深圳）有限公司 A kind of method of payment and terminal
CN110519570B (en) * 2019-09-11 2020-11-06 安徽璞华大数据技术有限公司 Data application system based on campus monitoring
CN110519570A (en) * 2019-09-11 2019-11-29 安徽璞华大数据技术有限公司 Data application based on campus monitoring
CN110599257A (en) * 2019-09-19 2019-12-20 福建工程学院 Method and system for calculating total amount of dishes based on image recognition technology
CN110874595A (en) * 2019-10-22 2020-03-10 杭州效准智能科技有限公司 Multi-dish dinner plate intelligent segmentation method based on deep learning
CN110852733A (en) * 2019-10-22 2020-02-28 杭州效准智能科技有限公司 Intelligent catering settlement system based on RFID fusion dish image matching identification
CN111047799A (en) * 2019-11-29 2020-04-21 北京三快在线科技有限公司 Cash register device, article identification method and storage medium
CN111178274A (en) * 2019-12-30 2020-05-19 秒针信息技术有限公司 Information prompting method and device, computer readable storage medium and electronic device
CN111369524A (en) * 2020-03-02 2020-07-03 北京推想科技有限公司 Nodule recognition model training method, nodule recognition method and device
CN111680603A (en) * 2020-05-28 2020-09-18 浙江师范大学 Dish detection and identification method
CN112115901A (en) * 2020-09-25 2020-12-22 广州市派客朴食信息科技有限责任公司 High-accuracy food identification method
CN112115903A (en) * 2020-09-25 2020-12-22 广州市派客朴食信息科技有限责任公司 Method for improving dish identification system identification precision based on deep learning
CN112115906A (en) * 2020-09-25 2020-12-22 广州市派客朴食信息科技有限责任公司 Open dish identification method based on deep learning target detection and metric learning
CN113255596A (en) * 2021-06-29 2021-08-13 中运科技股份有限公司 Intelligent video image analysis system and method based on big data
CN113343003A (en) * 2021-06-30 2021-09-03 上海歆广数据科技有限公司 Dining nutrition construction recording system and method
CN113470284A (en) * 2021-07-08 2021-10-01 广州五谷信息科技有限公司 Multi-screen ordering equipment and using method thereof

Similar Documents

Publication Publication Date Title
CN107122730A (en) Free dining room automatic price method
CN104077842B (en) Freestyle restaurant self-service payment device based on image identification and application method of device
CN110059654A (en) A kind of vegetable Automatic-settlement and healthy diet management method based on fine granularity identification
CN103208156B (en) A kind of food and drink Automated Clearing House system and method
CN107767590A (en) Automatic identification commercialization bar code electronic scale and Automatic identification method
CN108875669B (en) Commodity identification technology based on visible light and infrared image fusion
CN106096932A (en) The pricing method of vegetable automatic recognition system based on tableware shape
CN106952402A (en) A kind of data processing method and device
CN108345912A (en) Commodity rapid settlement system based on RGBD information and deep learning
CN109523428A (en) Intelligent food and drink settlement method, system, device and medium based on image recognition technology
CN101477729A (en) Self-help meal sale system and information processing method of the system
Aslan et al. Benchmarking algorithms for food localization and semantic segmentation
CN109508664A (en) A kind of vegetable identification pricing method based on deep learning
CN106056487A (en) Tableware-pattern-based pricing method of dish automatic identification system
CN110059586A (en) A kind of Iris Location segmenting system based on empty residual error attention structure
CN109615358B (en) Deep learning image recognition-based restaurant automatic settlement method and system
CN108550229A (en) A kind of automatic cash method of artificial intelligence
CN109918517A (en) A kind of wisdom purchase system
CN109145846A (en) Material microstructure intelligent recognition analysis system and analysis method
CN109754009A (en) Item identification method, device, vending system and storage medium
CN109712324A (en) A kind of automatic vending machine image-recognizing method, good selling method and vending equipment
CN110020603A (en) Self-checkout system and method based on convolutional neural networks
CN109377205A (en) A kind of cafeteria&#39;s intelligence settlement system based on depth convolutional network
CN105741139A (en) Automatic charging method for self-service restaurant and implementation apparatus thereof
CN102298134A (en) Batch detection method and device for withdrawn electric energy meters

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20180914

Address after: 200240 Room 301, 558 lane, Jinping Road, Minhang District, Shanghai, 301

Applicant after: SHANGHAI HAIJIAO NETWORK TECHNOLOGY CO., LTD.

Address before: 200240 room 702, Lane 58, Jinggu Middle Road, Minhang District, Shanghai, 702

Applicant before: Le Jinwei

TA01 Transfer of patent application right
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20170901

WD01 Invention patent application deemed withdrawn after publication